Calculus Explained: Derivatives, Differentiation and Integrals

Calculus is the mathematics of change. A derivative measures how fast a quantity is changing at one instant (the slope of a curve at a point), an integral adds up a quantity that accumulates over an interval (the area under a curve), and the Fundamental Theorem of Calculus proves the two are inverse operations. Newton and Leibniz developed it independently in the 1660s and 1670s, and it now runs underneath physics, engineering, economics and the gradient descent that trains machine learning models.

Below: limits, derivatives and their rules, integrals and the fundamental theorem, each with worked examples, then two short Python programs (numerical differentiation and gradient descent) and the floating point traps behind them. Safeguard builds security tools, not maths tools, and nothing here suggests otherwise.

Key takeaways

• A limit is the value a function approaches as its input approaches a point. Everything in calculus is defined with limits.
• The derivative of f at a is the limit of the difference quotient (f(a + h) - f(a)) / h as h shrinks to 0. It is the slope of the tangent line, or an instantaneous rate such as velocity.
• Four rules do most of the work: the power rule, the product rule, the quotient rule and the chain rule. The chain rule is the one machine learning depends on.
• The definite integral is the limit of sums of thin rectangles. The indefinite integral (antiderivative) is a family of functions whose derivative is the integrand, written with + C.
• The Fundamental Theorem of Calculus says that to compute a definite integral you can find any antiderivative F and evaluate F(b) - F(a).
• Newton worked out his "method of fluxions" in 1665 to 1666 but did not publish it; Leibniz published first, in 1684, and his d and ∫ notation is the one we still use.
• Numerical derivatives fight truncation error (step too big) and rounding error (step too small). For double precision, a central difference with h near 1e-6 beats a forward difference.
• Gradient descent computes the slope of a loss function and steps downhill. Too large a step makes it diverge, as the code below shows.

What is calculus, in one paragraph?

Calculus has two halves that turn out to be one subject. Differential calculus asks how fast something is changing right now: the speed of a car at 3.2 seconds, or how much a model's error changes when you nudge one weight. Integral calculus asks how much has built up: the distance a car covered, or the area of an irregular shape. The surprise is that the same machinery answers both questions. If you know a rate, integration recovers the total. If you know a total as a function of time, differentiation recovers the rate.

MIT's 18.01 course and OpenStax's free Calculus Volume 1 both follow that arc: differentiation and its applications, then the definite integral and the fundamental theorem, then techniques of integration. This guide does too.

Who invented calculus: Newton or Leibniz?

Both, independently, and neither from scratch. The ideas behind calculus are ancient; the general methods and notation arrived in the 1660s to 1680s; the rigorous foundations came in the nineteenth century.

The long run up

According to the MacTutor History of Mathematics archive at the University of St Andrews, the Greek method of exhaustion (approximating an area by filling it with shapes whose areas you know) was put on a scientific basis by Eudoxus around 370 BC. Around 225 BC Archimedes used it to show that a segment of a parabola has 4/3 the area of a triangle with the same base and vertex. That is an integral in everything but name.

In the seventeenth century Cavalieri, Fermat and Roberval pushed further. Isaac Barrow, Newton's teacher at Cambridge, treated the tangent as the limit of a chord, and MacTutor credits him as the first to recognise that integration and differentiation are inverse operations, though he never stated the fundamental theorem in general form.

Newton's fluxions, 1665 to 1671

MacTutor's Newton biography records that the plague closed Cambridge in summer 1665 and Newton, still under 25, went home to Lincolnshire, where in under two years he laid the foundations of differential and integral calculus. He called it the method of fluxions: a flowing quantity is a "fluent" and its rate of change is its "fluxion". His key insight was that integration is the inverse of differentiation.

He wrote a tract on fluxions in October 1666, and in July 1669 Barrow sent Newton's De Analysi to John Collins in London. A fuller treatment, De Methodis Serierum et Fluxionum, written in 1671, was not printed until John Colson's English translation of 1736. Cambridge University Library holds many of Newton's manuscripts, including the "Waste Book", digitised in the Cambridge Digital Library.

Leibniz's notation, 1675 to 1686

Gottfried Wilhelm Leibniz worked out his version in Paris. MacTutor's Leibniz biography dates the first use of the ∫ f(x) dx notation to a manuscript of 21 November 1675, which also states the product rule. By autumn 1676 he had the power rule, d(x^n) = n x^(n - 1) dx, for integer and fractional n.

In 1684 he published Nova Methodus pro Maximis et Minimis in the Leipzig journal Acta Eruditorum. MacTutor notes it contained the d notation and the rules for differentiating powers, products and quotients, but no proofs; Jacob Bernoulli called it "an enigma rather than an explanation". The ∫ symbol first appeared in print in a 1686 paper, and Jacob Bernoulli suggested the name "integral calculus" in 1690.

The priority dispute

According to MacTutor, in 1711 Leibniz read a paper by John Keill in the Royal Society's Transactions accusing him of plagiarism. The Royal Society set up a committee that did not ask Leibniz for his account; its report, finding for Newton, was written by Newton himself and published as Commercium epistolicum near the beginning of 1713. MacTutor's Newton biography calls Leibniz's discovery independent, and that is the settled view today: Newton first, Leibniz first in print, and Leibniz's notation the one that won.

Rigour arrives later

Early calculus rested on shaky "infinitely small" quantities, and George Berkeley attacked its logic in The Analyst in 1734. A satisfactory foundation waited for Augustin Louis Cauchy, whose Cours d'analyse of 1821 developed the basic theorems rigorously. In 1872, according to MacTutor, Karl Weierstrass found a function that is continuous everywhere yet has no derivative at any point.

A dated timeline

All dates below come from the MacTutor pages cited in the Sources list.

What is a limit, and why does calculus need it?

A limit answers the question "what value does f(x) get close to as x gets close to a?" It does not care what happens exactly at a, only nearby. That is the trick that lets calculus divide by something that is about to become zero without ever actually dividing by zero.

Worked example: a limit you cannot get by plugging in

Take f(x) = (x^2 - 9) / (x - 3). At x = 3 the formula gives 0/0, which is undefined. Look at nearby values instead:

The values close in on 6 from both sides. Algebra confirms it: x^2 - 9 factors as (x - 3)(x + 3), so for every x other than 3 the function equals x + 3, which approaches 6. We write:

lim (x → 3) of (x^2 - 9) / (x - 3) = 6

Continuity and when limits fail

A function is continuous at a when three things hold: f(a) exists, the limit as x approaches a exists, and the two are equal. Polynomials, sine, cosine and exponentials are continuous everywhere. A limit can fail to exist when the left and right sides disagree (a step function), when the function grows without bound (1/x near 0), or when it oscillates forever (sin(1/x) near 0).

The modern definition, the one Cauchy and Weierstrass made precise, says: the limit of f(x) as x approaches a is L if, for every tolerance you choose around L, there is a window around a inside which f(x) stays within that tolerance. You do not need that machinery to use calculus, but it is why the subject is logically sound.

What is a derivative?

The derivative of a function at a point is its instantaneous rate of change there. OpenStax's Calculus Volume 1 defines it as:

f'(a) = lim (h → 0) of [ f(a + h) - f(a) ] / h      (provided the limit exists)

The fraction inside is the difference quotient: the slope of the secant line through two points on the curve, a distance h apart. As h shrinks, the secant line pivots into the tangent line, and its slope becomes the derivative. NIST's Digital Library of Mathematical Functions (DLMF, section 1.4) gives the same definition and builds the standard results on it.

Worked example: the derivative of x^2 at x = 3, from the definition

Step 1. Write the difference quotient with f(x) = x^2 and a = 3:

[ (3 + h)^2 - 3^2 ] / h

Step 2. Expand the square: (3 + h)^2 = 9 + 6h + h^2. Subtract 9:

[ 6h + h^2 ] / h

Step 3. Divide by h (allowed, because h is not zero while we take the limit):

6 + h

Step 4. Let h approach 0. The result is 6. So f'(3) = 6: at x = 3 the parabola is rising 6 units for each unit to the right.

A numerical check makes the limit visible: h = 0.1 gives 6.1, h = 0.01 gives 6.01, h = 0.001 gives 6.001.

Worked example: the derivative of x^2 everywhere

Repeat the steps with a general x instead of 3:

[ (x + h)^2 - x^2 ] / h = [ 2xh + h^2 ] / h = 2x + h  →  2x

So the derivative function is f'(x) = 2x. At x = 3 it gives 6, matching the previous example. Doing the same for x^3 uses (x + h)^3 = x^3 + 3x^2 h + 3x h^2 + h^3, and the limit is 3x^2. The pattern (bring the power down, reduce it by one) is the power rule.

When a derivative does not exist

The absolute value function |x| is continuous at 0 but has a corner there. Coming from the right the slope is +1; from the left it is -1. The two one sided limits of the difference quotient disagree, so |x| has no derivative at 0. Every differentiable function is continuous, but not every continuous function is differentiable, and Weierstrass's 1872 example is the extreme case: continuous everywhere, differentiable nowhere.

Notation you will meet

The same derivative appears as f'(x) (prime notation), dy/dx (Leibniz), ẏ with a dot over the letter (Newton's fluxion notation, still used in physics for time derivatives) and ∂f/∂x for a partial derivative of a function of several variables. Second derivatives are written f''(x) or d²y/dx².

Derivative rules, with a reference table

The rules below are each proved once from the definition, then reused. OpenStax covers them in sections 3.3 (basic rules) and 3.6 (chain rule); DLMF section 1.4 states the chain rule as: for h(x) = f(g(x)), h'(x) = f'(g(x)) g'(x).

Angles are in radians. The sine and cosine rules are false in degrees unless you add a conversion factor of π/180, which is one of the most common sources of wrong answers in code.

Worked example: the power rule on a polynomial

Differentiate f(x) = 4x^5 - 3x^2 + 7x - 2. Use the sum rule to go term by term, the constant multiple rule to keep the coefficients, and the power rule on each power:

d/dx [4x^5]  = 4 · 5x^4 = 20x^4
d/dx [-3x^2] = -3 · 2x  = -6x
d/dx [7x]    = 7
d/dx [-2]    = 0

f'(x) = 20x^4 - 6x + 7

The power rule also covers roots and reciprocals once you rewrite them as powers. The square root of x is x^(1/2), whose derivative is (1/2) x^(-1/2), or 1 / (2 √x). The function 1/x^2 is x^(-2), whose derivative is -2x^(-3), or -2 / x^3.

Worked example: why the product rule is not "multiply the derivatives"

OpenStax makes this point directly: the derivative of a product is not the product of the derivatives. A quick counterexample shows why. Let f(x) = x and g(x) = x. Their product is x^2, whose derivative is 2x. The product of the derivatives is 1 · 1 = 1. Those are different.

Now apply the real rule to f(x) = x^2 sin(x):

first factor  u = x^2      u' = 2x
second factor v = sin(x)   v' = cos(x)

f'(x) = u'v + uv' = 2x sin(x) + x^2 cos(x)

Worked example: the quotient rule

Differentiate f(x) = (x^2 + 1) / (x - 1):

top    = x^2 + 1    top'    = 2x
bottom = x - 1      bottom' = 1

f'(x) = [ 2x (x - 1) - (x^2 + 1)(1) ] / (x - 1)^2
      = [ 2x^2 - 2x - x^2 - 1 ] / (x - 1)^2
      = (x^2 - 2x - 1) / (x - 1)^2

A memory aid: "low d high minus high d low, over low squared". The order matters, because subtraction is not symmetric.

Worked example: the chain rule

The chain rule handles a function inside another function. Differentiate the outer function with the inner one left alone, then multiply by the derivative of the inner function.

f(x) = (3x^2 + 1)^5
outer: u^5        derivative 5u^4
inner: u = 3x^2 + 1   derivative 6x

f'(x) = 5 (3x^2 + 1)^4 · 6x = 30x (3x^2 + 1)^4

Two more, because the chain rule is where most mistakes happen:

d/dx [ sin(x^2) ]        = cos(x^2) · 2x
d/dx [ e^(-x^2 / 2) ]    = e^(-x^2 / 2) · (-x) = -x e^(-x^2 / 2)

In Leibniz notation the chain rule reads dy/dx = (dy/du)(du/dx). These are not really fractions, but the notation makes the bookkeeping come out right.

Worked example: rules in combination

Differentiate f(x) = x e^(2x). It is a product, and the second factor needs the chain rule:

u = x        u' = 1
v = e^(2x)   v' = e^(2x) · 2

f'(x) = 1 · e^(2x) + x · 2e^(2x) = e^(2x) (1 + 2x)

Setting f'(x) = 0 gives 1 + 2x = 0, so x = -1/2 is the only place the slope is flat. That is how derivatives find maxima and minima, covered next.

What are derivatives used for?

Velocity and acceleration

If s(t) is position at time t, then the derivative s'(t) is velocity and the second derivative s''(t) is acceleration.

Worked example: a dropped object. Ignoring air resistance and using g ≈ 9.8 m/s², an object dropped from rest falls s(t) = 4.9 t^2 metres in t seconds.

s(t)  = 4.9 t^2          position (metres fallen)
s'(t) = 9.8 t            velocity (m/s)
s''(t) = 9.8             acceleration (m/s²), constant

At t = 2 s: fallen 19.6 m, moving at 19.6 m/s

Worked example: a car that stops and reverses. Let s(t) = t^3 - 6t^2 + 9t metres for t in seconds.

v(t) = s'(t) = 3t^2 - 12t + 9 = 3 (t - 1)(t - 3)
a(t) = v'(t) = 6t - 12

The velocity is zero at t = 1 and t = 3, so the car stops twice. Between those times v(t) is negative (it reverses). The acceleration is zero at t = 2, the moment it is reversing fastest.

Optimisation: finding the best value

At a smooth maximum or minimum the tangent line is flat, so the derivative is zero. DLMF section 1.4 states the condition: if f has a local maximum or minimum at an interior point c and is differentiable there, then f'(c) = 0. The second derivative tells you which: negative means a peak, positive means a valley.

Worked example: the biggest open box. Cut a square of side x from each corner of a 30 cm by 30 cm sheet and fold up the sides. What x gives the largest volume?

Step 1. Volume as a function of x. The base is (30 - 2x) on each side and the height is x:

V(x) = x (30 - 2x)^2 = 4x^3 - 120x^2 + 900x,   for x between 0 and 15

Step 2. Differentiate and set to zero:

V'(x) = 12x^2 - 240x + 900 = 12 (x^2 - 20x + 75) = 12 (x - 5)(x - 15)

Step 3. The critical points are x = 5 and x = 15. At x = 15 the base has zero width, so the volume is 0, a minimum.

Step 4. Check x = 5 with the second derivative: V''(x) = 24x - 240, so V''(5) = -120, which is negative. That makes x = 5 a maximum.

Step 5. Volume: V(5) = 5 · 20^2 = 2,000 cubic centimetres.

Newton's method: using tangents to solve equations

MIT's 18.01 course covers Newton's method alongside optimisation. To solve f(x) = 0, start with a guess, follow the tangent line down to where it crosses zero, and repeat:

x_next = x - f(x) / f'(x)

Worked example: the square root of 2. Solve f(x) = x^2 - 2 = 0, so f'(x) = 2x. Starting at x = 1:

step 1: 1.5
step 2: 1.4166666666666667
step 3: 1.4142156862745099
step 4: 1.4142135623746899
step 5: 1.4142135623730951

The number of correct digits roughly doubles each step once you are close.

Partial derivatives and the gradient

Most real functions depend on more than one input. A partial derivative differentiates with respect to one variable while holding the others fixed:

f(x, y) = x^2 y + 3y
∂f/∂x = 2xy          (y treated as a constant)
∂f/∂y = x^2 + 3      (x treated as a constant)

Stack the partial derivatives into a vector and you have the gradient. At any point the gradient points in the direction of steepest increase, and its length says how steep that increase is. Head the opposite way and you go downhill as fast as possible, which is the whole idea of gradient descent.

What is an integral?

An integral adds up infinitely many infinitely thin pieces. There are two related kinds.

• The definite integral of f from a to b, written ∫ from a to b of f(x) dx, is a number: the signed area between the curve and the x axis over that interval (area below the axis counts as negative).
• The indefinite integral, written ∫ f(x) dx, is a family of functions: every F whose derivative is f. Because any constant has derivative zero, the family is written F(x) + C.

Worked example: area under x^2 by rectangles

Find the area under f(x) = x^2 from x = 0 to x = 3. Split the interval into n equal strips of width 3/n and build a rectangle on each strip, using the function value at the strip's right edge as the height. Adding them up gives a Riemann sum.

With n = 3 the widths are 1 and the heights are 1, 4 and 9, so the sum is 14. With n = 6 the widths are 0.5 and the heights are 0.25, 1, 2.25, 4, 6.25 and 9, which add to 22.75; times 0.5 that is 11.375. Using left edges instead gives lower estimates.

Both columns squeeze toward 9. The definite integral is defined as that limit. Exact algebra agrees: the right edge sum is (27/n^3) · n(n + 1)(2n + 1)/6, and as n grows without bound that tends to 27 · 2/6 = 9.

Antiderivatives you should know

Each row is a derivative rule read backwards. Check any antiderivative by differentiating it: you should get the original function back.

What does the Fundamental Theorem of Calculus say?

The theorem joins the two halves of the subject. OpenStax states it in two parts; DLMF section 1.4 gives the second part as: if F'(x) = f(x) with f continuous, then the integral from a to b of f(x) dx equals F(b) - F(a).

Part 1. If you define a function by accumulating area, A(x) = ∫ from a to x of f(t) dt, then A'(x) = f(x). The rate at which the area grows is the height of the curve at the moving edge. Integration builds a function whose derivative is the original.

Part 2. To compute a definite integral, find any antiderivative F and subtract: ∫ from a to b of f(x) dx = F(b) - F(a). The + C cancels in the subtraction, which is why you can drop it here.

Worked example: the box of rectangles, done in one line

The area under x^2 from 0 to 3, which took a thousand rectangles to approximate, is now:

F(x) = x^3 / 3
∫ from 0 to 3 of x^2 dx = F(3) - F(0) = 27/3 - 0 = 9

Worked example: Part 1 with the chain rule

Find the derivative of G(x) = ∫ from 0 to x^2 of cos(t) dt. Part 1 gives the derivative with respect to the upper limit as cos(upper limit), and because the upper limit is x^2 rather than x, the chain rule multiplies by its derivative:

G'(x) = cos(x^2) · 2x

Check: G(x) = sin(x^2), whose derivative by the chain rule is the same.

Worked example: displacement versus total distance

Return to the car with velocity v(t) = 3t^2 - 12t + 9 and integrate from t = 0 to t = 4. An antiderivative is s(t) = t^3 - 6t^2 + 9t.

displacement = s(4) - s(0) = (64 - 96 + 36) - 0 = 4 metres

The car ends 4 metres from its start, but it moved forward until t = 1, backward until t = 3, then forward again:

s(0) = 0,  s(1) = 4,  s(3) = 0,  s(4) = 4
total distance = |4 - 0| + |0 - 4| + |4 - 0| = 12 metres

The integral of velocity gives net change; the integral of speed gives distance.

Two integration techniques that undo derivative rules

Substitution undoes the chain rule

When the integrand contains a function and its derivative, substitute u for the inner function.

∫ 2x cos(x^2) dx
let u = x^2, so du = 2x dx
= ∫ cos(u) du = sin(u) + C = sin(x^2) + C

Integration by parts undoes the product rule

Rearranging the product rule gives ∫ u dv = u v - ∫ v du.

∫ x e^x dx
u = x      du = dx
dv = e^x dx   v = e^x

= x e^x - ∫ e^x dx = x e^x - e^x + C = e^x (x - 1) + C

Differentiate e^x (x - 1) with the product rule and you get e^x (x - 1) + e^x = x e^x, so the answer checks.

How do computers calculate derivatives?

There are three approaches, and they make different trade offs.

PyTorch's autograd documentation describes the automatic approach: during the forward pass it records operations into a directed acyclic graph whose leaves are the inputs and whose roots are the outputs, and by tracing that graph from roots to leaves it "can automatically compute the gradients using the chain rule". In other words, the chain rule from the table above, applied millions of times, is what trains a modern model.

Numerical differentiation in Python

The difference quotient translates straight into code. The forward difference uses f(x + h) and f(x). The central difference uses f(x + h) and f(x - h), which cancels more of the error. NIST's DLMF covers numerical differentiation formulas of this kind in section 3.4.

The program below differentiates sin at x = 1, where the exact answer is cos(1), and prints the error of both formulas for a range of step sizes. It uses only the standard library and was tested with python3 -I on Python 3.9.

import math

def forward_diff(f, x, h):
    return (f(x + h) - f(x)) / h

def central_diff(f, x, h):
    return (f(x + h) - f(x - h)) / (2 * h)

x0 = 1.0
exact = math.cos(x0)  # the true derivative of sin at x0
print(f"exact derivative: {exact:.15f}")
for h in [1e-1, 1e-2, 1e-4, 1e-6, 1e-8, 1e-10, 1e-12]:
    fwd = abs(forward_diff(math.sin, x0, h) - exact)
    cen = abs(central_diff(math.sin, x0, h) - exact)
    print(f"h={h:.0e}  forward error={fwd:.1e}  central error={cen:.1e}")

Output on a standard machine with IEEE 754 double precision:

exact derivative: 0.540302305868140
h=1e-01  forward error=4.3e-02  central error=9.0e-04
h=1e-02  forward error=4.2e-03  central error=9.0e-06
h=1e-04  forward error=4.2e-05  central error=9.0e-10
h=1e-06  forward error=4.2e-07  central error=2.8e-11
h=1e-08  forward error=1.4e-08  central error=3.0e-09
h=1e-10  forward error=5.8e-08  central error=5.8e-08
h=1e-12  forward error=4.3e-05  central error=1.2e-05

Reading the output: two errors pulling in opposite directions

For large h the error comes from the formula itself (truncation error). The forward difference error shrinks roughly in proportion to h: divide h by 100 and the error drops by about 100. The central difference error shrinks in proportion to h squared: divide h by 100 and the error drops by about 10,000. That is why the central column improves so much faster at first.

Then something odd happens. Past a certain point, making h smaller makes the answer worse. At h = 1e-12 the forward difference is about as wrong as it was at h = 1e-4. The cause is rounding error in floating point arithmetic.

The Python tutorial explains why: almost all platforms map Python floats to IEEE 754 binary64 values with 53 bits of precision, so most decimals are stored approximately. The gap between 1.0 and the next representable float, sys.float_info.epsilon in Python's documentation, is 2.220446049250313e-16. When h is tiny, f(x + h) and f(x) agree in almost every digit, the subtraction cancels those digits, and what is left is mostly rounding noise. Dividing by a tiny h then magnifies that noise.

The best step balances the two errors. A standard rule of thumb, consistent with the output above:

• Forward difference: h near the square root of epsilon, about 1.5e-8. The table's best forward result is at h = 1e-8.
• Central difference: h near the cube root of epsilon, about 6e-6. The table's best central result is at h = 1e-6, with an error near 3e-11.

Scale h to the size of x when x is far from 1, and prefer the central formula.

Gradient descent in Python

Gradient descent minimises a function by repeatedly stepping against its gradient. Google's Machine Learning Crash Course describes it as an iterative optimisation algorithm that finds the weights and bias with the lowest loss by repeatedly calculating the loss, working out which direction reduces it, and updating the parameters. It also notes that linear models have convex loss functions, so gradient descent finds the global minimum for them.

The program below fits a straight line y = w x + b to five points by minimising mean squared error. It computes the gradient twice: once with formulas derived by the chain rule, and once numerically with central differences, as a gradient check. Then it runs gradient descent at three learning rates. Tested with python3 -I on Python 3.9; standard library only.

# Fit y = w*x + b to five points by gradient descent on mean squared error.
xs = [0.0, 1.0, 2.0, 3.0, 4.0]
ys = [1.1, 2.9, 5.2, 7.1, 8.8]   # roughly y = 2x + 1
n = len(xs)

def loss(w, b):
    return sum((w * x + b - y) ** 2 for x, y in zip(xs, ys)) / n

def grad(w, b):
    # Partial derivatives from the chain rule:
    # dL/dw = (2/n) * sum((w*x + b - y) * x)
    # dL/db = (2/n) * sum(w*x + b - y)
    dw = 2 / n * sum((w * x + b - y) * x for x, y in zip(xs, ys))
    db = 2 / n * sum((w * x + b - y) for x, y in zip(xs, ys))
    return dw, db

def numeric_grad(w, b, h=1e-6):
    dw = (loss(w + h, b) - loss(w - h, b)) / (2 * h)
    db = (loss(w, b + h) - loss(w, b - h)) / (2 * h)
    return dw, db

# Gradient check: analytic and numeric gradients should agree closely.
print("analytic:", grad(0.5, 0.5))
print("numeric: ", numeric_grad(0.5, 0.5))

for lr in (0.05, 0.12, 0.16):
    w, b = 0.0, 0.0
    for step in range(1, 201):
        dw, db = grad(w, b)
        w, b = w - lr * dw, b - lr * db
        if step in (1, 10, 50, 200):
            print(f"lr={lr} step={step:3d}  w={w:9.4f}  b={b:9.4f}  loss={loss(w, b):.4g}")

Output:

analytic: (-19.92, -7.040000000000001)
numeric:  (-19.919999999729043, -7.040000001623525)
lr=0.05 step=  1  w=   1.3960  b=   0.5020  loss=3.634
lr=0.05 step= 10  w=   2.0551  b=   0.8287  loss=0.04306
lr=0.05 step= 50  w=   1.9883  b=   1.0193  loss=0.02058
lr=0.05 step=200  w=   1.9603  b=   1.0991  loss=0.0184
lr=0.12 step=  1  w=   3.3504  b=   1.2048  loss=12.21
lr=0.12 step= 10  w=   2.0068  b=   0.9202  loss=0.03021
lr=0.12 step= 50  w=   1.9631  b=   1.0911  loss=0.01843
lr=0.12 step=200  w=   1.9600  b=   1.1000  loss=0.0184
lr=0.16 step=  1  w=   4.4672  b=   1.6064  loss=43.07
lr=0.16 step= 10  w=  -6.0477  b=  -1.8601  loss=488.3
lr=0.16 step= 50  w=-1779.1519  b=-623.6830  loss=2.388e+07
lr=0.16 step=200  w=-1104215934283.0420  b=-387338035187.6110  loss=9.177e+24

What the output shows

The gradient check passes. The analytic and numerical gradients agree to about nine significant figures. If they disagreed beyond rounding noise, the hand derived formula would have a bug.

The derivation uses the chain rule. The loss is an average of squares of (w x + b - y). Differentiating a square brings down a 2 (power rule) and then multiplies by the derivative of the inside with respect to w, which is x, or with respect to b, which is 1 (chain rule). That gives the two formulas in the comments.

Learning rates 0.05 and 0.12 converge. The exact least squares solution is w = 1.96 and b = 1.1; at 0.12 the program has it to four decimal places by step 200, while 0.05 is still slightly short.

Learning rate 0.16 diverges. The loss explodes from 43 to about 9 × 10^24. For a quadratic loss like this one, gradient descent is stable only if the step size is below 2 divided by the largest curvature (the largest eigenvalue of the matrix of second derivatives). Here the second derivatives form the matrix (2/n) times [[30, 10], [10, 5]], that is [[12, 4], [4, 2]], whose largest eigenvalue is about 13.4. The limit is about 2 / 13.4, roughly 0.149, so 0.12 is safe and 0.16 overshoots further on every step.

Real training adds momentum, adaptive step sizes and mini batches on non convex loss surfaces, but the core is the loop above.

Where calculus meets software security

Safeguard's models are single purpose security models, and Safeguard's own benchmarks page says plainly that a frontier general purpose LLM will out-write any Safeguard model on math word problems. Nothing in this post suggests Safeguard does calculus for you. The overlap with security is in the code that does calculus.

Floating point is a correctness and security surface. The cancellation that ruined the h = 1e-12 derivative is the same effect that breaks equality checks on computed values. The Python tutorial recommends math.isclose() for comparing inexact values. Comparing floats with == in a financial check or permission threshold produces bugs on certain inputs; our post on CWE-1077 floating point comparison with the wrong operator covers that weakness class.

Divergence and non finite values propagate. The diverging run above produced numbers near 10^24 in a few hundred steps. In real code the same instability yields infinities and NaN values that flow silently into downstream decisions unless you check for them with math.isfinite().

Numerical libraries are dependencies. Production gradient descent runs inside open source numerical and machine learning libraries, installed from registries and shipped in container images. They receive security fixes like any other code, and an environment built months ago carries whatever versions were current then. This is where Safeguard does apply: it scans repositories, container images and SBOM files, including Python projects, matches components against vulnerability databases, shows which projects carry a given component and version, and uses reachability analysis to rank the vulnerable code paths your application actually calls. For the reachability idea in plain words, see what reachability analysis is.

The mathematics in this post has no CVEs, but the libraries that compute it are open source dependencies that need patching like anything else in your build. The section below is our standard guidance for closing that gap; it applies to those libraries and images, not to calculus itself.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

Common mistakes

• Treating the derivative of a product as the product of derivatives. It is f'g + fg'.
• Forgetting the inner derivative in the chain rule. The derivative of sin(3x) is 3 cos(3x), not cos(3x).
• Using degrees in trigonometric derivatives. d/dx sin(x) = cos(x) holds only in radians.
• Dropping + C on indefinite integrals. An antiderivative is a family, not a single function.
• Confusing displacement with distance. The integral of velocity gives net change; integrate the absolute value to get total distance, as the car example showed (4 metres versus 12).
• Assuming continuous means differentiable. |x| is continuous at 0 with no derivative there. Check corners, cusps and jumps before applying derivative based tests.
• Shrinking h as far as possible in numerical derivatives. Below the sweet spot, rounding error grows. For double precision, use roughly 1e-8 for forward differences and 1e-6 for central differences, scaled to the size of x.
• Comparing computed floats with ==. Use a tolerance, such as math.isclose() in Python.
• Picking a learning rate by habit. Too large diverges, as the 0.16 run showed; too small crawls. Watch the loss curve.
• Skipping the gradient check. If you derive gradients by hand, compare them with central differences on a few random points before trusting a training run.

FAQ

What is calculus in simple terms?

Calculus is the maths of change and accumulation. Derivatives measure how fast something changes at an instant, integrals add up how much it has accumulated, and the Fundamental Theorem of Calculus shows they undo each other.

Who invented calculus, Newton or Leibniz?

Both, independently. Newton developed fluxions in 1665 to 1666 but published little for decades; Leibniz published first, in 1684. Our notation (dy/dx and ∫) is Leibniz's.

What is a derivative used for in real life?

Velocity and acceleration, maximum profit or minimum cost, sensitivity of an output to an input, Newton's method, and gradient descent in machine learning.

What is the difference between differentiation and integration?

Differentiation takes a function and returns its rate of change. Integration takes a rate and returns the accumulated total, or an area. By the Fundamental Theorem of Calculus each reverses the other, up to a constant.

What is the chain rule and why does it matter?

The chain rule differentiates a function inside another: the derivative of f(g(x)) is f'(g(x)) times g'(x). Backpropagation is the chain rule applied through every layer of a network.

Why do we add + C to an integral?

Because constants vanish when you differentiate. x^2, x^2 + 5 and x^2 - 100 all have derivative 2x, so the antiderivative of 2x is the whole family x^2 + C. Definite integrals do not need it because it cancels.

What is the derivative of x^2?

2x. From the definition, [(x + h)^2 - x^2] / h simplifies to 2x + h, which approaches 2x as h approaches 0. The power rule gives the same result directly.

How do you calculate a derivative numerically in Python?

Use the central difference (f(x + h) - f(x - h)) / (2h) with h around 1e-6 times the scale of x. Smaller steps make results worse because of floating point rounding, as the example program shows.

How is calculus used in machine learning?

Training computes the gradient of a loss function with respect to every parameter, using the chain rule through automatic differentiation, then steps each parameter downhill with gradient descent or a variant.

See how Safeguard would help

None of these demos do calculus; they show how Safeguard handles the libraries your numerical code imports.

The first demo is a real Safeguard CLI session: asked to fix a lodash CVE, it reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, and pushes nothing. The same approve each step flow applies when a numerical library pinned in a Python project needs an upgrade and you want tests to run before anything changes.

Click through it at your own pace, or open it full screen.

The second demo sets up Safeguard's local runner on your own machine, approves it with its device code, and runs a scan from the terminal. On a workstation or build host used for training jobs, that scan lists the packages that went in, so you know what versions your experiments actually depend on.

Click through it at your own pace, or open it full screen.

The third demo connects Claude to the Safeguard MCP server with OAuth and lets you choose which tools Claude can use. You can then ask which of your projects carry a particular version of a library while keeping control over what the assistant can touch.

Click through it at your own pace, or open it full screen.

To try it on your own projects, download Safeguard or read the CLI documentation.

Quick check

Sources

• MIT OpenCourseWare: 18.01SC Single Variable Calculus, Fall 2010
• MIT OpenCourseWare: 18.01 Calculus I, Single Variable Calculus, Fall 2020
• OpenStax Calculus Volume 1, 3.1 Defining the Derivative
• OpenStax Calculus Volume 1, 3.3 Differentiation Rules
• OpenStax Calculus Volume 1, 3.6 The Chain Rule
• OpenStax Calculus Volume 1, 5.3 The Fundamental Theorem of Calculus
• NIST Digital Library of Mathematical Functions, 1.4 Calculus of One Variable
• NIST Digital Library of Mathematical Functions, 3.4 Differentiation (numerical methods)
• MacTutor History of Mathematics: The rise of calculus
• MacTutor History of Mathematics: Isaac Newton
• MacTutor History of Mathematics: Gottfried Wilhelm Leibniz
• MacTutor History of Mathematics: Isaac Barrow
• MacTutor History of Mathematics: Augustin Louis Cauchy
• MacTutor History of Mathematics: Karl Weierstrass
• Cambridge Digital Library: Newton Papers
• Python documentation: Floating point arithmetic, issues and limitations
• Python documentation: sys.float_info
• Python documentation: math module
• Google Machine Learning Crash Course: Gradient descent
• PyTorch documentation: Autograd mechanics
• Safeguard: Model benchmarks

Last reviewed: 23 October 2026.