Derivatives
If you take a closer look at how functions work, there's a really important characteristic that became a cornerstone of mathematics and really important for AI.
Take a look at how some functions change at every step
For the function :

but the rate of change between functions can be different
And this function :

How to discover the rate of change of this functions and why is it important?
The theory behind this is Calculus. It's a huge field of mathematics, so I'm not going to try to explain all of it here. Instead, we only need one idea for now: the derivative.
A good intuition is to think about the velocity of a car.
Imagine a car traveling along a road. We want to know its velocity at a specific moment.
We could measure how long it takes the car to travel from point A to point B:
If the car travels 100 meters in 5 seconds, its average velocity over that interval is:

But that's only the average velocity. The car could have accelerated, slowed down, or changed its speed somewhere between A and B. So what if we make the distance between A and B smaller?
As point B gets closer and closer to point A, the average rate of change over that tiny interval becomes a better approximation of the car's velocity at that particular moment.
In calculus, this idea is pushed further: what happens as the distance between the two points approaches zero? (this is called a limit)
The resulting value is the "instantaneous rate of change"
The formal math representation of a derivative is:
Finally, derivatives are used to make models learn through a technique called backpropagation. Over the years, a bunch of “shortcuts” have been developed to make calculating derivatives easier. One of them, called the chain rule, is essentially what backpropagation is all about.
This was just a short introduction, but learning this concept will make some algorithms really easier to interpret.
Sources:
- I really like this channel to learn more: https://www.youtube.com/watch?v=AegzQ_dip8k
- The James Stewart's Calculus book
- Khan academy : https://www.khanacademy.org/math/differential-calculus/dc-diff-intro