In machine learning, we use the dot product every day. However, its...

@TivadarDanka
Tivadar Danka@TivadarDanka
10 views Jun 05, 2025 ~2 min read
Advertisement
1
In machine learning, we use the dot product every day.

However, its definition is far from revealing. For instance, what does it have to do with similarity?

There is a beautiful geometric explanation behind.
Media image
2
By definition, the dot product (or inner product) of two vectors is defined by the sum of coordinate products.
Media image
3
To peek behind the curtain, there are three key properties that we have to understand.

First, the dot product is linear in both variables. This property is called bilinearity.
Media image
4
Second, the dot product is zero if the vectors are orthogonal.

(In fact, the dot product generalizes the concept of orthogonality beyond Euclidean spaces. But that's for another day :) )
Media image
5
Third, the dot product of a vector with itself equals the square of its magnitude.
Media image
6
Now comes the interesting part.

Given a vector y, we can decompose any x into two components xₒ and xₚ. One is parallel to y, while the other is orthogonal to it.

In physics, we apply the same decomposition to various forces all the time.
Media image
7
The vectors xₒ and xₚ are characterized by two properties:

1. xₚ is a scalar multiple of y,
2. and xₒ is orthogonal to xₚ (and thus to y).

We are going to use these properties to find an explicit formula for xₚ.

Spoiler alert: it is related to the dot product.
Media image
8
Due to xₒ being orthogonal to y, we can use the bilinearity of the dot product to express the c in xₚ = c y.
Media image
9
By solving for c, we get that it is the ratio of the dot product and the magnitude of y.
Media image
10
If both x and y are unit vectors, the dot product is the magnitude of the orthogonal projection!
Media image
11
Do you recall how the famous trigonometric functions sine and cosine are defined?

Let's say that the hypotenuse of our right triangle is a unit vector and one of the legs is on the x-axis.

Then the trigonometric functions equal the magnitudes of the projections to the axes.
Media image
12
Using trigonometric functions, we see that the dot product of two unit vectors is the cosine of their enclosed angle α!

This is how the dot product relates to cosine.
Media image
13
If x and y are not unit vectors, we can scale them and use our previous discovery to get the cosine of α.
Media image
14
The closer to its value to 1, the more similar x and y are. (In a sense.)

In machine learning, we call this quantity the cosine similarity.

Now you understand why.
Media image
15
This explanation is directly from my new Mathematics of Machine Learning book, which you can get right now!

I have packed 20 years of math studies into 700 pages full of intuitive and application-oriented lessons, the ultimate learning resource for you.

amazon.com/Mathematics-Ma…
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement