The L2 Norm and Inner Products

Following up on our previous exploration of why signals are vectors…

This is where of the dark matter of digital signal processing starts — the invisible mathematical foundation that underlies everything you do, yet most practitioners never truly understand it.

Hilbert spaces aren’t just abstract math. They’re the core mathematical framework behind:

• Quantum mechanics (wave functions live in Hilbert spaces)
• Machine learning (optimization in function spaces)
• Modern DSP (everything from adaptive filters to compressed sensing)
• Communication theory (channel capacity and optimal detection)

Yet most engineers learn just enough to apply cookbook methods without understanding the underlying structure. This creates a fundamental ceiling — you can’t invent at the core level if you don’t understand the mathematical foundations that make everything work.

My attempt here is to demystify this math in a simple, elegant way.
Yes, this article is demanding. But it’s far less demanding than struggling through traditional textbooks that bury these insights under layers of geometric analogies and unmotivated definitions.

The payoff? Once you understand this foundation, you’ll see the deep connections between seemingly unrelated DSP (Digital Signal Processing) concepts. You’ll understand why certain algorithms work, when they fail, and most importantly — you’ll have the mathematical tools to innovate at deep level rather than just apply.

Now that we’ve established signals as vectors through the 8 axioms (Click here to read the previous blog post), let’s tackle the next fundamental question:

What makes a “length” a “length”?

The key insight comes from mathematicians like Hilbert who asked: What are the essential mathematical properties that make a “length” actually work as a length? Just as we discovered that vectors aren’t about arrows and directions but about algebraic properties, length and inner products aren’t about geometric intuition but about mathematical structure.

The Problem with Geometric Intuition

Many DSP texts try to extend geometric intuition from 2D → 3D → nD → ∞D. This approach completely misses the real power of mathematical abstraction. Too much geometric analogy addiction is really bad!

The profound insight is this: When we abstract to mathematical properties, we capture a vast universe of objects that can behave as “vectors,” “lengths,” or “inner products” — regardless of how weird or counterintuitive they seem.

Consider what can satisfy our vector axioms:

• Signals (our focus)
• Functions
• Matrices
• Probability distributions
• Solution sets to differential equations
• Polynomials
• Even abstract algebraic structures with no geometric interpretation whatsoever

The geometric 2D→3D→nD progression traps you in thinking about arrows and spatial relationships. But mathematical abstraction liberates you to recognize vector-like behavior in places that have nothing to do with geometry.

If you don’t grasp this freedom, pause here and think !!!

Try to feel the shift from “vectors are arrows in space” to “vectors are anything satisfying these 8 algebraic rules.” The universe of mathematical objects that can be “vectors” is breathtakingly vast and often completely divorced from geometric intuition.

The same applies to length and inner products. They’re not about spatial distance and angles — they’re about algebraic structures that happen to satisfy certain rules.

This isn’t just philosophical — it’s practical power. Once you recognize that signals satisfy vector axioms, you immediately inherit decades of linear algebra results, regardless of whether you can “visualize” your signal space.

What Makes a “Length” Actually Work?

Hilbert and others looked at length in 2D/3D geometry and asked: what mathematical properties are essential? They identified four requirements.

The Four Essential Properties of Length

For any operation ||·|| that measures “length” or “size”:

1. Non-negativity: ||x|| ≥ 0 for all x
👉Length can’t be negative

2. Definiteness: ||x|| = 0 if and only if x = 0
👉Only the zero vector has zero length

3. Homogeneity: ||αx|| = |α| ||x|| for any scalar α
👉Scaling input scales length proportionally

4. Triangle inequality: ||x + y|| ≤ ||x|| + ||y||
👉 Direct path is shortest

These aren’t geometric properties — they’re algebraic requirements.

Why L2 Specifically?

Many operations satisfy these four properties: L1 (sum of absolute values), L∞ (maximum absolute value), L2 (sum of squares), etc.

L2 is special because it satisfies an additional property — the parallelogram law:

||x + y||² + ||x — y||² = 2(||x||² + ||y||²)

This extra property is what enables inner products.

L2 Definition and Verification

For signals, L2 is defined as:

• Discrete: $||x||_2^2 = \sum_n |x[n]|^2$
• Continuous: $||x||_2^2 = \int |x(t)|^2 dt$

Think of the continuous not as a “limit of sums” in some geometric sense, but as another mathematical operation that happens to satisfy our four essential properties (Let us try to limit our geometric analogy weak mental models here.). Integration and summation are different operations that both produce valid length measures.

Verification for discrete signals:

1. Non-negativity:
   Since $|x[n]|^2 \ge 0$, their sum $\sum_{n} |x[n]|^2 \ge 0$

2. Definiteness:
   $$\sum_{n} |x[n]|^2 = 0 \implies |x[n]|^2 = 0 \text{ for all } n \implies x[n] = 0 \text{ for all } n$$

3. Homogeneity:
   $||\alpha x||_2^2 = \sum_n |\alpha x[n]|^2 = |\alpha|^2 \sum_n |x[n]|^2 = |\alpha|^2 ||x||_2^2$

4. Triangle inequality:
   Follows from Cauchy–Schwarz inequality

Verification for continuous signals:

1. Non-negativity:
   Since $|x(t)|^2 \ge 0$, their integral $\int |x(t)|^2 dt \ge 0$

2. Definiteness:
   $\int |x(t)|^2 dt = 0$ requires $|x(t)|^2 = 0$ almost everywhere,
   so $x(t) = 0$ almost everywhere

3. Homogeneity:
   $$ \ ||\alpha x||_2^2 = \int |\alpha x(t)|^2 dt = |\alpha|^2 \int |x(t)|^2 dt = |\alpha|^2 ||x||_2^2 $$

4. Triangle inequality:
   Follows from Cauchy–Schwarz inequality

Both summation and integration satisfy the same algebraic requirements. No geometric visualization required.

What Makes an Inner Product Actually Work?

Similarly, mathematicians asked: what are the essential properties of the dot product from 2D/3D that we need to preserve?

The Dot Product Pattern

In 2D/3D, the dot product $a \cdot b = a_1b_1 + a_2b_2 $ gives us:

• Projections: projection of $a$ onto $b$ is $\frac{a \cdot b}{|b|} b$
• Orthogonality: $a \cdot b = 0$ means vectors are perpendicular
• Angles: $\cos \theta = \frac{a \cdot b}{|a| |b|}$

The Three Essential Properties of Inner Products

For any operation defined as “⟨·,·⟩” between two signals to work like a dot product (and this could be ANY operation, not just pairwise multiplication and summation. It could be some completely bizarre mathematical operation that has nothing to do with multiplying corresponding components.) it has to satisfy the following three properties:

1. Conjugate symmetry: ⟨x, y⟩ = ⟨y, x⟩*

The asterisk (*) means “complex conjugate” — if you have a complex number a + bi, its conjugate is a — bi. For real numbers, the conjugate is just the number itself.

Why do we need this? When vectors can be complex, we need ⟨x, x⟩ to always give a real, positive number for measuring “length squared.” Without conjugation, ⟨x, x⟩ could be complex, which makes no sense for a length measure.

For real vectors: ⟨x, y⟩ = ⟨y, x⟩ (order doesn’t matter)
For complex vectors: ⟨x, y⟩ = ⟨y, x⟩* (order matters due to conjugation)

2. Linearity in first argument: ⟨αx + βz, y⟩ = α⟨x, y⟩ + β⟨z, y⟩

The inner product distributes over addition and scaling in the first slot.
As an example, if you scale a signal by α or add two signals together, the inner product behaves predictably.

3. Positive definiteness: ⟨x, x⟩ ≥ 0, with equality only for x = 0

The inner product of a vector with itself gives a non-negative number. Only the zero vector gives zero.

L2 Inner Product

For L2 spaces, the inner product is:

• Discrete: $||x||_2^2 = \sum_n |x[n]|^2$
• Continuous: $||x||_2^2 = \int |x(t)|^2 dt$

This directly extends the dot product pattern: sum of products of corresponding components.

The Polarization Identity: Connecting Length and Inner Products

Here’s the beautiful mathematical connection: Any length measure that satisfies the parallelogram law can be used to construct an inner product. And conversely, any inner product automatically gives you a length measure that satisfies the parallelogram law.

This is a deep mathematical theorem, not an obvious fact. It tells us that length measures and inner products are intimately connected — they’re essentially two sides of the same mathematical coin.

The polarization identity shows how to construct the inner product from any length measure satisfying the parallelogram law:

• For real numbers:
  $$ \langle x, y \rangle = \tfrac{1}{4} \left( ||x + y||_2^2 - ||x - y||_2^2 \right) $$

• For complex numbers:
  $$ \langle x, y \rangle = \tfrac{1}{4} \left( ||x + y||_2^2 - ||x - y||_2^2 + i \big( ||x + i y||_2^2 - ||x - i y||_2^2 \big) \right) $$

Deriving Inner Products from L2 Norms

Let’s see this in action. We defined our L2 length measures as (Remember that you can define any other L2 length measure if you want. But we chose the most common ones.):

• Discrete: $||x||_2^2 = \sum_n |x[n]|^2$
• Continuous: $||x||_2^2 = \int |x(t)|^2 dt$

Using the polarization identity, we can derive the corresponding inner products:

For discrete signals (For real signals):

$$ \langle x, y \rangle = \tfrac{1}{4} \left( ||x + y||_2^2 - ||x - y||_2^2 \right) $$

$$ = \tfrac{1}{4} \left( \sum_{n} (x[n] + y[n])^2 - \sum_{n} (x[n] - y[n])^2 \right) $$

$$ = \sum_{n} x[n]y[n] $$

$$ = \sum_{n} x[n]y[n]^* \quad \text{(\textit{Added the conjugate because this example is with real numbers})} $$

For continuous signals (For real signals):

$$ \langle x, y \rangle = \tfrac{1}{4} \left( ||x + y||_2^2 - ||x - y||_2^2 \right) $$

$$ = \tfrac{1}{4} \left( \int \big( |x|^2 + |y|^2 + 2xy \big) dt - \int \big( |x|^2 + |y|^2 - 2xy \big) dt \right) $$

$$ = \int x(t) y(t) dt $$

$$ = \int x(t) y^*(t) dt \quad \text{(\textit{Added the conjugate because this example is with real numbers})} $$

👉 Try it for complex signals yourself !

This isn’t geometric intuition — it’s pure algebra. Starting from our L2 length measures, the polarization identity automatically gives us the familiar inner product formulas.

Why This Matters for DSP

This axiomatic approach directly enables:

Projections: Just like in 2D/3D, but now for signals

• proj_y(x) = (⟨x,y⟩/||y||)y
• Optimal approximation in signal subspaces

Orthogonality: Signals with ⟨x,y⟩ = 0 are orthogonal

• Fourier basis functions are orthogonal
• Uncorrelated signals in stochastic processes

Correlation: Inner products measure signal relationships

• Cross-correlation functions
• Matched filtering

Energy: $||x||_2^2$ directly gives signal energy

• Parseval’s theorem
• Energy-based optimization

The Key Insight

The power isn’t in visualizing high-dimensional spaces or extending geometric intuition (Such visualizations have limitations, not because you are not creative enough to visualize, because there is no direct geometric meaning available sometimes.). The power is in the algebraic structure that these axioms guarantee.

Whether you’re working with:

• 10-sample discrete signals
• 10,000-sample audio files
• Continuous-time waveforms
• Stochastic processes, etc.

If they satisfy the axioms, you get all the same mathematical machinery.

Next: Hilbert Spaces

In our next post, we’ll complete this framework by exploring what happens when we add one more crucial property: completeness. This final piece creates Hilbert spaces — the mathematical foundation that makes modern signal processing or quantum mechanics possible.

The geometric intuition was just the starting point. The real power comes from understanding what mathematical properties actually matter.