Skip to content

Function Approximation๐Ÿ”—

An interactive guide to Fourier basis, RBF, and neural networks.

1. Introduction to Function Approximation๐Ÿ”—

Function approximation is the task of finding a simpler function that closely matches a complex target function. This is fundamental in machine learning, signal processing, and numerical analysis.

Key Idea: Represent any function f(x) as a combination of simpler "basis functions":

\[f(x) \approx \sum_i w_i \phi_i(x)\]

where \(\phi_i(x)\) are basis functions and \(w_i\) are weights/coefficients.

Why Function Approximation?

  • Compression: Store complex functions using fewer parameters
  • Generalization: Learn patterns from data and predict on new inputs
  • Analysis: Understand function properties through basis decomposition
  • Computation: Replace expensive function evaluations with fast approximations

2. Fourier Basis Approximation๐Ÿ”—

Fourier approximation represents functions as sums of sine and cosine waves at different frequencies.

Fourier Series:

\[f(x) = a_0 + \sum_n [a_n\cos(nx) + b_n\sin(nx)]\]

Coefficients:

\[a_0 = \frac{1}{2\pi} \int f(x)dx\]
\[a_n = \frac{1}{\pi} \int f(x)\cos(nx)dx\]
\[b_n = \frac{1}{\pi} \int f(x)\sin(nx)dx\]

Interactive Fourier Approximation

5

Approximation

Individual Basis Functions

Key Properties
  • Global basis: Each term affects the entire domain
  • Frequency interpretation: Low frequencies = trends, high frequencies = details
  • Best for: Periodic signals, smooth functions
  • Gibbs phenomenon: Overshoot at discontinuities (~9%)

3. Radial Basis Functions (RBF)๐Ÿ”—

RBF networks represent functions as weighted sums of "bumps" centered at different points.

RBF Approximation:

\[f(x) = \sum_i w_i \phi(||x - c_i||)\]

Common RBF Types:

  • Gaussian: \(\phi(r) = \exp(-r^2/\sigma^2)\)
  • Multiquadric: \(\phi(r) = \sqrt{r^2 + \sigma^2}\)
  • Inverse multiquadric: \(\phi(r) = 1/\sqrt{r^2 + \sigma^2}\)

Interactive RBF Approximation

7
0.5

Approximation

Individual RBF Basis Functions

Key Properties
  • Local basis: Each RBF has limited influence
  • Spatial interpretation: Centers determine which regions are captured
  • Best for: Scattered data, non-periodic functions, interpolation
  • Meshless: Works in multiple dimensions without structured grid