Kolmogorov-Arnold Networks

KAN: Kolmogorov-Arnold Networks proposes a different way to place the nonlinear functions in a neural network. A Multilayer Perceptron (MLP) applies a fixed activation function at each node after a linear transformation. A KAN places a learned univariate function on each edge, and each node sums its incoming values.

KAN and MLP

The paper is inspired by the Kolmogorov-Arnold representation theorem. For a continuous function on a suitable bounded domain, the theorem gives a representation built from univariate functions and sums. The paper turns that structure into a trainable network, then generalizes the two-layer representation to wider and deeper KANs. The theorem is an architectural motivation, not a guarantee that a practical KAN will beat an MLP on every task.

The comparison is easier to read as a table:

For a KAN layer with n_in inputs and n_out outputs, the paper writes the layer as a matrix of functions Phi = {phi_q,p}. Function phi_q,p receives input p and contributes to output q; the node then sums those contributions. This is the part that the phrase “activation on the edge” describes.

B-spline parameterization

The paper parameterizes each learned univariate function with a residual basis function and a B-spline:

phi(x) = w_b * b(x) + w_s * spline(x)
spline(x) = sum_i c_i * B_i(x)

Here, x is the one scalar entering an edge, b(x) is a simple basis function, and w_b and w_s are learned scale factors. The paper commonly uses b(x) = SiLU(x). Each B_i(x) is a local B-spline basis function defined over a grid, and c_i is its learned coefficient. If the grid has G intervals and the spline order is k, the implementation has G + k basis functions for that edge.

The grid matters because a basis function is active only over a local part of the input range. Increasing the number of intervals gives the spline more local degrees of freedom, but it also increases the parameter count and can lead to overfitting or difficult optimization. The paper updates grids from observed activations because a fixed bounded grid can become a poor fit as training changes the input range.

Open figure at full size

The paper’s theoretical parameter count is larger per layer than an equally wide MLP because each edge stores several spline coefficients. Its practical comparisons therefore vary width, depth, and grid size rather than matching only the number of layers. A smaller KAN can have fewer total parameters on a particular task, but that is an experimental result of the chosen architecture.

What the paper measured

The paper reports KAN and MLP comparisons on small-scale function fitting, special functions, partial differential equations, and interpretability tasks. The results are tied to those datasets, model shapes, grids, and training procedures.

One concrete comparison uses the knot signature classification task. The paper reports the following test accuracy and parameter counts:

The values come from the paper’s setup. They show one case where the KAN used fewer parameters and scored higher. They do not establish a general ranking of KANs over MLPs. In the same paper, the authors leave comparisons with more advanced MLP training setups for future work.

The edge functions also gave the authors a direct way to inspect which input variables affected the output in that experiment. That can make a fitted model easier to examine, but a plotted function is still evidence about the fitted model and dataset, not a complete explanation of every prediction.

References

• KAN: Kolmogorov-Arnold Networks
• KAN implementation and documentation
• The anatomy of a spline