Herath is a graduate student in Mathematics in the College of Arts & Sciences.

Abstract : Currently, artificial neural networks are the most popular approach to machine learning problems such as high-dimensional multivariate regression. Methods using sums of separable functions are designed to represent functions in high dimensions and can be applied to high-dimensional multivariate regression. Here we compare the ability of these two methods to approximate function spaces in order to assess their relative expressive power.

We show that translation of approximation results from one method to the other is possible under certain conditions. We identify general approximation schemes in both the single-layer and deep-layer settings that apply to both methods for approximating certain function classes. In particular, we show that sums of separable functions give the same error rates as neural networks for function classes such as Barron's functions and band-limited functions. Inspired by deep neural networks, we also introduce deep layer sums of separable functions that shows similar results as deep neural networks for functions with compositional structure.