The mathematics under the hood: optimization methods, convergence guarantees, and the theory that explains when and why deep learning works. Explainers that take proofs seriously without losing the reader.
Jia He and Maggie X. Cheng from Illinois Institute of Technology have found a way to build GNNs with the same k-WL and k-FWL expressive power as the best known high-order networks — using a fraction of the parameters and…
Orthogonal Bases for Equivariant Graph Learning with Provable k-WL Expressive Power Read More »
Two researchers from the University of Minnesota and Rice University have cracked open a new frontier: bilevel optimization on Riemannian manifolds. Their algorithms, RieBO and RieSBO, achieve the same theoretical complexity as flat-space methods — unlocking meta-learning, robust estimation, and…
Riemannian Bilevel Optimization — When Machine Learning Leaves Flat Space Behind Read More »
A team from UC San Diego and EPFL’s Biomedical Imaging Group rigorously proved that shallow ReLU networks with random weights are well-defined non-Gaussian processes — and that even in the infinite-width limit, they can stay non-Gaussian depending on how the…
Random ReLU Neural Networks as Non-Gaussian Processes Read More »
A team from Renmin University of China, Tsinghua University, and the University of Hong Kong proved that the classical sparse-to-dense transition in functional data analysis shifts when the number of functional variables grows large — and derived the exact non-asymptotic…
From Sparse to Dense Functional Data in High Dimensions: Phase Transitions Revisited Read More »
A team from Seoul National University and Ewha Womans University pinpointed a root cause of robust overfitting — the model memorizes tricky outliers during adversarial training rather than learning from them. Their remedy, difficulty proportional label smoothing, costs almost nothing…
Why Hard Training Examples Hurt Neural Networks — And How DPLS Fixes It Read More »
Verónica Álvarez, Santiago Mazuelas, and Jose A. Lozano show that a single Kalman-filter-based methodology can unify multi-task learning, continual learning, domain adaptation, and concept drift — while being the first to provide computable, tight error-probability guarantees and a closed-form characterization…
Ery Arias-Castro and Elizabeth Coda from UC San Diego take an approach inspired by Lebesgue integration — starting from first principles — to show that three intuitive axioms about what a cluster ought to be uniquely recover Hartigan’s cluster tree,…
An Axiomatic Definition of Hierarchical Clustering: Why Hartigan Was Right All Along Read More »
A research team from three leading institutions has proven something nobody expected: when learning a prior across many tasks using the Gibbs algorithm, fast convergence rates are always guaranteed at the meta level — even when individual tasks refuse to…
Bayes Meets Bernstein: Why Meta-Learning Finally Gets Fast — A PAC-Bayes Breakthrough Read More »
A research team from three major U.S. universities has proven for the first time that bilevel optimization algorithms can reliably escape saddle points and find true local minima — solving a problem that has quietly undermined meta-learning, hyperparameter tuning, and…
Tianyi Lin, Chi Jin, and Michael I. Jordan from Columbia, Princeton, and UC Berkeley deliver the first rigorous nonasymptotic analysis of two-timescale gradient descent ascent — the algorithm that has powered GAN training in practice for years — establishing tight…
TTGDA: Two-Timescale Gradient Descent Ascent for Nonconvex Minimax Optimization Read More »