The mathematics under the hood: optimization methods, convergence guarantees, and the theory that explains when and why deep learning works. Explainers that take proofs seriously without losing the reader.
Support vector machines have been a workhorse of classification since Cortes and Vapnik introduced them in 1995. Feed the model a set of labeled examples and it finds a hyperplane that separates the classes with the widest possible margin. That…
Why Support Vector Machines Finally Get Honest Confidence Intervals Read More »
A biologist staring at a flow cytometry readout from a single cell experiment wants to know which protein in a signaling cascade is actually driving which. Not correlated with. Driving. That distinction, cause versus association, is the entire reason causal…
Dimension Reduction Speeds Up Causal Graph Discovery Read More »
Stochastic Bregman proximal gradient methods drop the Lipschitz assumption behind SGD’s convergence proofs, match optimal sample complexity, and resist gradient explosion where standard optimizers collapse.
The idea that batch size shapes generalization dates back at least to Keskar et al. (2017), who showed empirically that large-batch training reliably finds sharper minima and those minima tend to generalize worse. The explanation became common wisdom: stochastic noise…
Why Batch Size Changes What Your Neural Network Learns Read More »
Expected Improvement has a lot going for it. It has a closed-form expression, it naturally trades off exploration and exploitation, and decades of practitioners have found that it just works. Jones, Schonlau, and Welch popularized it in 1998, and it…
When Expected Improvement Falls Short and What EIC Does About It Read More »
A research team from the Chinese University of Hong Kong and Florida State University has delivered the first unified convergence theory for a broad class of score based generative models in 2-Wasserstein distance, and it shows that the forward process…
Wasserstein Convergence Guarantees for Score-Based Generative Models Read More »
Paul Dommel and Alois Pichler from TU Chemnitz developed a Taylor series based approach to approximate kernel functions in reproducing kernel Hilbert spaces. Their work produces the first non-exponential eigenfunction bounds ever recorded and opens the door to regularization parameters…
Bonet, Drumetz, and Courty from ENSAE, IMT Atlantique, and Universite Bretagne Sud extend the Sliced-Wasserstein distance to Cartan-Hadamard manifolds, covering hyperbolic spaces and symmetric positive definite matrices with both geodesic and horospherical projections, then prove theoretical guarantees and derive non-parametric…
Sliced-Wasserstein Distances and Flows on Cartan-Hadamard Manifolds Read More »
Sen Na at Georgia Tech and Michael Mahoney at UC Berkeley prove that a sketched, adaptive Stochastic SQP method achieves asymptotic normality for constrained nonlinear stochastic optimization — the first online estimator that handles equality constraints without ever computing a…
A team from the University of Virginia and Scaled Foundations has extended the celebrated Borkar-Meyn theorem to handle Markovian noise, unlocking the first rigorous almost-sure convergence guarantees for GTD(λ) and ETD(λ) — the two principal algorithms for tackling the deadly…