Optimization & Learning Theory

The mathematics under the hood: optimization methods, convergence guarantees, and the theory that explains when and why deep learning works. Explainers that take proofs seriously without losing the reader.

Random ReLU Neural Networks as Non-Gaussian Processes.

Random ReLU Neural Networks as Non-Gaussian Processes

Random ReLU Neural Networks as Non-Gaussian Processes | AI Trend Blend AITrendBlend Machine Learning Computer Vision About Neural Network Theory · Journal of Machine Learning Research 26 (2025) 1–31 · 16 min read Wide Neural Networks Are Not Always Gaussian — Here’s the Proof A team from UC San Diego and EPFL’s Biomedical Imaging Group […]

Random ReLU Neural Networks as Non-Gaussian Processes Read More »

From Sparse to Dense Functional Data in High Dimensions: Phase Transitions Revisited.

From Sparse to Dense Functional Data in High Dimensions: Phase Transitions Revisited

From Sparse to Dense Functional Data in High Dimensions: Phase Transitions Revisited | AI Trend Blend AITrendBlend Machine Learning Math About Functional Data Analysis · Journal of Machine Learning Research 26 (2025) 1–40 · 18 min read When Does Sampling Density Actually Matter? Phase Transitions in High-Dimensional Functional Data, Revisited A team from Renmin University

From Sparse to Dense Functional Data in High Dimensions: Phase Transitions Revisited Read More »

Why Hard Training Examples Hurt Neural Networks — And How DPLS Fixes It.

Why Hard Training Examples Hurt Neural Networks — And How DPLS Fixes It

Why Hard Training Examples Hurt Neural Networks — And How DPLS Fixes It | AI Trend Blend AITrendBlend Machine Learning Adversarial AI About Adversarial Robustness · Journal of Machine Learning Research 26 (2025) 1–48 · 16 min read Why Hard Training Examples Are Secretly Sabotaging Your Neural Network’s Robustness A team from Seoul National University

Why Hard Training Examples Hurt Neural Networks — And How DPLS Fixes It Read More »

Teaching Machines That the World Keeps Changing: Supervised Learning with Evolving Tasks and Performance Guarantees

Teaching Machines That the World Keeps Changing: Supervised Learning with Evolving Tasks and Performance Guarantees

Teaching Machines That the World Keeps Changing: Supervised Learning with Evolving Tasks and Performance Guarantees | AI Trend Blend AITrendBlend Machine Learning Math About Continual Learning · Journal of Machine Learning Research 26 (2025) 1–59 · BCAM · University of the Basque Country · 22 min read Teaching Machines That the World Keeps Changing: One

Teaching Machines That the World Keeps Changing: Supervised Learning with Evolving Tasks and Performance Guarantees Read More »

An Axiomatic Definition of Hierarchical Clustering: Why Hartigan Was Right All Along.

An Axiomatic Definition of Hierarchical Clustering: Why Hartigan Was Right All Along

An Axiomatic Definition of Hierarchical Clustering: Why Hartigan Was Right All Along | AI Trend Blend AITrendBlend Machine Learning Math Computer Vision About Statistical Learning Theory · Journal of Machine Learning Research 26 (2025) 1–26 · 18 min read Three Rules That Define What a Cluster Actually Is: The Axiomatic Case for Hartigan’s Cluster Tree

An Axiomatic Definition of Hierarchical Clustering: Why Hartigan Was Right All Along Read More »

Bayes Meets Bernstein: Why Meta-Learning Finally Gets Fast — A PAC-Bayes Breakthrough.

Bayes Meets Bernstein: Why Meta-Learning Finally Gets Fast — A PAC-Bayes Breakthrough

Bayes Meets Bernstein: Why Meta-Learning Finally Gets Fast — A PAC-Bayes Breakthrough | AI Trend Blend AITrendBlend Machine Learning Math About Machine Learning Theory · Journal of Machine Learning Research 26 (2025) · University of Tokyo, ESSEC Business School & CNRS / Sorbonne Université · 16 min read Why Meta-Learning Suddenly Gets Smarter the More

Bayes Meets Bernstein: Why Meta-Learning Finally Gets Fast — A PAC-Bayes Breakthrough Read More »

Escaping Saddle Points in Bilevel Optimization — A Breakthrough in Machine Learning Theory.

Escaping Saddle Points in Bilevel Optimization — A Breakthrough in Machine Learning Theory

Escaping Saddle Points in Bilevel Optimization — A Breakthrough in Machine Learning Theory | AI Trend Blend AITrendBlend Machine Learning Math About Machine Learning Theory · Journal of Machine Learning Research 26 (2025) · UC Davis, University at Buffalo & Rice University · 18 min read Why Your Machine Learning Model Gets Stuck — And

Escaping Saddle Points in Bilevel Optimization — A Breakthrough in Machine Learning Theory Read More »

TTGDA: Two-Timescale Gradient Descent Ascent for Nonconvex Minimax Optimization.

TTGDA: Two-Timescale Gradient Descent Ascent for Nonconvex Minimax Optimization

TTGDA: Two-Timescale Gradient Descent Ascent for Nonconvex Minimax Optimization | AI Trend Blend Optimization Theory · Journal of Machine Learning Research 26 (2025) 1–45 · 19 min read The Two Clocks That Fixed GAN Training: A Complete Theory of Two-Timescale Gradient Descent Ascent Tianyi Lin, Chi Jin, and Michael I. Jordan from Columbia, Princeton, and

TTGDA: Two-Timescale Gradient Descent Ascent for Nonconvex Minimax Optimization Read More »

How to Tune a Robust Regression Model Without Knowing the Noise: Adaptive Error Estimation for Unregularized M-Estimators.

How to Tune a Robust Regression Model Without Knowing the Noise: Adaptive Error Estimation for Unregularized M-Estimators

How to Tune a Robust Regression Model Without Knowing the Noise: Adaptive Error Estimation for Unregularized M-Estimators | AI Trend Blend AITrendBlend Machine Learning Computer Vision About High-Dimensional Statistics · Journal of Machine Learning Research 26 (2025) 1–40 · Rutgers University · University of Chicago · 18 min read You Can Tune a Robust Regression

How to Tune a Robust Regression Model Without Knowing the Noise: Adaptive Error Estimation for Unregularized M-Estimators Read More »

Why Your AI Says It's Confident When It Shouldn't Be — And How MaxWEnt Fixes It.

Why Your AI Says It’s Confident When It Shouldn’t Be — And How MaxWEnt Fixes It

Why Your AI Says It’s Confident When It Shouldn’t Be — And How MaxWEnt Fixes It | AI Trend Blend AITrendBlend Machine Learning Math Applications About Machine Learning Safety · Journal of Machine Learning Research 26 (2025) · Michelin & ENS Paris-Saclay · 18 min read Why Your AI Says It’s Confident When It Shouldn’t

Why Your AI Says It’s Confident When It Shouldn’t Be — And How MaxWEnt Fixes It Read More »