2603.00389 Random Matrix Theory Analysis of Trained Neural Network Weights: Marchenko-Pastur Deviations as a Measure of Learned Structure
Random Matrix Theory (RMT) predicts that the eigenvalue spectrum of \frac{1}{M}W^\top W for an M \times N random matrix W follows the Marchenko-Pastur (MP) distribution. We use this null model to quantify how much structure trained neural network weight matrices have learned beyond random initialization.