Deconstructing What Makes a Good Optimizer for Language Models von Zhao, Rosie, Morwani, Depen, Brandfonbrener, David, Vyas, Nikhil, Kakade, Sham

Volltext bestellen

Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets von Morwani, Depen, Ramaswamy, Harish G

Volltext bestellen

How Does Critical Batch Size Scale in Pre-training? von Zhang, Hanlin, Morwani, Depen, Vyas, Nikhil, Wu, Jingfeng, Zou, Difan, Ghai, Udaya, Foster, Dean, Kakade, Sham

Volltext bestellen

Simplicity Bias in 1-Hidden Layer Neural Networks von Morwani, Depen, Batra, Jatin, Jain, Prateek, Netrapalli, Praneeth

Volltext bestellen

SOAP: Improving and Stabilizing Shampoo using Adam von Vyas, Nikhil, Morwani, Depen, Zhao, Rosie, Shapira, Itai, Brandfonbrener, David, Janson, Lucas, Kakade, Sham

Volltext bestellen

A New Perspective on Shampoo's Preconditioner von Morwani, Depen, Shapira, Itai, Vyas, Nikhil, Malach, Eran, Kakade, Sham, Janson, Lucas

Volltext bestellen

Using noise resilience for ranking generalization of deep neural networks von Morwani, Depen, Vashisht, Rahul, Ramaswamy, Harish G

Volltext bestellen

Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets von Morwani, Depen, Ramaswamy, Harish G

Volltext

Feature emergence via margin maximization: case studies in algebraic tasks von Morwani, Depen, Edelman, Benjamin L, Oncescu, Costin-Andrei, Zhao, Rosie, Kakade, Sham

Volltext bestellen

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning von Vyas, Nikhil, Morwani, Depen, Zhao, Rosie, Kaplun, Gal, Kakade, Sham, Barak, Boaz

Volltext bestellen

Feature-Learning Networks Are Consistent Across Widths At Realistic Scales von Vyas, Nikhil, Atanasov, Alexander, Bordelon, Blake, Morwani, Depen, Sainathan, Sabarish, Pehlevan, Cengiz

Volltext bestellen

Deconstructing What Makes a Good Optimizer for Language Models von Zhao, Rosie, Morwani, Depen, Brandfonbrener, David, Vyas, Nikhil, Kakade, Sham

Volltext

How Does Critical Batch Size Scale in Pre-training? von Zhang, Hanlin, Morwani, Depen, Vyas, Nikhil, Wu, Jingfeng, Zou, Difan, Ghai, Udaya, Foster, Dean, Kakade, Sham

Volltext

Simplicity Bias in 1-Hidden Layer Neural Networks von Morwani, Depen, Batra, Jatin, Jain, Prateek, Praneeth Netrapalli

Volltext

SOAP: Improving and Stabilizing Shampoo using Adam von Vyas, Nikhil, Morwani, Depen, Zhao, Rosie, Shapira, Itai, Brandfonbrener, David, Janson, Lucas, Kakade, Sham

Volltext

A New Perspective on Shampoo's Preconditioner von Morwani, Depen, Shapira, Itai, Vyas, Nikhil, Malach, Eran, Kakade, Sham, Janson, Lucas

Volltext

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning von Vyas, Nikhil, Morwani, Depen, Zhao, Rosie, Kaplun, Gal, Kakade, Sham, Barak, Boaz

Volltext

Using noise resilience for ranking generalization of deep neural networks von Morwani, Depen, Vashisht, Rahul, Ramaswamy, Harish G

Volltext

Feature emergence via margin maximization: case studies in algebraic tasks von Morwani, Depen, Edelman, Benjamin L, Costin-Andrei Oncescu, Zhao, Rosie, Kakade, Sham

Volltext

Feature-Learning Networks Are Consistent Across Widths At Realistic Scales von Vyas, Nikhil, Atanasov, Alexander, Bordelon, Blake, Morwani, Depen, Sainathan, Sabarish, Pehlevan, Cengiz

Volltext