ZARZA We are Zarza, the prestigious firm behind major projects in information technology.

Optimal Linear Decay Learning Rate Schedules and Further Refinements

Mechanical Dreams

05/01/2025 11 min Temporada 1 Episodio 30

Optimal Linear Decay Learning Rate Schedules and Further Refinements

Listen "Optimal Linear Decay Learning Rate Schedules and Further Refinements"

Descargar episodio

More episodes of the podcast Mechanical Dreams

DeepSeek OCR 24/10/2025

The Coverage Principle - How Pre-training Enables Post-Training 23/10/2025

Continual Learning via Sparse Memory Finetuning 22/10/2025

Untitled Episode 10/10/2025

Characterization and Mitigation of Training Instabilities in Microscaling Formats 08/10/2025

Demystifying Synthetic Data in LLM Pre-training- A Systematic Study of Scaling Laws, Benefits, and Pitfalls 07/10/2025

Data Mixing Can Induce Phase Transitions in Knowledge Acquisition 06/10/2025

Apertus Tech Report 21/09/2025

Learning Facts at Scale with Active Reading 20/09/2025

Fantastic Pretraining Optimizers and Where to Find Them 19/09/2025

Ver todos los episodios