Playgrounds › Noise Schedules › Cosine Schedule
Cosine Schedule
Originating Paper
Improved Denoising Diffusion Probabilistic Models🧑🎓 Alex Nichol, Prafulla Dhariwal
📍 ICML 2021
📄 4,892 citations
Why this schedule?
The cosine schedule was designed to fix the linear schedule's tendency to destroy information too quickly. The cosine-shaped curve provides a more gradual noise progression: the rate of change is highest near the midpoint (resembling a linear drop-off), but smooths out near and , ensuring that only minimal noise is added at the very beginning and very end. This delays the hardest denoising tasks until after the midpoint, leading to improved sample quality, better training efficiency, and faster convergence compared to the linear schedule.
Formula
- • = small offset () preventing from collapsing to zero at
- • = current timestep
- • = total number of timesteps
- • = cumulative signal retention
1.00001.00000.0000∞
across all schedules
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic
across all schedules
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic