Playgrounds › Noise Schedules › Cosine Schedule

Cosine Schedule

Originating Paper

Improved Denoising Diffusion Probabilistic Models

🧑‍🎓 Alex Nichol, Prafulla Dhariwal

📍 ICML 2021

📄 4,892 citations

Why this schedule?

The cosine schedule was designed to fix the linear schedule's tendency to destroy information too quickly. The cosine-shaped αˉt\bar\alpha_t curve provides a more gradual noise progression: the rate of change is highest near the midpoint (resembling a linear drop-off), but smooths out near t=0t = 0 and t=Tt = T, ensuring that only minimal noise is added at the very beginning and very end. This delays the hardest denoising tasks until after the midpoint, leading to improved sample quality, better training efficiency, and faster convergence compared to the linear schedule.

Formula

αˉt=f(t)f(0),f(t)=cos⁡2 ⁣(t/T+s1+s⋅π2)\bar\alpha_t = \frac{f(t)}{f(0)},\quad f(t) = \cos^2\!\left(\frac{t/T + s}{1 + s}\cdot\frac{\pi}{2}\right)
  • •ss = small offset (≈0.008\approx 0.008) preventing βt\beta_t from collapsing to zero at t=0t = 0
  • •tt = current timestep
  • •TT = total number of timesteps
  • •αˉt=∏i=1tαi\bar\alpha_t = \prod_{i=1}^{t} \alpha_i = cumulative signal retention
normalized values0.000.250.500.751.000.000.250.500.751.00
tt
αˉt\bar\alpha_t1.0000αt\alpha_t1.0000βt\beta_t0.0000SNR\mathrm{SNR}∞

βt\beta_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic

αˉt\bar\alpha_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic