Playgrounds › Noise Schedules › Linear Schedule

Linear Schedule

Originating Paper

Denoising Diffusion Probabilistic Models

🧑‍🎓 Jonathan Ho, Ajay Jain, Pieter Abbeel

📍 NeurIPS 2020

📄 27,291 citations

Why this schedule?

The linear schedule is the simplest and most straightforward noise schedule, introduced in the original DDPM paper. It adds noise at a constant rate throughout the diffusion process: βt\beta_t grows uniformly from a small initial value to a final value. While easy to implement and reason about, it is quite inefficient for training, especially at higher resolutions. The issue is: the signal-to-noise ratio (SNR) plummets very quickly. By the time the forward process is only ~70% complete, the original image is practically indistinguishable from noise. This means the model wastes the last 30% of the process essentially adding noise to noise, which is an inefficient use of its training capacity. The cosine schedule was specifically designed to address this problem.

Formula

βt=β1+t−1T−1(βT−β1)\beta_t = \beta_1 + \frac{t - 1}{T - 1}(\beta_T - \beta_1)
  • •tt = current timestep
  • •TT = total number of timesteps
  • •βt\beta_t = noise variance added at step tt
normalized values0.000.250.500.751.000.000.250.500.751.00
tt
αˉt\bar\alpha_t1.0000αt\alpha_t1.0000βt\beta_t0.0000SNR\mathrm{SNR}∞

βt\beta_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic

αˉt\bar\alpha_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic