Playgrounds › Noise Schedules › Exponential Schedule

Exponential Schedule

Originating Paper

Generative Modeling by Estimating Gradients of the Data Distribution

🧑‍🎓 Yang Song, Stefano Ermon

📍 NeurIPS 2019

📄 5,800 citations

⚠️ While not explicitly named "exponential" in the paper, this is the first paper where the noise follows a geometric progression (which scales exponentially).

Why this schedule?

The exponential schedule increases the noise variance βt\beta_t geometrically, following an exponential growth curve. At the beginning, it adds noise a lot slower than the linear schedule—which might sound unintuitive at first, since the name "exponential" suggests rapid growth. But this becomes apparent if you look at the comparative βt\beta_t in the plot below. During the late steps, the noise curve hits a steep upwards trajectory, rapidly destroying whatever signal is left so it safely reaches pure Gaussian noise at the final step.

By preserving the image longer, the model gets more "useful" timesteps with a healthy mix of signal and noise. In spirit, the exponential and cosine schedule are pursuing this same goal. But they differ in how they achieve it. The exponential schedule is defined directly in terms of βt\beta_t and the smooth shape of αˉt\bar\alpha_t is a consequence of its geometric progression. In contrast, the cosine schedule defines αˉt\bar\alpha_t directly to have the desired shape.

Formula

βt=βmin⁡ ⁣(βmax⁡βmin⁡) ⁣t−1T−1\beta_t = \beta_{\min}\!\left(\frac{\beta_{\max}}{\beta_{\min}}\right)^{\!\frac{t-1}{T-1}}
  • •tt = current timestep
  • •TT = total number of timesteps
  • •βt\beta_t = noise variance added at step tt
normalized values0.000.250.500.751.000.000.250.500.751.00
tt
αˉt\bar\alpha_t1.0000αt\alpha_t1.0000βt\beta_t0.0000SNR\mathrm{SNR}∞

βt\beta_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic

αˉt\bar\alpha_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic