Playgrounds › Noise Schedules › Sigmoid Schedule

Sigmoid Schedule

Originating Paper

Scalable Adaptive Computation for Iterative Generation

🧑‍🎓 Allan Jabri, David Fleet, Ting Chen

📍 arXiv 2022

📄 157 citations

Why this schedule?

The sigmoid schedule offers a smooth S-shaped transition similar to cosine, but with greater flexibility through its temperature parameter τ\tau. It begins with a gradual noise increase, accelerates in the middle, and smooths out again at the end. This characteristic enhances stability and mitigates abrupt transitions that could hurt sample quality. The sigmoid schedule is particularly effective for high-resolution images, where neighboring pixels exhibit greater redundancy. In practice, it demonstrates greater training stability than the cosine schedule on such images, though the difference becomes less significant at sampling time.

Formula

αˉt=−sigmoid ⁣(t(e−s)+sτ)+sigmoid ⁣(eτ)sigmoid ⁣(eτ)−sigmoid ⁣(sτ)\bar\alpha_t = \frac{-\mathrm{sigmoid}\!\bigl(\frac{t(e{-}s)+s}{\tau}\bigr) + \mathrm{sigmoid}\!\bigl(\frac{e}{\tau}\bigr)}{\mathrm{sigmoid}\!\bigl(\frac{e}{\tau}\bigr) - \mathrm{sigmoid}\!\bigl(\frac{s}{\tau}\bigr)}
  • •sigmoid(x)=1/(1+e−x)\mathrm{sigmoid}(x) = 1 / (1 + e^{-x})
  • •τ\tau = temperature coefficient that sharpens (τ<1\tau < 1) or smooths (τ>1\tau > 1) the transition
  • •tt = current timestep
  • •αˉt=∏i=1tαi\bar\alpha_t = \prod_{i=1}^{t} \alpha_i = cumulative signal retention
normalized values0.000.250.500.751.000.000.250.500.751.00
tt
αˉt\bar\alpha_t1.0000αt\alpha_t1.0000βt\beta_t0.0000SNR\mathrm{SNR}∞

βt\beta_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic

αˉt\bar\alpha_t across all schedules

0.000.250.500.751.000.000.250.500.751.00
LinearCosineSigmoidExponentialCauchyLaplaceSquare RootLogistic