torchcell.scheduler
Learning-rate schedulers. CosineAnnealingWarmupRestarts is cosine annealing with a linear warmup and cyclic restarts.
cosine_annealing_warmup
Cosine annealing learning-rate scheduler with warmup and cyclic restarts.
Classes
Cosine annealing scheduler with linear warmup and decaying cyclic restarts. |