U C^ @szdZddlmZmZddlZddZddZddd Zdd d Zd dZ dddZ d ddZ d!ddZ ddZ ddZdS)"zEGenerators that provide different rates, schedules, decays or series.)unicode_literalsdivisionNccs(t|D] }|Vq|D] }|VqdS)z>Yield a constant rate for N steps, before starting a schedule.N)range)rateZstepsscheduleivaluer ./tmp/pip-install-6_kvzl1k/thinc/thinc/rates.py constant_thens r ccs |VqdSNr )rr r r constantsr ccs$|dd||V|d7}qdS)aYield an infinite series of linearly decaying values, following the schedule: base_rate * 1/(1+decay*t) Example: >>> learn_rates = linear_decay(0.001, 1e-4) >>> next(learn_rates) 0.001 >>> next(learn_rates) 0.00999 ?Nr )Z base_ratedecaytr r r decayingsrccs$t|}t|||V||9}qdS)aZYield an infinite series of compounding values. Each time the generator is called, a value is produced by multiplying the previous value by the compound rate. EXAMPLE: >>> sizes = compounding(1., 10., 1.5) >>> assert next(sizes) == 1. >>> assert next(sizes) == 1 * 1.5 >>> assert next(sizes) == 1.5 * 1.5 N)float_clip)startstopZcompoundrcurrr r r compounding's rcCs||krt||St||Sr )maxmin)rrrr r r r8srccs0|dkr|Vq||||V|d7}qdS)Nrrr )rrZ decay_stepsrr r r annealing<srMbP?ccs<|}ttj|d}|||d|V||7}qdS)Nr)numpycospi)rendstepZpctZcos_outr r r annealing_cosEsr$皙? rc csjt||}|d7}||kr&||}nd|||d|d}|d||dd|}|Vq dS)zxYield an infinite series of values according to Howard and Ruder's "slanted triangular learning rate" schedule. rN)int) Zmax_rateZ num_stepsZcut_fracZratiorrZcutpZ learn_rater r r slanted_triangularNs  r)ccsPd}||kr|td|}ntd||td||}||V|d7}qdS)zGenerate a series, starting from an initial rate, and then with a warmup period, and then a linear decline. Used for learning rates. rrrrN)r)Z initial_rateZ warmup_stepsZ total_stepsr#Zfactorr r r warmup_linear^s r*ccsbd}td|d|}t||d|d}tdd|}||||V|d7}qdS)Nrrr)rfloorabsr)Zmin_lrZmax_lrZperioditcyclexrelativer r r cyclic_triangularns r1)r)r)r)r)r%r&rr)__doc__ __future__rrrr r rrrrr$r)r*r1r r r r s