U C^S @sddlmZddlZddlmZddlmZddlmZmZm Z m Z ddd Z e e ej ed ed ed ed dddde ddde de ddGdddeZdS))unicode_literalsN)describe)Model) DimensionSynapsesBiasesGradientcCsZ|jdkr|jd|_|jdkrV|dk rVt|jdkrD|jd|_nt|d|_dS)Nr)nIshapenOlenintmax)modelXyr=/tmp/pip-install-6_kvzl1k/thinc/thinc/neural/_classes/mish.py_set_dimensions_if_needed s   rz Batch sizez Input sizez Output sizezWeights matrixcCs |j|jfSN)rr objrrrrcCs ||Sr)Zxavier_uniform_init)Wopsrrrrrz Bias vectorcCs|jfSr)rrrrrrrrb)nBr rrrd_Wd_bc@sHeZdZdZdZeddZeddZddd Zd d Z dd dZ dS)MishzTDense layer with mish activation. https://arxiv.org/pdf/1908.08681.pdf mishcCs |j|jfSr)r r selfrrr input_shape)szMish.input_shapecCs |j|jfSr)r rr%rrr output_shape-szMish.output_shapeNcKs,tj|f|||_||_|dd|_dS)N drop_factorg?)r__init__rr getr))r&rr kwargsrrrr*1sz Mish.__init__cCs$|j|j|j|}|j|}|Sr)raffinerrr$)r&rYrrrpredict7s z Mish.predictcsp|dkrdfSjjjj}|j9}j||\}}dfdd }|||fS)Ncsnj|}jj|djdj|jdd7_j|j}|dk rj|jjjj j d|S)NT)Ztrans1outr)Zaxis)key) rZ backprop_mishZgemmr!r"sumrZ_memweightsZgradientid)ZdY2ZsgdZdY1ZdXrZY1r&rr finish_updateDsz(Mish.begin_update..finish_update)N)r/rr-rrr$r)Zdropout)r&rZdropZY2ZY3Z bp_dropoutr7rr6r begin_update<s   zMish.begin_update)NN)r0) __name__ __module__ __qualname____doc__namepropertyr'r(r*r/r8rrrrr#s   r#)N) __future__rZnumpyrrrrrr r rZon_data attributesr#rrrrs(