[c@sdZdddddgZddlZddlZddlZddlZddlZddlZddlZyddl Z Wn-e k rde fd YZ e Z nXddl Zd d lmZd ed Zd edZedZdZdZejjds%dZnXddlZdZdZeZejd d kr\ene Z!dZ"dZ#dZde%ddedZ&dZ'dZ(ddZ)de fd YZ*dS(!s"Parallelization utility optimizer.t split_datatsplit_and_loadtclip_global_normt check_sha1tdownloadiNtrequests_failed_to_importcBseZRS((t__name__t __module__(((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyR"si(tndarrayicCsy|j|}||kr>tdt|j||fn|r|||dkr|tdt|j|||fn||}|dkrgt|D]@}||dkr||||d|!n||||!^q}n|r tj|d|d|}nigt|D]V}||dkrStj|||||d|ntj|||||^q}|S(sSplits an NDArray into `num_slice` slices along `batch_axis`. Usually used for data parallelism where each slices is sent to one device (i.e. GPU). Parameters ---------- data : NDArray A batch of data. num_slice : int Number of desired slices. batch_axis : int, default 0 The axis along which to slice. even_split : bool, default True Whether to force all slices to have the same number of elements. If `True`, an error will be raised when `num_slice` does not evenly divide `data.shape[batch_axis]`. Returns ------- list of NDArray Return value is a list even if `num_slice` is 1. sUToo many slices for data with shape %s. Arguments are num_slice=%d and batch_axis=%d.isdata with shape %s cannot be evenly split into %d slices along axis %d. Use a batch size that's multiple of %d or set even_split=False to allow uneven partitioning of data.it num_outputstaxis(tshapet ValueErrortstrtrangeRtsplitt slice_axis(tdatat num_slicet batch_axist even_splittsizetsteptitslices((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyR*s$  "  SfcCst|tjs.tj|d|d}nt|dkrT|j|dgSt|t|||}gt||D]\}}|j|^qS(sSplits an NDArray into `len(ctx_list)` slices along `batch_axis` and loads each slice to one context in `ctx_list`. Parameters ---------- data : NDArray A batch of data. ctx_list : list of Context A list of Contexts. batch_axis : int, default 0 The axis along which to slice. even_split : bool, default True Whether to force all slices to have the same number of elements. Returns ------- list of NDArray Each corresponds to a context in `ctx_list`. tctxii(t isinstanceRtNDArraytarraytlent as_in_contextRtzip(Rtctx_listRRRRR((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyR[s cCs%d}t|dks!t|dj}tjg|D]}||j|^q;}tj|}|rtj|j st j t dddqn||d}tj tj|tjdd|d d}x$|D]}||j|j9}qW|r|j S|Sd S( sRescales NDArrays so that the sum of their 2-norm is smaller than `max_norm`. Parameters ---------- arrays : list of NDArray max_norm : float check_isfinite : bool, default True If True, check that the total_norm is finite (not nan or inf). This requires a blocking .asscalar() call. Returns ------- NDArray or float Total norm. Return type is NDArray of shape (1,) if check_isfinite is False. Otherwise a float is returned. cSs>|jdkr.|jd}tj||S|jjS(Ntdefaulti(i(tstypetreshapeRtdottnormtsquare(Rtx((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt_normsis;nan or inf is detected. Clipping results will be undefined.t stacklevelig:0yE>iRtdimN(RtAssertionErrortcontextRtadd_nRtsqrttnptisfinitetasscalartwarningstwarnt UserWarningtmintconcattones(tarraystmax_normtcheck_isfiniteR(Rtarrt total_normtscale((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyRxs"  1  0  cCso|jd}t|dkr%|S|jd}|gg|D]}|d|^qA}dj|}|S(sIndent string s iit (RRtpoptjoin(ts_t numSpaceststfirsttline((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt_indents(cCsgtj}t|d:}x0trP|jd}|s@Pn|j|q!WWdQX|j|kS(s?Check whether the sha1 hash of the file content matches the expected hash. Parameters ---------- filename : str Path to the file. sha1_hash : str Expected sha1 hash in hexadecimal digits. Returns ------- bool Whether the file content matches the expected hash. trbiN(thashlibtsha1topentTruetreadtupdatet hexdigest(tfilenamet sha1_hashRItfR((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyRs  twin32cCsqytj||WnVtk rlz)ytj|Wntk rKnXWdtdj||XnXdS(sAImplement atomic os.replace with linux and OSX. Internal use onlyNs^Moving downloaded temp file - {}, to {} failed. Please retry the download.(tostrenametOSErrortremovetformat(tsrctdst((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt_replace_atomics  iicCs&t|ts"|jtjS|S(s'Handle text decoding. Internal use only(Rt text_typetdecodetsystgetfilesystemencoding(R'((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt_str_to_unicodescCs[|sWtjtj}z)ytj|Wntk rBnXWdt|XndS(s"Handle WinError. Internal use onlyN(tctypest FormatErrort GetLastErrorRSRVRU(trvRXtmsg((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt_handle_errorss cCs6ttjjjt|t|ttB|dS(sImplement atomic os.replace with windows. refer to https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-movefileexw The function fails when one of the process(copy, flush, delete) fails. Internal use onlyN(ReR`twindlltkernel32t MoveFileExWR_t_windows_default_flagst_MOVEFILE_REPLACE_EXISTING(RXRY((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyRZs ic Cs7|dkr4|jdd}|stdnOtjj|}tjj|r}tjj||jdd}n|}|dkstdj||st j dn|stjj | s|r3t || r3tjj tjjtjj|}tjj |s5tj|nx|ddkr/ydj||GHtj|d td |}|jd krtd j|nttj} td j|| d:} x0|jddD]} | r| j| qqWWdQXtjj | s0|rLt || rLtd j|| |nOz5ytjd j|| Wntk rnXWdt j dj|X|rt || rtdj|nPWq8tk r+} |d8}|dkr| q,djt | ||dkr dndGHq8Xq8Wn|S(sDownload an given URL Parameters ---------- url : str URL to download path : str, optional Destination path to store downloaded file. By default stores to the current directory with same name as in url. overwrite : bool, optional Whether to overwrite destination file if already exists. sha1_hash : str, optional Expected sha1 hash in hexadecimal digits. Will ignore existing file when hash is specified but doesn't match. retries : integer, default 5 The number of times to attempt the download in case of failure or non 200 return codes verify_ssl : bool, default True Verify SSL certificates. Returns ------- str The file path of the downloaded file. t/isOCan't construct file-name from this URL. Please set the `path` option manually.is9Number of retries should be at least 0, currently it's {}soUnverified HTTPS request is being made (verify_ssl=False). Adding certificate verification is strongly advised.isDownloading {} from {}...tstreamtverifyisFailed downloading url {}s{}.{}twbt chunk_sizeiNs?File {} exists in file system so the downloaded file is deletedsFile {} is downloaded but the content hash does not match. The repo may be outdated or download may be incomplete. If the "repo_url" is overridden, consider switching to the default repo.s6download failed due to {}, retrying, {} attempt{} leftRCt(!tNoneRR+RStpatht expandusertisdirR@RWR2R3texistsRtdirnametabspathtmakedirstrequeststgetRKt status_codet RuntimeErrorR tuuidtuuid4RJt iter_contenttwriteRZRVRUR4t Exceptiontrepr( turlRrt overwriteRPtretriest verify_ssltfnameRvtrt random_uuidRQtchunkte((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyRs` %  /*)    1cCs<d}tjjd|}|ddkr8|d}n|S(s;Return the base URL for Gluon dataset and model repository.s;https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/tMXNET_GLUON_REPOiRk(RStenvironRz(t default_repotrepo_url((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt _get_repo_url]s  cCsdjdtd|d|S(sReturn the URL for hosted file in Gluon repository. Parameters ---------- namespace : str Namespace of the file. filename : str Name of the file s {base_url}{namespace}/{filename}tbase_urlt namespaceRO(RWR(RRO((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt_get_repo_file_urles icCswt|}t||krMt||d |dt|| d|Sdjg|D]}dt|^qZS(s'Print at most `limit` elements of list.is, ..., s, s'%s'(tlistRt_brief_print_listR@R (tlsttlimitR((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyRss  t HookHandlecBsMeZdZdZdZdZdZdZdZdZ RS(s'A handle that can attach/detach a hook.cCsd|_d|_dS(N(Rqt_hooks_dict_reft_id(tself((sQ/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/utils.pyt__init__s cCsH|j stdt||_|||jsH            1 *     ]