ó ùµÈ[c@sÂdZddddgZddlZddlmZmZdefd „ƒYZdefd „ƒYZd efd „ƒYZ defd „ƒYZ defd„ƒYZ defd„ƒYZ dS(sDataset container.tDatasett SimpleDatasett ArrayDatasettRecordFileDatasetiÿÿÿÿNi(trecordiotndarraycBs8eZdZd„Zd„Zed„Zed„ZRS(sAbstract dataset class. All datasets should have this interface. Subclasses need to override `__getitem__`, which returns the i-th element, and `__len__`, which returns the total number elements. .. note:: An mxnet or numpy array can be directly used as a dataset. cCs t‚dS(N(tNotImplementedError(tselftidx((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyt __getitem__%scCs t‚dS(N(R(R((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyt__len__(scCs6t||ƒ}|r|Stg|D] }|^q#ƒS(s{Returns a new dataset with each sample transformed by the transformer function `fn`. Parameters ---------- fn : callable A transformer function that takes a sample as input and returns the transformed sample. lazy : bool, default True If False, transforms all samples at once. Otherwise, transforms each sample on demand. Note that if `fn` is stochastic, you must set lazy to True or you will get the same result on all epochs. Returns ------- Dataset The transformed dataset. (t_LazyTransformDatasetR(Rtfntlazyttransti((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyt transform+scs‡fd†}|j||ƒS(sReturns a new dataset with the first element of each sample transformed by the transformer function `fn`. This is useful, for example, when you only want to transform data while keeping label as is. Parameters ---------- fn : callable A transformer function that takes the first elemtn of a sample as input and returns the transformed element. lazy : bool, default True If False, transforms all samples at once. Otherwise, transforms each sample on demand. Note that if `fn` is stochastic, you must set lazy to True or you will get the same result on all epochs. Returns ------- Dataset The transformed dataset. cs!|rˆ|ƒf|Sˆ|ƒS(N((txtargs(R (sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pytbase_fn[s(R(RR R R((R sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyttransform_firstDs(t__name__t __module__t__doc__R R tTrueRR(((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyRs    cBs)eZdZd„Zd„Zd„ZRS(s¤Simple Dataset wrapper for lists and arrays. Parameters ---------- data : dataset-like object Any object that implements `len()` and `[]`. cCs ||_dS(N(t_data(Rtdata((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyt__init__jscCs t|jƒS(N(tlenR(R((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR mscCs |j|S(N(R(RR((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ps(RRRRR R (((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyRbs  R cBs)eZdZd„Zd„Zd„ZRS(sLazily transformed dataset.cCs||_||_dS(N(Rt_fn(RRR ((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyRvs cCs t|jƒS(N(RR(R((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR zscCs6|j|}t|tƒr)|j|ŒS|j|ƒS(N(Rt isinstancettupleR(RRtitem((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR }s  (RRRRR R (((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ts  cBs)eZdZd„Zd„Zd„ZRS(sA dataset that combines multiple dataset-like objects, e.g. Datasets, lists, arrays, etc. The i-th sample is defined as `(x1[i], x2[i], ...)`. Parameters ---------- *args : one or more dataset-like objects The data arrays. cGsÜt|ƒdkstdƒ‚t|dƒ|_g|_x›t|ƒD]\}}t|ƒ|jksŽtd|j|dt|ƒfƒ‚t|tjƒrÄt|jƒdkrÄ|j ƒ}n|jj |ƒqGWdS(NisNeeds at least 1 arrayssTAll arrays must have the same length; array[0] has length %d while array[%d] has %d.i( RtAssertionErrort_lengthRt enumerateRRtNDArraytshapetasnumpytappend(RRRR((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyRs  'csEt|jƒdkr$|jdˆSt‡fd†|jDƒƒSdS(Niic3s|]}|ˆVqdS(N((t.0R(R(sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pys Ÿs(RRR(RR((RsX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ›scCs|jS(N(R"(R((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ¡s(RRRRR R (((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR„s  cBs)eZdZd„Zd„Zd„ZRS(sÈA dataset wrapping over a RecordIO (.rec) file. Each sample is a string representing the raw content of an record. Parameters ---------- filename : str Path to rec file. cCsHtjj|ƒdd|_||_tj|j|jdƒ|_dS(Nis.idxtr(tostpathtsplitexttidx_filetfilenameRtMXIndexedRecordIOt_record(RR.((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR¯s cCs|jj|jj|ƒS(N(R0tread_idxtkeys(RR((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ´scCst|jjƒS(N(RR0R2(R((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ·s(RRRRR R (((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR¥s   t_DownloadedDatasetcBs2eZdZd„Zd„Zd„Zd„ZRS(s#Base class for MNIST, cifar10, etc.cCsytt|ƒjƒ||_d|_d|_tjj |ƒ}||_ tjj |ƒsktj |ƒn|j ƒdS(N(tsuperR3Rt _transformtNoneRt_labelR*R+t expandusert_roottisdirtmakedirst _get_data(RtrootR((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR½s    cCsE|jdk r-|j|j||j|ƒS|j||j|fS(N(R5R6RR7(RR((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ÈscCs t|jƒS(N(RR7(R((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR ÍscCs t‚dS(N(R(R((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR<Ðs(RRRRR R R<(((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyR3»s   ( Rt__all__R*tRRtobjectRRR RRR3(((sX/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/gluon/data/dataset.pyts   E!