[c@@sdZddlmZddlmZddlZddlZddlZddlZddl Z ddl m Z ddl m Z mZmZddl mZmZdd l mZdd l mZmZdd lmZdd lmZdd lmZddlmZddlmZddlmZmZm Z dedddgfdYZ!de"fdYZ#de"fdYZ$de$fdYZ%de$fdYZ&de$fdYZ'd e$fd!YZ(d"Z)d#Z*e*dS($s'Data iterators for common data formats.i(tabsolute_import(t namedtupleNi(t_LIB(t c_str_arraytmx_uinttpy_str(tDataIterHandlet NDArrayHandle(t mx_real_t(t check_calltbuild_param_doc(tNDArray(t CSRNDArray(t _ndarray_cls(tarray(tconcati(t _init_datat _has_instancet_getdata_by_idxtDataDesctnametshapecB@sDeZdZeddZdZedZedZRS(s3DataDesc is used to store name, shape, type and layout information of the data or the label. The `layout` describes how the axes in `shape` should be interpreted, for example for image data setting `layout=NCHW` indicates that the first axis is number of examples in the batch(N), C is number of channels, H is the height and W is the width of the image. For sequential data, by default `layout` is set to ``NTC``, where N is number of examples in the batch, T the temporal axis representing time and C is the number of channels. Parameters ---------- cls : DataDesc The class. name : str Data name. shape : tuple of int Data shape. dtype : np.dtype, optional Data type. layout : str, optional Data layout. tNCHWcC@s4t|tj|||}||_||_|S(N(tsuperRt__new__tdtypetlayout(tclsRRRRtret((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRCs  cC@s d|j|j|j|jfS(NsDataDesc[%s,%s,%s,%s](RRRR(tself((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt__repr__IscC@s|dkrdS|jdS(sGet the dimension that corresponds to the batch size. When data parallelism is used, the data will be automatically split and concatenated along the batch-size dimension. Axis can be -1, which means the whole array will be copied for each data-parallelism device. Parameters ---------- layout : str layout string. For example, "NCHW". Returns ------- int An axis indicating the batch_size dimension. itNN(tNonetfind(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pytget_batch_axisMs cC@sw|dk rKt|}g|D](}t|d|d||d^qSg|D]}t|d|d^qRSdS(sGet DataDesc list from attribute lists. Parameters ---------- shapes : a tuple of (name, shape) types : a tuple of (name, type) iiN(R tdictR(tshapesttypest type_dicttx((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pytget_listcs  3( t__name__t __module__t__doc__RRRt staticmethodR"R((((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR)s  t DataBatchcB@s2eZdZdddddddZdZRS(sA data batch. MXNet's data iterator returns a batch of data for each `next` call. This data contains `batch_size` number of examples. If the input data consists of images, then shape of these images depend on the `layout` attribute of `DataDesc` object in `provide_data` parameter. If `layout` is set to 'NCHW' then, images should be stored in a 4-D matrix of shape ``(batch_size, num_channel, height, width)``. If `layout` is set to 'NHWC' then, images should be stored in a 4-D matrix of shape ``(batch_size, height, width, num_channel)``. The channels are often in RGB order. Parameters ---------- data : list of `NDArray`, each array containing `batch_size` examples. A list of input data. label : list of `NDArray`, each array often containing a 1-dimensional array. optional A list of input labels. pad : int, optional The number of examples padded at the end of a batch. It is used when the total number of examples read is not divisible by the `batch_size`. These extra padded examples are ignored in prediction. index : numpy.array, optional The example indices in this batch. bucket_key : int, optional The bucket key, used for bucketing module. provide_data : list of `DataDesc`, optional A list of `DataDesc` objects. `DataDesc` is used to store name, shape, type and layout information of the data. The *i*-th element describes the name and shape of ``data[i]``. provide_label : list of `DataDesc`, optional A list of `DataDesc` objects. `DataDesc` is used to store name, shape, type and layout information of the label. The *i*-th element describes the name and shape of ``label[i]``. cC@s|dk r0t|ttfs0tdn|dk r`t|ttfs`tdn||_||_||_||_||_ ||_ ||_ dS(NsData must be list of NDArrayssLabel must be list of NDArrays( R t isinstancetlistttupletAssertionErrortdatatlabeltpadtindext bucket_keyt provide_datat provide_label(RR2R3R4R5R6R7R8((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt__init__s $ $      cC@sig|jD]}|j^q }|jrJg|jD]}|j^q2}nd}dj|jj||S(Ns${}: data shapes: {} label shapes: {}(R2RR3R tformatt __class__R)(Rtdt data_shapestlt label_shapes((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt__str__s " N(R)R*R+R R9R@(((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR-rs% tDataItercB@skeZdZddZdZdZdZdZdZdZ d Z d Z d Z RS( sThe base class for an MXNet data iterator. All I/O in MXNet is handled by specializations of this class. Data iterators in MXNet are similar to standard-iterators in Python. On each call to `next` they return a `DataBatch` which represents the next batch of data. When there is no more data to return, it raises a `StopIteration` exception. Parameters ---------- batch_size : int, optional The batch size, namely the number of items in the batch. See Also -------- NDArrayIter : Data-iterator for MXNet NDArray or numpy-ndarray objects. CSVIter : Data-iterator for csv data. LibSVMIter : Data-iterator for libsvm data. ImageIter : Data-iterator for images. icC@s ||_dS(N(t batch_size(RRB((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR9scC@s|S(N((R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt__iter__scC@sdS(s,Reset the iterator to the begin of the data.N((R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pytresetsc C@sM|jrCtd|jd|jd|jd|jStdS(sGet next data batch from iterator. Returns ------- DataBatch The data of next batch. Raises ------ StopIteration If the end of the data is reached. R2R3R4R5N(t iter_nextR-tgetdatatgetlabeltgetpadtgetindext StopIteration(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pytnexts cC@s |jS(N(RK(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt__next__scC@sdS(s}Move to the next batch. Returns ------- boolean Whether the move is successful. N((R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyREscC@sdS(sGet data of current batch. Returns ------- list of NDArray The data of the current batch. N((R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRFscC@sdS(sGet label of the current batch. Returns ------- list of NDArray The label of the current batch. N((R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRGscC@sdS(sGet index of the current batch. Returns ------- index : numpy.array The indices of examples in the current batch. N(R (R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRIscC@sdS(sGet the number of padding examples in the current batch. Returns ------- int Number of padding examples in the current batch. N((R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRHs( R)R*R+R9RCRDRKRLRERFRGRIRH(((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRAs      t ResizeItercB@sPeZdZedZdZdZdZdZdZ dZ RS(sMResize a data iterator to a given number of batches. Parameters ---------- data_iter : DataIter The data iterator to be resized. size : int The number of batches per epoch to resize to. reset_internal : bool Whether to reset internal iterator on ResizeIter.reset. Examples -------- >>> nd_iter = mx.io.NDArrayIter(mx.nd.ones((100,10)), batch_size=25) >>> resize_iter = mx.io.ResizeIter(nd_iter, 2) >>> for batch in resize_iter: ... print(batch.data) [] [] cC@stt|j||_||_||_d|_d|_|j |_ |j |_ |j |_ t |dr|j |_ ndS(Nitdefault_bucket_key(RRMR9t data_itertsizetreset_internaltcurR t current_batchR7R8RBthasattrRN(RRORPRQ((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR9.s        cC@s&d|_|jr"|jjndS(Ni(RRRQRORD(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRD<s  cC@sr|j|jkrtSy|jj|_Wn0tk r^|jj|jj|_nX|jd7_tS(Ni( RRRPtFalseRORKRSRJRDtTrue(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyREAs  cC@s |jjS(N(RSR2(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRFMscC@s |jjS(N(RSR3(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRGPscC@s |jjS(N(RSR5(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRISscC@s |jjS(N(RSR4(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRHVs( R)R*R+RVR9RDRERFRGRIRH(((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRMs     tPrefetchingItercB@seZdZd d dZdZedZedZdZ dZ dZ dZ d Z d Zd ZRS( sPerforms pre-fetch for other data iterators. This iterator will create another thread to perform ``iter_next`` and then store the data in memory. It potentially accelerates the data read, at the cost of more memory usage. Parameters ---------- iters : DataIter or list of DataIter The data iterators to be pre-fetched. rename_data : None or list of dict The *i*-th element is a renaming map for the *i*-th iter, in the form of {'original_name' : 'new_name'}. Should have one entry for each entry in iter[i].provide_data. rename_label : None or list of dict Similar to ``rename_data``. Examples -------- >>> iter1 = mx.io.NDArrayIter({'data':mx.nd.ones((100,10))}, batch_size=25) >>> iter2 = mx.io.NDArrayIter({'data':mx.nd.ones((100,10))}, batch_size=25) >>> piter = mx.io.PrefetchingIter([iter1, iter2], ... rename_data=[{'data': 'data_1'}, {'data': 'data_2'}]) >>> print(piter.provide_data) [DataDesc[data_1,(25, 10L),,NCHW], DataDesc[data_2,(25, 10L),,NCHW]] cC@stt|jt|ts.|g}nt||_|jdksRt||_||_ ||_ |j ddd|_ gt |jD]}tj^q|_gt |jD]}tj^q|_x|jD]}|jqWt|_gt |jD] }d^q|_gt |jD] }d^q7|_d}gt |jD]$}tjd|d||g^qe|_x(|jD]}|jt|jqWdS(NiicS@sxtr|j|j|js'Pny|j|j|j|>> data = np.arange(40).reshape((10,2,2)) >>> labels = np.ones([10, 1]) >>> dataiter = mx.io.NDArrayIter(data, labels, 3, True, last_batch_handle='discard') >>> for batch in dataiter: ... print batch.data[0].asnumpy() ... batch.data[0].shape ... [[[ 36. 37.] [ 38. 39.]] [[ 16. 17.] [ 18. 19.]] [[ 12. 13.] [ 14. 15.]]] (3L, 2L, 2L) [[[ 32. 33.] [ 34. 35.]] [[ 4. 5.] [ 6. 7.]] [[ 24. 25.] [ 26. 27.]]] (3L, 2L, 2L) [[[ 8. 9.] [ 10. 11.]] [[ 20. 21.] [ 22. 23.]] [[ 28. 29.] [ 30. 31.]]] (3L, 2L, 2L) >>> dataiter.provide_data # Returns a list of `DataDesc` [DataDesc[data,(3, 2L, 2L),,NCHW]] >>> dataiter.provide_label # Returns a list of `DataDesc` [DataDesc[softmax_label,(3, 1L),,NCHW]] In the above example, data is shuffled as `shuffle` parameter is set to `True` and remaining examples are discarded as `last_batch_handle` parameter is set to `discard`. Usage of `last_batch_handle` parameter: >>> dataiter = mx.io.NDArrayIter(data, labels, 3, True, last_batch_handle='pad') >>> batchidx = 0 >>> for batch in dataiter: ... batchidx += 1 ... >>> batchidx # Padding added after the examples read are over. So, 10/3+1 batches are created. 4 >>> dataiter = mx.io.NDArrayIter(data, labels, 3, True, last_batch_handle='discard') >>> batchidx = 0 >>> for batch in dataiter: ... batchidx += 1 ... >>> batchidx # Remaining examples are discarded. So, 10/3 batches are created. 3 >>> dataiter = mx.io.NDArrayIter(data, labels, 3, False, last_batch_handle='roll_over') >>> batchidx = 0 >>> for batch in dataiter: ... batchidx += 1 ... >>> batchidx # Remaining examples are rolled over to the next iteration. 3 >>> dataiter.reset() >>> dataiter.next().data[0].asnumpy() [[[ 36. 37.] [ 38. 39.]] [[ 0. 1.] [ 2. 3.]] [[ 4. 5.] [ 6. 7.]]] (3L, 2L, 2L) `NDArrayIter` also supports multiple input and labels. >>> data = {'data1':np.zeros(shape=(10,2,2)), 'data2':np.zeros(shape=(20,2,2))} >>> label = {'label1':np.zeros(shape=(10,1)), 'label2':np.zeros(shape=(20,1))} >>> dataiter = mx.io.NDArrayIter(data, label, 3, True, last_batch_handle='discard') `NDArrayIter` also supports ``mx.nd.sparse.CSRNDArray`` with `last_batch_handle` set to `discard`. >>> csr_data = mx.nd.array(np.arange(40).reshape((10,4))).tostype('csr') >>> labels = np.ones([10, 1]) >>> dataiter = mx.io.NDArrayIter(csr_data, labels, 3, last_batch_handle='discard') >>> [batch.data[0] for batch in dataiter] [ , , ] Parameters ---------- data: array or list of array or dict of string to array The input data. label: array or list of array or dict of string to array, optional The input label. batch_size: int Batch size of data. shuffle: bool, optional Whether to shuffle the data. Only supported if no h5py.Dataset inputs are used. last_batch_handle : str, optional How to handle the last batch. This parameter can be 'pad', 'discard' or 'roll_over'. If 'pad', the last batch will be padded with data starting from the begining If 'discard', the last batch will be discarded If 'roll_over', the remaining elements will be rolled over to the next iteration and note that it is intended for training and can cause problems if used for prediction. data_name : str, optional The data name. label_name : str, optional The label name. iR4R2t softmax_labelc C@s]tt|j|t|dtd||_t|dtd||_t|jt spt|jt r|dkrt dnt j |jddj d|_||_||_||_|j |_|jj d|_|jg|jD]}|d^qg|jD]}|d^q|_t|j|_d|_d|_dS(Nt allow_emptyt default_nametdiscardsU`NDArrayIter` only supports ``CSRNDArray`` with `last_batch_handle` set to `discard`.ii(RRwR9RRUR2RVR3RR tNotImplementedErrortnptarangeRtidxtshuffletlast_batch_handleRBtcursortnum_dataRDt data_listRdt num_sourceR t _cache_datat _cache_label( RR2R3RBRRt data_namet label_nameR'((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR9]s$ $     A c C@sLg|jD]>\}}t|t|jgt|jd|j^q S(s5The name and shape of data provided by this iterator.i(R2RR0RBR/RR(Rtktv((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR7zsc C@sLg|jD]>\}}t|t|jgt|jd|j^q S(s6The name and shape of label provided by this iterator.i(R3RR0RBR/RR(RRR((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR8scC@s9|jr|jn|j |_d|_d|_dS(s'Ignore roll over data and set to start.N(Rt _shuffle_dataRBRR RR(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt hard_resets     cC@s|jr|jn|jdkrn|j|j|jkoL|jknrn|j|j|j|_n |j |_dS(s1Resets the iterator to the beginning of the data.t roll_overN(RRRRRBR(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRDs   ,cC@s"|j|j7_|j|jkS(s|Increments the coursor by batch_size for next batch and check current cursor if it exceed the number of data points.(RRBR(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyREsc C@s|jstn|j}|j}|djd|jkrb||_||_tntd|d|d|j ddS(sReturns the next batch of data.iR2R3R4R5N( RERJRFRGRRBRRR-RHR (RR2R3((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRKs       c C@s|dk s$|dk s$td|dk r6|nd}|dkrl|rc|ddjdnd}nt||}g|D]}t|dtjtfr|d|nXt|dt |j |gt |j |D]"}t |j |j |^q^qS(s!Load data from underlying arrays.s$should at least specify start or endiiN( R R1RtsliceR.R}tndarrayR RtsortedRR/R5(Rt data_sourceRntendtsR'R`((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt_getdatas$ $cC@st|t|ks$td|rj|rjgtt|D]#}t||||dd^qCS| r|| r|gSgtt|D] }|r|dn|d^qSdS(s'Helper function to concat two NDArrays.s(data source should contain the same sizetdimiN(RdR1RhR(Rt first_datat second_dataR'((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt_concats  7cC@s|j|jkstd|jdkr|j |jkoKdknr|jdk sz|jdk sztd|jdk r|jn|j}|j|d|j|j}|jdk rd|_n d|_|j ||S|jdkrj|j|j|jkrj|j|j|j}|j|d|j}|j|d|}|j ||S|j|j|jkr|j|j}n |j}|j||j|SdS( s4Load data from underlying arrays, internal use only.sDataIter needs reset.Ris"next epoch should have cached dataRR4RnN( RRR1RRBRR RRR(RRt cache_dataRR4Rtend_idx((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt _batchifys,#! !   cC@s|j|jS(s Get data.(RR2(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRFscC@s|j|jS(s Get label.(RR3(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRGscC@s|jdkr=|j|j|jkr=|j|j|jS|jdkrw|j |jkojdknrw|j SdSdS(sGet pad value of DataBatch.R4RiN(RRRBR(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRHs#cC@sGtjj|jt|j|j|_t|j|j|_dS(sShuffle the data.N(R}trandomRRRR2R3(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR sN(R)R*R+R RUR9RvR7R8RRDRERKRRRRFRGRHR(((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRws"s         t MXDataItercB@sneZdZdddZdZdZdZdZdZd Z d Z d Z d Z RS( sA python wrapper a C++ data iterator. This iterator is the Python wrapper to all native C++ data iterators, such as `CSVIter`, `ImageRecordIter`, `MNISTIter`, etc. When initializing `CSVIter` for example, you will get an `MXDataIter` instance to use in your Python code. Calls to `next`, `reset`, etc will be delegated to the underlying C++ data iterators. Usually you don't need to interact with `MXDataIter` directly unless you are implementing your own data iterators in C++. To do that, please refer to examples under the `src/io` folder. Parameters ---------- handle : DataIterHandle, required The handle to the underlying C++ Data Iterator. data_name : str, optional Data name. Default to "data". label_name : str, optional Label name. Default to "softmax_label". See Also -------- src/io : The underlying C++ data iterator implementation, e.g., `CSVIter`. R2RxcK@stt|j||_t|_d|_|j|_|jj d}|jj d}t ||j |j g|_t ||j |j g|_|j d|_dS(Ni(RRR9thandleRUt_debug_skip_loadR t first_batchRKR2R3RRRR7R8RB(RRRRt_R2R3((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyR9.s   cC@sttj|jdS(N(R RtMXDataIterFreeR(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRq?scC@st|_tjddS(Ns>Set debug_skip_load to be true, will simply return first batch(RVRtloggingtinfo(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pytdebug_skip_loadBs cC@s,t|_d|_ttj|jdS(N(RVt_debug_at_beginR RR RtMXDataIterBeforeFirstR(R((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRDIs  c C@s|jrP|j rPtd|jgd|jgd|jd|jS|jdk ru|j}d|_|St |_t j d}t t j|jt j||jrtd|jgd|jgd|jd|jStdS(NR2R3R4R5i(RRR-RFRGRHRIRR RUtctypestc_intR RtMXDataIterNextRtbyreftvalueRJ(RRutnext_res((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRKNs0    " 0 cC@sK|jdk rtStjd}ttj|jtj ||j S(Ni( RR RVRRR RRRRR(RR((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRE_s "cC@s8t}ttj|jtj|t|tS(N( RR RtMXDataIterGetDataRRRR RU(Rthdl((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRFfs "cC@s8t}ttj|jtj|t|tS(N( RR RtMXDataIterGetLabelRRRR RU(RR((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRGks "cC@stjd}tjtj}ttj|jtj|tj||jrtj |j }tj|jj |}t j |dt j}|jSdSdS(NiR(Rtc_uint64tPOINTERR RtMXDataIterGetIndexRRRt addressoftcontentst from_addressR}t frombuffertuint64tcopyR (Rt index_sizet index_datataddresstdbuffertnp_index((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRIps   cC@s8tjd}ttj|jtj||jS(Ni(RRR RtMXDataIterGetPadNumRRR(RR4((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRH~s"( R)R*R+R9RqRRDRKRERFRGRIRH(((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyRs        c @stj}tj}t}tjtj}tjtj}tjtj}ttjtj|tj|tj|tj|tj|tj|t|j t |j }t gt |D]}t||^qgt |D]}t||^qgt |D]}t||^q5} d ddd} | |j | f} fd} | _ | | _| S( s Create an io iterator by handle.s%s s%s sReturns s------- s MXDataIter s The result iterator.c@sg}g}x:|jD],\}}|j||jt|qWt|}t|}t}ttjtt |||t j |t |rt dnt ||S(sECreate an iterator. The parameters listed below can be passed in as keyword arguments. Parameters ---------- name : string, required. Name of the resulting data iterator. Returns ------- dataiter: Dataiter The resulting data iterator. s$%s can only accept keyword arguments(titemstappendtstrRRR RtMXDataIterCreateIterRRdRRt TypeErrorR(Rctkwargst param_keyst param_valsRtvalt iter_handle(Rt iter_name(sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pytcreators       s%s %s s%s %s Returns (Rtc_char_pRRR RtMXDataIterGetIterInfoRRRtintt_build_param_docRhR)R+( RRtdesctnum_argst arg_namest arg_typest arg_descstnargR`t param_strtdoc_strR((RRsK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt_make_io_iterators2       &&, #  cC@stjtj}tj}ttjtj|tj|tj t }xIt |j D]8}tj||}t |}t||j |qfWdS(s6List and add all the data iterators to current module.N(RRtc_void_ptc_uintR RtMXListDataItersRtsystmodulesR)RhRRtsetattr(tplistRPt module_objR`Rtdataiter((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyt_init_io_modules (  (+R+t __future__Rt collectionsRRRRRitnumpyR}tbaseRRRRRRRR R RRR tndarray.sparseR R RRtutilsRRRRtobjectR-RARMRWRwRRR(((sK/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/io/io.pyts:     %I@fA,o F