ó ùµÈ[c@@sMdZddlmZddlmZddlZddlZddlZddlZ ddl m Z ddl m Z ddl m Z dd l mZyddlZWnek r¿dZnXd efd „ƒYZd efd „ƒYZedddddgƒZdZejeƒZd„Zd„Zdd„Zddd„ZdS(s,Read and write for the RecordIO data format.i(tabsolute_import(t namedtupleNi(t_LIB(tRecordIOHandle(t check_call(tc_strt MXRecordIOcB@s_eZdZd„Zd„Zd„Zd„Zd„Zd„Zd„Z d„Z d „Z RS( s¥Reads/writes `RecordIO` data format, supporting sequential read and write. Example usage: ---------- >>> record = mx.recordio.MXRecordIO('tmp.rec', 'w') >>> for i in range(5): ... record.write('record_%d'%i) >>> record.close() >>> record = mx.recordio.MXRecordIO('tmp.rec', 'r') >>> for i in range(5): ... item = record.read() ... print(item) record_0 record_1 record_2 record_3 record_4 >>> record.close() Parameters ---------- uri : string Path to the record file. flag : string 'w' for write or 'r' for read. cC@s;t|ƒ|_tƒ|_||_t|_|jƒdS(N(RturiRthandletflagtFalsetis_opentopen(tselfRR ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyt__init__@s    cC@s |jdkr@ttj|jtj|jƒƒƒt|_ nS|jdkr€ttj |jtj|jƒƒƒt |_ nt d|jƒ‚t|_ dS(sOpens the record file.twtrsInvalid flag %sN(R RRtMXRecordIOWriterCreateRtctypestbyrefRtTruetwritabletMXRecordIOReaderCreateR t ValueErrorR (R ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyR Gs% % cC@s|jƒdS(N(tclose(R ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyt__del__SscC@st|j}|jƒt|jƒ}||d<|jj}y|jdƒ}Wntk r^nX|d=||d<|S(sOverride pickling behavior.R sutf-8RR(R Rtdictt__dict__RtvaluetdecodetAttributeError(R R tdR((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyt __getstate__Vs      cC@sQ||_|d}t|_tƒ|_t|jƒ|_|rM|jƒndS(sRestore from pickled.R N(RR R RRRRR (R RR ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyt __setstate__fs    cC@sR|js dS|jr/ttj|jƒƒnttj|jƒƒt|_dS(sCloses the record file.N(R RRRtMXRecordIOWriterFreeRtMXRecordIOReaderFreeR (R ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyRps   cC@s|jƒ|jƒdS(s Resets the pointer to first item. If the record is opened with 'w', this function will truncate the file to empty. Example usage: ---------- >>> record = mx.recordio.MXRecordIO('tmp.rec', 'r') >>> for i in range(2): ... item = record.read() ... print(item) record_0 record_1 >>> record.reset() # Pointer is reset. >>> print(record.read()) # Started reading from start again. record_0 >>> record.close() N(RR (R ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pytresetzs cC@sG|jst‚ttj|jtj|ƒtjt |ƒƒƒƒdS(smInserts a string buffer as a record. Example usage: ---------- >>> record = mx.recordio.MXRecordIO('tmp.rec', 'w') >>> for i in range(5): ... record.write('record_%d'%i) >>> record.close() Parameters ---------- buf : string (python2), bytes (python3) Buffer to write. N( RtAssertionErrorRRtMXRecordIOWriterWriteRecordRRtc_char_ptc_size_ttlen(R tbuf((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pytwrites cC@s“|j st‚tjƒ}tjƒ}ttj|jtj |ƒtj |ƒƒƒ|r‹tj |tj tj |j ƒƒ}|jjSdSdS(s«Returns record as a string. Example usage: ---------- >>> record = mx.recordio.MXRecordIO('tmp.rec', 'r') >>> for i in range(5): ... item = record.read() ... print(item) record_0 record_1 record_2 record_3 record_4 >>> record.close() Returns ---------- buf : string Buffer read. N(RR%RR'R(RRtMXRecordIOReaderReadRecordRRtcasttPOINTERtc_charRtcontentstrawtNone(R R*tsize((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pytread£s   % ( t__name__t __module__t__doc__RR RR R!RR$R+R4(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyR$s     tMXIndexedRecordIOcB@sYeZdZed„Zd„Zd„Zd„Zd„Zd„Z d„Z d„Z RS( sCReads/writes `RecordIO` data format, supporting random access. Example usage: ---------- >>> for i in range(5): ... record.write_idx(i, 'record_%d'%i) >>> record.close() >>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r') >>> record.read_idx(3) record_3 Parameters ---------- idx_path : str Path to the index file. uri : str Path to the record file. Only supports seekable file types. flag : str 'w' for write or 'r' for read. key_type : type Data type for keys. cC@sJ||_i|_g|_||_d|_tt|ƒj||ƒdS(N( tidx_pathtidxtkeystkey_typeR2tfidxtsuperR8R(R R9RR R<((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyRÛs      cC@s¼tt|ƒjƒi|_g|_t|j|jƒ|_|js¸xot |jj dƒD]U}|j ƒj dƒ}|j |dƒ}t|dƒ|j|<|jj|ƒq\WndS(Nts ii(R>R8R R:R;R9R R=RtitertreadlinetstriptsplitR<tinttappend(R tlinetkey((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyR ãs   cC@s1|js dStt|ƒjƒ|jjƒdS(sCloses the record file.N(R R>R8RR=(R ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyRïs cC@s#tt|ƒjƒ}d|d<|S(sOverride pickling behavior.R=N(R>R8R R2(R R((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyR ös cC@sC|j st‚tj|j|ƒ}ttj|j|ƒƒdS(s·Sets the current read pointer position. This function is internally called by `read_idx(idx)` to find the current reader pointer position. It doesn't return anything.N( RR%RR(R:RRtMXRecordIOReaderSeekR(R R:tpos((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pytseeküscC@sD|jst‚tjƒ}ttj|jtj|ƒƒƒ|j S(s†Returns the current position of write head. Example usage: ---------- >>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w') >>> print(record.tell()) 0 >>> for i in range(5): ... record.write_idx(i, 'record_%d'%i) ... print(record.tell()) 16 32 48 64 80 ( RR%RR(RRtMXRecordIOWriterTellRRR(R RI((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyttells "cC@s|j|ƒ|jƒS(sReturns the record at given index. Example usage: ---------- >>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w') >>> for i in range(5): ... record.write_idx(i, 'record_%d'%i) >>> record.close() >>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r') >>> record.read_idx(3) record_3 (RJR4(R R:((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pytread_idxs cC@si|j|ƒ}|jƒ}|j|ƒ|jjdt|ƒ|fƒ||j|<|jj|ƒdS(sFInserts input record at given index. Example usage: ---------- >>> for i in range(5): ... record.write_idx(i, 'record_%d'%i) >>> record.close() Parameters ---------- idx : int Index of a file. buf : Record to write. s%s %d N(R<RLR+R=tstrR:R;RE(R R:R*RGRI((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyt write_idx+s     ( R5R6R7RDRR RR RJRLRMRO(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyR8Äs     tHEADERR tlabeltidtid2tIfQQcC@s–t|Œ}t|jtjƒr6|jddƒ}nFtj|jdtjƒ}|jd|j ddƒ}|j ƒ|}t j t |Œ|}|S(snPack a string into MXImageRecord. Parameters ---------- header : IRHeader Header of the image record. ``header.label`` can be a number or an array. See more detail in ``IRHeader``. s : str Raw image string to be packed. Returns ------- s : str The packed string. Examples -------- >>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3] >>> id = 2574 >>> header = mx.recordio.IRHeader(0, label, id, 0) >>> with open(path, 'r') as file: ... s = file.read() >>> packed_s = mx.recordio.pack(header, s) R itdtypeRQ(tIRHeadert isinstanceRQtnumberstNumbert_replacetnptasarraytfloat32R3ttostringtstructtpackt _IR_FORMAT(theadertsRQ((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyR`Us cC@szttjt|t ƒŒ}|t}|jdkrp|jdtj|tj |jƒƒ}||jd}n||fS(sÙUnpack a MXImageRecord to string. Parameters ---------- s : str String buffer from ``MXRecordIO.read``. Returns ------- header : IRHeader Header of the image record. s : str Unpacked string. Examples -------- >>> record = mx.recordio.MXRecordIO('test.rec', 'r') >>> item = record.read() >>> header, s = mx.recordio.unpack(item) >>> header HEADER(flag=0, label=14.0, id=20129312, id2=0) iRQi( RVR_tunpackRat_IR_SIZER RZR[t frombufferR](RcRb((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyRdxs  'iÿÿÿÿcC@sXt|ƒ\}}tj|dtjƒ}tdk s<t‚tj||ƒ}||fS(scUnpack a MXImageRecord to image. Parameters ---------- s : str String buffer from ``MXRecordIO.read``. iscolor : int Image format option for ``cv2.imdecode``. Returns ------- header : IRHeader Header of the image record. img : numpy.ndarray Unpacked image. Examples -------- >>> record = mx.recordio.MXRecordIO('test.rec', 'r') >>> item = record.read() >>> header, img = mx.recordio.unpack_img(item) >>> header HEADER(flag=0, label=14.0, id=20129312, id2=0) >>> img array([[[ 23, 27, 45], [ 28, 32, 50], ..., [ 36, 40, 59], [ 35, 39, 58]], ..., [[ 91, 92, 113], [ 97, 98, 119], ..., [168, 169, 167], [166, 167, 165]]], dtype=uint8) RUN(RdR[Rftuint8tcv2R2R%timdecode(RctiscolorRbtimg((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyt unpack_img–s %i_s.jpgc C@sµtdk st‚ddg}dg}d}|jƒ|krQtj|g}n$|jƒ|krutj|g}ntj|||ƒ\}}|s¢tdƒ‚t||jƒƒS(sPack an image into ``MXImageRecord``. Parameters ---------- header : IRHeader Header of the image record. ``header.label`` can be a number or an array. See more detail in ``IRHeader``. img : numpy.ndarray Image to be packed. quality : int Quality for JPEG encoding in range 1-100, or compression for PNG encoding in range 1-9. img_fmt : str Encoding of the image (.jpg for JPEG, .png for PNG). Returns ------- s : str The packed string. Examples -------- >>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3] >>> id = 2574 >>> header = mx.recordio.IRHeader(0, label, id, 0) >>> img = cv2.imread('test.jpg') >>> packed_s = mx.recordio.pack_img(header, img) s.JPGs.JPEGs.PNGsfailed to encode imageN( RhR2R%tuppertIMWRITE_JPEG_QUALITYtIMWRITE_PNG_COMPRESSIONtimencodeR`R^( RbRktqualitytimg_fmtt jpg_formatst png_formatst encode_paramstretR*((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pytpack_imgÁs  (R7t __future__Rt collectionsRRR_RXtnumpyR[tbaseRRRRRht ImportErrorR2tobjectRR8RVRatcalcsizeReR`RdRlRw(((sN/usr/local/lib/python2.7/site-packages/mxnet-1.3.1-py2.7.egg/mxnet/recordio.pyts.        #  +