U C^m @sdZddlmZddlmZddlmZddlZddl m Z ddZ de iZ d!d d Z d"d dZe dede dede ddddz,ddlZe dejde dejdddWnek rYnXz0ddlZe dejddde dejdddWnek rYnXGddde ZzddlZeje degWneefk rZYnXzddlZe dejjdWnek rYnXz"ddlZd#ddZe ded Wnek rYnXdS)$z9Helper functions for a standard streaming compression API)BZ2FileGzipFile)ZipFileN)AbstractBufferedFilecKs|SN)filemodekwargsrr6/tmp/pip-install-6_kvzl1k/fsspec/fsspec/compression.py noop_file sr FcCstt|tr|g}|tkr(|s(td||D]$}|tjjkr,|s,td||fq,|t|<|D]}|tjj|<q^dS)aARegister an "inferable" file compression type. Registers transparent file compression type for use with fsspec.open. Compression can be specified by name in open, or "infer"-ed for any files ending with the given extensions. Args: name: (str) The compression type name. Eg. "gzip". callback: A callable of form (infile, mode, **kwargs) -> file-like. Accepts an input file-like object, the target mode and kwargs. Returns a wrapped file-like object. extensions: (str, Iterable[str]) A file extension, or list of file extensions for which to infer this compression scheme. Eg. "gz". force: (bool) Force re-registration of compression type or extensions. Raises: ValueError: If name or extensions already registered, and not force. z&Duplicate compression registration: %sz-Duplicate compression file extension: %s (%s)N) isinstancestrcompr ValueErrorfsspecutils compressions)namecallback extensionsforceextrrr register_compressions    rrbc s|d|krJ|pd}t|fddi|j|dd}|jffdd |_|St||dkrfd}j|fddi|S) Nrr r w)r cs|p Sr)close)closerzrr >zunzip..r)ropenrnamelist)infiler filenamer forr r unzip9s r)zipbz2gzipcKstfd|i|S)Nfileobjr)fr rrr r"Hr#r"gzlzmaxzT)rcs@eZdZfddZd ddZdddZd d Zd d ZZS) SnappyFilec s^ddl}ddi|_tjfdd|ddd|||_d|krP||_n ||_dS)Nrsizeiɚ;snappyb)fspathr r) r4detailssuper__init__stripr&ZStreamDecompressorcodecZStreamCompressor)selfr&r r r4 __class__rr r:\s $ zSnappyFile.__init__FcCs.|jd|j|j}|j|dS)NrT)bufferseekr<Z add_chunkreadr&write)r=finaloutrrr _upload_chunkgs  zSnappyFile._upload_chunkrcCs tddS)NzSnappyFile is not seekable)NotImplementedError)r=locwhencerrr rAmszSnappyFile.seekcCsdS)NFr)r=rrr seekablepszSnappyFile.seekablecCs|j||}|j|S)z*Get the specified set of bytes from remote)r&rBr< decompress)r=startenddatarrr _fetch_rangesszSnappyFile._fetch_range)F)r) __name__ __module__ __qualname__r:rFrArJrO __classcell__rrr>r r2[s   r2r4lz4cCs4d|krt}||Stjdd}||SdS)Nr )level)zstdZZstdDecompressorZ stream_readerZZstdCompressorZ stream_writer)r&r Zcctxrrr zstandard_files   rXrWZzst)F)rN)r)__doc__r+rr,rzipfilerZ fsspec.utilsrZ fsspec.specrr rrr)r0LZMAFile ImportErrorZlzmaffir2r4compress NameErrorZ lz4.framerTframer$Z zstandardrWrXrrrr sT     '