U Dx`(@sdZddlmZmZddlmZmZmZmZm Z m Z m Z m Z m Z eZgZzddlmZWnek rvedYnXzddlmZmZmZmZWnek redYnXedd Zd d Zdd dZdddZGddde ZdS)zO FileSystem abstraction to interact with various local and remote filesystems. ) _is_path_like_stringify_path) FileSelectorFileTypeFileInfo FileSystemLocalFileSystemSubTreeFileSystem_MockFileSystemFileSystemHandler PyFileSystem)HadoopFileSystemr ) S3FileSystem S3LogLevel initialize_s3 finalize_s3rcCs(|tkrtd|td|dS)Nz>> PyFileSystem(FSSpecHandler(fsspec_fs)) cCs ||_dSN)fs)selfr4rrr__init__szFSSpecHandler.__init__cCst|tr|j|jkStSr3r&r*r4NotImplementedr5otherrrr__eq__s  zFSSpecHandler.__eq__cCst|tr|j|jkStSr3r7r9rrr__ne__s  zFSSpecHandler.__ne__cCs$|jj}t|tr|d}d|S)Nrz fsspec+{0})r4protocolr&listr)r5r=rrr get_type_names zFSSpecHandler.get_type_namecCs|Sr3rr5r rrrrszFSSpecHandler.normalize_pathcCsR|d}|ddkrtj}n|ddkr4tj}d}ntj}t||||dddS)Nsizerfile directorymtime)rArD)rFilerUnknownrget)r inforAZftyperrr_create_file_infos  zFSSpecHandler._create_file_infoc Cs\g}|D]N}z|j|}Wn&tk rB|t|tjYqX||||q|Sr3)r4rHFileNotFoundErrorappendrrr/rI)r5pathsinfosr rHrrrrszFSSpecHandler.get_file_infocCs|j|js<|j|jr(t|jn|jr2gSt|j|jrHd}nd}g}|jj|j|ddd}| D]\}}| | ||qn|S)NT)maxdepthZwithdirsdetail) r4isdirbase_direxistsNotADirectoryErrorZallow_not_foundrJ recursivefinditemsrKrI)r5selectorrOrMZselected_filesr rHrrrget_file_info_selectors&  z$FSSpecHandler.get_file_info_selectorcCs|jj||ddS)N)Zcreate_parents)r4mkdir)r5r rUrrr create_dirszFSSpecHandler.create_dircCs|jj|dddSNTrU)r4rmr@rrr delete_dirszFSSpecHandler.delete_dircCsP|jj|ddD]:}|j|r2|jj|ddq|j|r|j|qdS)NF)rPTr])r4listdirrQr^isfile)r5r subpathrrr_delete_dir_contentss   z"FSSpecHandler._delete_dir_contentscCs(|ddkrtd|d||dS)N/z$delete_dir_contents called on path '')striprrcr@rrrdelete_dir_contentssz!FSSpecHandler.delete_dir_contentscCs|ddS)Nrd)rc)r5rrrdelete_root_dir_contentssz&FSSpecHandler.delete_root_dir_contentscCs$|j|st||j|dSr3)r4rSrJr^r@rrr delete_files zFSSpecHandler.delete_filecCs|jj||dddSr\)r4mvr5srcdestrrrmoveszFSSpecHandler.movecCs|j||dSr3)r4copyrlrrr copy_file"szFSSpecHandler.copy_filecCs8ddlm}|j|s t|||jj|ddddSNr PythonFilerbmoderpyarrowrtr4rarJopenr5r rtrrropen_input_stream's  zFSSpecHandler.open_input_streamcCs8ddlm}|j|s t|||jj|ddddSrrryr|rrropen_input_file/s  zFSSpecHandler.open_input_filecCs$ddlm}||jj|ddddS)Nrrswbrvwrzrtr4r{r|rrropen_output_stream7s z FSSpecHandler.open_output_streamcCs$ddlm}||jj|ddddS)Nrrsabrvrrr|rrropen_append_stream<s z FSSpecHandler.open_append_streamN)r) __module__ __qualname____doc__r6r;r<r?r staticmethodrIrrYr[r_rcrhrirjrorqr}r~rrrrrrr*s,  r*)FF)NF)rZ pyarrow.utilrrZ pyarrow._fsrrrrrr r r r Z FileStatsrZ pyarrow._hdfsr rrKZ pyarrow._s3fsrrrrrr$r-r2r*rrrrs,,   * 2