U C^4@shddlmZddlmZddlZddlmZmZddlm Z ddZ Gdd d eZ Gd d d eZ dS) ) get_worker)_get_global_clientN)AbstractFileSystemAbstractBufferedFile) filesystemcCs|||}||SN)_determine_worker)clsargskwargsinstr ?/tmp/pip-install-6_kvzl1k/fsspec/fsspec/implementations/dask.py make_instances rcsleZdZdZdfdd ZddZddZd d Zd d Zd dZ ddZ ddZ dddZ ddZ ZS)DaskWorkerFileSystema)View files accessible to a worker as any other remote file-system When instances are run on the worker, uses the real filesystem. When run on the client, they call the worker to provide information or data. **Warning** this implementation is experimental, and read-only for now. Nc s8tjf|||_||_d|_d|_d|_|dSr)super__init__protocolremote_optionsworkerclientfsr)selfZremote_protocolrr  __class__r rrszDaskWorkerFileSystem.__init__cCsZz&td|_t|jf|jpi|_Wn.tk rTd|_t|_t ||_ YnXdS)NTF) rrrrrr ValueErrorrrdaskZdelayedrfsrr r rr sz&DaskWorkerFileSystem._determine_workercCstt||j|jffSr)rtypeZ storage_argsZstorage_optionsrr r r __reduce__*szDaskWorkerFileSystem.__reduce__cOs,|jr|jj||n|jj||dSr)rrmkdirrcomputerr r r r rr!-szDaskWorkerFileSystem.mkdircOs,|jr|jj||n|jj||dSr)rrrmrr"r#r r rr$3szDaskWorkerFileSystem.rmcOs,|jr|jj||n|jj||dSr)rrcopyrr"r#r r rr%9szDaskWorkerFileSystem.copycOs,|jr|jj||n|jj||dSr)rrmvrr"r#r r rr&?szDaskWorkerFileSystem.mvcOs*|jr|jj||S|jj||SdSr)rrlsrr"r#r r rr'EszDaskWorkerFileSystem.lsrbTcKsD|jr$|jj|f||||d|St|||f|||d|SdS)N)mode block_size autocommit cache_options)r*r+r,)rr_openDaskFile)rpathr)r*r+r,r r r rr-Ks,  zDaskWorkerFileSystem._openc CsZ|jr@|||&}|||||W5QRSQRXn|j||||SdSr)rr-seekreadr fetch_ranger")rr/r)startendfr r rr2hs  "z DaskWorkerFileSystem.fetch_range)N)r(NTN)__name__ __module__ __qualname____doc__rrr r!r$r%r&r'r-r2 __classcell__r r rrrs    rc@s&eZdZd ddZddZddZdS) r.FcCsdSrr )rfinalr r r _upload_chunkrszDaskFile._upload_chunkcCsdS)z Create remote file/upload Nr rr r r_initiate_uploaduszDaskFile._initiate_uploadcCs|j|j|j||S)z*Get the specified set of bytes from remote)rr2r/r))rr3r4r r r _fetch_rangeyszDaskFile._fetch_rangeN)F)r6r7r8r<r=r>r r r rr.qs r.) Zdistributed.workerrZdistributed.clientrrZ fsspec.specrrZfsspecrrrr.r r r rs   c