B t `1?@sdddgZddlZddlZddlZddlZddlZddlZddlmZm Z m Z ddlm Z ddlm Z mZddlmZmZmZdd lmZdZe re jje jjfZn e re jjZeZGd d d eZGd ddeZGd ddeZddZdS) TaskWorkerMultiTaskWorkerTaskWorkerServerN)get_nameserver using_pyro3 using_pyro4)Pyro)get_dispatchers_connection_problem)advance_iterator iteritems itervalues)xrangec@s&eZdZd ddZdd Zd d ZdS) TaskWorkerBase:PyUtilibServerN Task WorkerFc Cs||_d|_d|_t|_d|_tdkr.tdtrr9r9r9r:r)s Y rc@seZdZdddZddZdS)rNTcOs4||_||_||_d|_d|_tj|f||dS)NF)typeblocktimeout_contiguous_task_processing_next_processing_idrr;)r3rBrCrDargskwdsr9r9r:r;s zTaskWorker.__init__c Cstdxd|_d|_yxd}|jrh|j|j|j|jff}t |dksNt |j|ks\t ||j}n|jj |j|j|jdf}|dk st Wnlt k r}zNt d}tdtt|dt|tdtdd }t|Wdd}~XYq Xt |d kr |jr(td t ||jfi}xZt|d d dD]D}|jrdtd|d|jf|jr|jdkr|d|_|j|dkrtd|d|jf|jd7_t|_|d|_||d|d<|j|d<|j}|tkr|j}|jr<||krg||<|j|d<|||td|dP|jrP|dS|drz||krlg||<||||jr@Pq@Wt |r |j |q WdS)Nz%Listening for work from dispatcher...Fr)rBrCrDzF***WARNING: Connection to dispatcher server denied - exception type: z - message: z]A potential remedy may be to increase PYUTILIB_PYRO_MAXCONNECTIONS in your shell environment.g?g333333?rz"Collected %s task(s) from queue %scSs|dS)Nidr9)xr9r9r:z TaskWorker.run..)keyz(Processing task with id=%s from queue %srIz#Got task with id=%s, expected id=%sclientdataresult processedByztTask worker reported error during processing of task with id=%s. Any remaining tasks in local queue will be ignored.generateResponse)!r)rrrr* get_tasksrBrCrDlenAssertionErrorZget_task_worker_connection_problemsysexc_infostrr+r/r0r1rsortedrErFr(rr_current_task_clientprocessr'appendr< add_results) r3tasksZtasks_erJr8resultstaskreturn_type_namer9r9r:r>s              zTaskWorker.run)NTN)r?r@rAr;r>r9r9r9r:rs c@sVeZdZdddZddZdd Zd d Zd d ZddZddZ ddZ ddZ dS)rNTcOs0tj|f||d|_d|_||||dS)Nr)rr; _num_types _type_cyclepush_request_type)r3Z type_defaultZ block_defaultZtimeout_defaultrGrHr9r9r:r;szMultiTaskWorker.__init__cCs4d}yt|j}Wntk r.tdYnX|S)Nz*MultiTaskWorker has no work request types!)r rf StopIterationr()r3newr9r9r:_get_request_type s z!MultiTaskWorker._get_request_typecCs:|jdkrgSg}x"t|jD]}|t|jqW|S)zj return the full work type list, starting from the current location in the cycle. r)rerr]r rf)r3Z type_listZcntr9r9r:current_type_orders  z"MultiTaskWorker.current_type_ordercCst|jdS)N)r rf)r3r9r9r:cycle_type_order!sz MultiTaskWorker.cycle_type_ordercCs|jS)N)re)r3r9r9r:num_request_types$sz!MultiTaskWorker.num_request_typescCstg|_d|_dS)Nr) itertoolscyclerfre)r3r9r9r:clear_request_types's z#MultiTaskWorker.clear_request_typescCs.t||||fg|_|jd7_dS)z^ add a request type to the end relative to the current cycle location rN)rnrorkrfre)r3 type_namerCrDr9r9r:rg+s z!MultiTaskWorker.push_request_typecCs.|}|}t||_|jd8_|S)z] delete the last type request relative to the current cycle location r)rkpoprnrorfre)r3Z new_type_listelr9r9r:pop_request_type4s  z MultiTaskWorker.pop_request_typec Cstdxd|_d|_y|j|}Wnltk r}zNtd}tdt t |dt |tdt dd}t |Wdd}~XYq Xt|d kr |jrtd td d t|Dt|fi}xt|D]\}}g}||<xt|d ddD]} t|_| d|_|| d| d<|j| d<|j} | tkrJ|} |jr| |krdg|| <|| | td| dP|jr|dS| dr| |krg|| <|| | qW|jrPqWt|r |j|q WdS)Nz%Listening for work from dispatcher...FrzF***WARNING: Connection to dispatcher server denied - exception type: z - message: z]A potential remedy may be to increase PYUTILIB_PYRO_MAXCONNECTIONS in your shell environment.g?g333333?rz%Collected %s task(s) from %s queue(s)css|]}t|VqdS)N)rT).0Z_tlr9r9r: Zsz&MultiTaskWorker.run..cSs|dS)NrIr9)rJr9r9r:rKarLz%MultiTaskWorker.run..)rMrNrOrPrQztTask worker reported error during processing of task with id=%s. Any remaining tasks in local queue will be ignored.rIrR)r)rrr*rSrkrVrWrXrYrBr+r/r0r1rTrsumr r rZrrr[r\r'r]r<r^) r3r_r`rJr8rarqZ type_tasksZ type_resultsrbrcr9r9r:r>?s`           zMultiTaskWorker.run)NTrd) r?r@rAr;rjrkrlrmrprgrtr>r9r9r9r:rs    cKs^|f|}|jdkrdSy |Wn4tjjk rDtdYn|YnXdS)Nz4Lost connection to dispatch server. Shutting down...)rr>r errorsZConnectionClosedErrorr)r<)clsrHZworkerr9r9r:rs    )__all__rWr#r%r0rnr+Zpyutilib.pyro.utilrrrrr r r sixr r r Z six.movesrrVrx TimeoutErrorZConnectionDeniedErrorobjectrrrrrr9r9r9r: s.    na