3 Nb@svddlZddlZddlZdddgZGdddeZejddgZejdd d dgZGd ddeZ Gd ddeZ dS) N FutureResult SlavePipe SyncMasterc@s(eZdZdZddZddZddZdS) rzBA thread-safe future implementation. Used only as one-to-one pipe.cCs"d|_tj|_tj|j|_dS)N)_result threadingLock_lock Condition_cond)selfr G/home/ec2-user/SageMaker/lama/models/ade20k/segm_lib/nn/modules/comm.py__init__s zFutureResult.__init__c Cs8|j(|jdkstd||_|jjWdQRXdS)Nz#Previous result has't been fetched.)rrAssertionErrorr notify)r resultr r r putszFutureResult.putc Cs6|j&|jdkr|jj|j}d|_|SQRXdS)N)rrr wait)r resr r r get s   zFutureResult.getN)__name__ __module__ __qualname____doc__rrrr r r r rsZMasterRegistryr_SlavePipeBase identifierqueuec@seZdZdZddZdS)rz$Pipe for master-slave communication.cCs,|jj|j|f|jj}|jjd|S)NT)rrrrr)r msgretr r r run_slave1s  zSlavePipe.run_slaveN)rrrrrr r r r r.sc@s4eZdZdZddZddZddZedd Zd S) ra An abstract `SyncMaster` object. - During the replication, as the data parallel will trigger an callback of each module, all slave devices should call `register(id)` and obtain an `SlavePipe` to communicate with the master. - During the forward pass, master device invokes `run_master`, all messages from slave devices will be collected, and passed to a registered callback. - After receiving the messages, the master device should gather the information and determine to message passed back to each slave devices. cCs$||_tj|_tj|_d|_dS)z Args: master_callback: a callback to be invoked after having collected messages from slave devices. FN)_master_callbackrQueue_queue collections OrderedDict _registry _activated)r Zmaster_callbackr r r rCs  zSyncMaster.__init__cCsJ|jr(|jjstdd|_|jjt}t||j|<t||j|S)z Register an slave device. Args: identifier: an identifier, usually is the device id. Returns: a `SlavePipe` object which can be used to communicate with the master device. z.Queue is not clean before next initialization.F) r&r"emptyrr%clearr_MasterRegistryr)r rfuturer r r register_slaveNs  zSyncMaster.register_slavecCsd|_d|fg}x"t|jD]}|j|jjqW|j|}|dddksVtdx,|D]$\}}|dkrnq\|j|j j |q\Wx$t|jD]}|jjdkstqW|ddS)ad Main entry for the master device in each forward pass. The messages were first collected from each devices (including the master device), and then an callback will be invoked to compute the message to be sent back to each devices (including the master device). Args: master_msg: the message that the master want to send to itself. This will be placed as the first message when calling `master_callback`. For detailed usage, see `_SynchronizedBatchNorm` for an example. Returns: the message to be sent back to the master device. Trz.The first result should belongs to the master.) r&range nr_slavesappendr"rr rr%rr)r Z master_msg intermediatesiresultsrr r r run_master`s  zSyncMaster.run_mastercCs t|jS)N)lenr%)r r r r r.szSyncMaster.nr_slavesN) rrrrrr+r3propertyr.r r r r r8s   !) rr#r__all__objectr namedtupler)rrrr r r r  s