ó —Àv]c@s@dZddlZddlZd„Zd„Zdd„ZdS(s.Command line options of job submission script.iÿÿÿÿNcCsÕtƒ}g}|jr•xztt|jƒƒD]`}|j|}tjj|ƒr|j|ƒ|j d|j dƒdƒq.|j |ƒq.Wnx3|j D](}tjj|ƒrŸ|j|ƒqŸqŸW||fS(suGet the list of files to be cached. Parameters ---------- args: ArgumentParser.Argument The arguments returned by the parser. Returns ------- cache_file_set: set of str The set of files to be cached to local execution environment. command: list of str The commands that get rewritten after the file cache is used. s./t/iÿÿÿÿ( tsettauto_file_cachetrangetlentcommandtostpathtexiststaddtappendtsplittfiles(targstfsettcmdstitfname((s2/tmp/pip-install-Qvdv_2/mxnet/dmlc_tracker/opts.pytget_cache_file_sets    !cCsp|jƒ}|jdƒr3tt|d ƒdƒS|jdƒrVtt|d ƒƒSd|}t|ƒ‚dS(s¿Get the memory in MB from memory string. mem_str: str String representation of memory requirement. Returns ------- mem_mb: int Memory requirement in MB. tgiÿÿÿÿitmsCInvalid memory specification %s, need to be a number follows g or mN(tlowertendswithtinttfloatt RuntimeError(tmem_strtmsg((s2/tmp/pip-install-Qvdv_2/mxnet/dmlc_tracker/opts.pyt get_memory_mb's  c Cs>tjddƒ}|jddtddddd d d d d gdddƒ|jddtdtddƒ|jddddtddƒ|jddddtdddƒ|jddddtddƒ|jddddtdd d!ƒ|jd"dddtdd#d$ƒ|jd%ddwdtdd&ƒ|jd'dddtdd(ƒ|jd)dd*dtdd*d+gdd,ƒ|jd-ddwdtdd.d/ƒ|jd0ddwdtdd1d2ƒ|jd3dd4dtdd5d6ƒ|jd7ddwdtdd8ƒ|jd9ddwdtdd:ƒ|jd;dtdtdd<d=d>d?ƒ|jd@dgdAdBddCdDdEƒ|jdFdgdAdBddGdHdIdJƒ|jdKdAdBdgddLƒ|jdMdtddNdOƒ|jdPdtdtj j tj j t ƒtj dƒddQƒ|jdRdtddSƒf|jdTddwdtddUdVdWƒ|jdXdtddYƒ|jdZd[d\dd]ƒ|jd^ddwdtdd_d`ƒ|jdaddwdtddbdcƒ|jdddddtddedfƒ|jdgddhdtddidjƒ|jdkddhdtddldjƒ|jdmddwdtddndOƒ|jdoddwdtddpdOƒ|jdqdddtddrƒ|j|ƒ\}}|j|7_|jdwkrðtjdsdwƒ|_n|jdwkrtdtdudvƒ‚nt|jƒ|_t|jƒ|_|S(xsòGet options to launch the job. Returns ------- args: ArgumentParser.Argument The arguments returned by the parser. cache_file_set: set of str The set of files to be cached to local execution environment. t descriptionsDMLC job submission.s --clusterttypetchoicestyarntslurmtmpitsgetlocaltsshtmesost kubernetesthelps Cluster type of this submission,s/default to env variable ${DMLC_SUBMIT_CLUSTER}.s --num-workerstrequireds)Number of worker proccess to be launched.s--worker-corestdefaultis8Number of cores to be allocated for each worker process.s--worker-memoryt1gs,Memory need to be allocated for each worker,s need to ends with g or ms --num-serversis>Number of server process to be launched. Only used in PS jobs.s--server-coress8Number of cores to be allocated for each server process.sOnly used in PS jobs.s--server-memorys-Memory need to be allocated for each server, sneed to ends with g or m.s --jobnamesName of the job.s--queues,The submission queue the job should goes to.s --log-leveltINFOtDEBUGsLogging level of the logger.s --log-files%Output log to the specific log file, s#the log is still printed on stderr.s --host-ips'Host IP addressed, this is only needed s/if the host IP cannot be automatically guessed.s--hdfs-tempdirs/tmpsTemporary directory in HDFS, s only needed in YARN mode.s --host-files@The file contains the list of hostnames, needed for MPI and ssh.s --sge-log-dirs3Log directory of SGD jobs, only needed in SGE mode.s--auto-file-caches6Automatically cache files appeared in the command linesto local executor folder.sE This will also cause rewritten of all the file names in the command,sO e.g. `../../kmeans ../kmeans.conf` will be rewritten to `./kmeans kmeans.conf`s--filestactionR s?The cached file list which will be copied to local environment,s4 You may need this option to cache additional files.s You --auto-file-cache is offs --archivessSame as cached files,s@ but corresponds to archieve files that will be unziped locally,s2 You can use this option to ship python libraries.s Only valid in yarn jobs.s--envs3Client and ApplicationMaster environment variables.s--yarn-app-classpaths*Explicit YARN ApplicationMaster classpath.s!Can be used to override defaults.s--yarn-app-dirs4Directory to YARN appmaster. Only used in YARN mode.s--mesos-masters(Mesos master, default to ${MESOS_MASTER}s --ship-libcxxs*The path to the customized gcc lib folder.s4You can use this option to ship customized libstdc++s library to the workers.s--sync-dst-dirsmif specificed, it will sync the current directory into remote machines's SYNC_DST_DIRRtnargst+sCommand to be launcheds--slurm-worker-nodessBNumber of nodes on which workers are run. Used only in SLURM mode.s8If not explicitly set, it defaults to number of workers.s--slurm-server-nodessLNumber of nodes on which parameter servers are run. Used only in SLURM mode.sBIf not explicitly set, it defaults to number of parameter servers.s--kube-namespacesFA namespace in whitch all tasks are run. Used only in Kubernetes mode.s.If not explicitly set, it defaults to default.s--kube-worker-images mxnet/pythons9Container image of workers. Used only in Kubernetes mode.s3If not explicitly set, it defaults to mxnet/python.s--kube-server-images9Container image of servers. Used only in Kubernetes mode.s--kube-worker-templates<Manifest template for workers. Used only in Kubernetes mode.s--kube-server-templates<Manifest template for servers. Used only in Kubernetes mode.s--local-num-attempts2Number of attempt local tracker can restart slave.tDMLC_SUBMIT_CLUSTERs--cluster is not specified, s.you can also specify the default behavior via s(environment variable DMLC_SUBMIT_CLUSTERN(targparsetArgumentParsert add_argumenttstrtTrueRtNonetboolRRtjointdirnamet__file__tpardirtparse_known_argsRtclustertgetenvRRt worker_memorytworker_memory_mbt server_memorytserver_memory_mb(R tparsertunknown((s2/tmp/pip-install-Qvdv_2/mxnet/dmlc_tracker/opts.pytget_opts<sÈ    '  (t__doc__RR2RRR7RF(((s2/tmp/pip-install-Qvdv_2/mxnet/dmlc_tracker/opts.pyts    !