B u `_|@sddlZddlZddlZddlZddlZddlZddlmZmZddl m Z ddl m Z ddl mZmZmZmZmZmZmZmZddlmZmZmZmZmZmZddlZddlm Z m!Z!ddl"m#Z#dd l$m%Z%dd l&m'Z'dd l(m)Z)m*Z*m+Z+dd l,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:dd l;mZ>ddl?m@Z@ddlAmBZBmCZCmDZDddlEmFZFe d\ZGZHeIdZJGdddeKZLddZMGdddeKZNGdddeNZOdS)N)fabssqrt)ApplicationError)attempt_import)VarSet BooleanSet IntegerSetSuffixvalueminimizemaximize) UndefinedData undefined SolverFactory SolverStatusTerminationConditionSolutionStatus) _PHBoundBase$ExtractInternalNodeSolutionsforInner) DualPHModel)create_ef_instance)2scenario_tree_node_variables_generator_noinstances)add_ph_objective_weight_termsadd_ph_objective_proximal_terms%form_linearized_objective_constraints)create_block_symbol_mapsreset_nonconverged_variablesreset_stage_cost_variablesreset_linearization_variables indexToStringcreate_ph_parameterscreate_nodal_ph_parameterspreprocess_scenario_instancepreprocess_bundle_instancefind_active_objectivepreprocess_block_objectivespreprocess_block_constraintsextract_solve_times _OLD_OUTPUT)load_external_module)phsolverserverutils)SolverManager_Serial)iterkeys itervalues iteritems)xrangeguppyz pyomo.pyspc@sneZdZddZdddZdddZddd Zdd d Zdd d ZdddZ dddZ dddZ dddZ dS) ProblemStatescCst|d|_t|d|_t|d|_t|d|_tdd|D|_tdd|D|_tdd|D|_tdd|D|_ t|d|_ t|d|_ t|d|_ dS)NFcss|]}|gfVqdS)N).0 inst_namer3r31/tmp/pip-unpacked-wheel-n62dbgi3/pyomo/pysp/ph.py _sz)ProblemStates.__init__..css|]}|gfVqdS)Nr3)r4r5r3r3r6r7`scss|]}|gfVqdS)Nr3)r4r5r3r3r6r7iscss|]}|gfVqdS)Nr3)r4r5r3r3r6r7js) dictfromkeyshas_ph_objective_weight_termshas_ph_objective_proximal_terms!ph_objective_proximal_expressionsph_objective_weight_expressionsph_constraints ph_variablesfixed_variablesfreed_variablesobjective_updatedph_constraints_updateduser_constraints_updated)selfZ instancesr3r3r6__init__Xs zProblemStates.__init__NcCs|dk r(d|j|<d|j|<d|j|<nZxt|jD]}d|j|<q4Wxt|jD]}d|j|<qRWxt|jD]}d|j|<qpWdS)NF)rBrCrDr-)rEnamekeyr3r3r6clear_update_flagsqs   z ProblemStates.clear_update_flagscCsF|dkr0x"t|jD]}t|dkrdSqWdSt|j|dkSdS)NrTF)r.r@len)rErGvalr3r3r6has_fixed_variabless  z!ProblemStates.has_fixed_variablescCsF|dkr0x"t|jD]}t|dkrdSqWdSt|j|dkSdS)NrTF)r.rArJ)rErGrKr3r3r6has_freed_variabless  z!ProblemStates.has_freed_variablescCsF|dkr0x"t|jD]}t|dkrdSqWdSt|j|dkSdS)NrTF)r.r>rJ)rErGrKr3r3r6has_ph_constraintss  z ProblemStates.has_ph_constraintscCsF|dkr0x"t|jD]}t|dkrdSqWdSt|j|dkSdS)NrTF)r.r?rJ)rErGrKr3r3r6has_ph_variabless  zProblemStates.has_ph_variablescCsJ|dkr$x<|jD]}g|j|<qWn"||jkr:g|j|<n td|dS)Nz KeyError: %s)r@KeyError)rErGrHr3r3r6clear_fixed_variabless    z#ProblemStates.clear_fixed_variablescCsJ|dkr$x<|jD]}g|j|<qWn"||jkr:g|j|<n td|dS)Nz KeyError: %s)rArP)rErGrHr3r3r6clear_freed_variabless    z#ProblemStates.clear_freed_variablescCsJ|dkr$x<|jD]}g|j|<qWn"||jkr:g|j|<n td|dS)Nz KeyError: %s)r?rP)rErGrHr3r3r6clear_ph_variabless    z ProblemStates.clear_ph_variablescCsJ|dkr$x<|jD]}g|j|<qWn"||jkr:g|j|<n td|dS)Nz KeyError: %s)r>rP)rErGrHr3r3r6clear_ph_constraintss    z"ProblemStates.clear_ph_constraints)N)N)N)N)N)N)N)N)N) __name__ __module__ __qualname__rFrIrLrMrNrOrQrRrSrTr3r3r3r6r2Ss ! r2cCs ||_dS)N)_aggregate_user_data)ph scenario_treeZscenario_tree_objectZaggregate_datar3r3r6assign_aggregate_datasr[c@s eZdZddZeddZeddZddZd d Zd d Z d dZ ddZ ddZ ddZ ddZddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0ZdEd3d4Zd5d6Zd7d8Zd9d:Zd;d<Z d=d>Z!dFd?d@Z"dAdBZ#dCdDZ$d2S)G_PHBasecCsi|_d|_d|_d|_d|_i|_d|_i|_i|_d|_ d|_ d|_ i|_ i|_ i|_i|_d|_i|_i|_i|_d|_d|_d|_d|_d|_d|_d|_i|_i|_i|_d|_d|_d|_ dS)NZcplexgh㈵>TFgr)! _solver_map _solver_type _solver_io$_comparison_tolerance_for_fixed_vars_problem_states_modules_imported_write_fixed_variablesrX _instances_scenario_instance_factory_objective_sense_objective_sense_option_gaps_solution_status _solve_times_pyomo_solve_times_scenario_tree_cached_solutions_cached_scenariotree_solutions_solver_results_verbose _output_times_rho_retain_quadratic_binary_terms"_linearize_nonbinary_penalty_terms_breakpoint_strategy_integer_tolerance_bundle_binding_instance_map_bundle_scenario_instance_map_scenario_to_bundle_map_best_reported_inner_bound_best_reported_outer_bound _initialized)rEr3r3r6rFsB  z_PHBase.__init__cCs|jS)N)rl)rEr3r3r6rZsz_PHBase.scenario_treecCs|jS)N)r|)rEr3r3r6 initializedsz_PHBase.initializedcOs tddS)Nz+_PHBase::initialize() is an abstract method)NotImplementedError)rEargskwdsr3r3r6 initializesz_PHBase.initializecCs"xt|jD]}t||q WdS)N)r.rdr)rEctypesinstancer3r3r6_create_instance_symbol_mapssz$_PHBase._create_instance_symbol_mapscCstdd|jjD|_x|jjD]t}|j}|j|jkssz5_PHBase._setup_scenario_instances..z;ScenarioTree has not been linked with Pyomo model instancesT)activeF)r2rl _scenariosra _instancerGAssertionError RuntimeErrorrBrrDblock_data_objects _gen_obj_repn _gen_con_repnrd)rEscenarioscenario_instanceblockr3r3r6_setup_scenario_instancess  z!_PHBase._setup_scenario_instancescOs tddS)Nz&_PHBase::solve() is an abstract method)r~)rErrr3r3r6solvesz _PHBase.solvecCs|j|d}|dkr"td||r6|dk r6|j|=x |jjD]}|||jq@Wt|jspt|j spdS|j |d}|dkrtd||r|dk r|j |=|j rxt |jD]\}}||\}}xRt |D]F\} } |j | } | j tj} x&| D]\} }}| | }||qWqW|dk r\|\}}|j||jj||j|jdx\t |D]P\} } |j | } | j tj} x.| D]&\} }}| | }|jst||_qWqfWqWnxt |j D]\} } || \}}| j tj} x&|D]\} }}| | }||qW|dk rH|\}}| j|| jj||j|jd| j tj} x.|D]&\} }}| | }|jsxt||_qZWqWdS)Nz9PH scenario tree solution cache with id %s does not exist)&allow_consistent_values_for_fixed_vars#comparison_tolerance_for_fixed_vars)rngetrrlr set_solutionrrJrwrdrmcontains_bundlesr/_PHInstanceSymbolMapsrbySymbolZfix solutionsZadd_symbol_map load_fromrcr`fixedrstale)rEcache_id release_cachecacher bundle_namebundle_ef_instanceZsolver_results fixed_results scenario_nameZscenario_fixed_resultsrr instance_idZvarvalueZ stale_flagvardataresults results_smr3r3r6restoreCachedSolutionsst                     z_PHBase.restoreCachedSolutionscCsx(|jjD]}||j|i|j<q W|jrxt|jD]b\}}i}x6t|D]*\}}t ddt|j t j D||<qVW|j ||f|j|i|<q@WnTxRt|jD]D\}}t ddt|j t j D}|j ||f|j|i|<qWdS)Ncss&|]\}}|jr||j|jfVqdS)N)rr r)r4rrr3r3r6r7)sz)_PHBase.cacheSolutions..css&|]\}}|jr||j|jfVqdS)N)rr r)r4rrr3r3r6r7:s)rlrZ copy_solutionrn setdefaultrrr/rxtuplerrrrorrmrd)rErrrZ scenario_maprrrr3r3r6cacheSolutionss"   z_PHBase.cacheSolutionsc Cst}|jrtd|j|j|jsPH linearized objective constraint formation time=%.2f seconds)rr/rdrar;rrlrtrurvr>extendrCrqr)rErrr new_attrsrr3r3r6(form_ph_linearized_objective_constraintss  z0_PHBase.form_ph_linearized_objective_constraintsFNc Cs&t}|jr|rxt|jD]\}}|dkr<||krd}t| dksZt| dkrj|j sjd}d} t|| | |jj ||jj ||jj || |j | |j||j||j|qWi}|rt||d| rt||d|rt|| qWt}|jr"td||dS)NFTr)rz1Scenario instance preprocessing time=%.2f seconds)rrlrr/rdr#rar@rArDrCr>rBrcr]rIrQrRrwrxrJr&r'r$rqr)rEZignore_bundles subproblemsrrrZscenario_bundle_namerZpreprocess_bundle_objectiveZpreprocess_bundle_constraints bundle_solverZ fixed_varsZ freed_varsrBrrr3r3r6_preprocess_scenario_instancess                   z&_PHBase._preprocess_scenario_instancescCsTt|jxDt|jD]6\}}t||j|j|j}|gkr|jj| |qWdS)N) r"rlr/rdr!rrrtrar?r)rErrZnew_penalty_variable_namesr3r3r6_create_scenario_ph_parameters s  z&_PHBase._create_scenario_ph_parameterscCs||jkr||jkr>|j|\}}td|t|||jfxH|jjD]<}||jkrH|j|\}}td|t|||j|jfqHWtd|dS)NzCannot access rho for variable '%s' with scenario tree id '%s' on tree node '%s'. The variable is derived and therefore exclued from nonanticipativity conditions.zCannot access rho for variable '%s' with scenario tree id '%s' on tree node '%s' because the variable is tracked by a different tree node (%s).z~Invalid scenario tree id '%s' for accessing rho. No tree nodes were found with an associated instance variable having that id.)_standard_variable_idsZ_derived_variable_idsr ValueErrorr rrlr)rErrrrZother_tree_noder3r3r6 _rho_checks"      z_PHBase._rho_checkcCs8|||t|}x|jD]}||j|j|<qWdS)N)rr rrrr)rErrrho_expressionZ new_rho_valuerr3r3r6setRhoAllScenarios>s  z_PHBase.setRhoAllScenarioscCs$|||t||j|j|<dS)N)rr rrr)rErrrrr3r3r6setRhoOneScenarioHs z_PHBase.setRhoOneScenariocCs||||j|j|S)N)rrrr)rErrrr3r3r6getRhoOneScenarioNs z_PHBase.getRhoOneScenariocsXfdd}|dk r0jdks*||jr0|_|dk rTjdksN|j|rT|_dS)Ncsjtkr||kS||kSdS)N)rfr )ab)rEr3r6isbetterZs z1_PHBase._update_reported_bounds..isbetter)rzr{)rEinnerouterrr3)rEr6_update_reported_boundsWs     z_PHBase._update_reported_boundscCs6x0|jD]&}|jj|}||||qWdS)N)rr_ScenarioTreeSymbolMap getObjectsetlbsetub)rErr lower_bound upper_boundrrr3r3r6setVariableBoundsAllScenariosms   z%_PHBase.setVariableBoundsAllScenarioscCs&|jj|}||||dS)N)rrrrr)rErrrrrrr3r3r6setVariableBoundsOneScenarioys z$_PHBase.setVariableBoundsOneScenario)FN)NN)%rUrVrWrFpropertyrZr}rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr3r3r3r6r\sF   " P8R      !" j    r\c@sbeZdZddZddZddZddZd d ZdWd d ZddZ ddZ ddZ ddZ ddZ ddZddZddZddZd d!Zd"d#ZdXd%d&ZdYd(d)Zd*d+Zd,d-Zd.d/Zd0d1Zd2d3Zd4d5ZdZd6d7Zd[d8d9Zd:d;Zd\dd?Z d@dAZ!dBdCZ"dDdEZ#dFdGZ$dHdIZ%dJdKZ&dLdMZ'dNdOZ(dPdQZ)dRdSZ*d]dUdVZ+d'S)^ProgressiveHedgingcCs|jS)N)rl)rEr3r3r6get_scenario_treesz$ProgressiveHedging.get_scenario_treecCs|jS)N)rf)rEr3r3r6get_objective_sensesz&ProgressiveHedging.get_objective_sensecs>jrtfddjDp<j o.c3s|]}|VqdS)N)r)r4r)rEr3r6r7s)_or_convergersany _convergersall)rEr3)rEr6 is_convergedszProgressiveHedging.is_convergedcCs d|_dS)NT) _dual_mode)rEr3r3r6 set_dual_modesz ProgressiveHedging.set_dual_modecCs d|_dS)NF)Z_set_dual_mode)rEr3r3r6 primal_modeszProgressiveHedging.primal_moderYcCs,|jdk r(x|jD]}||j|qWdS)N)_solution_pluginswriterl)rElabelpluginr3r3r6 save_solutions  z ProgressiveHedging.save_solutioncCsN|js dSt|jtjjjjjr*t ||j | |d|_dS)NF)r| isinstance_solver_managerpyomosolverspluginssmanagerphpyroSolverManager_PHPyror+Zrelease_phsolverserversr]r_cleanup_scenario_instances_clear_bundle_instances)rEr3r3r6release_componentss  z%ProgressiveHedging.release_componentscCsTt}t|t|jtjjjj j r2t |t}|j rPt d||dS)Nz6Activate PH objective proximal terms time=%.2f seconds)rr\rrrr r r r r rr+rqr)rErrr3r3r6rs  z7ProgressiveHedging.activate_ph_objective_proximal_termscCsTt}t|t|jtjjjj j r2t |t}|j rPt d||dS)Nz8Deactivate PH objective proximal terms time=%.2f seconds)rr\rrrr r r r r rr+rqr)rErrr3r3r6rs  z9ProgressiveHedging.deactivate_ph_objective_proximal_termscCsTt}t|t|jtjjjj j r2t |t}|j rPt d||dS)Nz4Activate PH objective weight terms time=%.2f seconds)rr\rrrr r r r r rr+rqr)rErrr3r3r6rs  z5ProgressiveHedging.activate_ph_objective_weight_termscCsTt}t|t|jtjjjj j r2t |t}|j rPt d||dS)Nz6Deactivate PH objective weight terms time=%.2f seconds)rr\rrrr r r r r rr+rqr)rErrr3r3r6rs  z7ProgressiveHedging.deactivate_ph_objective_weight_termsc Cs|jsttdd|jjD}ttd|f}dt|dd}|dd7}|jrr|dd 7}|dd 7}t|xl|jjD]^}|j |j }|j |j }d }d }xL|jj D]@}|j |j j} |j |j j} || |j7}|| |j7}qWdt|d}|d 7}t|ts2|dk r2|d 7}n d}|d7}|d7}||j ||||f;}|jr|j|j } t| ts| dk r|d| 7}n |dd7}|j|j } t| ts| dk r|d| 7}n |dd7}t|qWtddS)Ncss|]}tt|jVqdS)N)rJstrr)r4Z_scenario_bundler3r3r6r7sz>ProgressiveHedging.report_bundle_objectives..zScenario Bundlez %-zs z%-20s %-20s %-20s %20s)Cost Objectivez Objective GapzSolution Statusz %10sz Solve TimezPyomo Solve Timegz%-20.4f %-20.4fz %-20.4fz None Reportedz %-20sz %20sz %-10.2fz %-10s)rlrrmaxrrJrrqrrhrrirZ get_scenario _objective_costrrrrjrrk) rE max_name_lenlinerZ bundle_gapZ bundle_statusZbundle_objective_valueZbundle_cost_valuerZscenario_objective scenario_cost solve_timepyomo_solve_timer3r3r6report_bundle_objectivessf                 z+ProgressiveHedging.report_bundle_objectivesc Cstdd|jjD}ttd|f}dt|dd}|dd7}|jrd|dd 7}|dd 7}t|x$|jjD]}|j}|j}|j |j }|j |j d }dt|d}|d 7}t |ts|dk r|d 7}n d }|d7}|d7}||j ||||f;}|jr|j |j }t |ts>|dk r>|d|7}n |dd 7}|j |j } t | ts|| dk r||d| 7}n |dd 7}t|qxWtddS)Ncss|]}tt|jVqdS)N)rJrr)r4Z _scenarior3r3r6r7asz@ProgressiveHedging.report_scenario_objectives..ZScenarioz %-zs z%-20s %-20s %-20s %20s)rrz Objective GapzSolution Statusz %10sz Solve TimezPyomo Solve Timez None Reportedz%-20.4f %-20.4fz %-20.4fz %-20sz %20sz %-10.2fz %-10sr)rrlrrJrrqrrrrhrrrirrrjrk) rErrrZobjective_valuergapstatusrrr3r3r6report_scenario_objectives_sT            z-ProgressiveHedging.report_scenario_objectivescCs0t|jtjjjjjr"t |n t |dS)N) rrr r r r r rr+Ztransmit_weightsr\r)rEr3r3r6rs z'ProgressiveHedging._push_w_to_instancescCs0t|jtjjjjjr"t |n t |dS)N) rrr r r r r rr+Z transmit_rhosr\r)rEr3r3r6rs z)ProgressiveHedging._push_rho_to_instancescCs0t|jtjjjjjr"t |n t |dS)N) rrr r r r r rr+Ztransmit_xbarsr\r)rEr3r3r6rs z*ProgressiveHedging._push_xbar_to_instancescCsJt|jtjjjjjrrrTr?rSrhasattrrrr)rErrZconstraint_namerrr3r3r6r.s     z.ProgressiveHedging._cleanup_scenario_instancesc Csd}d}d}d}d}|j}|jj}xZ|jD]P}|j|} | dk r.|d7}|| 7}|rfd}| }| }q.| |krr| }| |kr.| }q.W|dkr||}|||fS)NgrTr%F)rl findRootNodeZ_stagerr _stage_costs) rE maximum_value minimum_valueZ sum_values num_values first_timeZ root_nodeZfirst_stage_namer this_valuer3r3r6$_extract_first_stage_cost_statisticsIs.   z7ProgressiveHedging._extract_first_stage_cost_statisticsc Cs(t}||||||t|}||||j|j dd}d}t|rt dt d| |t d|j t d|j ddddd|j|jddd}t d |j |jrt d |j nFt|jtjjjjjrt|tjjtjjBtjj Btjj!B|j"d d |j#ks4t$|%||j&sN|j'rrt d |j (rj|)|*d}x"|j j+D]}||j,|j-7}qWt d|j.t/krdnd|f|j 0|j t d|j ddddd|j|jddt d|j |jr t d|j ||fS)NF) warmstartexception_on_failurerzoFailed to compute bound at xhat due to one or more solve failures. Restoring PH to solution at final iteration.z8Generating scenario tree solution from scenario averagesz Scenario tree variable values: T)output_only_statisticsoutput_only_nonconvergedoutput_no_statisticszScenario tree costs:z.Scenario tree solution (scenario tree format):rrz/Successfully completed xhat inner bound solves gz# Computed objective %s bound=%12.4fupperlowerz X-hat variable values: z X-hat costs: z' X-hat solution (scenario tree format):)1rZCachePHSolutionZ"DeactivatePHObjectiveProximalTermsZ DeactivatePHObjectiveWeightTermsrZFixScenarioTreeVariablessolve_subproblems_disable_warmstartsrJrZ RestorePHrlZsnapshotSolutionFromScenariospprint_report_only_statistics#_report_only_nonconverged_variablesZ pprintCosts_output_scenario_tree_solutionZpprintSolutionrrr r r r r rr+collect_full_results TransmitType all_stagesblendedderivedr_stackZTREE_VARIABLE_FIXINGrZRestoreLastPHChangerp_report_subproblem_objectivesrrr!rrrrfr ZupdateNodeStatistics)rEZ ph_bound_baseZ xhat_solutionfailuresobjective_boundrr3r3r6)compute_and_report_inner_bound_using_xhathsv                 zd|_?d|_@d|_Ad|_Bd|_Cd|_Dd|_Ed|_F|jG|_|jH|_|jI|_"|jJ|_|jK|_|jL|_#|jM|_$|jN|_O|jP|_Q|jR|_S|jT|_U|jV|_W|jX|_Y|jZ|_[|j\|_]dd }i|_^x|j_D]}xx|`D]l}|a`d }tb|d krt|d}||d}||j^|<n*tb|dkrd|j^|d<n tcd |q4Wq&W|jd|_e|jf|_F|jg|_h|ji|_7|jj|_8|jk|_0|jl|_9|jm|_n|jo|_'|jp|_(|jq|_+|jr|_)|js|_*|jt|_,|ju|_-|jv|_.|jw|_x|jy|_z|j{|_E|j||_}|j~|_|j|_|j|_1|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j |_/|j|_|j|_t|d r|j|_2nd|_2|jr"|jtjjO_|jOdkr.convert_value_string_to_number=z3Illegally formed scenario solver option=%s detectedprofile_memoryrzPH detected an invalid value for default rho: %s. Use --default-rho=X to specify a positive number X for default rho. A value of 1.0 is no longer assumed.Nonea9***WARNING***: PH is using a default rho value of None for all blended scenario tree variables. This will result in error during weight updates following PH iteration 0 solves unless rho is changed. This option indicates that a user intends to set rho for all blended scenario tree variables using a PH extension.z***WARNING***: PH is using a default rho value of 0 for all blended scenario tree variables. This will effectively disable non-anticipativity for all variables unless rho is change using a PH extensionzFMaximum number of PH iterations must be non-negative; value specified=zPValue of the nu parameter in PH must be on the interval (0, 2); value specified=g?zRValue of the mipgap parameter in PH must be on the unit interval; value specified=zSA breakpoint distribution strategy was specified, but linearization is not enabled!zcValue of linearization parameter for nonbinary penalty terms must be non-negative; value specified=z^Value of the breakpoint distribution strategy parameter must be non-negative; value specified=zrUnknown breakpoint distribution strategy specified - valid values are between 0 and 2, inclusive; value specified=ph_aggregategetter_callbackpysp_aggregategetter_callbackph_rhosetter_callbackpysp_phrhosetter_callbackph_boundsetter_callbackpysp_boundsetter_callback))_aggregate_getter_file_aggregate_getterrX)_rho_setter_file _rho_setterrZ)_bound_setter_file _bound_setterr\) clear_cacheverbosez@PH callback with name '%s' could not be found in module file: %sZ PHIterations)rGzPH solver configuration: z Max iterations=z Async mode=z Async buffer length=z Default global rho=z Over-relaxation enabled=z Nu=z" Aggregate getter callback file=z Rho setter callback file=z Bound setter callback file=z Sub-problem solver type='%s'z Keep solver files? z Output solver results? z Output solver log? z Output times? )r\rF_optionsrZ_phpyro_worker_jobs_map_phpyro_job_worker_mapset_queued_solve_action_handles _cost_history_incumbent_cost_history_best_incumbent_key_incumbent_cache_id_xhatr&_total_discrete_vars_total_continuous_vars_total_fixed_discrete_vars_total_fixed_continuous_varsrr+rDZnonleaf_stagesrGrF#_phpyro_variable_transmission_flags_ph_warmstart_file_ph_warmstart_index_ph_warmstarted_iteration_0_has_warmstart _overrelax_nur`r^rb_max_iterations _async_mode_async_buffer_length_ph_weight_updates_enabled_ph_xbar_updates_enabled_report_solutions_report_weights_report_rhos_each_iteration_report_rhos_first_iterationrIr@ _report_for_zero_variable_valuesrA!_output_continuous_variable_stats_output_solver_resultsrB_profile_memoryr _time_since_last_garbage_collect$_minimum_garbage_collection_interval_current_iteration_keep_solver_files_symbolic_solver_labels_output_solver_log_extensions_suffix_listr _ph_pluginsr_init_start_time_init_end_time_solve_start_time_solve_end_time_cumulative_solve_time_cumulative_xbar_time_cumulative_weight_timer>_mipgapZph_warmstart_fileZph_warmstart_indexZmax_iterations overrelaxnuZ async_modeZasync_buffer_lengthZ default_rhorrZ rho_cfgfiler_Z xhat_methodZ _xhat_methodZdisable_xhat_computation_disable_xhat_computationZaggregate_cfgfiler]Zbounds_cfgfileraZ solver_typer^ solver_ior__scenario_solver_optionsZscenario_solver_optionssplitstriprJrZhandshake_with_phpyroZ_handshake_with_phpyroZscenario_mipgapZwrite_fixed_variablesrcZkeep_solver_filessymbolic_solver_labelsZoutput_solver_resultsZoutput_solver_logrdrpZreport_solutionsZreport_weightsZreport_subproblem_objectivesZreport_rhos_each_iterationZreport_rhos_first_iterationZreport_only_statisticsZreport_for_zero_variable_valuesZ"report_only_nonconverged_variablesZ output_timesrqZ!output_instance_construction_timeZ"_output_instance_construction_timeZdisable_warmstartsZretain_quadratic_binary_termsrsZ!linearize_nonbinary_penalty_termsrtZbreakpoint_strategyruZoutput_scenario_tree_solutionZ#phpyro_transmit_leaf_stage_solutionZ$_phpyro_transmit_leaf_stage_solutionZ or_convergersrZtermdiff_threshold_termdiff_thresholdZ&enable_free_discrete_count_convergence'_enable_free_discrete_count_convergenceZfree_discrete_count_threshold_free_discrete_count_thresholdZ&enable_normalized_termdiff_convergence'_enable_normalized_termdiff_convergenceZenable_termdiff_convergence_enable_termdiff_convergenceZenable_outer_bound_convergence_enable_outer_bound_convergenceZenable_inner_outer_convergence_enable_inner_outer_convergenceZ'enable_primal_dual_residual_convergence(_enable_primal_dual_residual_convergenceZ!outer_bound_convergence_threshold"_outer_bound_convergence_thresholdZ!inner_outer_convergence_threshold"_inner_outer_convergence_thresholdZ*primal_dual_residual_convergence_threshold+_primal_dual_residual_convergence_thresholdZshutdown_pyro_workersZ_shutdown_pyro_workersZ#suppress_continuous_variable_outputobjective_senserfrgr-rTrErrrOZscenario_bundle_specificationZ_scenario_bundle_specificationZcreate_random_bundlesZ_create_random_bundlesZscenario_tree_random_seedZ_scenario_tree_random_seedr_callback_function_mapped_module_namekeysrgetattrrbr*inspect getmembers ImportErrorsetattrrZ_iteration_index_setZ constructrangeadd)rEoptionsrQZthis_option_stringZ this_optionZthis_option_pieces option_key option_valueZrenamedZ ph_attr_fileZph_attrZ callback_nameZdeprecated_callback_name module_namemoduleZsys_modules_keycallbackZonameobjir3r3r6rFs                   $                         zProgressiveHedging.__init__c" s ddlddlddlm}t|_tdtd|dk rD||_|dk rR||_|j |_ }|j dk rv|j |j |j <|j dk r|j |j |j<fdd|jD}fdd|jD}fdd|jD} fd d|jD} g} x`|jD]V} t| jjjjst| jjjjst| jjjjst| jjjjs| | qWg}|||||| || || ||_|jrtd x|jD]} | |qW|dkrtd |dkrtd ||_||_t|jjjjjj } g}| r.|jr td |t!"||jrtdn|jrDtd|j#x|jj$D]}t%|j#|j&d}|j'|j(<|dkrtd|j#dt)|j*dkr|jrtdt+|j*x*t,|j*D]\}}||j'|j(j-|<qW|j.rNd|_/qNW|0t}x|jD]} | 1|q Wt}|j.r@td||| s|jrVtdt}|2t}|j.rtd|||jrtdt}t3t4f}|5|t}|j.rtd|||6|7|8|9|j:r|;t|j'tD]*}|j'|j(j?|j@|j(|jA|jBdq6WnHxF|jjCD]:}|jrtd|j(|j'|j(j?|jD|jA|jBdqpW|jEdk r| rPtd|j:r|x|jjFD]}t!jG||jH|jI|jEdt!jJjKd|jLfd}xJ|jM}||kr:|N||jO|n||kr|jO|}PqWt)|dksjtP|d|_LqWnx|jjQD]}t!jG||jH|jI|jEdt!jJjRd|jLfd}xJ|jM}||kr|N||jO|n||kr|jO|}PqWt)|dkstP|d|_LqWtd |t!jS|d!d"t!jJjRd|jLfd#nRtd$xH|jjQD]<}|jT|jE||j||jL}t)|dkstP|d|_LqbW|jUdk rz| rHtd%|j:rx||jjFD]0}|t!jG||jH|jI|jUd&t!jJjVdd'qWn>xp|jjQD]0}|t!jG||jH|jI|jUd&t!jJjRdd'qWn2td(x(|jjQD]}|jT|jU||j|qZW|jWdk rR| r td)|j:rx||jjFD]0}|t!jG||jH|jI|jWd*t!jJjVdd'qWn>xp|jjQD]0}|t!jG||jH|jI|jWd*t!jJjRdd'qWn2td+x(|jjQD]}|jT|jW||j|q2W| r|jrhtd,t!X||t)|dkstP|jrtd-|jjQdjY|_Y|jZr|jrtd.jj[j\|j]d/}|j^||j_ r8|jrtd0|j`dk rtad1jj[jb|j`|jYtck r$d2ndd3}|j^||jd r|j rPtd4|jedk rdtad5jj[jf|jed/}|j^||jg r|j rtd6jj[jh|jid/}|j^||jj r|jg sjj[jk|jid/}|j^||jl r|j rtd7|j^jj[jm|jnd/d|_o|j r.td8|j r>td9|j. r\td:t|j|j rltd;t} x|jD]} | p| q|Wt}!|j. rtd<|!| |j. rtd=t|jt|_q|j. rtd>|jq|jtddS)?Nr)PersistentSolverzInitializing PHrcs"g|]}t|jjjjr|qSr3)rpyspr phboundextension)r4r)r r3r6rsz1ProgressiveHedging.initialize..cs"g|]}t|jjjjr|qSr3)rrr convexhullboundextension)r4r)r r3r6rscs"g|]}t|jjjjr|qSr3)rrr  wwphextension)r4r)r r3r6rscs"g|]}t|jjjjr|qSr3)rrr phhistoryextension)r4r)r r3r6rsz&Invoking pre-initialization PH pluginsz>A scenario tree must be supplied to the PH initialize() methodz?A solver manager must be supplied to the PH initialize() methodz5Broadcasting requests to initialize PH solver serverszAPH solver server initialization requests successfully transmittedzConstructing solvers of type=)rzUnknown solver type=z$ specified in call to PH constructorz-Initializing sub-problem solver with options=Tz2PH post-instance plugin callback time=%.2f secondszJCreating weight, average, and rho parameter vectors for scenario instancesz2PH parameter vector construction time=%.2f secondsz8Creating deterministic SymbolMaps for scenario instancesz'PH SymbolMap creation time=%.2f seconds)routput_fixed_variable_boundsz?Setting instance for scenario=%s in persistent solver interfacezCTransmitting user aggregate callback invocations to phsolverserversrW)invocation_typereturn_action_handle function_argsr%z4Broadcasting final aggregate data to phsolverserversz pyomo.pysp.phr[)rZreturn_action_handlesrz1Executing user aggregate getter callback functionz=Transmitting user rho callback invocations to phsolverserversrY)rrz+Executing user rho setter callback functionz?Transmitting user bound callback invocations to phsolverserversr[z-Executing user bound setter callback functionzSBroadcasting requests to collect scenario tree instance data from PH solver serversz2Scenario tree instance data successfully collectedzBEnabling convergence based on a fixed number of discrete variables)convergence_thresholdz3Enabling convergence based on outer bound criterionzYA convergence threshold must be specified when using the outer bound convergence criteronF)rZconvergence_threshold_sensez9Enabling convergence based on inner outer bound criterionz_A convergence threshold must be specified when using the inner-outer bound convergence criteronzfEnabling convergence based on non-normalized term diff criterion, as opposed to the normalized variantz.bundle_in_subproblemscSs|dkr dS||kSdS)NTr3)rrr3r3r6scenario_in_subproblems szEProgressiveHedging.queue_subproblems..scenario_in_subproblems)r)teeZ keepfilesrrrrmipgapsolver_optionsZsolver_suffixesr6Zvariable_transmissionFZload_solutionsz$Queuing solve for scenario bundle=%sr)actionZ queue_namerGTz"Solver manager queuing instance=%soptzQueuing solve for scenario=%s)rsuffixes)3rrrlrrrrrrrr r r r r rrrrhrirjrkr,rr.r]rOrrrrrrcrr-rrrZ begin_bulkrprqueuerfrGrqZwarm_start_capablerwrhrrrZend_bulk)rErr6r7rrrrrrscenario_action_handle_mapaction_handle_scenario_mapbundle_action_handle_mapaction_handle_bundle_mapZcommon_solve_kwdsZ some_solverrZnew_action_handlerrZscenario_solverr3r3r6queue_subproblems s                                     z$ProgressiveHedging.queue_subproblemscsRg}g}g}|jr|jr&tdd} x| |kr|j} |j| } y || } Wndtk r| |jkr|j | td| j w.n(t dd|D} t d| j t | fYnX|| | d7} t|jtjjjjjrt| dkr|| q.t}x0t| dD] \}}|jj|}||qW| d}d |kr^|d |j| <tt|d |j| <|d dk r|d |j| <|d dk r|d |j | <t}|j!r||||n|j"| }|jrtd | t| j#dksV| #dj$tj%ksV| #dj$tj&ksV| #dj$tj'ksV| j(j$t)j*ksV| j(j+t,j%kr|jrv| -td| | f|| q.|j.rtd| | j-ddt}| j/}|j0j1| |j2|j3|j2 d| |f|j4| <| #d}t5|d r|j6dk r|j6|j| <|j$|j| <t7| \|j| <|j | <|jj8| }x$|j9D]}|jj|}|:q@Wt}|j!r|||||jr.td| q.WnV|jrtdd} x>| |kr|j} |j| }y || }Wnjtk rD| |jkr|j | td| j wn(t dd|D} t d| j t | fYnX|jj|}||| d7} t|jtjjjjjr|t?t|t@t>fdd|Dt?t|}tdtA|tB||f||fS)Nz%Waiting for bundle sub-problem solvesrz_WARNING: Discarding uncollected solve action handle with id=%d encountered during bundle solvescss|] }|jVqdS)N)id)r4rr3r3r6r7 szFProgressiveHedging.wait_for_and_process_subproblems..z`PH client received an unknown action handle=%d from the dispatcher; known action handles are: %sr%rSrZsolution_statusrrzResults obtained for bundle=%sz>Solve failed for scenario bundle=%s; no solutions generated %szResults for bundle=%s)num)rrZignore_fixed_varsz*Successfully loaded solution for bundle=%sz'Waiting for scenario sub-problem solveszaWARNING: Discarding uncollected solve action handle with id=%d encountered during scenario solvescss|] }|jVqdS)N)r)r4rr3r3r6r7 sz Results obtained for scenario=%sz7Solve failed for scenario=%s; no solutions generated %szResults for scenario=zASuccessfully loaded solution for scenario=%s - waiting on %d morec3s|]}t|dVqdS)g@N)pow)r4x)meanr3r6r7 szVResult load time statistics - Min: %0.2f Avg: %0.2f Max: %0.2f StdDev: %0.2f (seconds))CrlrrprrrrrPrhdiscardrsortedrrrrr r r r r rrJrr/rrrhrrrirjrkrqrwZsolutionr Z infeasibleerrorZ unboundedZsolverrokZtermination_conditionrrrZ_smaprrrcr`ror-rr(Z_scenario_bundle_maprZupdate_solution_from_instancerrrsumrOrminr)rEsubproblem_countrrrrrJrZresult_load_timesZnum_results_so_farrZbundle_resultsrZknown_action_handlesrrZscenario_solutionrZauxilliary_valuesrbundle_instanceZbundle_results_smZ solution0rrrrstd_devr3)rr6 wait_for_and_process_subproblemsk sr                                                                & z3ProgressiveHedging.wait_for_and_process_subproblemscst}t}|j|||d\}}}} t} |jrFtd| ||dkrt|jrft|jj} q|t|jj} nt|} t} | | |||| \}} t}|jrtd|| t|j dkr~d}x t |j D]}t |t rd}qW|rtdnt|j tt|j ttfdd |j Dtt|j }|jr~td t|j t|j |ft|jdkrJd}x$t |jD]}t |t rd}qW|rtdnt|jtt|jttfd d |jDtt|j}|jrJtd t|jt|j|ft}|j||7_|jr|td ||t| rtdtdx"t| D]}tdt|qW|rtdt| | S)N)rr6r7z'Time queueing subproblems=%0.2f secondsz*Time waiting for subproblems=%0.2f secondsrFTzNAt least one sub-problem solve time was undefined - skipping timing statisticsc3s|]}t|dVqdS)g@N)r)r4r)rr3r6r7 sz7ProgressiveHedging.solve_subproblems..z\Sub-problem solve time statistics - Min: %0.2f Avg: %0.2f Max: %0.2f StdDev: %0.2f (seconds)c3s|]}t|dVqdS)g@N)r)r4r)rr3r6r7 szbSub-problem pyomo solve time statistics - Min: %0.2f Avg: %0.2f Max: %0.2f StdDev: %0.2f (seconds)z-Aggregate sub-problem solve time=%.2f secondsz1 ** At least one sub-problem failed to solve! ** z Failed sub-problems:z z.FgTr%z-Variable statistics compute time=%.2f seconds)rrwrrlrr_xbarsrrrrr _minimumsr _maximumsr}rx _averagesrrqr)rErrZcurrent_iterationrZxbarsZscenario_solutionsrrrZ avg_valueZ probabilityr(rKrr3)rr6update_variable_statisticsF s>       $z-ProgressiveHedging.update_variable_statisticsc Cst}t}|j}|j}x|jjddD]}x|jD]}d}|jdkr\|j}n|j}|j }t dd|j D}|_ xL|j D]@} | j} | j|j} | j|j} | j|j} x|jD]}| |}|dk rd}|r|j}|jsX|tkr*| |||| |||||7<n,| |||| |||||8<nR||dksjt|dksxt|tkst||| |||||| |<||| j| ||j7<qWqWq@Wq0Wt}|j||7_|jr td||dS)NrTcss|]}|dfVqdS)rNr3)r4Zvar_idr3r3r6r7 sz4ProgressiveHedging.update_weights..g?zWeight update time=%.2f seconds)rrwrfrlrrrrr_blendr8rZ_wbarsrr_wrrrrrrxr rrrrqr)rEr over_relaxingrrrtree_node_xbars blend_valuesZtree_node_wbarsrr weight_values rho_valuesr(rvarvalnu_valuerr3r3r6update_weightsz sZ      (z!ProgressiveHedging.update_weightsc Cst}|j}|j}xN|jddD]:}|j|j}|j|j}|j|j}d} |jdkrf|j } n|j } |j } x|j D]} || } | dk rzd} |r|j } |jdkr |tkr|| | | || | | | | 7<n,|| | | || | | | | 8<qz| | dkst| dks,t|tks:t| | || | | | | || <qzWq&Wt}|j||7_dS)NrTg?F)rrwrfZ _node_listrrrrrrrrrrrxr rr)rErrrrrrrr(rrrrrrr3r3r6update_weights_for_scenario sH       z.ProgressiveHedging.update_weights_for_scenariocCs|jrtdtd|j||||t|jt j j j j jrr|jdksf|jrt|nH|jdkr||jrt|j|jt|j|jnt|j|j|j|j d}|js|jrtd|j|jr|||S)Nz0------------------------------------------------zStarting PH iteration %s solvesr)r6zESuccessfully completed PH iteration %s solves - solution statistics: )rprrrrrrrrr r r r r rrtrlrr+Ztransmit_tree_node_statisticsrrrdrr=r>rIrr!)rErJr3r3r6iteration_k_solves.s6         z%ProgressiveHedging.iteration_k_solvesc Csb|jdks|jt|jjkr6td|jt|jjf|jrJtd|jg}d}i}x|jjD]}d||j<q`Wi}|| | | | |js|j s|jrtd|jddddd|j|jddg}x|jD]}||}qWt|dksttd|t|}x,|jD]"}x|D]}|||q*Wq Wi} i} td td||j||j d \} } } }| | | | td xtd |d | i| i\}}tdt|d kst|j|d}|j}||d 7<|d 7}t|t||jkr,d}|jd 7_nd}|jrLtd|||f|jrftd|ddf||t||kr|jrtd|x,|D]$}|j|}|jr| |qWx|jD]}|!|||qW|rx|jD]}|"|qW|#\|_$|_%|jrDtd|jddddd|j&|jdd|jsT|j'rxtd|jddddd|j|jd|(\}}}td|||fx&|j)D]}||j||j|j*qW|+|j,-}t.std|||j/|j<|0rt|j1dksH|j2t3kr,|t4|j1ksH|j2t5krj|t6|j1krjt.sVtd|7|j8|j|_9||j1|j<d}x,|jD]"}t:|dr|;|sd}qW|rP|t|jj|j|t?|j|j=|j@|jA}|jBjC|D|d|jBjE|<x|jD]}|||qVW|j|g|j d \} } } }| | | | t|}|jrtd||d |f|jrx&tF|D]\}}td||fqWtd|td|j|jrtd|jddddd|j|jdqW|jdkrTtdg}qWdS) NrzhAsync buffer length parameter=%d is invalid - must be less than or equal to the number of subproblems=%dzEStarting PH iteration k+ solves - running async with buffer length=%szAsync starting rhos:FT)r8r9report_stage_costszSUBPROBLEMS TO QUEUE=zABOUT TO QUEUE SUBPROLEMS)rr6z(Entering PH asynchronous processing loopzCALLING WAIT FOR AND PROCESSr%zDONE WITH CALLzJSolve for subproblem=%s completed - new solve count for this subproblem=%sz%20s %18.4f %14.4fgzProcessing async bufferz(Async Reportable Iteration Current rhos:zAAsync Reportable Iteration Current variable averages and weights:)r8r9z)First stage cost avg=%12.4f Max-Min=%8.2fzExpected Cost=%14.4fz*Caching results for new incumbent solutionph_convergence_checkz!Queued solve k=%s for scenario=%szScenario=%s was solved %s timesz'Cumulative number of scenario solves=%sz PH Iteration Count (computed)=%sz*Variable values following scenario solves:z$Emptying the asynch scenario buffer.)Gr{rJrlrrrprrGrrrrrrrr?r@rArZ!asynchronous_subproblems_to_queuerZasynchronous_pre_scenario_queuerr>updaterZget_subproblemrNrrrr|rZpost_asynchronous_var_w_updatepost_iteration_k_solvesr,rprqZ_report_only_statisticrr5rrdprintConvergerStatusr.computeExpectedNodeCostr)rirrjrfr rr rrrlrkr-rryrtrrrurvrar>rrCr/)rEZsubproblem_bufferZtotal_subproblem_solve_countZsubproblem_solve_countsrZaction_handle_subproblem_mapZsubproblems_to_queuerZnumber_subproblems_queuedZ%integrated_action_handle_scenario_mapZ#integrated_action_handle_bundle_maprrrrZsolved_subproblemsrJZsolved_subproblemZsolved_subproblem_nameZnew_reportable_iterationZsubproblem_namefirst_stage_minfirst_stage_avgfirst_stage_maxr expected_costplugin_convergencerZsnameZscenario_countr3r3r6async_iteration_k_plus_solvesusZ                                          z0ProgressiveHedging.async_iteration_k_plus_solvesc CsP t|_d|_d|_d|_d|_t}tt d|j dkrLt dd}|j r^t |}t d|js|jdkrt dt|j|}t|r|jrt dt||S|\|_|_|jrt d t|jt d t|j|j|jdkr t d ||js"|jrFt d |jddd dd|j|jdx|jD]}||qNW|\|_|_ t dt|jdt|jdt dt|j dt|jd|!\}}}t d|||fx&|j"D]}|#|j||j$|j%qW|&|j$'(} t)s,t d| | |j*|j<|+rpt)sPt d|,|j-|j|_.| |j/|j<x|jD]}|0|qxW|j1r|2|rt|j3|j4krt5t|_6t dt|jn|js|jdk st7ddl8m9} m:} t d|j| |j\} } }|d}|j;dk rh|j;|krP|j;}ntd|j?dD]}|dkr|@|j s|A|jBdkr|C|jd|_t dt dt|jx|jD]}|D|q Wt)sp|jEsJ|js>|jFrp|jdkrpt d|jddddd |j|jdd|js|jGrt d|jd d ddd|j|jd|jBdkr|Cy |HWntIk rt dt dt d ttJKdt d|jd8_PYnVtLk rnt dt d!t d ttJKdt d|jd8_PYnX|j dkr|n|M}|N|jdd"|j1r|2x|jD]}|O|qW|js|jrt d |jddd dd|j|jd|jPs|\|_|_|\|_|_ t dt|jdt|jdt dt|j dt|jd|jQdks|jRdkrt d#|jR|jQf|!\}}}t d|||fx&|j"D]}|#|j||j$|j%qW|&|j$'(} t)st d| | |j*|j<|+rt|j/dksv|jStTkrV| tU|j/Vksv|jStWkr| tX|j/Vkrt)st d|,|j-|j|_.| |j/|j<x|jD]}|Y|qWt dt|j|j s$|+r0d }x,|jD]"}tZ|d$r|[|sd}qW|r0Pn |d kr0P||j?krNt d%t|j?|r|t|j3|j4kr|t5t|_3t\r|j]dkrt t^_`qWn|j d krtad&|@|j s|A|jBdkr|Cx|jD]}|b|qW|cx|jD]}|d| qW| r4tet dt d't|jdt|jdt d(t|j dt|jd|j.dk r|j.|jk rt) st dt d)|j/|j.|f|j-tg|jhtijjjkjljmjn r top|tojqjrtojqjsBtojqjtBtojqjuBx|jD]}|v| qW|\|_|_ t|_wt d*t) rt dt d+x.|j"D]$}t d,|jx|t d qhWnt dt d-d.}|d/d07}|d1d27}|d3d47}|d3d57}t |d}x0t>|jdD]}d6}|d7|7}|jyjz|}|jyj{d8k r4|d9|7}n |d:|7}|j*|} |d;| 7}||j/k r||d<}|j/|} d}|dk r| }d }n2|jStTk r| |k r| }d }n| |k r| }d}|d;|7}| r |d=7}n.||d>}|dk r|d;|7}n |d3d?7}t | qWt dt d@t|jdt|jdt dAt|j dt|jd|j} st dt dB|~\}|_t dn6t dt dCt dD|jddd d d|j|jddE|j r|j rt dF|jw|jx0|j$jD]$}| s| rt dGP qWtg|jhtijjjkjljmjn sL||dS)HNgrz Starting PHFz4PH is not initialized - cannot invoke solve() methodrzInitiating PH iteration=z/Iteration zero reports trouble with scenarios: z6Total number of non-stale discrete instance variables=z8Total number of non-stale continuous instance variables=z***ERROR: The total number of non-anticipative discrete and continuous variables equals 0! Did you set the StageVariables set(s) in ScenarioStructure.datz*Variable values following scenario solves:T)r8r9z#Number of discrete variables fixed=z (total=)z%Number of continuous variables fixed=z)First stage cost avg=%12.4f Max-Min=%8.2fzExpected Cost=%14.4fz*Caching results for new incumbent solutionz Cumulative run-time=%.2f seconds)load_ph_warmstart load_historyz Loading PH warmstart from file: rz?'%s' is not a valid index in warmstart file: %s Choices are: %sr%zRhos prior to scenario solves:)r8r9rz7Variable averages and weights prior to scenario solves:zA ** Caught SystemExit exception. Attempting to gracefully exit PHz Signal: zG ** Caught ApplicationError exception. Attempting to gracefully exit PH)Z compute_xbarsz!Outer bound=%20s Inner bound=%20srz-Halting PH - reached maximal iteration count=z8The 'async' option has not been implemented for dual ph.z=Number of discrete variables fixed before final plugin calls=z?Number of continuous variables fixed before final plugin calls=zURestoring scenario tree solution to best incumbent solution with expected cost=%14.4fz PH completezConvergence history:zConverger=%20szAlgorithm History: z z%10sZ Iterationz%14sz Metric Valuez%17sz Expected CostzBest Convergedz{0} z%10dg-C6?z%14.4fz%14.3ez%17.4f*z (new incumbent) -z)Final number of discrete variables fixed=z+Final number of continuous variables fixed=z0Computing objective inner bound at xhat solutionz***WARNING: Computation and evaluation of xhat solution is disabled - the reported final solution may not be valid / non-anticipativez+ Final scenario solution variable values: )r8r9r:zOverall run-time=%.2f secondsa***WARNING***: PH exiting with fixed or freed variables in the scenario tree queue whose statuses have not been pushed to the scenario instances. This is because these requests were placed in the queue after the most recent solve. If these statuses are pushed to the scenario instances, a new solution should be obtained in order to reflect accurate solution data within the scenario tree. This warning can be safely ignored in most cases.)rrrrrrgc isenableddisablerr|rrrrursrrrJrpr+rnrorr~r?r@rArZpost_iteration_0_solvesr,rprqr5rrrlrdrr.rr)rirrrlrkrjZpost_iteration_0r|rrrZcollectZ_time_last_garbage_collectrZ%pyomo.pysp.plugins.phhistoryextensionrrrtrrzr0ryrrrtrZpre_iteration_k_solvesrrrr SystemExitsysexc_inforZadd_cutrrr&rzr{rfr rrr rZpost_iteration_kr-rguppy_availablerr1Zhpyheapr~Zpre_asynchronous_solvesr Zpost_asynchronous_solvesenablerrrr r r r r rr+rCrDrErFrGrZpost_ph_executionrrZ _convergerZ_metric_history_convergence_thresholdformatrrLrmrqrZhas_fixed_in_queueZhas_freed_in_queuerr)rEZ re_enable_gcZ dual_modelZ iter0retvalrrr r rr rrZscenario_tree_dicthistoryZ iterations_indexrZdual_rcr Z label_stringZbest_incumbent_costZ row_stringZmetricZ updated_bestrKrr3r3r6rs     ""                         ""                       ""                           ""   zProgressiveHedging.solvecCsFx4t|jD]&\}}x|j|D]}||q Wq Wi|_i|_dS)N)r/rwrxr)rErrrr3r3r6rs z*ProgressiveHedging._clear_bundle_instancescCs^tdtd|j|jtd|j|jtd|jtd|jtd|jdS)NzPH run-time statistics:z"Initialization time= %.2f secondsz"Overall solve time= %.2f secondsz"Scenario solve time= %.2f secondsz"Average update time= %.2f secondsz"Weight update time= %.2f seconds)rrrrrrrr)rEr3r3r6print_time_statss z#ProgressiveHedging.print_time_statscCs4|jdkr0|j}t|tdkr0t|tdkr0dSdS)NFT)rdomainrr r)rErvariableZ variable_typer3r3r6 should_prints  zProgressiveHedging.should_printcCsfx`|jD]V}d}|jdkr$|d7}n|d7}td|j|dkrDdn ||||fqWdS)Nrg-C6?z%14.4fz%14.3ez3Converger=%20s value is %12s - threshold reached=%srU)rrrrZ lastMetricr)rErZmetric_format_stringr3r3r6rs   z'ProgressiveHedging.printConvergerStatusTc  s jdkrtdd# f dd d$fdd } x@ jjddD]* td jd} x t jD]} | | | 7} qxWxlt jd d d D]V} t | jdkrtd | jxt| jD]} | | }qW|dkrtd| |7} qW| dkrtd|sqT j dj d} j dj d}td|t |x2 jD]&} t jd| jdkrt jdxB| jD]8}t jt|jd|| jdkrt jdqWd}d}d}d}d}d}dkrt jdn t jdxt| jD]h}|j j}|dkr,d}dkr||dk rld|}t|dkr^dd}t j|nt jdd|dk r|d7}||j|7}||j7}|rd}|}|}n||kr|}||kr|}|| jdkr |dkrhr6t jd||dkr$t jd||t jd|n2t jd |||dkrht jd!||t jd"q WqTWqTWdS)%NFz5PH is not initialized - cannot invoke pprint() methodrcssPtjdx>jD]4}tjt|jd|jdkrtjdqWj|}d}x,|D]$}|dkrdtt|}||krd|}qdWxt |D] }d}j ||f } t fddjD} r| s| r| sj } j} j} t|  jks,t|  jks, jdkrt| |  jkrLdkrLq|d 7}|d 7}|d krztjd |d |d kr|dk rd t|d }tj|t|t jd krtjd tjdjrdkrtjdjd}xjD]}|j}|jj}|dkr6d}sfd|}t|dkrZdd}tj|||krrtjd| tjd| tjd| rtdn*stjd| | tjd| tjd qW r2tjdx,jD]"}tjd|jjq Wrrtjdx,jD]"}tjd|jjqLWrtjd|  ssrtjd qW|S)Nz (Scenarios: z rz) rc3s|]}| VqdS)N)Zis_variable_stale)r4r)rrr3r6r7eszEProgressiveHedging.pprint.._print_node_var..Tr%z Variable:  z Index: %rPz Tree Node: %sFz Values: z%12.4fz Min: %12.4fz Avg: %12.4fz Max: %12.4fzFoutput_only_statistics and output_no_statistics are both set in pprintz Max-Min: %12.4fz Weights: z Rhos: z Average: %12.4f)rstdoutrrrrGZ_variable_indicesrJr rZ_name_index_to_idZis_variable_fixedrrrrrrvrrrrrOrrrr)rrnum_outputs_this_variablerZvariable_indicesZmax_index_string_lengthrZthis_index_lengthZnum_outputs_this_indexr)Z is_not_staler1Z average_valuer0 format_stringZ last_scenarioZscenario_probabilityr4valstr) output_averages output_fixedr:r9r8 output_rhos output_valuesoutput_weightsrEr)rrr6_print_node_var7s                                     z2ProgressiveHedging.pprint.._print_node_varcs>|dk|dkAstd}x |jD]}||||d7}q W|S)Nr)r')rr)rrnoder'r)r/r3r6_print_stage_vars z3ProgressiveHedging.pprint.._print_stage_varrz Stage: %scSs|jS)N)rG)rr3r3r6z+ProgressiveHedging.pprint..)rHz Node: %sz$ No non-converged variables in nodez% No non-converged variables in stager%z Cost Variable: z Tree Node: %sz (Scenarios: rz) gTz Values: z z%12.4fz%12sz Not Reportedz Min: %12.4fz Avg: %12.4fz Max: %12.4fz Max-Min: %12.4fz Avg: %12.4fr%)r)NN)r|rrlrrrGrZ_variable_templatesrrJZnodesZ_cost_variabler rr&rrrr/rOr)rEr*r.r-r+r,r8r9rr:r1Znum_outputs_this_stagerrZnum_outputs_this_nodeZcost_variable_nameZcost_variable_indexrr0r1Zprob_sum_valuesZsum_probr2r3r4r)r3) r/r*r+r:r9r8r,r-r.rErr6r?(s  /                          zProgressiveHedging.pprint)rY)F)N)NNNN)NFF)NFF)FFTF),rUrVrWrrrrrrrrrrrrr!rrrrrrr+r,rr5rLrFrrrr=rrrrrr rrr!r$rr?r3r3r3r6rs H4    p*  ^D q&4lHG-L r)Prloggingrrrr"mathrrZpyomo.common.errorsrZpyomo.common.dependenciesrZ pyomo.corerrrr r r r r Z pyomo.optrrrrrrZpyomo.pysp.convergencer Zpyomo.pysp.phboundbaserrZpyomo.pysp.dualphmodelrZ pyomo.pysp.efrZpyomo.pysp.generatorsrZpyomo.pysp.phobjectiverrrZpyomo.pysp.phutilsrrrrr r!r"r#r$r%r&r'r(r)Zpyomo.pysp.util.miscr*Z pyomo.pyspr+Zpyomo.opt.parallel.localr,sixr-r.r/Z six.movesr0r1r getLoggerloggerobjectr2r[r\rr3r3r3r6 sH  (    @      $&