B @`J@spdZddlmZddlmZmZmZmZmZm Z ddl m Z ddl m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZddl m!Z!dd l"m#Z#dd l$m%Z%m&Z&m'Z'dd l(m)Z)m*Z*m+Z+dd l,m-Z-dd l.m/Z/ddl0Z0e01dZ2iZ3ddZ4ddZ5ddZ6ddZ7ddZ8ddZ9ddZ:ddZ;d d!ZdS)'z| Cutting plane-based GDP reformulation. Implements a general cutting plane-based reformulation for linear and convex GDPs. )division) ConfigBlock ConfigValue PositiveFloatNonNegativeFloat PositiveIntIn)unique_component_name)AnyBlock Constraint ObjectiveParamVarSortComponentsTransformationTransformationFactoryvalueNonNegativeIntegersRealsNonNegativeRealsSuffix ComponentMap) differentiate) ComponentSet) SolverFactory)generate_standard_repn)Disjunct Disjunction GDP_Error)verify_successful_solveNORMAL#clone_without_expression_components)*Fourier_Motzkin_Elimination_Transformation) iteritemsNzpyomo.gdp.cuttingplanecCs|S)N)mr%r%B/tmp/pip-unpacked-wheel-bi3529v6/pyomo/gdp/plugins/cuttingplane.pydo_not_tighten.sr(cCs*g}x |D]}|t|j|dq W|S)zReturns a list of expressions which are constrain.expr translated into the bigm space, for each constraint in constraints. ) substitute)appendr"expr) constraintshull_to_bigm_mapcutsconsr%r%r'_get_constraint_exprs1s  r0cCsnt|j}ddg}|jdk r>|t|j|kr>|dd8<|jdk rjt|j||krj|dd7<|S)z Returns a list [a,b] where a is -1 if the lower bound is tight or slightly violated, b is 1 if the upper bound is tight of slightly violated, and [a,b]=[-1,1] if we have an exactly satisfied (or slightly violated) equality. rN)rbodylowerupper)model constraintTOLvalZansr%r%r'_constraint_tight;s   r9csHd}x"t|D]\}}|||8}qW|tfddt|D kS)zVReturns constraint linearly approximating constraint normal to normal_vec at pointrc3s |]\}}||jVqdS)N)r).0idxv) normal_vecr%r' Vsz1_get_linear_approximation_expr..)zipsum enumerate)r=Zpointr2Zcoefr<r%)r=r'_get_linear_approximation_exprPs rBcCsn|}g}|_xV|jtdttjdD]>}t|j}x.|j |j |j D]}||krJ| |PqJWq(WdS)NT)active descend_intosort) r5constraints_for_FMEcomponent_data_objectsr r r deterministicrr2Z linear_varsZquadratic_varsZnonlinear_varsr*)transBlock_rHulldisaggregated_varsinstance_rHullr,r6Zrepnr<r%r%r'*_precompute_potentially_useful_constraintsYs   rLc s|} |ddkr t||t} tt} | _| d}x|jD]x}t | || }xf|D]^r^d}|j }t ||d}fdd|D}| dkr|j | t| <q^t||| t| <q^WqHW|sdS| td t| jt|ftd j| ||| d d | j}d dt|D}t||d}d}g}xtD]\}}td|t|rntdq@||t|jdkstt|jdt|jd}||kr@||kr@|}|}td|q@Wfdd|D|dk r|gSdS)aGReturns a cut which removes x* from the relaxed bigm feasible region. Finds all the constraints which are tight at xhat (assumed to be the solution currently in instance_rHull), and calculates a composite normal vector by summing the vectors normal to each of these constraints. Then Fourier-Motzkin elimination is used to project the disaggregated variables out of the polyhedron formed by the composite normal and the collection of tight constraints. This results in multiple cuts, of which we select one that cuts of x* by the greatest margin, as long as that margin is more than cut_threshold. If no cut satisfies the margin specified by cut_threshold, we return None. Parameters ----------- transBlock_rHull: transformation blcok on relaxed hull instance var_info: List of tuples (rBigM_var, rHull_var, xstar_param) hull_to_bigm_map: For expression substition, maps id(hull_var) to coresponding bigm var rBigM_linear_constraints: list of linear constraints in relaxed bigM rHull_vars: list of all variables in relaxed hull disaggregated_vars: ComponentSet of disaggregated variables in hull reformulation norm: norm used in the separation problem cut_threshold: Amount x* needs to be infeasible in generated cut in order to consider the cut for addition to the bigM model. zero_tolerance: Tolerance at which a float will be treated as 0 during Fourier-Motzkin elimination integer_arithmetic: boolean, whether or not to require Fourier-Motzkin Elimination does integer arithmetic. Only possible when all data is integer. constraint_tolerance: Tolerance at which we will consider a constraint tight. rFNFT)Zwrt_listcsg|]}t|qSr%)r)r:_) multiplierr%r' sz#create_cuts_fme..r1zFCalling FME transformation on %s constraints to eliminate %s variablesz#contrib.fourier_motzkin_eliminationfme_constraints)Zvars_to_eliminatezero_tolerancedo_integer_arithmeticZprojected_constraints_namecSsg|] \}}|qSr%r%)r:ir/r%r%r'rOsrzFME: Post-processing cut %szFME: Doesn't cut off x*z&FME: New best cut: Cuts off x* by %s.csg|] }|qSr%r%)r:rS)r.r%r'rOs)r5 componentrLr r rr,Z constructrFr9r2rpolynomial_degreer+lenrBloggerinforapply_torPr$r0rArr*argsAssertionError)rIvar_infor-rBigM_linear_constraints rHull_varsrJnorm cut_thresholdrQinteger_arithmeticconstraint_tolerancerKZtight_constraintsZconslistZsomething_interestingr6Z multipliersfZ firstDerivsr=Z fme_resultsZprojected_constraintsbestZbest_cutZ cuts_to_keeprScutcut_offr%)r.rNr'create_cuts_fmeksn%           rhc Csd} |dkred?d@dedAedBe dCdDdfdEdFZd[dGdHZdIdJZdKdLZdMdNZdOdPZdQdRZdSdTZdUdVZdWdXZdYdZZZS)\CuttingPlane_TransformationaRelax convex disjunctive model by forming the bigm relaxation and then iteratively adding cuts from the hull relaxation (or the hull relaxation after some basic steps) in order to strengthen the formulation. Note that gdp.cuttingplane is not a structural transformation: If variables on the model are fixed, they will be treated as data, and unfixing them after transformation will very likely result in an invalid model. This transformation accepts the following keyword arguments: Parameters ---------- solver : Solver name (as string) to use to solve relaxed BigM and separation problems solver_options : dictionary of options to pass to the solver stream_solver : Whether or not to display solver output verbose : Enable verbose output from cuttingplanes algorithm cuts_name : Optional name for the IndexedConstraint containing the projected cuts (must be a unique name with respect to the instance) minimum_improvement_threshold : Stopping criterion based on improvement in Big-M relaxation. This is the minimum difference in relaxed BigM objective values between consecutive iterations separation_objective_threshold : Stopping criterion based on separation objective. If separation objective is not at least this large, cut generation will terminate. cut_filtering_threshold : Stopping criterion based on effectiveness of the generated cut: This is the amount by which a cut must be violated at the relaxed bigM solution in order to be added to the bigM model max_number_of_cuts : The maximum number of cuts to add to the big-M model norm : norm to use in the objective of the separation problem tighten_relaxation : callback to modify the GDP model before the hull relaxation is taken (e.g. could be used to perform basic steps) create_cuts : callback to create cuts using the solved relaxed bigM and hull problems post_process_cut : callback to perform post-processing on created cuts back_off_problem_tolerance : tolerance to use while post-processing zero_tolerance : Tolerance at which a float will be considered 0 when using Fourier-Motzkin elimination to create cuts. do_integer_arithmetic : Whether or not to require Fourier-Motzkin elimination to do integer arithmetic. Only possible when all data is integer. tight_constraint_tolerance : Tolerance at which a constraint is considered tight for the Fourier-Motzkin cut generation procedure By default, the callbacks will be set such that the algorithm performed is as presented in [1], but with an additional post-processing procedure to reduce numerical error, which calculates the maximum violation of the cut subject to the relaxed hull constraints, and then pads the constraint by this violation plus an additional user-specified tolerance. In addition, the create_cuts_fme function provides an (exponential time) method of generating cuts which reduces numerical error (and can eliminate it if all data is integer). It collects the hull constraints which are tight at the solution of the separation problem. It creates a cut in the extended space perpendicular to a composite normal vector created by summing the directions normal to these constraints. It then performs fourier-motzkin elimination on the collection of constraints and the cut to project out the disaggregated variables. The resulting constraint which is most violated by the relaxed bigM solution is then returned. References ---------- [1] Sawaya, N. W., Grossmann, I. E. (2005). A cutting plane method for solving linear generalized disjunctive programming problems. Computers and Chemical Engineering, 29, 1891-1913 zgdp.cuttingplanesolverZipoptzISolver to use for relaxed BigM problem and the separation problema This specifies the solver which will be used to solve LP relaxation of the BigM problem and the separation problem. Note that this solver must be able to handle a quadratic objective because of the separation problem. )defaultdomain descriptionrminimum_improvement_thresholdg{Gz?ziThreshold value for difference in relaxed bigM problem objectives used to decide when to stop adding cutsz If the difference between the objectives in two consecutive iterations is less than this value, the algorithm terminates without adding the cut generated in the last iteration. separation_objective_thresholdzThreshold value used to decide when to stop adding cuts: If separation problem objective is not at least this quantity, cut generation will terminate.z If the separation problem objective (distance between relaxed bigM solution and its projection onto the relaxed hull feasible region) does not exceed this threshold, the algorithm will terminate. max_number_of_cutsdzBThe maximum number of cuts to add before the algorithm terminates.z If the algorithm does not terminate due to another criterion first, cut generation will stop after adding this many cuts. r`rTriz9Norm to use in the separation problem: 2, or float('inf')a Norm used to calculate distance in the objective of the separation problem which finds the nearest point on the hull relaxation region to the current solution of the relaxed bigm problem. Supported norms are the Euclidean norm (specify 2) and the infinity norm (specify float('inf')). Note that the first makes the separation problem objective quadratic and the latter makes it linear. verboseFzFlag to enable verbose outputz If True, prints subproblem solutions, as well as potential and added cuts during algorithm. If False, only the relaxed BigM objective and minimal information about cuts is logged. rzZIf true, sets tee=True for every solve performed over "the course of the algorithm)rrrsolver_optionsTzDictionary of solver optionsz Dictionary of solver options that will be set for the solver for both the relaxed BigM and separation problem solves. )Zimplicitrrtighten_relaxationzeCallback which takes the GDP formulation and returns a GDP formulation with a tighter hull relaxationax Function which accepts the GDP formulation of the problem and returns a GDP formulation which the transformation will then take the hull reformulation of. Most typically, this callback would be used to apply basic steps before taking the hull reformulation, but anything which tightens the GDP can be performed here. )rrr create_cutszCallback which generates a list of cuts, given the solved relaxed bigM and relaxed hull solutions. If no cuts can be generated, returns Nonea Callback to generate cuts to be added to the bigM problem based on solutions to the relaxed bigM problem and the separation problem. Arguments --------- transBlock_rBigm: transformation block on relaxed bigM instance transBlock_rHull: transformation blcok on relaxed hull instance var_info: List of tuples (rBigM_var, rHull_var, xstar_param) hull_to_bigm_map: For expression substition, maps id(hull_var) to coresponding bigm var rBigM_linear_constraints: list of linear constraints in relaxed bigM rHull_vars: list of all variables in relaxed hull disaggregated_vars: ComponentSet of disaggregated variables in hull reformulation cut_threshold: Amount x* needs to be infeasible in generated cut in order to consider the cut for addition to the bigM model. zero_tolerance: Tolerance at which a float will be treated as 0 Returns ------- list of cuts to be added to bigM problem (and relaxed bigM problem), represented as expressions using variables from the bigM model post_process_cutzCallback which takes a generated cut and post processes it, presumably to back it off in the case of numerical error. Set to None if not post-processing is desired.a Callback to adjust a cut returned from create_cuts before adding it to the model, presumably to make it more conservative in case of numerical error. Arguments --------- cut: the cut to be made more conservative, a Constraint transBlock_rHull: the relaxed hull model's transformation Block. bigm_to_hull_map: Dictionary mapping ids of bigM variables to the corresponding variables on the relaxed hull instance. opt: SolverFactory object for subproblem solves in this procedure stream_solver: Whether or not to set tee=True while solving. TOL: A tolerance Returns ------- None, modifies the cut in place back_off_problem_toleranceg:0yE>z3Tolerance to pass to the post_process_cut callback.z Tolerance passed to the post_process_cut callback. Depending on the callback, different values could make sense, but something on the order of the solver's optimality or constraint tolerances is appropriate. cut_filtering_thresholdgMbP?zuTolerance used to decide if a cut removes x* from the relaxed BigM problem by enough to be added to the bigM problem.z Absolute tolerance used to decide whether to keep a cut. We require that, when evaluated at x* (the relaxed BigM optimal solution), the cut be infeasible by at least this tolerance. rQg& .>zZTolerance at which floats are assumed to be 0 while performing Fourier-Motzkin eliminationz Only relevant when create_cuts=create_cuts_fme, this sets the zero_tolerance option for the Fourier-Motzkin elimination transformation. rRzOnly relevant if using Fourier-Motzkin Elimination (FME) and if all problem data is integer, requires FME transformation to perform integer arithmetic to eliminate numerical error.a Only relevant when create_cuts=create_cuts_fme and if all problem data is integer, this sets the do_integer_arithmetic flag to true for the FME transformation, meaning that the projection to the big-M space can be done with exact precision. cuts_nameNzzOptional name for the IndexedConstraint containing the projected cuts. Must be a unique name with respect to the instance.a Optional name for the IndexedConstraint containing the projected constraints. If not specified, the cuts will be stored on a private block created by the transformation, so if you want access to them after the transformation, use this argument. Must be a string which is a unique component name with respect to the Block on which the transformation is called. tight_constraint_tolerancegư>zeTolerance at which a constraint is considered tight for the Fourier-Motzkin cut generation procedure.a_ For a constraint a^Tx <= b, the Fourier-Motzkin cut generation procedure will consider the constraint tight (and add it to the set of constraints being projected) when a^Tx - b is less than this tolerance. It is recommended to set this tolerance to the constraint tolerance of the solver being used. cstt|dS)N)superr__init__)self) __class__r%r'rsz$CuttingPlane_Transformation.__init__c Kstj}t}ztrt||di|_|j||jj r^|t j kr^t t j d|_ n|t j krpd|_ nd|_ | |||jj\}}}} } ||||| | tdj|ddWd|`|` tt |XdS)NoptionsTFzcore.relax_integer_vars)Zundo)rXlevelgetEffectiveLevel NAME_BUFFERr\CONFIGpop_config set_valuerloggingINFOsetLevel_setup_subproblemsr_generate_cuttingplanesrrZclear) rinstancebigMkwdsZoriginal_log_levelZ log_levelinstance_rBigMcuts_objrKr]transBlockNamer%r%r' _apply_tos.     z%CuttingPlane_Transformation._apply_tocsF||\}}tdd|jtttftjdD|_|j j }|dkrTt t }|_ n4||dk rntd|||t t ||}td}td} td} ||} | | } | j| dd |j||d | j|dd | |ttt|jdd td _t} _| t t| _ fd dt|jD}||| ||fS)Ncss|]}|s|VqdS)N)Zis_fixed)r:r<r%r%r'r>szACuttingPlane_Transformation._setup_subproblems..)rDrEzocuts_name was specified as '%s', but this is already a component on the instance! Please specify a unique name.zgdp.bigmzgdp.hullzcore.relax_integer_varsT)Ztransform_deactivated_blocks)rr)ZmutablerZwithincs&g|]\}}|j|j|fqSr%)all_varsxstar)r:rSr<)rIr%r'rO!szBCuttingPlane_Transformation._setup_subproblems..)_add_transformation_blocklistrGrr rrrHrrrr rr.rUr add_componentrZ create_usingrZrrangerWrrextendedSpaceCutsrur rA)rrrZtighten_relaxation_callbackr transBlocknmrZbigMRelaxationZhullRelaxationZrelaxIntegralityZtighter_instancerKrr]r%)rIr'rs>       z.CuttingPlane_Transformation._setup_subproblemscstfddttDS)Nc3s*|]"}t|d|dfVqdS)r1rN)id)r:rS)r]r%r'r>/szTCuttingPlane_Transformation._create_hull_to_bigm_substitution_map..)dictrrW)rr]r%)r]r'%_create_hull_to_bigm_substitution_map.s zACuttingPlane_Transformation._create_hull_to_bigm_substitution_mapcstfddttDS)Nc3s*|]"}t|d|dfVqdS)rr1N)r)r:rS)r]r%r'r>4szRCuttingPlane_Transformation._create_bigm_to_hull_substition_map..)rrrW)rr]r%)r]r'#_create_bigm_to_hull_substition_map3s z?CuttingPlane_Transformation._create_bigm_to_hull_substition_mapcCspt}td}x\|jtttfdD]F}x@|jD]6}|jdk r.|}x|jt D]}| |qRWq.Wq"W|S)Nzgdp.hull)rD) rrrGrrr Z disjunctsZtransformation_blockdisaggregatedVarsradd)rZhullrZ hull_xformZ disjunctionZdisjunctrr<r%r%r'_get_disaggregated_vars7s   z3CuttingPlane_Transformation._get_disaggregated_varscCszt|jtddd}|dkr$tdtd}g}x@|jtttjddD](}|j }| dkr^qF| | |qFW||fS)NT)rCzTCannot apply cutting planes transformation without an active objective in the model!z#contrib.fourier_motzkin_elimination)rDrErCr1) nextrGr rrr r rrHr2rVextendZ_process_constraint)rr rBigM_objZfmer^r/r2r%r%r'_get_rBigM_obj_and_constraintsFs    z:CuttingPlane_Transformation._get_rBigM_obj_and_constraintsc Cst|jj}|jj}t|jj|_d}d} |jj} d} ||} | |\} }dd|j t t t jdD}||}||}||}t}x|r|j||dd}t|tk rtd|fdS|j|t| }td|f| d dkr||| td xR|D]J\}}}|js:|j|_|j|_|jrtd |jdt d |jfqW| dkrvd}n2| |}t!|d krt!|| knt!|| | k}|j||dd}t|tk rtd|fdS|j|tdt| j"|jr tdxF|D]>\}}}t|||<|jrtd |jdt d |jfqWt| j"|jj#kr|td|jj#P|j$| ||||||jj%|jj&|jj'|jj(|jj) } | dkrtdP|std| Px^| D]V}t*|}td|f|+|||jj,dk r|j,||| ||||jj-qW|d |jj.krZtdP|} x|D]\}}}|||_qdWqWdS)NTcSsg|]}|qSr%r%)r:rSr%r%r'rOszGCuttingPlane_Transformation._generate_cuttingplanes..)rDrEF)rrrszVRelaxed BigM subproblem did not solve normally. Stopping cutting plane generation. %szrBigM objective = %srtzx* is:z %s = %s)fully_qualified name_bufferr1zYHull separation subproblem did not solve normally. Stopping cutting plane generation. %sz&separation problem objective value: %sz xhat is: zLSeparation problem objective below threshold of %s: Stopping cut generation.z6Did not generate a valid cut, stopping cut generation.zDifference in relaxed BigM problem objective values from past two iterations is below threshold of %s: Stopping cut generation.zAdding cut %s to BigM model.zReached maximum number of cuts.)/rrrrrrrrrUrrGrr rrHrrrrrvr r!rXrmryrzr_add_separation_objectiverYstalergetnamerr|rtrrr`rrQrRrrWrrrr)rrrrKr]rr~rZ improvingZprev_objepsilonr.rIrr^r_rJr-r}ZxhatrZ rBigM_objValrnrorpZobj_diffrfZ cut_numberr%r%r'rns                           z3CuttingPlane_Transformation._generate_cuttingplanescCs$t|d}t}|||||fS)NZ&_pyomo_gdp_cuttingplane_transformation)r r r)rrrrr%r%r'r s  z5CuttingPlane_Transformation._add_transformation_blockcCsx|tD] }|qW|jj}g}|dkrd}xZt|D]N\}\}} } |jsh|| | d7}qB|jrt d|j dt d| |qBWn|tdkrVttd} |_tt} |_d}x|t|D]p\} \}} } |js| | | k| |<| | | k| |d<|d7}q|jr6t d|j dt d| | qW||| }xt|dd D] }||=qdWt|d |_dS) NrTrz`The variable %s will not be included in the separation problem: It was stale in the rBigM solve.T)rrri)rr1)reverse)r+)r5rGr rurr`rArrrXrYrrr*rjrrur rrl_add_dual_suffixsortedrt)rr]rIor`Z to_deleteZobj_exprrSrnrorprZinf_consjr%r%r'rsH       z5CuttingPlane_Transformation._add_separation_objectivecCs^|d}|dkr"ttjd|_n8|jtkr0dS||ttjd|_|t|d|dS)Nrk) direction)rUrZIMPORTrkctyperwrr )rZrHullrkr%r%r'rPs   z,CuttingPlane_Transformation._add_dual_suffix)N)__name__ __module__ __qualname____doc__rrZdeclarerstrrrrrjboolr(rqrrrrrrrrrrrr __classcell__r%r%)rr'rzsK                   $W( ;r)?r __future__rZpyomo.common.configrrrrrrZpyomo.common.modelingr Z pyomo.corer r r r rrrrrrrrrrrZpyomo.core.exprrZpyomo.common.collectionsrZ pyomo.optrZ pyomo.repnrZ pyomo.gdprrrZpyomo.gdp.utilr r!r"Z-pyomo.contrib.fme.fourier_motzkin_eliminationr#sixr$r getLoggerrXrr(r0r9rBrLrhrqrrregisterrr%r%r%r's8   D         C5