B y `{@sUdZddlmZmZmZmZmZmZmZm Z m Z m Z m Z m Z mZddlmZdZddlZddlmZdZeed<iaeee eeffed <ee eefd d d Ze d Ze dZe ee eeffZe eeeeeeeefZGdddeZGdddeZ GdddeZ!eeedddZ"eeefZ#GdddeZ$Gddde$Z%Gddde$Z&Gddde$Z'Gd d!d!e$Z(ee$eeee ee#fd"d#d$Z)eeegefZ*dS)%z Python parse tree definitions. This is a very concrete parse tree; we need to keep every token and even the comments and whitespace between tokens. There's also a pattern matching implementation here. ) AnyCallableDictIteratorListOptionalTextTupleTypeVarUnionSetIterableSequence)Grammarz#Guido van Rossum N)StringIOiHUGE _type_reprs)type_numreturncCsLts@ddlm}x.t|D]"}t||}t|tkr|t|<qWt||S)N)python_symbols)rZpygramrdirgetattrtypeint setdefault)rrnamevalr3/tmp/pip-unpacked-wheel-qq0fnpma/blib2to3/pytree.py type_repr*s    r _P)NodeLeafc@sjeZdZUdZeed<dZeded<ee ed<dZ e ed<dZ e ed <d d Z ee d d dZdZeedddZeee dddZeedddZee dddZee dddZee ee fddddZeeddd Zddd!d"Zeedd#d$Zeee dd%d&Zeee dd'd(Z ed)dd*d+Z!edd,d-Z"edd.d/Z#dS)0Basez Abstract base class for Node and Leaf. This provides some default functionality and boilerplate using the template pattern. A node may be a subnode of at most one parent. rNr"parentchildrenF was_changed was_checkedcOs|tk stdt|S)z7Constructor that prevents Base from being instantiated.zCannot instantiate Base)r$AssertionErrorobject__new__)clsargskwdsrrrr+Qsz Base.__new__)otherrcCs|j|jk rtS||S)zW Compare two nodes for equality. This calls the method _eq(). ) __class__NotImplemented_eq)selfr/rrr__eq__Vs z Base.__eq__)rcCstdS)N)NotImplementedError)r3rrrprefixbsz Base.prefix)r3r/rcCstdS)a_ Compare two nodes for equality. This is called by __eq__ and __ne__. It is only called if the two nodes have the same type. This must be implemented by the concrete subclass. Nodes should be considered equal if they have the same structure, ignoring the prefix string and other context information. N)r5)r3r/rrrr2fs zBase._eq)r3rcCstdS)zr Return a cloned (deep) copy of self. This must be implemented by the concrete subclass. N)r5)r3rrrcloneqsz Base.clonecCstdS)zx Return a post-order iterator for the tree. This must be implemented by the concrete subclass. N)r5)r3rrr post_orderyszBase.post_ordercCstdS)zw Return a pre-order iterator for the tree. This must be implemented by the concrete subclass. N)r5)r3rrr pre_orderszBase.pre_order)newrcCs|jdk stt||dk s"tt|ts2|g}g}d}xP|jjD]D}||kr~|rft|jj||f|dk rx||d}qD||qDW|st|j||f||j_|j|j x|D] }|j|_qWd|_dS)z/Replace this node with a new one in the parent.NFT) r%r)str isinstancelistr&extendappendchangedinvalidate_sibling_maps)r3r:Z l_childrenfoundchxrrrreplaces(       z Base.replacecCs.|}x"t|ts&|jsdS|jd}qW|jS)z9Return the line number which generated the invocant node.Nr)r<r#r&lineno)r3noderrr get_linenos  zBase.get_linenocCs$|jr dS|jr|jd|_dS)NT)r'r%r@)r3rrrr@s  z Base.changedcCsT|jrPxHt|jjD]8\}}||kr|jj|=|j|jd|_|SqWdS)z Remove the node from the tree. Returns the position of the node in its parent's children before it was removed. N)r% enumerater&r@rA)r3irGrrrremoves   z Base.removecCsD|jdkrdS|jjdkr$|j|jjdk s4t|jjt|S)z The node immediately following the invocant in their parent's children list. If the invocant does not have a next sibling, it is None N)r%next_sibling_mapupdate_sibling_mapsr)id)r3rrr next_siblings    zBase.next_siblingcCsD|jdkrdS|jjdkr$|j|jjdk s4t|jjt|S)z The node immediately preceding the invocant in their parent's children list. If the invocant does not have a previous sibling, it is None. N)r%prev_sibling_maprMr)rN)r3rrr prev_siblings    zBase.prev_siblingr#ccs"x|jD]}|EdHqWdS)N)r&leaves)r3childrrrrRs z Base.leavescCs|jdkrdSd|jS)Nrr)r%depth)r3rrrrTs z Base.depthcCs|j}|dkrdS|j}|S)z Return the string immediately following the invocant node. This is effectively equivalent to node.next_sibling.prefix N)rOr6)r3Znext_sibr6rrr get_suffixs zBase.get_suffix)$__name__ __module__ __qualname____doc__r__annotations__r%rrNLr'boolr(r+rr4__hash__propertyrr6r!r2r7rr8r9r rErHr@rKrOrQrRrTrVrrrrr$?s4         r$c@s2eZdZUdZeeeed<eee ed<d$e ee eeee eeeddddZ e dd d Z e dd d Zedd dZddddZee dddZee dddZee dddZejddddZe e ddddZe e ddddZe ddddZddd d!Zddd"d#ZdS)%r"z+Concrete implementation for interior nodes.fixers_applied used_namesN)rr&contextr6r`rcCs|dkst|||_t||_x*|jD] }|jdksBtt|||_q(W||dk rb||_|rv|dd|_nd|_dS)z Initializer. Takes a type constant (a symbol number >= 256), a sequence of child nodes, and an optional context keyword argument. As a side effect, the parent pointers of the children are updated. N) r)rr=r&r%reprrAr6r`)r3rr&rbr6r`rCrrr__init__s   z Node.__init__)rcCs(|jdk std|jjt|j|jfS)z)Return a canonical string representation.Nz %s(%s, %r))rr)r0rWr r&)r3rrr__repr__s z Node.__repr__cCsdtt|jS)zk Return a pretty string representation. This reproduces the input source exactly. rU)joinmapr;r&)r3rrr__str__sz Node.__str__cCs|j|jf|j|jfkS)zCompare two nodes for equality.)rr&)r3r/rrrr2'szNode._eqcCs,|jdk stt|jdd|jD|jdS)NcSsg|] }|qSr)r7).0rCrrr 0szNode.clone..)r`)rr)r"r&r`)r3rrrr7+s z Node.cloneccs(x|jD]}|EdHqW|VdS)z*Return a post-order iterator for the tree.N)r&r8)r3rSrrrr84s zNode.post_orderccs(|Vx|jD]}|EdHqWdS)z)Return a pre-order iterator for the tree.N)r&r9)r3rSrrrr9:s zNode.pre_ordercCs|js dS|jdjS)zO The whitespace and comments preceding this node in the input. rUr)r&r6)r3rrrr6@sz Node.prefixcCs|jr||jd_dS)Nr)r&r6)r3r6rrrr6Is)rJrSrcCs0||_d|j|_||j|<||dS)z Equivalent to 'node.children[i] = child'. This method also sets the child's parent attribute appropriately. N)r%r&r@rA)r3rJrSrrr set_childNs   zNode.set_childcCs(||_|j||||dS)z Equivalent to 'node.children.insert(i, child)'. This method also sets the child's parent attribute appropriately. N)r%r&insertr@rA)r3rJrSrrr insert_childYszNode.insert_child)rSrcCs&||_|j|||dS)z Equivalent to 'node.children.append(child)'. This method also sets the child's parent attribute appropriately. N)r%r&r?r@rA)r3rSrrr append_childcs zNode.append_childcCsd|_d|_dS)N)rPrL)r3rrrrAmszNode.invalidate_sibling_mapscCsTi}i}||_||_d}x*|jD] }||t|<||t|<|}q Wd|t|<dS)N)rPrLr&rN)r3_prev_nextpreviouscurrentrrrrMqs   zNode.update_sibling_maps)NNN)rWrXrYrZrrrr[r rrr\rerfrir]r2r7rr8r9r_r6setterrlrnrorArMrrrrr"s,       r"c@s"eZdZUdZeed<eeed<eed<ded<e e eed<dZ dZ eed <dZ eed <d d gfeee ee eeed d d dZedddZedddZedddZddddZeddddZeddddZeddddZeedddZejd dd dZd S)!r#z'Concrete implementation for leaf nodes.valuer`Z bracket_depthZopening_bracketrarUrrFcolumnN)rrurbr6r`rcCsjd|krdksnt||dk r8|\|_\|_|_||_||_|dk rR||_|dd|_g|_dS)z Initializer. Takes a type constant (a token number < 256), a string value, and an optional context keyword argument. rrcN)r)_prefixrFrvrrur`r&)r3rrurbr6r`rrrresz Leaf.__init__)rcCs:ddlm}|jdk std|jj||j|j|jfS)z)Return a canonical string representation.r)tok_nameNz %s(%s, %r))Z pgen2.tokenrxrr)r0rWgetru)r3rxrrrrfs  z Leaf.__repr__cCs|jt|jS)zk Return a pretty string representation. This reproduces the input source exactly. )r6r;ru)r3rrrrisz Leaf.__str__cCs|j|jf|j|jfkS)zCompare two nodes for equality.)rru)r3r/rrrr2szLeaf._eqcCs2|jdk stt|j|j|j|j|jff|jdS)N)r`)rr)r#rur6rFrvr`)r3rrrr7s z Leaf.cloneccs |VdS)Nr)r3rrrrRsz Leaf.leavesccs |VdS)z*Return a post-order iterator for the tree.Nr)r3rrrr8szLeaf.post_orderccs |VdS)z)Return a pre-order iterator for the tree.Nr)r3rrrr9szLeaf.pre_ordercCs|jS)zP The whitespace and comments preceding this token in the input. )rw)r3rrrr6sz Leaf.prefixcCs|||_dS)N)r@rw)r3r6rrrr6s)rWrXrYrZrr[rrrrr rwrFrvContextrer;rfrir]r2r7rrRr8r9r_r6rtrrrrr#~s0      r#)grraw_nodercCs^|\}}}}|s||jkrH|dk s&tt|dkr:|dSt|||dSt||pRd|dSdS)z Convert raw node information to a Node or Leaf instance. This is passed to the parser driver which calls it whenever a reduction of a grammar rule produces a new complete node, so that the tree is build strictly bottom-up. Nrr)rbrU)Z number2symbolr)lenr"r#)r{r|rrurbr&rrrconverts   r~c@seZdZUdZeeed<dZdZe ed<dZ ee ed<ddZ e dd d Z dedd d Zddd dZdeeeedddZdeeeeedddZeeeeeefdddZdS) BasePatterna A pattern is a tree matching pattern. It looks for a specific node type (token or symbol), and optionally for a specific content. This is an abstract base class. There are three concrete subclasses: - LeafPattern matches a single leaf node; - NodePattern matches a single node (usually non-leaf); - WildcardPattern matches a sequence of nodes of variable length. rNcontentrcOs|tk stdt|S)z>Constructor that prevents BasePattern from being instantiated.zCannot instantiate BasePattern)rr)r*r+)r,r-r.rrrr+ szBasePattern.__new__)rcCsZ|jdk stt|j|j|jg}x|r<|ddkr<|d=q$Wd|jjdtt |fS)Nz%s(%s)z, ) rr)r rrr0rWrgrhrd)r3r-rrrrfs  zBasePattern.__repr__cCstdS)N)r5)r3rGresultsrrr _submatchszBasePattern._submatchcCs|S)z A subclass can define this as a hook for optimizations. Returns either self or another node with the same effect. r)r3rrroptimizeszBasePattern.optimize)rGrrcCsz|jdk r|j|jkrdS|jdk r^d}|dk r4i}|||sDdS|r^|dk sTt|||dk rv|jrv|||j<dS)a# Does this pattern exactly match a node? Returns True if it matches, False if not. If results is not None, it must be a dict which will be updated with the nodes matching named subpatterns. Default implementation for non-wildcard patterns. NFT)rrrr)updater)r3rGrrrrrmatch"s      zBasePattern.match)nodesrrcCs t|dkrdS||d|S)z Does this pattern exactly match a sequence of nodes? Default implementation for non-wildcard patterns. rFr)r}r)r3rrrrr match_seq<s zBasePattern.match_seq)rrccs&i}|r"||d|r"d|fVdS)z} Generator yielding all matches for this pattern. Default implementation for non-wildcard patterns. rrN)r)r3rrrrrgenerate_matchesFszBasePattern.generate_matches)N)N)N)rWrXrYrZrrr[rrrrrr+rfr]rrr\_Resultsrrrrr rrrrrrs    rc@sHeZdZd eeeeeeddddZd edddZd dd Z dS) LeafPatternN)rrrrcCsZ|dk r&d|krdks&nt||dk rDt|tsDtt|||_||_||_dS)ap Initializer. Takes optional type, content, and name. The type, if given must be a token type (< 256). If not given, this matches any *leaf* node; the content may still be required. The content, if given, must be a string. If a name is given, the matching node is stored in the results dict under that key. Nrrc)r)r<r;rdrrr)r3rrrrrrreRszLeafPattern.__init__)rGcCst|tsdSt|||S)z*Override match() to insist on a leaf node.F)r<r#rr)r3rGrrrrrks zLeafPattern.matchcCs |j|jkS)a Match the pattern's content to the node's children. This assumes the node type matches and self.content is not None. Returns True if it matches, False if not. If results is not None, it must be a dict which will be updated with the nodes matching named subpatterns. When returning False, the results dict may still be updated. )rru)r3rGrrrrrqs zLeafPattern._submatch)NNN)N)N) rWrXrYrrrrer\rrrrrrrQs rc@sPeZdZUdZeed<d eeeee ee ddddZ d eddd Z dS) NodePatternF wildcardsN)rrrrcCs|dk r|dkst||dk rzt|tr6tt|t|}x:t|D].\}}t|tsft||ft|trHd|_qHW||_ ||_ ||_ dS)ad Initializer. Takes optional type, content, and name. The type, if given, must be a symbol type (>= 256). If the type is None this matches *any* single node (leaf or not), except if content is not None, in which it only matches non-leaf nodes that also match the content pattern. The content, if not None, must be a sequence of Patterns that must match the node's children exactly. If the content is given, the type must not be None. If a name is given, the matching node is stored in the results dict under that key. NrcT) r)r<r;rdr=rIrWildcardPatternrrrr)r3rrrZ newcontentrJitemrrrres  zNodePattern.__init__)rcCs|jrJx>t|j|jD],\}}|t|jkr|dk r>||dSqWdSt|jt|jkrbdSx*t|j|jD]\}}|||srdSqrWdS)a Match the pattern's content to the node's children. This assumes the node type matches and self.content is not None. Returns True if it matches, False if not. If results is not None, it must be a dict which will be updated with the nodes matching named subpatterns. When returning False, the results dict may still be updated. NTF)rrrr&r}rzipr)r3rGrcr subpatternrSrrrrs   zNodePattern._submatch)NNN)N) rWrXrYrr]r[rrr rrerrrrrrs  rc@seZdZUdZeed<eed<ddedfeeeeeeddddZ e d d d Z de d d d Z de d ddZeeeefd ddZeeeefd ddZeeefd ddZeeeefd ddZdS)ra A wildcard pattern can match zero or more nodes. This has all the flexibility needed to implement patterns like: .* .+ .? .{m,n} (a b c | d e | f) (...)* (...)+ (...)? (...){m,n} except it always uses non-greedy matching. minmaxNr)rrrrrcCsd|kr|krtks,nt||f|dk rdd}tt||}t|s^tt|x |D]}t|sdtt|qdW||_||_||_||_ dS)a Initializer. Args: content: optional sequence of subsequences of patterns; if absent, matches one node; if present, each subsequence is an alternative [*] min: optional minimum number of times to match, default 0 max: optional maximum number of times to match, default HUGE name: optional name assigned to this match [*] Thus, if content is [[a, b, c], [d, e], [f, g, h]] this is equivalent to (a b c | d e | f g h); if content is None, this is equivalent to '.' in regular expression terms. The min and max parameters work as follows: min=0, max=maxint: .* min=1, max=maxint: .+ min=0, max=1: .? min=1, max=1: . If content is not None, replace the dot with the parenthesized list of alternatives, e.g. (a b c | d e | f g h)* rNcSst|S)N)tuple)srrrz*WildcardPattern.__init__..) rr)rrhr}rdrrrr)r3rrrrfZwrapped_contentaltrrrres,  zWildcardPattern.__init__)rcCsd}|jdk r\}}|t|kr |dk rF|||jrFt|||j<dSq WdS)z4Does this pattern exactly match a sequence of nodes?NTF)rr}rrr=)r3rrrrrrrr s  zWildcardPattern.match_seqc cs:|jdkrXxJt|jdtt||jD]*}i}|jrH|d|||j<||fVq(Wn|jdkrp||Vnttdrtj }t t_ zy@x:| |dD]*\}}|jr|d|||j<||fVqWWnRt k rx:| |D],\}}|jr |d|||j<||fVqWYnXWdttdr4|t_ XdS)a" Generator yielding matches for a sequence of nodes. Args: nodes: sequence of nodes Yields: (count, results) tuples where: count: the match comprises nodes[:count]; results: dict containing named submatches. NrZ bare_name getrefcountr)rrangerr}rr_bare_name_matcheshasattrsysstderrr_recursive_matches RuntimeError_iterative_matches)r3rcountrZ save_stderrrrrr+s. "   z WildcardPattern.generate_matchesc cs t|}d|jkrdifVg}x>|jD]4}x.t||D] \}}||fV|||fq8Wq(Wx|rg}x|D]\}} ||krr||jkrrxn|jD]d}x^t|||dD]H\} } | dkri}|| || || |fV||| |fqWqWqrW|}qbWdS)z(Helper to iteratively yield the matches.rN)r}rrrr?rr) r3rZnodelenrrrrZ new_resultsc0r0c1r1rrrrXs*       z"WildcardPattern._iterative_matchescCsd}i}d}t|}xF|sZ||krZd}x0|jD]&}|d|||r.|d7}d}Pq.WqW|jdk sjt|d|||j<||fS)z(Special optimized matcher for bare_name.rFTrN)r}rrrr))r3rrrdonerZleafrrrrus  z"WildcardPattern._bare_name_matchesc cs|jdk st||jkr"difV||jkrxr|jD]h}xbt||D]T\}}xJ|||d|dD].\}}i}|||||||fVqfWqDWq4WdS)z(Helper to recursively yield the matches.Nrr)rr)rrrrr) r3rrrrrrrrrrrrs    "  z"WildcardPattern._recursive_matches)N)N)rWrXrYrZrr[rrrrerrr]rrrr rrrrrrrrrrs & -rc@s\eZdZd eeddddZd edddZdeddd Ze e e e fdd d Z dS)NegatedPatternN)rrcCs(|dk rt|tstt|||_dS)a Initializer. The argument is either a pattern or None. If it is None, this only matches an empty sequence (effectively '$' in regex lingo). If it is not None, this matches whenever the argument pattern doesn't have any matches. N)r<rr)rdr)r3rrrrres zNegatedPattern.__init__)rcCsdS)NFr)r3rGrrrrrszNegatedPattern.matchcCs t|dkS)Nr)r})r3rrrrrrszNegatedPattern.match_seqccsL|jdkr"t|dkrHdifVn&x|j|D] \}}dSWdifVdS)Nr)rr}r)r3rrrrrrrs    zNegatedPattern.generate_matches)N)N)N)rWrXrYrrrer]rrrr rrrrrrrrs r)patternsrrc cs|sdifVn|d|dd}}xl||D]^\}}|sJ||fVq2xDt|||dD].\}}i}|||||||fVq^Wq2WdS)aN Generator yielding matches for a sequence of patterns and nodes. Args: patterns: a sequence of patterns nodes: a sequence of nodes Yields: (count, results) tuples where: count: the entire sequence of patterns matches nodes[:count]; results: dict containing named submatches. rrN)rr) rrprestrrrrrrrrrs    r)+rZtypingrrrrrrrr r r r r rZblib2to3.pgen2.grammarr __author__riorrrr[rr r!r\rzZRawNoder*r$r"r#r~rrrrrrrZ_Convertrrrr s8<    3` \0BS#