ó <¿CVc@@sÑdZddlmZmZmZddlmZmZddlZddl Z yddl Z Wn%e k r‡e dƒe dƒnXddlZdefd„ƒYZd „Zd „Zd „Zd „Zd „Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Z d„Z!dd„Z"d„Z#d„Z$d„Z%d„Z&d „Z'd!„Z(d"„Z)e*d#„Z+d$„Z,d%„Z-d&„Z.e*d'„Z/e*d(„Z0dS()uï ============================================ TGrep search implementation for NLTK trees ============================================ This module supports TGrep2 syntax for matching parts of NLTK Trees. Note that many tgrep operators require the tree passed to be a ``ParentedTree``. External links: - `Tgrep tutorial `_ - `Tgrep2 manual `_ - `Tgrep2 source `_ Usage ===== >>> from nltk.tree import ParentedTree >>> from nltk.tgrep import tgrep_nodes, tgrep_positions >>> tree = ParentedTree.fromstring('(S (NP (DT the) (JJ big) (NN dog)) (VP bit) (NP (DT a) (NN cat)))') >>> list(tgrep_nodes('NN', [tree])) [[ParentedTree('NN', ['dog']), ParentedTree('NN', ['cat'])]] >>> list(tgrep_positions('NN', [tree])) [[(0, 2), (2, 1)]] >>> list(tgrep_nodes('DT', [tree])) [[ParentedTree('DT', ['the']), ParentedTree('DT', ['a'])]] >>> list(tgrep_nodes('DT $ JJ', [tree])) [[ParentedTree('DT', ['the'])]] This implementation adds syntax to select nodes based on their NLTK tree position. This syntax is ``N`` plus a Python tuple representing the tree position. For instance, ``N()``, ``N(0,)``, ``N(0,0)`` are valid node selectors. Example: >>> tree = ParentedTree.fromstring('(S (NP (DT the) (JJ big) (NN dog)) (VP bit) (NP (DT a) (NN cat)))') >>> tree[0,0] ParentedTree('DT', ['the']) >>> tree[0,0].treeposition() (0, 0) >>> list(tgrep_nodes('N(0,0)', [tree])) [[ParentedTree('DT', ['the'])]] Caveats: ======== - Link modifiers: "?" and "=" are not implemented. - Tgrep compatibility: Using "@" for "!", "{" for "<", "}" for ">" are not implemented. - The "=" and "~" links are not implemented. Known Issues: ============= - There are some issues with link relations involving leaf nodes (which are represented as bare strings in NLTK trees). For instance, consider the tree:: (S (A x)) The search string ``* !>> S`` should select all nodes which are not dominated in some way by an ``S`` node (i.e., all nodes which are not descendants of an ``S``). Clearly, in this tree, the only node which fulfills this criterion is the top node (since it is not dominated by anything). However, the code here will find both the top node and the leaf node ``x``. This is because we cannot recover the parent of the leaf, since it is stored as a bare string. A possible workaround, when performing this kind of search, would be to filter out all leaf nodes. Implementation notes ==================== This implementation is (somewhat awkwardly) based on lambda functions which are predicates on a node. A predicate is a function which is either True or False; using a predicate function, we can identify sets of nodes with particular properties. A predicate function, could, for instance, return True only if a particular node has a label matching a particular regular expression, and has a daughter node which has no sisters. Because tgrep2 search strings can do things statefully (such as substituting in macros, and binding nodes with node labels), the actual predicate function is declared with three arguments:: pred = lambda n, m, l: return True # some logic here ``n`` is a node in a tree; this argument must always be given ``m`` contains a dictionary, mapping macro names onto predicate functions ``l`` is a dictionary to map node labels onto nodes in the tree ``m`` and ``l`` are declared to default to ``None``, and so need not be specified in a call to a predicate. Predicates which call other predicates must always pass the value of these arguments on. The top-level predicate (constructed by ``_tgrep_exprs_action``) binds the macro definitions to ``m`` and initialises ``l`` to an empty dictionary. i(tabsolute_importtprint_functiontunicode_literals(t binary_typet text_typeNuAWarning: nltk.tgrep will not work without the `pyparsing` packageu installed.tTgrepExceptioncB@seZdZRS(uTgrep exception type.(t__name__t __module__t__doc__(((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyR|scC@sUg}y|jƒ}Wntk r*|SXx#|rP|j|ƒ|jƒ}q.W|S(u§ Returns the list of all nodes dominating the given tree node. This method will not work with leaf nodes, since there is no way to recover the parent. (tparenttAttributeErrortappend(tnodetresultstcurrent((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt ancestors€s   cC@sgg}y|jƒ}Wntk r*|SXx5|rbt|ƒdkrb|j|ƒ|jƒ}q.W|S(ut Returns the list of all nodes dominating the given node, where there is only a single path of descent. i(R R tlenR (R R R((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pytunique_ancestors‘s  cC@sDy|jƒ}Wntk r$gSXg|dD]}||^q0S(ue Returns the list of all nodes which are descended from the given tree node in some way. i(t treepositionsR (R ttreepostx((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt _descendants¡s  cC@sZy|jƒ}Wntk r$gSXg|dD]&}td„|Dƒƒr0||^q0S(uf Returns the set of all nodes descended in some way through left branches from this node. ics@s|]}|dkVqdS(iN((t.0ty((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys µs(RR tall(R RR((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_leftmost_descendants¬s  cC@s]yt|jƒƒ}Wntk r*gSXgtdt|ƒdƒD]}||| ^qES(ug Returns the set of all nodes descended in some way through right branches from this node. i(tmaxRR trangeR(R trightmost_leafti((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_rightmost_descendants·s  cC@st|tjjƒS(u5Predicate to check whether `obj` is a nltk.tree.Tree.(t isinstancetnltkttreetTree(tobj((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_istreeÂscC@sRg}|}x?|rMt|ƒrMt|ƒdkrM|d}|j|ƒqW|S(ux Returns the list of all nodes descended from the given node, where there is only a single path of descent. ii(R$RR (R R R((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_unique_descendantsÆs ' cC@sry|jƒ}|jƒ}Wntk r0gSXg|jƒD]0}|t|ƒ |t|ƒ kr>||^q>S(uF Returns the set of all nodes that are before the given node. (t treepositiontrootR RR(R tposR!R((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_beforeÒs  cC@s½y|jƒ}|jƒ}Wntk r0gSXt|ƒd}x*d|krm||dkrm|d8}qDW|dkr~gSt||d ƒ}|dcd8<||}|gt|ƒS(uú Returns the set of all nodes that are immediately before the given node. Tree node A immediately precedes node B if the last terminal symbol (word) produced by A immediately precedes the first terminal symbol produced by B. iiiÿÿÿÿ(R&R'R RtlistR(R R(R!tidxtbefore((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_immediately_beforeÞs     cC@sry|jƒ}|jƒ}Wntk r0gSXg|jƒD]0}|t|ƒ |t|ƒ kr>||^q>S(uE Returns the set of all nodes that are after the given node. (R&R'R RR(R R(R!R((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_after÷s  cC@sßy(|jƒ}|jƒ}|jƒ}Wntk r<gSXt|ƒd}x@d|kr||t|ƒdkr|d8}|jƒ}qPW|dkr gSt||d ƒ}|dcd7<||}|gt|ƒS(u÷ Returns the set of all nodes that are immediately after the given node. Tree node A immediately follows node B if the first terminal symbol (word) produced by A immediately follows the last terminal symbol produced by B. iiiÿÿÿÿ(R&R'R R RR*R(R R(R!RR+tafter((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_immediately_afters    )   cC@s t|ƒr|jƒSt|ƒS(uw Gets the string value of a given parse tree node, for comparison using the tgrep node literal predicates. (R$tlabelR(R ((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_node_literal_valuesc@sYt|ƒdkst‚|dddks2t‚|dd‰dd‡fd†}|S(uF Builds a lambda function which looks up the macro name used. iiu@c@sD|dksˆ|kr0tdjˆƒƒ‚n|ˆ|||ƒS(Numacro {0} not defined(tNoneRtformat(tntmtl(t macro_name(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt macro_use-sN(RtAssertionErrorR3(t_st_lttokensR9((R8s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_macro_use_action&s cC@sô|ddkr|d}nt|ƒdkrŸtt|ddd…ƒƒdgks]t‚g|ddd…D]}tdd|gƒ^qq}d„|ƒSt|ddƒrº|dS|dd ksÚ|dd krçddd „S|djd ƒrI|djd ƒst‚|ddd !j dd ƒj ddƒ}d„|ƒS|djdƒrœ|djdƒsut‚|ddd !}d„t j |ƒƒS|djdƒrßt|||ddj ƒgƒ}d„|ƒSd„|dƒSdS(uq Builds a lambda function representing a predicate on a tree node depending on the name of its node. iu'iNiu|c@sdd‡fd†S(Nc@s t‡‡‡fd†ˆDƒƒS(Nc3@s!|]}|ˆˆˆƒVqdS(N((Rtf(R7R6R5(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys Cs(tany(R5R6R7(tt(R7R6R5s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pytCs(R3(RA((RAs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBCsu__call__u*u__cS@stS(N(tTrue(R5R6R7((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBJsu"iÿÿÿÿu\"u\\u\c@sdd‡fd†S(Nc@st|ƒˆkS(N(R2(R5R6R7(ts(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBNs(R3(RD((RDs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBNsu/c@sdd‡fd†S(Nc@sˆjt|ƒƒS(N(tsearchR2(R5R6R7(tr(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBRs(R3(RF((RFs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBRsui@c@sdd‡fd†S(Nc@sˆt|ƒjƒƒS(N(R2tlower(R5R6R7(R?(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBVs(R3(R?((R?s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBVsc@sdd‡fd†S(Nc@st|ƒˆkS(N(R2(R5R6R7(RD(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBYs(R3(RD((RDs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBYs( RR*tsetR:t_tgrep_node_actionR3thasattrt startswithtendswithtreplacetretcompileRG(R;R<R=R tnode_litt node_func((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRI3s4 .2   ) #cC@sLt|ƒdkst‚|ddks.t‚|ddksDt‚|dS(um Builds a lambda function representing a predicate on a tree node from a parenthetical notation. iiu(iu)i(RR:(R;R<R=((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_parens_action\scC@s#td„|Dƒƒ}d„|ƒS(u™ Builds a lambda function representing a predicate on a tree node which returns true if the node is located at a specific tree position. cs@s'|]}|jƒrt|ƒVqdS(N(tisdigittint(RR((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys nsc@sdd‡fd†S(Nc@st|dƒo|jƒˆkS(Nu treeposition(RJR&(R5R6R7(R(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBps(R3(R((Rs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBps(ttuple(R;R<R=tnode_tree_position((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_nltk_tree_pos_actiongsc@st}|ddkr)t}|d}n|ddkrtt|ƒdksQt‚|ddksgt‚|d}nwt|ƒdksŒt‚|\}‰|dkr¼dLdL‡fd †}n/|d kràdLdL‡fd †}n |d ksø|d krdLdL‡fd†}nÛ|dks(|dkr@dLdL‡fd†}n«|ddkrŒ|djƒrŒt|dƒ}‡fd†|dƒ}n_|dd krØ|djƒrØt|dƒ}‡fd†|dƒ}n|dksü|dksü|dkrdLdL‡fd†}n×|dks8|dks8|dkrPdLdL‡fd†}n›|d dkr™|djƒr™t|dƒ }‡fd†|ƒ}nR|d dkrâ|djƒrât|dƒ }‡fd†|ƒ}n |dkrdLdL‡fd†}nå|d kr*dLdL‡fd!†}nÁ|d"krNdLdL‡fd#†}n|d$krrdLdL‡fd%†}ny|d&ksŠ|d'kr¢dLdL‡fd(†}nI|d)krÆdLdL‡fd*†}n%|d+krêdLdL‡fd,†}n|d-krdLdL‡fd.†}nÝ|d/kr2dLdL‡fd0†}n¹|d1krVdLdL‡fd2†}n•|d3krzdLdL‡fd4†}nq|d5krždLdL‡fd6†}nM|d7krÂdLdL‡fd8†}n)|d9krædLdL‡fd:†}n|d;ksþ|d<krdLdL‡fd=†}nÕ|d>ks.|d?krFdLdL‡fd@†}n¥|dAks^|dBkrvdLdL‡fdC†}nu|dDksŽ|dEkr¦dLdL‡fdF†}nE|dGks¾|dHkrÖdLdL‡fdI†}ntdJj|ƒƒ‚|rþdK„|ƒS|SdLS(Mu„ Builds a lambda function representing a predicate on a tree node depending on its relation to other nodes in the tree. iu!iu[iiu]uŠs(R$R@(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB‰s u>c@s7t|dƒo6t|jƒƒo6ˆ|jƒ||ƒS(Nuparent(RJtboolR (R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu<,u<1c@s2t|ƒo1tt|ƒƒo1ˆ|d||ƒS(Ni(R$RYR*(R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB’s u>,u>1c@sMt|dƒoLt|jƒƒoL||jƒdkoLˆ|jƒ||ƒS(Nuparenti(RJRYR (R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB—sc@sdd‡‡fd†S(Nc@sTt|ƒoStt|ƒƒoSdˆko;t|ƒknoSˆ|ˆ||ƒS(Ni(R$RYR*R(R5R6R7(RRX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBŸs "(R3(R(RX(Rs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBŸsc@sdd‡‡fd†S(Nc@sut|dƒott|jƒƒotdˆkoDt|jƒƒknot||jƒˆkotˆ|jƒ||ƒS(Nuparenti(RJRYR R(R5R6R7(RRX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB§s((R3(R(RX(Rs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB§su<'u<-u<-1c@s2t|ƒo1tt|ƒƒo1ˆ|d||ƒS(Niÿÿÿÿ(R$RYR*(R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB¯su>'u>-u>-1c@sMt|dƒoLt|jƒƒoL||jƒdkoLˆ|jƒ||ƒS(Nuparentiÿÿÿÿ(RJRYR (R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB´sc@sdd‡‡fd†S(Nc@sht|ƒogtt|ƒƒogdˆt|ƒkoEt|ƒknogˆ|ˆt|ƒ||ƒS(Ni(R$RYR*R(R5R6R7(RRX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB¼s ,(R3(R(RX(Rs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB¼sc@sdd‡‡fd†S(Nc@s•t|dƒo”t|jƒƒo”dˆt|jƒƒkoTt|jƒƒkno”||jƒˆt|jƒƒko”ˆ|jƒ||ƒS(Nuparenti(RJRYR R(R5R6R7(RRX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBÄs 8&(R3(R(RX(Rs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBÄsu<:c@s2t|ƒo1t|ƒdko1ˆ|d||ƒS(Nii(R$R(R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBÌs u>:c@sOt|dƒoNt|jƒƒoNt|jƒƒdkoNˆ|jƒ||ƒS(Nuparenti(RJRYR R(R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBÑsu<Øs(R$R@R(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB×s u>>c@s&t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys Ûs(R@R(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBÛsu<<,u<<1c@s2t|ƒo1t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys ßs(R$R@R(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBÞs u>>,c@s)t‡‡‡‡fd†tˆƒDƒƒS(Nc3@s3|])}ˆ|ˆˆƒo*ˆt|ƒkVqdS(N(R(RR(R7R6R5RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys ãs(R@R(R5R6R7(RX(R7R6R5s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBãsu<<'c@s2t|ƒo1t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys és(R$R@R(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBès u>>'c@s)t‡‡‡‡fd†tˆƒDƒƒS(Nc3@s3|])}ˆ|ˆˆƒo*ˆt|ƒkVqdS(N(R(RR(R7R6R5RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys ís(R@R(R5R6R7(RX(R7R6R5s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBísu<<:c@s2t|ƒo1t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys ós(R$R@R%(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBòs u>>:c@s&t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys ÷s(R@R(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB÷su.c@s&t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys ús(R@R0(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBúsu,c@s&t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys þs(R@R-(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBþsu..c@s&t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys s(R@R.(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu,,c@s&t‡‡‡fd†t|ƒDƒƒS(Nc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys s(R@R)(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu$u%c@sJtˆdƒoItˆjƒƒoIt‡‡‡‡fd†ˆjƒDƒƒS(Nuparentc3@s-|]#}|ˆk rˆ|ˆˆƒVqdS(N((RR(R7R6R5RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys s(RJRYR R@(R5R6R7(RX(R7R6R5s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu$.u%.c@s7t|dƒo6t|jƒƒo6ˆ|jƒ||ƒS(Nu right_sibling(RJRYt right_sibling(R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu$,u%,c@s7t|dƒo6t|jƒƒo6ˆ|jƒ||ƒS(Nu left_sibling(RJRYt left_sibling(R5R6R7(RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu$..u%..c@sdt|dƒoct|dƒoct|jƒƒoct‡‡‡fd†|jƒ|jƒdDƒƒS(Nuparentu parent_indexc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys si(RJRYR R@t parent_index(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu$,,u%,,c@s`t|dƒo_t|dƒo_t|jƒƒo_t‡‡‡fd†|jƒ|jƒ DƒƒS(Nuparentu parent_indexc3@s!|]}ˆ|ˆˆƒVqdS(N((RR(R7R6RX(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys "s(RJRYR R@R\(R5R6R7(RX(R7R6s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBsu%cannot interpret tgrep operator "{0}"c@sdd‡fd†S(Nc@sˆ|||ƒ S(N((R5R6R7(RF(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB)s(R3(RF((RFs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRB)sN( tFalseRCRR:R3RSRTRR4(R;R<R=tnegatedtretvaltoperatorR+((RXs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_relation_actionss¢         $$                    u&cC@sPg|D]}||kr|^q}t|ƒdkr?|dSd„|ƒSdS(uÞ Builds a lambda function representing a predicate on a tree node from the conjunction of several other such lambda functions. This is prototypically called for expressions like (`tgrep_rel_conjunction`):: < NP & < AP < VP where tokens is a list of predicates representing the relations (`< NP`, `< AP`, and `< VP`), possibly with the character `&` included (as in the example here). This is also called for expressions like (`tgrep_node_expr2`):: NP < NN S=s < /NP/=n : s < /VP/=v : n .. v tokens[0] is a tgrep_expr predicate; tokens[1:] are an (optional) list of segmented patterns (`tgrep_expr_labeled`, processed by `_tgrep_segmented_pattern_action`). iic@sdd‡fd†S(Nc@s t‡‡‡fd†ˆDƒƒS(Nc3@s!|]}|ˆˆˆƒVqdS(N((RRX(R7R6R5(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys Js(R(R5R6R7(tts(R7R6R5s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBJs(R3(Rb((Rbs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyRBJsN(R(R;R<R=t join_charR((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_conjunction_action-s %c@s0|d‰|d‰dd‡‡fd†}|S(u Builds a lambda function representing a segmented pattern. Called for expressions like (`tgrep_expr_labeled`):: =s .. =v < =n This is a segmented pattern, a tgrep2 expression which begins with a node label. The problem is that for segemented_pattern_action (': =v < =s'), the first element (in this case, =v) is specifically selected by virtue of matching a particular node in the tree; to retrieve the node, we need the label, not a lambda function. For node labels inside a tgrep_node_expr, we need a lambda function which returns true if the node visited is the same as =v. We solve this by creating two copies of a node_label_use in the grammar; the label use inside a tgrep_expr_labeled has a separate parse action to the pred use inside a node_expr. See `_tgrep_node_label_use_action` and `_tgrep_node_label_pred_use_action`. iic@sZˆdksˆˆkr0tdjˆƒƒ‚nˆˆ‰t‡‡‡fd†ˆDƒƒS(u2This predicate function ignores its node argument.u$node_label ={0} not bound in patternc3@s!|]}|ˆˆˆƒVqdS(N((Rtpred(R7R6R (s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pys rsN(R3RR4R(R5R6R7(t node_labelt reln_preds(R7R6R s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pytpattern_segment_predjs   N(R3(R;R<R=Rh((RfRgs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_segmented_pattern_actionMs   cC@s=t|ƒdkst‚|djdƒs1t‚|ddS(uU Returns the node label used to begin a tgrep_expr_labeled. See `_tgrep_segmented_pattern_action`. Called for expressions like (`tgrep_node_label_use`):: =s when they appear as the first element of a `tgrep_expr_labeled` expression (see `_tgrep_segmented_pattern_action`). It returns the node label. iiu=(RR:RK(R;R<R=((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_node_label_use_actionusc@sXt|ƒdkst‚|djdƒs1t‚|dd‰dd‡fd†}|S(uÈ Builds a lambda function representing a predicate on a tree node which describes the use of a previously bound node label. Called for expressions like (`tgrep_node_label_use_pred`):: =s when they appear inside a tgrep_node_expr (for example, inside a relation). The predicate returns true if and only if its node argument is identical the the node looked up in the node label dictionary using the node's label. iiu=c@sD|dksˆ|kr0tdjˆƒƒ‚n|ˆ}||kS(Nu$node_label ={0} not bound in pattern(R3RR4(R5R6R7R (Rf(s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pytnode_label_use_pred˜s   N(RR:RKR3(R;R<R=Rk((Rfs\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt!_tgrep_node_label_pred_use_action‡s c@s|t|ƒdkr|dSt|ƒdks2t‚|ddksHt‚|d‰|d‰dd‡‡fd†}|SdS(uõ Builds a lambda function representing a predicate on a tree node which can optionally bind a matching node into the tgrep2 string's label_dict. Called for expressions like (`tgrep_node_expr2`):: /NP/ @NP=n iiiu=ic@sLˆ|||ƒrD|dkr6tdjˆƒƒ‚n||ˆôs(R@(R5R6R7(t tgrep_exprs(RtR6R5s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyttop_level_predñsN(RR3Rtdicttupdate( R;R<R=Rt macro_dictttokt macro_defst macro_defRv((RuR=s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_tgrep_exprs_actionÚs %( (cC@s tjdƒtjdƒ}tjdddddtƒ}tjdddddtƒ}tjd ƒ}tjd ƒ}tjd ƒ}tjƒ}tjƒ}tjd ƒ|d } tjdƒtjtjtjƒdtjtj tjtjƒddƒtjdƒƒƒd } tjdƒ} tj d| ƒ} | j ƒ} tjdƒ}|j dƒtj d|ƒ}| |B| B|B|B|B|BdB|B}|tjdƒj dƒ| j ƒj dƒ|B}| tjdƒ|tj d|ƒB}tjdƒd|d}|||B}tjƒ}||tj tjdƒ|ƒ>||tj d|ƒ>||tj|ƒ>| tj|ƒ}|tj d|ƒ}tjdƒtjƒjƒ||}tj|tj d|ƒdƒ|tj d||Bƒtj dƒjƒ}|rø| jtƒ| jtƒ|jtƒ|jtƒ|jtƒ| jtƒ| jtƒ|jtƒ|jtƒ|jtƒ|jtƒ|jtƒ|jtƒ|jtjtddƒƒ|jtƒn|j dtj!ƒS( uj Builds a pyparsing-based parser object for tokenizing and interpreting tgrep search strings. u!u[$%,.<>][%,.<>0-9-':]*t quoteCharu"tescCharu\tunquoteResultsu/ui@\"(?:[^"\n\r\\]|(?:\\.))*\"ui@\/(?:[^/\n\r\\]|(?:\\.))*\/u[^][ ;:.,&|<>()$!@%'^=]+u(u)uN(u,tdelimu [A-Za-z0-9]+u=u[^];:.,&|<>()[$!@%'^= ]+uu@u*u'u|u[u]u&u:u;Rcu#("t pyparsingtOptionaltRegext QuotedStringR]tForwardtLiteraltWordtnumst delimitedListtCombinetcopytsetWhitespaceCharst ZeroOrMoretWhitetsuppresstsetParseActionRjRlR>RIRoRRRWRaRdRrRsRit functoolstpartialR}tignoret restOfLine(tset_parse_actionsttgrep_opt tgrep_qstringttgrep_node_regexttgrep_qstring_icasettgrep_node_regex_icasettgrep_node_literalt tgrep_exprttgrep_relationst tgrep_parensttgrep_nltk_tree_posttgrep_node_labelttgrep_node_label_usettgrep_node_label_use_predR8R9ttgrep_node_exprttgrep_node_expr2t tgrep_nodettgrep_bracketsttgrep_relationttgrep_rel_conjunctionttgrep_expr_labeledt tgrep_expr2t macro_defnRu((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt_build_tgrep_parser÷s|       e       :              cC@s=ttƒ}t|tƒr*|jƒ}nt|j|ƒƒS(u? Tokenizes a TGrep search string into separate tokens. (R­R]RRtdecodeR*t parseString(t tgrep_stringtparser((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyttgrep_tokenizeQs cC@sGttƒ}t|tƒr*|jƒ}nt|j|dtƒƒdS(u` Parses (and tokenizes, if necessary) a TGrep search string into a lambda function. tparseAlli(R­RCRRR®R*R¯(R°R±((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt tgrep_compileZs cC@sw|jƒ}tƒ}x<|D]4}x+tt|ƒƒD]}|j|| ƒq5WqWg|D]}||kr[|^q[S(uX Returns all the tree positions in the given tree which are not leaf nodes. (RRHRRtadd(R!RtprefixesR(tlength((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyttreepositions_no_leavesds    cc@sŸt|ttfƒr$t|ƒ}nxt|D]l}yM|rI|jƒ}n t|ƒ}g|D]}|||ƒr\|^q\VWq+tk r–gVq+Xq+WdS(u£ Return the tree positions in the trees which match the given pattern. :param pattern: a tgrep search pattern :type pattern: str or output of tgrep_compile() :param trees: a sequence of NLTK trees (usually ParentedTrees) :type trees: iter(ParentedTree) or iter(Tree) :param search_leaves: whether ot return matching leaf nodes :type search_leaves: bool :rtype: iter(tree positions) N(RRRR´RR¸R (tpatternttreest search_leavesR!t positionstposition((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyttgrep_positionsrs     cc@s£t|ttfƒr$t|ƒ}nxx|D]p}yQ|rI|jƒ}n t|ƒ}g|D] }|||ƒr\||^q\VWq+tk ršgVq+Xq+WdS(u› Return the tree nodes in the trees which match the given pattern. :param pattern: a tgrep search pattern :type pattern: str or output of tgrep_compile() :param trees: a sequence of NLTK trees (usually ParentedTrees) :type trees: iter(ParentedTree) or iter(Tree) :param search_leaves: whether ot return matching leaf nodes :type search_leaves: bool :rtype: iter(tree nodes) N(RRRR´RR¸R (R¹RºR»R!R¼R½((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pyt tgrep_nodess    # (1Rt __future__RRRt nltk.compatRRR’t nltk.treeR R‚t ImportErrortprintRNt ExceptionRRRRRRR$R%R)R-R.R0R2R>RIRRRWRaRdRiRjRlRoRrRsR}RCR­R²R´R¸R¾R¿(((s\/private/var/folders/cc/xm4nqn811x9b50x1q_zpkmvdjlphkp/T/pip-build-FUwmDn/nltk/nltk/tgrep.pytosT            ) º (   "    Z