B y `<ã@sÐddlmZmZmZddlmZmZmZmZm Z m Z m Z m Z m Z mZmZmZddlmZddlmZddlZeedfZGdd „d ejƒZGd d „d eƒZGd d „d eƒZGdd„deƒZdeedœdd„ZdS)é)ÚgrammarÚtokenÚtokenizeé) ÚAnyÚDictÚIOÚIterableÚIteratorÚListÚOptionalÚTextÚTupleÚUnionÚSequenceÚNoReturn)r)Ú GoodTokenInfoNzos.PathLike[str]c@s eZdZdS)Ú PgenGrammarN)Ú__name__Ú __module__Ú __qualname__©rrú7/tmp/pip-unpacked-wheel-qq0fnpma/blib2to3/pgen2/pgen.pyrsrc@s´eZdZUeed<eeed<eeed<e ee e ee ffed<d8ee eeddœdd„Z e d œd d „Ze ee e e fd œd d„Ze ee dœdd„Zdd œdd„Zeddœdd„Zee eedfefd œdd„Zddeddœdd„Zeddddœdd „Zeeddd!œd"d#„Zeddd$œd%d&„Zed'd œd(d)„Zed'd œd*d+„Zed'd œd,d-„Zed'd œd.d/„Zd9e e eed0œd1d2„Z dd œd3d4„Z!e"ee#d5œd6d7„Z$dS):ÚParserGeneratorÚfilenameÚstreamÚ generatorÚfirstN)rrÚreturncCsld}|dkrt|ƒ}|j}||_||_t |j¡|_| ¡|  ¡\|_ |_ |dk rZ|ƒi|_ |  ¡dS)N)ÚopenÚcloserrrÚgenerate_tokensÚreadlinerÚgettokenÚparseÚdfasÚ startsymbolrÚ addfirstsets)ÚselfrrZ close_streamrrrÚ__init__(szParserGenerator.__init__)rc Cs*tƒ}t|j ¡ƒ}| ¡| |j¡| d|j¡x.|D]&}dt|j ƒ}||j |<||j |<qrcCsD|j|}|dk st‚i}x$t|ƒD]}| ||¡}d||<q$W|S)Nr)rÚAssertionErrorr2r6)r(r<r>ZrawfirstrrBÚilabelrrrr:Ps    zParserGenerator.make_first)r<rBrcCsbt|jƒ}|d ¡r¾||jkrZ||jkr4|j|S|j |j|df¡||j|<|Snbtt|dƒ}t|t ƒsxt |ƒ‚|tj ksŠt |ƒ‚||j krž|j |S|j |df¡||j |<|Sn |ddksÒt |ƒ‚t |ƒ}|d ¡r ||jkrþ|j|S|j tj|f¡||j|<|Sn>tj|}||j kr@|j |S|j |df¡||j |<|SdS)Nr)ú"ú')r0ÚlabelsÚisalphar1Z symbol2labelr5ÚgetattrrÚ isinstanceÚintrEÚtok_nameÚtokensÚevalÚkeywordsÚNAMErÚopmap)r(r<rBrFZitokenÚvaluerrrr6Zs<                 zParserGenerator.make_labelcCs<t|j ¡ƒ}| ¡x |D]}||jkr| |¡qWdS)N)r+r%r,r-rÚ calcfirst)r(r=r>rrrr'ˆs   zParserGenerator.addfirstsets)r>rc Cs|j|}d|j|<|d}i}i}x’|j ¡D]„\}}||jkr ||jkrl|j|}|dkrŒtd|ƒ‚n | |¡|j|}|dk sŒt‚| |¡|||<q0d||<|di||<q0Wi} xJ| ¡D]>\}} x4| D],} | | krøtd|| || | fƒ‚|| | <qÔWqÆW||j|<dS)Nrzrecursion for rule %rrzArule %s is ambiguous; %s is in the first sets of %s as well as %s)r%rr3r4Ú ValueErrorrUrEÚupdate) r(r>r@rAZtotalsetZ overlapcheckrBrCÚfsetZinverseZitsfirstÚsymbolrrrrUs6           zParserGenerator.calcfirstÚDFAStatec Cs´i}d}x–|jtjkržx|jtjkr.| ¡qW| tj¡}| tjd¡| ¡\}}| tj¡|  ||¡}t |ƒ}|  |¡t |ƒ}|||<|dkr |}q W|dk s¬t ‚||fS)Nú:) ÚtyperÚ ENDMARKERÚNEWLINEr#ÚexpectrRÚOPÚ parse_rhsÚmake_dfar0Ú simplify_dfarE) r(r%r&r>ÚaÚzr@ZoldlenZnewlenrrrr$±s$       zParserGenerator.parseÚNFAState)r;Úfinishrc st|tƒst‚t|tƒst‚ttttfdœ‡fdd„ }ttttfddœ‡fdd„ ‰t||ƒ|ƒg}x¢|D]š}i}x<|jD]2}x,|jD]"\}} |dk rˆˆ| | |i¡ƒqˆWq|WxRt |  ¡ƒD]B\}} x,|D]} | j| krÎPqÎWt| |ƒ} |  | ¡|  | |¡qÀWqlW|S)N)rArcsi}ˆ||ƒ|S)Nr)rAÚbase)Ú addclosurerrÚclosureÒs z)ParserGenerator.make_dfa..closure)rArhrcsLt|tƒst‚||krdSd||<x$|jD]\}}|dkr*ˆ||ƒq*WdS)Nr)rLrfrEr3)rArhrBrC)rirrri×sz,ParserGenerator.make_dfa..addclosure) rLrfrErrMrZÚnfasetr3Ú setdefaultr2r4r5Úaddarc) r(r;rgrjr9rAr3ZnfastaterBrCrkÚstr)rirrbÊs&       zParserGenerator.make_dfa)r>r;rgrc Cs¢td|ƒ|g}xŒt|ƒD]€\}}td|||kr4dp6dƒx^|jD]T\}}||kr^| |¡} nt|ƒ} | |¡|dkr†td| ƒqBtd|| fƒqBWqWdS)NzDump of NFA forz Statez(final)Úz -> %dz %s -> %d)ÚprintÚ enumerater3r7r0r5) r(r>r;rgÚtodor?rArBrCÚjrrrÚdump_nfañs   zParserGenerator.dump_nfa)r>r@rcCsltd|ƒx\t|ƒD]P\}}td||jr,dp.dƒx0t|j ¡ƒD]\}}td|| |¡fƒqBWqWdS)NzDump of DFA forz Statez(final)roz %s -> %d)rprqr8r2r3r4r7)r(r>r@r?rArBrCrrrÚdump_dfas  zParserGenerator.dump_dfa)r@rcCs~d}xt|rxd}xft|ƒD]Z\}}xPt|dt|ƒƒD]:}||}||kr4||=x|D]}| ||¡qTWd}Pq4WqWqWdS)NTFr)rqÚranger0Ú unifystate)r(r@Úchangesr?Zstate_irsZstate_jrArrrrcs zParserGenerator.simplify_dfa)rfrfcCs‚| ¡\}}|jdkr||fStƒ}tƒ}| |¡| |¡x6|jdkrt| ¡| ¡\}}| |¡| |¡q@W||fSdS)Nú|)Ú parse_altrTrfrmr#)r(rdreZaaÚzzrrrras       zParserGenerator.parse_rhscCsP| ¡\}}x:|jdks*|jtjtjfkrF| ¡\}}| |¡|}qW||fS)N)ú(ú[)Ú parse_itemrTr\rrRÚSTRINGrm)r(rdÚbr<Údrrrrz.s    zParserGenerator.parse_altcCsŽ|jdkr>| ¡| ¡\}}| tjd¡| |¡||fS| ¡\}}|j}|dkr`||fS| ¡| |¡|dkr‚||fS||fSdS)Nr}ú])ú+Ú*rƒ)rTr#rar_rr`rmÚ parse_atom)r(rdrerTrrrr~7s     zParserGenerator.parse_itemcCsŽ|jdkr4| ¡| ¡\}}| tjd¡||fS|jtjtjfkrpt ƒ}t ƒ}|  ||j¡| ¡||fS|  d|j|j¡dsŠt ‚dS)Nr|ú)z+expected (...) or NAME or STRING, got %s/%sF) rTr#rar_rr`r\rRrrfrmÚ raise_errorrE)r(rdrerrrr…Ks  zParserGenerator.parse_atom)r\rTrcCsD|j|ks|dk r2|j|kr2| d|||j|j¡|j}| ¡|S)Nzexpected %s/%s, got %s/%s)r\rTr‡r#)r(r\rTrrrr_^s zParserGenerator.expectcCsJt|jƒ}x"|dtjtjfkr,t|jƒ}q W|\|_|_|_|_|_ dS)Nr) rCrrÚCOMMENTÚNLr\rTÚbeginÚendÚline)r(Útuprrrr#gs zParserGenerator.gettoken)ÚmsgÚargsrc Gs^|r8y ||}Wn&d |gttt|ƒƒ¡}YnXt||j|jd|jd|jfƒ‚dS)Nú rr)Újoinr+ÚmapÚstrÚ SyntaxErrorrr‹rŒ)r(rŽrrrrr‡ns   zParserGenerator.raise_error)N)N)%rrrÚPathÚ__annotations__rr r rrr rMr)rrDr:r6r'rUrr r$rbrtrrurcrarzr~r…rr_r#r“rr‡rrrrr!s,    .!"'  rc@sLeZdZUeeeedfed<ddœdd„Zd deeddœdd„Z dS) rfr3N)rcCs g|_dS)N)r3)r(rrrr)zszNFAState.__init__)rCrBrcCs8|dkst|tƒst‚t|tƒs$t‚|j ||f¡dS)N)rLr“rErfr3r5)r(rCrBrrrrm}szNFAState.addarc)N) rrrr rr r r–r)rmrrrrrfws rfc@seZdZUeeefed<eed<eedfed<eeefeddœdd„Z deddœd d „Z dddd œd d „Z eedœdd„Z dZ eed<dS)rZrkr8r3N)rkÚfinalrcCsLt|tƒst‚ttt|ƒƒtƒs$t‚t|tƒs2t‚||_||k|_i|_dS)N) rLÚdictrErCÚiterrfrkr8r3)r(rkr—rrrr)ˆs  zDFAState.__init__)rCrBrcCs8t|tƒst‚||jkst‚t|tƒs*t‚||j|<dS)N)rLr“rEr3rZ)r(rCrBrrrrmszDFAState.addarc)ÚoldÚnewrcCs.x(|j ¡D]\}}||kr ||j|<q WdS)N)r3r4)r(ršr›rBrCrrrrw–szDFAState.unifystate)ÚotherrcCsft|tƒst‚|j|jkrdSt|jƒt|jƒkr6dSx*|j ¡D]\}}||j |¡k rBdSqBWdS)NFT)rLrZrEr8r0r3r4Úget)r(rœrBrCrrrÚ__eq__›s zDFAState.__eq__Ú__hash__)rrrrrfrr–Úboolr r)rmrwržrŸrrrrrZƒs rZú Grammar.txt)rrcCst|ƒ}| ¡S)N)rrD)rÚprrrÚgenerate_grammar¬sr£)r¡)rorrrÚtypingrrrr r r r r rrrrZblib2to3.pgen2Zblib2to3.pgen2.tokenizerÚosr“r•ZGrammarrÚobjectrrfrZr£rrrrÚs8   X )