B “äî\±iã@sÎddlmZddlmZddlZddlZddlZddlZddlm Z ddlm Z ddl m Z m Z e e¡ZGdd„deƒZGd d „d eƒZGd d „d eƒZGd d„deƒZGdd„deƒZGdd„deƒZdS)é)Útee)ÚsixN)ÚPaginationError)Úzip)Úset_value_from_jmespathÚ merge_dictsc@s8eZdZdZdd„Zdd„Zdd„Zdd „Zd d „Zd S) Ú TokenEncoderabEncodes dictionaries into opaque strings. This for the most part json dumps + base64 encoding, but also supports having bytes in the dictionary in addition to the types that json can handle by default. This is intended for use in encoding pagination tokens, which in some cases can be complex structures and / or contain bytes. c Cs`yt |¡}Wn:ttfk rH| |g¡\}}||d<t |¡}YnXt | d¡¡ d¡S)a.Encodes a dictionary to an opaque string. :type token: dict :param token: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. :rtype: str :returns: An opaque string Úboto_encoded_keyszutf-8) ÚjsonÚdumpsÚ TypeErrorÚUnicodeDecodeErrorÚ_encodeÚbase64Ú b64encodeÚencodeÚdecode)ÚselfÚtokenÚ json_stringZ encoded_tokenÚ encoded_keys©rú5/tmp/pip-build-uw_ogi45/botocore/botocore/paginate.pyr)s zTokenEncoder.encodecCsPt|tƒr| ||¡St|tƒr,| ||¡St|tjƒrD| ||¡S|gfSdS)z@Encode bytes in given data, keeping track of the path traversed.N)Ú isinstanceÚdictÚ _encode_dictÚlistÚ _encode_listrÚ binary_typeÚ _encode_bytes)rÚdataÚpathrrrrFs      zTokenEncoder._encodec CsTg}g}xBt|ƒD]6\}}||g}| ||¡\}} | |¡| | ¡qW||fS)z@Encode any bytes in a list, noting the index of what is encoded.)Ú enumeraterÚappendÚextend) rr r!Únew_dataÚencodedÚiÚvalueÚnew_pathÚ new_valueÚ new_encodedrrrrQs  zTokenEncoder._encode_listc CsRi}g}x@| ¡D]4\}}||g}| ||¡\}} |||<| | ¡qW||fS)z@Encode any bytes in a dict, noting the index of what is encoded.)Úitemsrr$) rr r!r%r&Úkeyr(r)r*r+rrrr\s zTokenEncoder._encode_dictcCst |¡ d¡|gfS)zBase64 encode a byte string.zutf-8)rrr)rr r!rrrrgszTokenEncoder._encode_bytesN) Ú__name__Ú __module__Ú __qualname__Ú__doc__rrrrrrrrrrs     rc@s0eZdZdZdd„Zdd„Zdd„Zdd „Zd S) Ú TokenDecoderz´Decodes token strings back into dictionaries. This performs the inverse operation to the TokenEncoder, accepting opaque strings and decoding them into a useable form. cCsHt | d¡¡ d¡}t |¡}| dd¡}|dkr8|S| ||¡SdS)adDecodes an opaque string to a dictionary. :type token: str :param token: A token string given by the botocore pagination interface. :rtype: dict :returns: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. zutf-8r N)rÚ b64decoderrr ÚloadsÚpopÚ_decode)rrrZ decoded_tokenrrrrrss   zTokenDecoder.decodecCs<x6|D].}| ||¡}t | d¡¡}| |||¡qW|S)z&Find each encoded value and decode it.zutf-8)Ú _path_getrr3rÚ _path_set)rrrr-r&Údecodedrrrr6Šs   zTokenDecoder._decodecCs|}x|D] }||}q W|S)z¨Return the nested data at the given path. For instance: data = {'foo': ['bar', 'baz']} path = ['foo', 0] ==> 'bar' r)rr r!ÚdÚsteprrrr7’s   zTokenDecoder._path_getcCs$| ||dd…¡}|||d<dS)zÖSet the value of a key in the given data. Example: data = {'foo': ['bar', 'baz']} path = ['foo', 1] value = 'bin' ==> data = {'foo': ['bar', 'bin']} Néÿÿÿÿ)r7)rr r!r(Ú containerrrrr8£s zTokenDecoder._path_setN)r.r/r0r1rr6r7r8rrrrr2ls r2c@seZdZdd„Zdd„ZdS)ÚPaginatorModelcCs|d|_dS)NZ pagination)Ú_paginator_config)rZpaginator_configrrrÚ__init__±szPaginatorModel.__init__cCs4y|j|}Wn tk r.td|ƒ‚YnX|S)Nz*Paginator for operation does not exist: %s)r?ÚKeyErrorÚ ValueError)rZoperation_nameZsingle_paginator_configrrrÚ get_paginator´s zPaginatorModel.get_paginatorN)r.r/r0r@rCrrrrr>°sr>c@s¾eZdZdd„Zedd„ƒZedd„ƒZejdd„ƒZedd „ƒZd d „Z d d „Z dd„Z dd„Z dd„Z dd„Zdd„Zdd„Zdd„Zdd„Zdd„Zd d!„Zd"d#„Zd$d%„Zd&d'„Zd(S))Ú PageIteratorc Csb||_||_||_||_||_||_||_| |_| |_| |_ d|_ ||_ i|_ t ƒ|_tƒ|_dS)N)Ú_methodÚ _input_tokenÚ _output_tokenÚ _more_resultsÚ _result_keysÚ _max_itemsÚ _limit_keyÚ_starting_tokenÚ _page_sizeÚ _op_kwargsÚ _resume_tokenÚ_non_aggregate_key_exprsÚ_non_aggregate_partrÚ_token_encoderr2Ú_token_decoder) rÚmethodÚ input_tokenÚ output_tokenÚ more_resultsÚ result_keysÚnon_aggregate_keysÚ limit_keyÚ max_itemsZstarting_tokenÚ page_sizeÚ op_kwargsrrrr@¾szPageIterator.__init__cCs|jS)N)rI)rrrrrXÑszPageIterator.result_keyscCs|jS)z&Token to specify to resume pagination.)rO)rrrrÚ resume_tokenÕszPageIterator.resume_tokencCsnt|tƒstd|ƒ‚d|kr0t|jdgƒ}n t|jƒ}t| ¡ƒ}||kr^|j |¡|_n td|ƒ‚dS)NzBad starting token: %sÚboto_truncate_amount) rrrBÚsortedrFÚkeysrRrrO)rr(Z token_keysÚ dict_keysrrrr^Ús    cCs|jS)N)rQ)rrrrÚnon_aggregate_partêszPageIterator.non_aggregate_partccsŠ|j}d}tdd„|jDƒƒ}|jdk r4| ¡d}d}d}|jd}d}| |¡x.| |¡}| |¡} |r˜|jdk rˆ|  | ||¡}d}|  | ¡nd}|  | ¡} | dkr²g} t | ƒ} d} |j dk rÖ|| |j } | dkrú| | || ||¡|VPqX|V|| 7}| | ¡}tdd„| ¡Dƒƒr,P|j dk rL||j krL||_P|dk rr||krrd|} t| d‚| ||¡|}qXWdS) Ncss|]}|dfVqdS)Nr)Ú.0r-rrrú ñsz(PageIterator.__iter__..rTFcss|]}|dkVqdS)Nr)rdÚtrrrre!sz*The same next token was received twice: %s)Úmessage)rNrrFrLÚ_parse_starting_tokenrXÚ_inject_starting_paramsÚ _make_requestÚ_extract_parsed_responseÚ_handle_first_requestÚ _record_non_aggregate_key_valuesÚsearchÚlenrJÚ_truncate_responseÚ_get_next_tokenÚallÚvaluesr^rÚ_inject_token_into_kwargs)rÚcurrent_kwargsZprevious_next_tokenÚ next_tokenZ total_itemsZ first_requestÚprimary_result_keyÚstarting_truncationÚresponseÚparsedZcurrent_responseZnum_current_responseÚtruncate_amountrgrrrÚ__iter__îsd                  zPageIterator.__iter__ccsLt |¡}x<|D]4}| |¡}t|tƒr>x|D] }|Vq.Wq|VqWdS)a…Applies a JMESPath expression to a paginator Each page of results is searched using the provided JMESPath expression. If the result is not a list, it is yielded directly. If the result is a list, each element in the result is yielded individually (essentially implementing a flatmap in which the JMESPath search is the mapping function). :type expression: str :param expression: JMESPath expression to apply to each page. :return: Returns an iterator that yields the individual elements of applying a JMESPath expression to each page of results. N)ÚjmespathÚcompilernrr)rÚ expressionZcompiledÚpageÚresultsÚelementrrrrn1s      zPageIterator.searchcCs |jf|ŽS)N)rE)rrurrrrjKszPageIterator._make_requestcCs|S)Nr)rryrrrrkNsz%PageIterator._extract_parsed_responsecCs6i}x&|jD]}| |¡}t||j|ƒq W||_dS)N)rPrnrrrQ)rryrYrÚresultrrrrmQs   z-PageIterator._record_non_aggregate_key_valuescCs<|jdk r"| ¡d}| ||¡|jdk r8|j||j<dS)Nr)rLrhrtrMrK)rr]rvrrrriZs     z$PageIterator._inject_starting_paramscCsBx<| ¡D]0\}}|dk r,|dkr,|||<q ||kr ||=q WdS)NÚNone)r,)rr]rvÚnamerrrrrtgs  z&PageIterator._inject_token_into_kwargsc Cs¶| ¡d}| |¡}t|ttjfƒr4||d…}nd}t||j|ƒxj|jD]`}||kr\qN| |¡}t|tƒrvg}n*t|tjƒrˆd}nt|t t fƒrœd}nd}t||j|ƒqNW|S)NéÚr) rhrnrrrÚ string_typesrrrXÚintÚfloat) rrzrwrxZall_datar rÚsampleZ empty_valuerrrrlns,      z"PageIterator._handle_first_requestc CsR| |¡}|dkrg}t|ƒ|}|d|…}t||j|ƒ|||d<||_dS)Nr_)rnrorrr^) rrzrwr{rxrvÚoriginalZamount_to_keepZ truncatedrrrrps    zPageIterator._truncate_responsecCs^|jdk r|j |¡siSi}x:t|j|jƒD](\}}| |¡}|rN|||<q.d||<q.W|S)N)rHrnrrGrF)rrzZ next_tokensrVZ input_keyrvrrrrqªs     zPageIterator._get_next_tokencCs&t|t|jƒƒ}dd„t||jƒDƒS)NcSsg|]\}}t||ƒ‘qSr)ÚResultKeyIterator)rdr'Ú result_keyrrrú ¼sz1PageIterator.result_key_iters..)rrorXr)rZ teed_resultsrrrÚresult_key_itersºszPageIterator.result_key_iterscCsØi}x®|D]¦}|}t|tƒr0t|ƒdkr0|d}x~|jD]t}| |¡}|dkrPq8| |¡}|dkrrt||j|ƒq8t|tƒrˆ| |¡q8t|t t t j fƒr8t||j||ƒq8Wq Wt ||jƒ|jdk rÔ|j|d<|S)Nér†Z NextToken)rÚtuplerorXrnrrrr$r‰rŠrrˆrrcr^)rZcomplete_resultryr€Zresult_expressionZ result_valueZexisting_valuerrrÚbuild_full_result¿s2         zPageIterator.build_full_resultc Csn|jdkrdS|j}y,|j |¡}d}d|kr>| d¡}|d=Wn$ttfk rd| ¡\}}YnX||fS)Nrr_)rLrSrÚgetrBr Ú _parse_starting_token_deprecated)rrvÚindexrrrrhìs    z"PageIterator._parse_starting_tokencCs¶t d|j¡|jdkrdS|j d¡}g}d}t|ƒt|jƒdkr|yt| ¡ƒ}Wn"tk rztd|jƒ‚YnXx*|D]"}|dkrš|  d¡q‚|  |¡q‚W|  |¡|fS)z| This handles parsing of old style starting tokens, and attempts to coerce them into the new style. zCAttempting to fall back to old starting token parser. For token: %sNZ___rr†zBad starting token: %sr„) ÚlogÚdebugrLÚsplitrorFr‰r5rBr#Ú"_convert_deprecated_starting_token)rÚpartsrvr–Úpartrrrr•üs$     z-PageIterator._parse_starting_token_deprecatedcCslt|ƒ}t|jƒ}||kr*td|jƒ‚n2||kr\t d¡xt||ƒD]}| d¡qJWtt |j|ƒƒS)zb This attempts to convert a deprecated starting token into the new style. zBad starting token: %szaOld format starting token does not contain all input tokens. Setting the rest, in order, as None.N) rorFrBrLr—r˜Úranger#rr)rZdeprecated_tokenZlen_deprecated_tokenZlen_input_tokenr'rrrršs  z/PageIterator._convert_deprecated_starting_tokenN)r.r/r0r@ÚpropertyrXr^Úsetterrcr|rnrjrkrmrirtrlrprqrr“rhr•ršrrrrrD½s(   C  !-rDc@sdeZdZeZdd„Zedd„ƒZdd„Zdd„Z d d „Z d d „Z d d„Z dd„Z dd„Zdd„ZdS)Ú PaginatorcCsj||_||_||_| |j¡|_| |j¡|_| |j¡|_|  |j¡|_ |  |j¡|_ |  |j¡|_dS)N)Ú_modelrEÚ_pagination_cfgÚ_get_output_tokensrGÚ_get_input_tokensrFÚ_get_more_results_tokenrHÚ_get_non_aggregate_keysÚ_non_aggregate_keysÚ_get_result_keysrIÚ_get_limit_keyrK)rrTÚpagination_configÚmodelrrrr@*s zPaginator.__init__cCs|jS)N)rI)rrrrrX6szPaginator.result_keyscCs.g}x$| dg¡D]}| t |¡¡qW|S)NrY)r”r#r}r~)rÚconfigrar-rrrr¦:sz!Paginator._get_non_aggregate_keyscCs>g}|d}t|tƒs|g}x|D]}| t |¡¡q"W|S)NrV)rrr#r}r~)rr¬ÚoutputrVrrrr£@s  zPaginator._get_output_tokenscCs|jd}t|tƒs|g}|S)NrU)r¢rr)rr¬rUrrrr¤Is  zPaginator._get_input_tokenscCs | d¡}|dk rt |¡SdS)NrW)r”r}r~)rr¬rWrrrr¥Os z!Paginator._get_more_results_tokencCs8| d¡}|dk r4t|tƒs"|g}dd„|Dƒ}|SdS)NrŽcSsg|]}t |¡‘qSr)r}r~)rdZrkrrrrYsz.Paginator._get_result_keys..)r”rr)rr¬rŽrrrr¨Ts   zPaginator._get_result_keyscCs | d¡S)NrZ)r”)rr¬rrrr©\szPaginator._get_limit_keyc KsB| |¡}| |j|j|j|j|j|j|j|d|d|d|¡ S)z»Create paginator object for an operation. This returns an iterable object. Iterating over this object will yield a single page of a response at a time. ÚMaxItemsÚ StartingTokenÚPageSize) Ú_extract_paging_paramsÚPAGE_ITERATOR_CLSrErFrGrHrIr§rK)rÚkwargsZ page_paramsrrrÚpaginate_s zPaginator.paginatecCs¢| di¡}| dd¡}|dk r(t|ƒ}| dd¡}|dk rŽ|jdkrPtdd‚|jjj}| |j¡}|jdkr†t |t j ƒsŽt |ƒ}nt|ƒ}|| dd¡|dœS) NZPaginationConfigr®r°zTPageSize parameter is not supported for the pagination interface for this operation.)rgÚstringr¯)r®r¯r°) r5r”r‰rKrr¡Z input_shapeÚmembersZ type_namerrrˆÚstr)rr³rªr[r\Z input_membersZlimit_key_shaperrrr±rs$          z Paginator._extract_paging_paramsN)r.r/r0rDr²r@ržrXr¦r£r¤r¥r¨r©r´r±rrrrr 's  r c@s eZdZdZdd„Zdd„ZdS)raŒIterates over the results of paginated responses. Each iterator is associated with a single result key. Iterating over this object will give you each element in the result key list. :param pages_iterator: An iterator that will give you pages of results (a ``PageIterator`` class). :param result_key: The JMESPath expression representing the result key. cCs||_||_dS)N)Ú_pages_iteratorrŽ)rZpages_iteratorrŽrrrr@™szResultKeyIterator.__init__ccs@x:|jD]0}|j |¡}|dkr$g}x|D] }|Vq*WqWdS)N)r¸rŽrn)rr€rrƒrrrr|s    zResultKeyIterator.__iter__N)r.r/r0r1r@r|rrrrr‹s r)Ú itertoolsrZbotocore.compatrr}r rÚloggingZbotocore.exceptionsrrZbotocore.utilsrrÚ getLoggerr.r—Úobjectrr2r>rDr rrrrrÚs"     ND ld