o ?ck@sddlZddlZddlZddlmZddlZddlmZddlm Z m Z e e Z GdddZGdddZGd d d ZGd d d ZGd ddZGdddZdS)N)tee)PaginationError) merge_dictsset_value_from_jmespathc@s8eZdZdZddZddZddZdd Zd d Zd S) TokenEncoderabEncodes dictionaries into opaque strings. This for the most part json dumps + base64 encoding, but also supports having bytes in the dictionary in addition to the types that json can handle by default. This is intended for use in encoding pagination tokens, which in some cases can be complex structures and / or contain bytes. c Cs^zt|}Wnttfy#||g\}}||d<t|}Ynwt|ddS)a.Encodes a dictionary to an opaque string. :type token: dict :param token: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. :rtype: str :returns: An opaque string boto_encoded_keysutf-8) jsondumps TypeErrorUnicodeDecodeError_encodebase64 b64encodeencodedecode)selftoken json_stringZ encoded_token encoded_keysrl/private/var/folders/cw/wlscbxl13mj6wd668h7l9g9sllkg5j/T/pip-target-b31awkwq/lib/python/botocore/paginate.pyr&s  zTokenEncoder.encodecCsJt|tr |||St|tr|||St|tr!|||S|gfS)z@Encode bytes in given data, keeping track of the path traversed.) isinstancedict _encode_dictlist _encode_listbytes _encode_bytesrdatapathrrrr Cs      zTokenEncoder._encodec CsPg}g}t|D]\}}||g}|||\}} |||| q||fS)z@Encode any bytes in a list, noting the index of what is encoded.) enumerater appendextend) rr r!new_dataencodedivaluenew_path new_value new_encodedrrrrNs   zTokenEncoder._encode_listc CsNi}g}|D]\}}||g}|||\}} |||<|| q||fS)z@Encode any bytes in a dict, noting the index of what is encoded.)itemsr r$) rr r!r%r&keyr(r)r*r+rrrrYs  zTokenEncoder._encode_dictcCst|d|gfS)zBase64 encode a byte string.r)rrrrrrrrdszTokenEncoder._encode_bytesN) __name__ __module__ __qualname____doc__rr rrrrrrrrs   rc@s0eZdZdZddZddZddZdd Zd S) TokenDecoderzDecodes token strings back into dictionaries. This performs the inverse operation to the TokenEncoder, accepting opaque strings and decoding them into a useable form. cCsDt|dd}t|}|dd}|dur|S|||S)adDecodes an opaque string to a dictionary. :type token: str :param token: A token string given by the botocore pagination interface. :rtype: dict :returns: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. rrN)r b64decoderrr loadspop_decode)rrrZ decoded_tokenrrrrrps    zTokenDecoder.decodecCs8|D]}|||}t|d}||||q|S)z&Find each encoded value and decode it.r) _path_getrr3r _path_set)rrrr-r&decodedrrrr6s  zTokenDecoder._decodecCs|}|D]}||}q|S)zReturn the nested data at the given path. For instance: data = {'foo': ['bar', 'baz']} path = ['foo', 0] ==> 'bar' r)rr r!dsteprrrr7s  zTokenDecoder._path_getcCs$|||dd}|||d<dS)zSet the value of a key in the given data. Example: data = {'foo': ['bar', 'baz']} path = ['foo', 1] value = 'bin' ==> data = {'foo': ['bar', 'bin']} N)r7)rr r!r( containerrrrr8s zTokenDecoder._path_setN)r.r/r0r1rr6r7r8rrrrr2is  r2c@seZdZddZddZdS)PaginatorModelcCs|d|_dS)NZ pagination)_paginator_config)rZpaginator_configrrr__init__szPaginatorModel.__init__cCs,z|j|}W|Stytd|w)Nz*Paginator for operation does not exist: %s)r?KeyError ValueError)rZoperation_nameZsingle_paginator_configrrr get_paginators  zPaginatorModel.get_paginatorN)r.r/r0r@rCrrrrr>s r>c@seZdZdZddZeddZeddZejddZed d Z d d Z d dZ ddZ ddZ ddZddZddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)S)* PageIteratorzAn iterable object to pagiante API results. Please note it is NOT a python iterator. Use ``iter`` to wrap this as a generator. c Csb||_||_||_||_||_||_||_| |_| |_| |_ d|_ ||_ i|_ t |_t|_dSN)_method _input_token _output_token _more_results _result_keys _max_items _limit_key_starting_token _page_size _op_kwargs _resume_token_non_aggregate_key_exprs_non_aggregate_partr_token_encoderr2_token_decoder) rmethod input_token output_token more_results result_keysnon_aggregate_keys limit_key max_itemsZstarting_token page_size op_kwargsrrrr@s zPageIterator.__init__cC|jSrErJrrrrrYzPageIterator.result_keyscCr_)z&Token to specify to resume pagination.)rPrarrr resume_tokenszPageIterator.resume_tokencCslt|ts td|d|vrt|jdg}nt|j}t|}||kr0|j||_dStd|)NBad starting token: %sboto_truncate_amount) rrrBsortedrGkeysrSrrP)rr(Z token_keys dict_keysrrrrcs     cCr_rE)rRrarrrnon_aggregate_partrbzPageIterator.non_aggregate_partccs||j}d}dd|jD}|jdur|d}d}d}|jd}d}|| ||}||} |rJ|jdurB|| ||}d}| | nd}| | } | durWg} t | } d} |j duri|| |j } | dkr{| | || |||VdS|V|| 7}|| }tdd|DrdS|j dur||j kr||_dS|dur||krd|} t| d ||||}q*) NcSsi|]}|dqSrEr).0r-rrr sz)PageIterator.__iter__..rTFcss|]}|duVqdSrEr)rjtrrr 5sz(PageIterator.__iter__..z(The same next token was received twice: message)rOrGrM_parse_starting_tokenrY_inject_starting_params _make_request_extract_parsed_response_handle_first_request _record_non_aggregate_key_valuessearchlenrK_truncate_response_get_next_tokenallvaluesrcr_inject_token_into_kwargs)rcurrent_kwargsZprevious_next_token next_tokenZ total_itemsZ first_requestprimary_result_keystarting_truncationresponseparsedZcurrent_responseZnum_current_responsetruncate_amountrorrr__iter__s~              zPageIterator.__iter__ccs@t|}|D]}||}t|tr|EdHq|VqdS)aApplies a JMESPath expression to a paginator Each page of results is searched using the provided JMESPath expression. If the result is not a list, it is yielded directly. If the result is a list, each element in the result is yielded individually (essentially implementing a flatmap in which the JMESPath search is the mapping function). :type expression: str :param expression: JMESPath expression to apply to each page. :return: Returns an iterator that yields the individual elements of applying a JMESPath expression to each page of results. N)jmespathcompilervrr)r expressioncompiledpageresultsrrrrvKs    zPageIterator.searchcCs|jdi|S)Nr)rF)rr}rrrrrdszPageIterator._make_requestcCs|SrEr)rrrrrrsgsz%PageIterator._extract_parsed_responsecCs2i}|jD]}||}t||j|q||_dSrE)rQrvrrrR)rrrZrresultrrrrujs   z-PageIterator._record_non_aggregate_key_valuescCs@|jdur|d}||||jdur|j||j<dSdS)Nr)rMrpr|rNrL)rr^r~rrrrqss    z$PageIterator._inject_starting_paramscCs>|D]\}}|dur|dkr|||<q||vr||=qdS)NNone)r,)rr^r~namerrrrr|s z&PageIterator._inject_token_into_kwargsc Cs|d}||}t|ttfr||d}nd}t||j||jD]/}||kr,q%||}t|tr9g}nt|trAd}n t|tt frKd}nd}t||j|q%|S)Nr) rprvrrstrrrrYintfloat) rrrrZall_datar rsampleZ empty_valuerrrrts&      z"PageIterator._handle_first_requestc CsR||}|dur g}t||}|d|}t||j||||d<||_dS)Nre)rvrwrrrc) rrrrrr~originalZamount_to_keep truncatedrrrrxs    zPageIterator._truncate_responsecCsZ|jdur |j|s iSi}t|j|jD]\}}||}|r&|||<qd||<q|SrE)rIrvziprHrG)rrZ next_tokensrWZ input_keyr~rrrrys     zPageIterator._get_next_tokencCs&t|t|j}ddt||jDS)NcSsg|] \}}t||qSr)ResultKeyIterator)rjr' result_keyrrr sz1PageIterator.result_key_iters..)rrwrYr)rZ teed_resultsrrrresult_key_iterss zPageIterator.result_key_iterscCsi}|D]P}|}t|trt|dkr|d}|jD]9}||}|dur&q||}|dur7t||j|qt|trB||qt|t t t frSt||j||qqt ||j |jdure|j|d<|S)NrZ NextToken)rtuplerwrYrvrrrr$rrrrrirc)rZcomplete_resultrrZresult_expressionZ result_valueZexisting_valuerrrbuild_full_results>        zPageIterator.build_full_resultc Csz|jdurdS|j}z|j|}d}d|vr$|d}|d=W||fSW||fSttfy<|\}}Y||fSw)Nrre)rMrTrgetrBr _parse_starting_token_deprecated)rr~indexrrrrps   z"PageIterator._parse_starting_tokencCstd|j|jdurdS|jd}g}d}t|t|jdkr:zt|}Wn ty9|jg}Ynw|D]}|dkrH| dq<| |q<| ||fS)z| This handles parsing of old style starting tokens, and attempts to coerce them into the new style. zCAttempting to fall back to old starting token parser. For token: %sNZ___rrr) logdebugrMsplitrwrGrr5rBr#"_convert_deprecated_starting_token)rpartsr~rpartrrrrs*      z-PageIterator._parse_starting_token_deprecatedcCsft|}t|j}||krtd|j||kr+tdt||D]}|dq#tt |j|S)zb This attempts to convert a deprecated starting token into the new style. rdzaOld format starting token does not contain all input tokens. Setting the rest, in order, as None.N) rwrGrBrMrrranger#rr)rZdeprecated_tokenZlen_deprecated_tokenZlen_input_tokenr'rrrr<s  z/PageIterator._convert_deprecated_starting_tokenN)r.r/r0r1r@propertyrYrcsetterrirrvrrrsrurqr|rtrxryrrrprrrrrrrDs4    O   1 rDc@sdeZdZeZddZeddZddZddZ d d Z d d Z d dZ ddZ ddZddZdS) PaginatorcCsj||_||_||_||j|_||j|_||j|_| |j|_ | |j|_ | |j|_dSrE)_modelrF_pagination_cfg_get_output_tokensrH_get_input_tokensrG_get_more_results_tokenrI_get_non_aggregate_keys_non_aggregate_keys_get_result_keysrJ_get_limit_keyrL)rrUpagination_configmodelrrrr@RszPaginator.__init__cCr_rEr`rarrrrY_rbzPaginator.result_keyscCs*g}|dgD] }|t|q|S)NrZ)rr#rr)rconfigrgr-rrrrcsz!Paginator._get_non_aggregate_keyscCs:g}|d}t|ts|g}|D] }|t|q|S)NrW)rrr#rr)rroutputrWrrrris zPaginator._get_output_tokenscCs|jd}t|ts |g}|S)NrV)rrr)rrrVrrrrrs  zPaginator._get_input_tokenscCs |d}|durt|SdS)NrX)rrr)rrrXrrrrxs  z!Paginator._get_more_results_tokencCs8|d}|durt|ts|g}dd|D}|SdS)NrcSsg|]}t|qSr)rr)rjZrkrrrrsz.Paginator._get_result_keys..)rrr)rrrrrrr}s  zPaginator._get_result_keyscCs |dS)Nr[)r)rrrrrrs zPaginator._get_limit_keyc KsB||}||j|j|j|j|j|j|j|d|d|d| S)zCreate paginator object for an operation. This returns an iterable object. Iterating over this object will yield a single page of a response at a time. MaxItems StartingTokenPageSize) _extract_paging_paramsPAGE_ITERATOR_CLSrFrGrHrIrJrrL)rkwargsZ page_paramsrrrpaginates zPaginator.paginatecCs|di}|dd}|durt|}|dd}|durF|jdur(tdd|jjj}||j}|jdkrBt |t sAt |}nt|}||dd|dS) NZPaginationConfigrrzTPageSize parameter is not supported for the pagination interface for this operation.rnstringr)rrr) r5rrrLrrZ input_shapemembers type_namerr)rrrr\r]Z input_membersZlimit_key_shaperrrrs*         z Paginator._extract_paging_paramsN)r.r/r0rDrr@rrYrrrrrrrrrrrrrOs   rc@s eZdZdZddZddZdS)raIterates over the results of paginated responses. Each iterator is associated with a single result key. Iterating over this object will give you each element in the result key list. :param pages_iterator: An iterator that will give you pages of results (a ``PageIterator`` class). :param result_key: The JMESPath expression representing the result key. cCs||_||_dSrE)_pages_iteratorr)rZpages_iteratorrrrrr@s zResultKeyIterator.__init__ccs4|jD]}|j|}|durg}|EdHqdSrE)rrrv)rrrrrrrs   zResultKeyIterator.__iter__N)r.r/r0r1r@rrrrrrs r)rr logging itertoolsrrZbotocore.exceptionsrZbotocore.utilsrr getLoggerr.rrr2r>rDrrrrrrs"    NDj