B ݁[i@sddlmZddlmZddlZddlZddlZddlZddlm Z ddlm Z ddl m Z m Z eeZGdddeZGd d d eZGd d d eZGd ddeZGdddeZGdddeZdS))tee)sixN)PaginationError)zip)set_value_from_jmespath merge_dictsc@s8eZdZdZddZddZddZdd Zd d Zd S) TokenEncoderabEncodes dictionaries into opaque strings. This for the most part json dumps + base64 encoding, but also supports having bytes in the dictionary in addition to the types that json can handle by default. This is intended for use in encoding pagination tokens, which in some cases can be complex structures and / or contain bytes. c Cs`yt|}Wn:ttfk rH||g\}}||d<t|}YnXt|ddS)a.Encodes a dictionary to an opaque string. :type token: dict :param token: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. :rtype: str :returns: An opaque string boto_encoded_keyszutf-8) jsondumps TypeErrorUnicodeDecodeError_encodebase64 b64encodeencodedecode)selftoken json_stringZ encoded_token encoded_keysrR/Users/kashii/Desktop/Projects/Workshops/worker-safety/lambda/botocore/paginate.pyr)s zTokenEncoder.encodecCsPt|tr|||St|tr,|||St|tjrD|||S|gfSdS)z@Encode bytes in given data, keeping track of the path traversed.N) isinstancedict _encode_dictlist _encode_listr binary_type _encode_bytes)rdatapathrrrrFs      zTokenEncoder._encodec CsTg}g}xBt|D]6\}}||g}|||\}} |||| qW||fS)z@Encode any bytes in a list, noting the index of what is encoded.) enumeraterappendextend) rr r!new_dataencodedivaluenew_path new_value new_encodedrrrrQs  zTokenEncoder._encode_listc CsRi}g}x@|D]4\}}||g}|||\}} |||<|| qW||fS)z@Encode any bytes in a dict, noting the index of what is encoded.)itemsrr$) rr r!r%r&keyr(r)r*r+rrrr\s zTokenEncoder._encode_dictcCst|d|gfS)zBase64 encode a byte string.zutf-8)rrr)rr r!rrrrgszTokenEncoder._encode_bytesN) __name__ __module__ __qualname____doc__rrrrrrrrrrs     rc@s0eZdZdZddZddZddZdd Zd S) TokenDecoderzDecodes token strings back into dictionaries. This performs the inverse operation to the TokenEncoder, accepting opaque strings and decoding them into a useable form. cCsHt|dd}t|}|dd}|dkr8|S|||SdS)adDecodes an opaque string to a dictionary. :type token: str :param token: A token string given by the botocore pagination interface. :rtype: dict :returns: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. zutf-8r N)r b64decoderrr loadspop_decode)rrrZ decoded_tokenrrrrrss   zTokenDecoder.decodecCs<x6|D].}|||}t|d}||||qW|S)z&Find each encoded value and decode it.zutf-8) _path_getrr3r _path_set)rrrr-r&decodedrrrr6s   zTokenDecoder._decodecCs|}x|D] }||}q W|S)zReturn the nested data at the given path. For instance: data = {'foo': ['bar', 'baz']} path = ['foo', 0] ==> 'bar' r)rr r!dsteprrrr7s   zTokenDecoder._path_getcCs$|||dd}|||d<dS)zSet the value of a key in the given data. Example: data = {'foo': ['bar', 'baz']} path = ['foo', 1] value = 'bin' ==> data = {'foo': ['bar', 'bin']} N)r7)rr r!r(Z containerrrrr8s zTokenDecoder._path_setN)r.r/r0r1rr6r7r8rrrrr2ls r2c@seZdZddZddZdS)PaginatorModelcCs|d|_dS)N pagination)_paginator_config)rpaginator_configrrr__init__szPaginatorModel.__init__cCs4y|j|}Wn tk r.td|YnX|S)Nz*Paginator for operation does not exist: %s)r?KeyError ValueError)roperation_nameZsingle_paginator_configrrr get_paginators zPaginatorModel.get_paginatorN)r.r/r0rArErrrrr=sr=c@seZdZddZeddZeddZejddZedd Zd d Z d d Z ddZ ddZ ddZ ddZddZddZddZddZddZd d!Zd"d#Zd$d%Zd&d'Zd(S)) PageIteratorc Csb||_||_||_||_||_||_||_| |_| |_| |_ d|_ ||_ i|_ t |_t|_dS)N)_method _input_token _output_token _more_results _result_keys _max_items _limit_key_starting_token _page_size _op_kwargs _resume_token_non_aggregate_key_exprs_non_aggregate_partr_token_encoderr2_token_decoder) rmethod input_token output_token more_results result_keysnon_aggregate_keys limit_key max_itemsZstarting_token page_size op_kwargsrrrrAszPageIterator.__init__cCs|jS)N)rK)rrrrrZszPageIterator.result_keyscCs|jS)z&Token to specify to resume pagination.)rQ)rrrr resume_tokenszPageIterator.resume_tokencCsnt|tstd|d|kr0t|jdg}n t|j}t|}||kr^|j||_n td|dS)NzBad starting token: %sboto_truncate_amount) rrrCsortedrHkeysrTrrQ)rr(Z token_keys dict_keysrrrr`s    cCs|jS)N)rS)rrrrnon_aggregate_partszPageIterator.non_aggregate_partccs|j}d}tdd|jD}|jdk r4|d}d}d}|jd}d}||x.||}||} |r|jdk r| | ||}d}| | nd}| | } | dkrg} t | } d} |j dk r|| |j } | dkr|| || |||VPqX|V|| 7}|| }tdd|Dr,P|j dk rL||j krL||_P|dk rr||krrd|} t| d||||}qXWdS) Ncss|]}|dfVqdS)Nr).0r-rrr sz(PageIterator.__iter__..rTFcss|]}|dkVqdS)Nr)rftrrrrg!sz*The same next token was received twice: %s)message)rPrrHrN_parse_starting_tokenrZ_inject_starting_params _make_request_extract_parsed_response_handle_first_request _record_non_aggregate_key_valuessearchlenrL_truncate_response_get_next_tokenallvaluesr`r_inject_token_into_kwargs)rcurrent_kwargsZprevious_next_token next_tokenZ total_itemsZ first_requestprimary_result_keystarting_truncationresponseparsedZcurrent_responseZnum_current_responsetruncate_amountrirrr__iter__sd                  zPageIterator.__iter__ccsLt|}x<|D]4}||}t|tr>x|D] }|Vq.Wq|VqWdS)aApplies a JMESPath expression to a paginator Each page of results is searched using the provided JMESPath expression. If the result is not a list, it is yielded directly. If the result is a list, each element in the result is yielded individually (essentially implementing a flatmap in which the JMESPath search is the mapping function). :type expression: str :param expression: JMESPath expression to apply to each page. :return: Returns an iterator that yields the individual elements of applying a JMESPath expression to each page of results. N)jmespathcompilerprr)r expressionZcompiledpageresultselementrrrrp1s      zPageIterator.searchcCs |jf|S)N)rG)rrwrrrrlKszPageIterator._make_requestcCs|S)Nr)rr{rrrrmNsz%PageIterator._extract_parsed_responsecCs6i}x&|jD]}||}t||j|q W||_dS)N)rRrprrrS)rr{r[rresultrrrroQs   z-PageIterator._record_non_aggregate_key_valuescCs<|jdk r"|d}||||jdk r8|j||j<dS)Nr)rNrjrvrOrM)rr_rxrrrrkZs     z$PageIterator._inject_starting_paramscCsBx<|D]0\}}|dk r,|dkr,|||<q ||kr ||=q WdS)NNone)r,)rr_rxnamerrrrrvgs  z&PageIterator._inject_token_into_kwargsc Cs|d}||}t|ttjfr4||d}nd}t||j|xj|jD]`}||kr\qN||}t|trvg}n*t|tjrd}nt|t t frd}nd}t||j|qNW|S)Nr) rjrprrr string_typesrrrZintfloat) rr|ryrzZall_datar rsampleZ empty_valuerrrrnns,      z"PageIterator._handle_first_requestc CsR||}|dkrg}t||}|d|}t||j||||d<||_dS)Nra)rprqrrr`) rr|ryr}rzrxoriginalZamount_to_keepZ truncatedrrrrrs    zPageIterator._truncate_responsecCs^|jdk r|j|siSi}x:t|j|jD](\}}||}|rN|||<q.d||<q.W|S)N)rJrprrIrH)rr|Z next_tokensrXZ input_keyrxrrrrss     zPageIterator._get_next_tokencCs&t|t|j}ddt||jDS)NcSsg|]\}}t||qSr)ResultKeyIterator)rfr' result_keyrrr sz1PageIterator.result_key_iters..)rrqrZr)rZ teed_resultsrrrresult_key_itersszPageIterator.result_key_iterscCsi}x|D]}|}t|tr0t|dkr0|d}x~|jD]t}||}|dkrPq8||}|dkrrt||j|q8t|tr||q8t|t t t j fr8t||j||q8Wq Wt ||j|jdk r|j|d<|S)Nr NextToken)rtuplerqrZrprrrr$rrrrrrer`)rZcomplete_resultr{rZresult_expressionZ result_valueZexisting_valuerrrbuild_full_results2         zPageIterator.build_full_resultc Csn|jdkrdS|j}y,|j|}d}d|kr>|d}|d=Wn$ttfk rd|\}}YnX||fS)Nrra)rNrUrgetrCr _parse_starting_token_deprecated)rrxindexrrrrjs    z"PageIterator._parse_starting_tokencCstd|j|jdkrdS|jd}g}d}t|t|jdkr|yt|}Wn"tk rztd|jYnXx*|D]"}|dkr| dq| |qW| ||fS)z| This handles parsing of old style starting tokens, and attempts to coerce them into the new style. zCAttempting to fall back to old starting token parser. For token: %sNZ___rrzBad starting token: %sr) logdebugrNsplitrqrHrr5rCr#"_convert_deprecated_starting_token)rpartsrxrpartrrrrs$     z-PageIterator._parse_starting_token_deprecatedcCslt|}t|j}||kr*td|jn2||kr\tdxt||D]}|dqJWtt |j|S)zb This attempts to convert a deprecated starting token into the new style. zBad starting token: %szaOld format starting token does not contain all input tokens. Setting the rest, in order, as None.N) rqrHrCrNrrranger#rr)rZdeprecated_tokenZlen_deprecated_tokenZlen_input_tokenr'rrrrs  z/PageIterator._convert_deprecated_starting_tokenN)r.r/r0rApropertyrZr`setterrer~rprlrmrorkrvrnrrrsrrrjrrrrrrrFs(   C  !-rFc@sdeZdZeZddZeddZddZddZ d d Z d d Z d dZ ddZ ddZddZdS) PaginatorcCsj||_||_||_||j|_||j|_||j|_| |j|_ | |j|_ | |j|_dS)N)_modelrG_pagination_cfg_get_output_tokensrI_get_input_tokensrH_get_more_results_tokenrJ_get_non_aggregate_keys_non_aggregate_keys_get_result_keysrK_get_limit_keyrM)rrVpagination_configmodelrrrrA*s zPaginator.__init__cCs|jS)N)rK)rrrrrZ6szPaginator.result_keyscCs.g}x$|dgD]}|t|qW|S)Nr[)rr#rr)rconfigrcr-rrrr:sz!Paginator._get_non_aggregate_keyscCs>g}|d}t|ts|g}x|D]}|t|q"W|S)NrX)rrr#rr)rroutputrXrrrr@s  zPaginator._get_output_tokenscCs|jd}t|ts|g}|S)NrW)rrr)rrrWrrrrIs  zPaginator._get_input_tokenscCs |d}|dk rt|SdS)NrY)rrr)rrrYrrrrOs z!Paginator._get_more_results_tokencCs8|d}|dk r4t|ts"|g}dd|D}|SdS)NrcSsg|]}t|qSr)rr)rfZrkrrrrYsz.Paginator._get_result_keys..)rrr)rrrrrrrTs   zPaginator._get_result_keyscCs |dS)Nr\)r)rrrrrr\szPaginator._get_limit_keyc KsB||}||j|j|j|j|j|j|j|d|d|d| S)zCreate paginator object for an operation. This returns an iterable object. Iterating over this object will yield a single page of a response at a time. MaxItems StartingTokenPageSize) _extract_paging_paramsPAGE_ITERATOR_CLSrGrHrIrJrKrrM)rkwargsZ page_paramsrrrpaginate_s zPaginator.paginatecCs|di}|dd}|dk r(t|}|dd}|dk r|jdkrPtdd|jjj}||j}|jdkrt |t j st |}nt|}||dd|dS) NPaginationConfigrrzTPageSize parameter is not supported for the pagination interface for this operation.)ristringr)rrr) r5rrrMrr input_shapemembers type_namerrrstr)rrrr]r^ input_membersZlimit_key_shaperrrrrs$          z Paginator._extract_paging_paramsN)r.r/r0rFrrArrZrrrrrrrrrrrrr's  rc@s eZdZdZddZddZdS)raIterates over the results of paginated responses. Each iterator is associated with a single result key. Iterating over this object will give you each element in the result key list. :param pages_iterator: An iterator that will give you pages of results (a ``PageIterator`` class). :param result_key: The JMESPath expression representing the result key. cCs||_||_dS)N)_pages_iteratorr)rZpages_iteratorrrrrrAszResultKeyIterator.__init__ccs@x:|jD]0}|j|}|dkr$g}x|D] }|Vq*WqWdS)N)rrrp)rrrrrrrr~s    zResultKeyIterator.__iter__N)r.r/r0r1rAr~rrrrrs r) itertoolsrbotocore.compatrrr rloggingbotocore.exceptionsrrbotocore.utilsrr getLoggerr.robjectrr2r=rFrrrrrrs"     ND ld