3 B\@s~dZddlZddlZddlZddlZddlZddlmZm Z ddl m Z m Z ddl mZmZmZmZejeZeZGdddeZdd Zd d ZGd d d eZGdddeZGdddeZGdddeZGdddeZGdddeZ GdddeZ!Gddde!e Z"Gddde!eZ#Gddde Z$Gd d!d!eZ%Gd"d#d#e%e Z&Gd$d%d%e%eZ'eee$e&e'd&Z(dS)'aResponse parsers for the various protocol types. The module contains classes that can take an HTTP response, and given an output shape, parse the response into a dict according to the rules in the output shape. There are many similarities amongst the different protocols with regard to response parsing, and the code is structured in a way to avoid code duplication when possible. The diagram below is a diagram showing the inheritance hierarchy of the response classes. :: +--------------+ |ResponseParser| +--------------+ ^ ^ ^ +--------------------+ | +-------------------+ | | | +----------+----------+ +------+-------+ +-------+------+ |BaseXMLResponseParser| |BaseRestParser| |BaseJSONParser| +---------------------+ +--------------+ +--------------+ ^ ^ ^ ^ ^ ^ | | | | | | | | | | | | | ++----------+-+ +-+-----------++ | | |RestXMLParser| |RestJSONParser| | +-----+-----+ +-------------+ +--------------+ +----+-----+ |QueryParser| |JSONParser| +-----------+ +----------+ The diagram above shows that there is a base class, ``ResponseParser`` that contains logic that is similar amongst all the different protocols (``query``, ``json``, ``rest-json``, ``rest-xml``). Amongst the various services there is shared logic that can be grouped several ways: * The ``query`` and ``rest-xml`` both have XML bodies that are parsed in the same way. * The ``json`` and ``rest-json`` protocols both have JSON bodies that are parsed in the same way. * The ``rest-json`` and ``rest-xml`` protocols have additional attributes besides body parameters that are parsed the same (headers, query string, status code). This is reflected in the class diagram above. The ``BaseXMLResponseParser`` and the BaseJSONParser contain logic for parsing the XML/JSON body, and the BaseRestParser contains logic for parsing out attributes that come from other parts of the HTTP response. Classes like the ``RestXMLParser`` inherit from the ``BaseXMLResponseParser`` to get the XML body parsing logic and the ``BaseRestParser`` to get the HTTP header/status code/query string parsing. Additionally, there are event stream parsers that are used by the other parsers to wrap streaming bodies that represent a stream of events. The BaseEventStreamParser extends from ResponseParser and defines the logic for parsing values from the headers and payload of a message from the underlying binary encoding protocol. Currently, event streams support parsing bodies encoded as JSON and XML through the following hierarchy. +--------------+ |ResponseParser| +--------------+ ^ ^ ^ +--------------------+ | +------------------+ | | | +----------+----------+ +----------+----------+ +-------+------+ |BaseXMLResponseParser| |BaseEventStreamParser| |BaseJSONParser| +---------------------+ +---------------------+ +--------------+ ^ ^ ^ ^ | | | | | | | | +-+----------------+-+ +-+-----------------+-+ |EventStreamXMLParser| |EventStreamJSONParser| +--------------------+ +---------------------+ Return Values ============= Each call to ``parse()`` returns a dict has this form:: Standard Response { "ResponseMetadata": {"RequestId": } } Error response { "ResponseMetadata": {"RequestId": } "Error": { "Code": , "Message": , "Type": , } } N)six XMLParseError) EventStreamNoInitialResponseError)parse_timestamp merge_dictsis_json_value_headerlowercase_dictc@s$eZdZddZddZddZdS)ResponseParserFactorycCs i|_dS)N) _defaults)selfr j/private/var/folders/pf/wv4htv3x0qs2c2mp0dnn0kchsvlck3/T/pip-install-emcbgzcf/botocore/botocore/parsers.py__init__szResponseParserFactory.__init__cKs|jj|dS)aOSet default arguments when a parser instance is created. You can specify any kwargs that are allowed by a ResponseParser class. There are currently two arguments: * timestamp_parser - A callable that can parse a timetsamp string * blob_parser - A callable that can parse a blob type N)r update)r kwargsr r rset_parser_defaultss z)ResponseParserFactory.set_parser_defaultscCst|}|f|jS)N)PROTOCOL_PARSERSr )r Z protocol_nameZ parser_clsr r r create_parsersz#ResponseParserFactory.create_parserN)__name__ __module__ __qualname__rrrr r r rr s r cCs tj|S)N)r r)protocolr r rrsrcsfdd}|S)Ncs.t|dr|j}|dkr"d}n|}|||S)Ntext)hasattrr)r shapenode_or_stringr)funcr r_get_text_contents  z(_text_content.._get_text_contentr )rrr )rr _text_contents r c@s eZdZdS)ResponseParserErrorN)rrrr r r rr!sr!c@sreZdZdZdZdZdddZddZdd Zd d Z d d Z ddZ ddZ ddZ ddZddZddZdS)ResponseParseraoBase class for response parsing. This class represents the interface that all ResponseParsers for the various protocols must implement. This class will take an HTTP response and a model shape and parse the HTTP response into a dictionary. There is a single public method exposed: ``parse``. See the ``parse`` docstring for more info. zutf-8NcCsH|dkr t}||_|dkr |j}||_d|_|jdk rD|j|||_dS)N)DEFAULT_TIMESTAMP_PARSER_timestamp_parser_default_blob_parser _blob_parser_event_stream_parserEVENT_STREAM_PARSER_CLS)r timestamp_parser blob_parserr r rrs zResponseParser.__init__cCs tj|S)N)base64 b64decode)r valuer r rr%sz#ResponseParser._default_blob_parsercCstjd|dtjd|d|ddkrP|j|rB|j|}q\|j||}n |j||}|rp|jjdrp|St|t r|jdi}|d|d <|d}t ||d <||d<|S) a>Parse the HTTP response given a shape. :param response: The HTTP response dictionary. This is a dictionary that represents the HTTP request. The dictionary must have the following keys, ``body``, ``headers``, and ``status_code``. :param shape: The model shape describing the expected output. :return: Returns a dictionary representing the parsed response described by the model. In addition to the shape described from the model, each response will also have a ``ResponseMetadata`` which contains metadata about the response, which contains at least two keys containing ``RequestId`` and ``HTTPStatusCode``. Some responses may populate additional keys, but ``RequestId`` will always be present. zResponse headers: %sheaderszResponse body: %sbody status_codei- eventstreamResponseMetadataZHTTPStatusCodeZ HTTPHeaders) LOGdebug_is_generic_error_response_do_generic_error_parse_do_error_parse _do_parse serializationget isinstancedictr )r responserparsedZresponse_metadatar.r r rparses         zResponseParser.parsecCsD|ddkr@d|ks |ddkr$dS|dj}|jdp>| SdS)Nr0ir/Ts)strip startswith)r r=r/r r rr5s  z)ResponseParser._is_generic_error_responsecCs4tjdt|dtjjjj|dddidS)NzlReceived a non protocol specific error response from the service, unable to populate error code and message.r0r)CodeMessage)Errorr2)r3r4strrmoves http_client responsesr:)r r=r r rr6s    z&ResponseParser._do_generic_error_parsecCstd|jjdS)Nz %s._do_parse)NotImplementedError __class__r)r r=rr r rr8$szResponseParser._do_parsecCstd|jjdS)Nz%s._do_error_parse)rIrJr)r r=rr r rr7'szResponseParser._do_error_parsecCst|d|j|j}|||S)Nz _handle_%s)getattr type_name_default_handle)r rnodehandlerr r r _parse_shape+s zResponseParser._parse_shapecCs.g}|j}x|D]}|j|j||qW|S)N)memberappendrP)r rrNr> member_shapeitemr r r _handle_list0s  zResponseParser._handle_listcCs|S)Nr )r rr-r r rrM9szResponseParser._default_handlecCs&|j}|djd}t|d|||S)NcontextZoperation_namer/)r'r:r)r r=rparsernamer r r_create_event_stream<sz#ResponseParser._create_event_stream)NN)rrr__doc__DEFAULT_ENCODINGr(rr%r?r5r6r8r7rPrUrMrYr r r rr"s  ,  r"cseZdZd fdd ZddZddZfdd Zd d Zd d ZddZ ddZ ddZ e ddZ e ddZe ddZe ddZe ddZe ddZeZeZeZZS)!BaseXMLResponseParserNcs"tt|j||tjd|_dS)Nz{.*})superr\rrecompile _namespace_re)r r)r*)rJr rrCs zBaseXMLResponseParser.__init__c Csi}|j}|j}|jjdpd}|jjdp.d}|jjdrNt|t rN|g}xd|D]\}xN|D]F} |j| } | |kr|j|| } q^| |kr|j|| } q^td| q^W| || <qTW|S)NrXkeyr- flattenedzUnknown tag: %s) rar-r9r:r;list _node_tagrPr!) r rrNr> key_shape value_shapeZkey_location_nameZvalue_location_nameZ keyval_nodeZ single_pairZtag_nameZkey_nameZval_namer r r _handle_mapHs"    z!BaseXMLResponseParser._handle_mapcCs|jjd|jS)Nr)r`subtag)r rNr r rrd]szBaseXMLResponseParser._node_tagcs0|jjdrt|t r|g}tt|j||S)Nrb)r9r:r;rcr]r\rU)r rrN)rJr rrU`sz"BaseXMLResponseParser._handle_listcCsi}|j}|j|}x|D]}||}d|jks|jjdr>q|j||}|j|} | dk rn|j|| ||<q|jjdri} |jd} x:|jjD],\} } |jj | j ddd| }| | |<qW| | kr| | ||<qW|S)Nlocation eventheaderZ xmlAttributerX:r) members_build_name_to_xml_noder9r:_member_key_namerPattribitemsr`rhsplit)r rrNr>rmxml_dict member_namerSZxml_nameZ member_nodeZattribsZ location_namerar-Znew_keyr r r_handle_structurejs.         z'BaseXMLResponseParser._handle_structurecCsL|jdkr0|jjdr0|jjjd}|dk r0|S|jjd}|dk rH|S|S)NrcrbrX)rLr9r:rQ)r rrtZlist_member_serialized_nameZserialized_namer r rros z&BaseXMLResponseParser._member_key_namecCsxt|tr|j|dSi}xV|D]N}|j|}||krht||trV||j|qp|||g||<q"|||<q"W|S)Nr)r;rcrnrdrR)r Z parent_nodersrTrar r rrns    z-BaseXMLResponseParser._build_name_to_xml_nodecCsjy2tjjjtjjj|jd}|j||j}Wn2tk rd}zt d||fWYdd}~XnX|S)N)targetencodingz7Unable to parse response (%s), invalid XML received: %s) xmletree cElementTree XMLParser TreeBuilderr[feedcloserr!)r xml_stringrWrooter r r_parse_xml_string_to_doms    z.BaseXMLResponseParser._parse_xml_string_to_domcCsFx@|jD]4\}}t|r4|j|}|j|||<q |j||<q W|S)N)rqrcrn_replace_nodesr)r r>rar-Zsub_dictr r rrs  z$BaseXMLResponseParser._replace_nodescCs|dkr dSdSdS)NtrueTFr )r rrr r r_handle_booleansz%BaseXMLResponseParser._handle_booleancCst|S)N)float)r rrr r r _handle_floatsz#BaseXMLResponseParser._handle_floatcCs |j|S)N)r$)r rrr r r_handle_timestampsz'BaseXMLResponseParser._handle_timestampcCst|S)N)int)r rrr r r_handle_integersz%BaseXMLResponseParser._handle_integercCs|S)Nr )r rrr r r_handle_stringsz$BaseXMLResponseParser._handle_stringcCs |j|S)N)r&)r rrr r r _handle_blobsz"BaseXMLResponseParser._handle_blob)NN)rrrrrgrdrUrurornrrr rrrrrrZ_handle_characterZ_handle_doubleZ _handle_long __classcell__r r )rJrr\Bs$        r\c@s,eZdZddZddZddZddZd S) QueryParsercCs\|d}|j|}|j|}|j|d|kr>|j|jdd|krXd|jdi|d<|S)Nr/ZErrors RequestIdr2)rrnrrpop)r r=r xml_contentsrr>r r rr7s   zQueryParser._do_error_parsecCsZ|d}|j|}i}|dk rJ|}d|jkr>|j|jd|}|j||}|j|||S)Nr/Z resultWrapper)rr9_find_result_wrapped_shaperP_inject_response_metadata)r r=rrrr>startr r rr8s    zQueryParser._do_parsecCs|j|}||S)N)rn)r Z element_nameZ xml_root_nodemappingr r rrs z&QueryParser._find_result_wrapped_shapecCsR|j|}|jd}|dk rN|j|}x|jD]\}}|j||<q0W||d<dS)Nr2)rnr:rqr)r rN inject_intor child_nodeZ sub_mappingrar-r r rrs   z%QueryParser._inject_response_metadataN)rrrr7r8rrr r r rrsrcs$eZdZddZfddZZS)EC2QueryParsercCs.|j|}|jd}|dk r*d|ji|d<dS)NZ requestIdrr2)rnr:r)r rNrrrr r rrs  z(EC2QueryParser._inject_response_metadatacs(tt|j||}d|jdi|d<|S)NrZ RequestIDr2)r]rr7r)r r=roriginal)rJr rr7s zEC2QueryParser._do_error_parse)rrrrr7rr r )rJrrsrc@sDeZdZddZddZddZddZd d Zd d Zd dZ dS)BaseJSONParserc Csd|j}|dkrdSi}xH|D]@}||}|jjd|}|j|}|dk r|j|||||<qW|S)NrX)rmr9r:rP) r rr- member_shapes final_parsedrtrSZ json_nameZ raw_valuer r rru-s  z BaseJSONParser._handle_structurec CsJi}|j}|j}x4|jD](\}}|j||}|j||}|||<qW|S)N)rar-rqrP) r rr-r>rerfraZ actual_keyZ actual_valuer r rrg?s   zBaseJSONParser._handle_mapcCs |j|S)N)r&)r rr-r r rrIszBaseJSONParser._handle_blobcCs |j|S)N)r$)r rr-r r rrLsz BaseJSONParser._handle_timestampcCs|j|d}dddid}|jd|jdd|dd<|jd}|jd |oTt|}|dk rd |krx|jd d d }||dd <|j||d |S)Nr/r)rCrB)rDr2messagerCrDr0Z__type#rBr.)_parse_body_as_jsonr:rErsplitr)r r=rr/errorZ response_codecoder r rr7Os   zBaseJSONParser._do_error_parsecCs d|kr|d|jdid<dS)Nzx-amzn-requestidr2r) setdefault)r r>r.r r rrjsz(BaseJSONParser._inject_response_metadatac Cs@|siS|j|j}ytj|}|Stk r:d|iSXdS)Nr)decoder[jsonloads ValueError)r body_contentsr/original_parsedr r rros  z"BaseJSONParser._parse_body_as_jsonN) rrrrurgrrr7rrr r r rr+s rc@s4eZdZddZddZddZddZd d Zd S) BaseEventStreamParsercCshi}|jjdr@|djd}|jj|}|rd|j||||<n$|j|||j||j|||j||S)Nr1r.z :event-type)r9r:rmr8_parse_non_payload_attrs_parse_payload)r r=rrZ event_typeZ event_shaper r rr8~s   zBaseEventStreamParser._do_parsecCs|djd}|jj|}|dk r\|j|d}|j||}d||jd|jdddi}n&d|djd d|djd ddi}|S) Nr.z:exception-typer/rDrCrr)rBrCz :error-codez:error-message)r:rm_initial_body_parserP)r r=rZexception_typeZexception_shaperr/rr r rr7s  z%BaseEventStreamParser._do_error_parsec Cs|jjdrxr|D]j}||}|jjdr|d}|jdkrB|}n.|jdkrZ|j|j}n|j|} |j|| }|||<dSqW|j|d} |j|| } |j| dS)NeventZ eventpayloadr/blobstring)r9r:rLrr[rrPr) r r=rrrrXrSr/Z parsed_bodyZ raw_parser body_parsedr r rrs         z$BaseEventStreamParser._parse_payloadc Cs^|d}xP|D]H}||}|jjdr||kr||}|jdkrN|j|d}|||<qWdS)Nr.rk timestampg@@)r9r:rLr$) r r=rrrr.rXrSr-r r rrs   z.BaseEventStreamParser._parse_non_payload_attrscCs tddS)Nr)rI)r rr r rrsz)BaseEventStreamParser._initial_body_parseN)rrrr8r7rrrr r r rr|s  rc@seZdZddZdS)EventStreamJSONParsercCs |j|S)N)r)r rr r rrsz)EventStreamJSONParser._initial_body_parseN)rrrrr r r rrsrc@seZdZddZdS)EventStreamXMLParsercCs|stjjjdS|j|S)Nr)rxryrzElementr)r rr r rrsz(EventStreamXMLParser._initial_body_parseN)rrrrr r r rrsrc@s(eZdZeZddZddZddZdS) JSONParsercCsJi}|dk r6|j}|r&|j|||}n|j|d|}|j||d|S)Nr/r.)Zevent_stream_name_handle_event_stream_handle_json_bodyr)r r=rr> event_namer r rr8szJSONParser._do_parsec Cs^|j|}|j||}y |j}Wn tk rBd}t|YnX|j|j|}|||<|S)Nz,First event was not of type initial-response)rmrYZget_initial_responserr!rpayload) r r=rrZevent_stream_shapeZ event_streamr error_msgr>r r rrs   zJSONParser._handle_event_streamcCs|j|}|j||S)N)rrP)r Zraw_bodyrZ parsed_jsonr r rrs zJSONParser._handle_json_bodyN)rrrrr(r8rrr r r rrs  rc@sDeZdZddZddZddZddZd d Zd d Zd dZ dS)BaseRestParsercCsHi}|j||d<|dkr|S|j}|j|||||j|||||S)Nr2)_populate_response_metadatarmrr)r r=rrrr r rr8s zBaseRestParser._do_parsecCsJi}|d}d|kr"|d|d<n$d|krF|d|d<|jdd|d<|S)Nr.zx-amzn-requestidrzx-amz-request-idz x-amz-id-2rHostId)r:)r r=metadatar.r r rr s z*BaseRestParser._populate_response_metadatac Csd|jkr|jd}||}|jjdr>|j||}|||<q|jdkrp|d}t|trf|j|j}|||<q|j|d}|j ||||<n$|j|d}|j ||} |j | dS)Nrr1rrr/)rr) r9r:rYrLr;bytesrr[rrPr) r r=rrrZpayload_member_nameZ body_shaper/rrr r rrs"          zBaseRestParser._parse_payloadc Cs|d}x|D]}||}|jjd}|dkr2qq|dkrP|j||d||<q|dkrj|j||||<q|dkr|jjd|} | |kr|j||| ||<qWdS)Nr.rjZ statusCoder0headerrX)r9r:rP_parse_header_map) r r=rrrr.rXrSrj header_namer r rr/s"   z'BaseRestParser._parse_non_payload_attrscCsRi}|jjddj}x6|D].}|jj|r|t|d}||||<qW|S)NrXr)r9r:lowerrAlen)r rr.r>prefixrrXr r rrCs z BaseRestParser._parse_header_mapcCs tddS)Nr)rI)r rr r rrPsz"BaseRestParser._initial_body_parsecCs,|}t|r(tj|j|j}tj|}|S)N)rr+r,rr[rr)r rr-r>decodedr r rrWs  zBaseRestParser._handle_stringN) rrrr8rrrrrrr r r rrs   rcs0eZdZeZddZfddZddZZS)RestJSONParsercCs |j|S)N)r)r rr r rrcsz"RestJSONParser._initial_body_parsecs"tt|j||}|j|||S)N)r]rr7_inject_error_code)r r=rr)rJr rr7fs zRestJSONParser._do_error_parsecCsr|j|d}d|dkrB|dd}|jdd}||dd<n,d|ksRd|krn|jd|jdd |dd<dS) Nr/zx-amzn-errortyper.rlrrDrBrr)rrrr:)r rr=r/rr r rrks  z!RestJSONParser._inject_error_code) rrrrr(rr7rrr r )rJrr_s rcsDeZdZeZddZddZddZddZe fd d Z Z S) RestXMLParsercCs|stjjjdS|j|S)Nr)rxryrzrr)r rr r rr~sz!RestXMLParser._initial_body_parsecCsN|drDy |j|Stk rB}ztjdddWYdd}~XnX|j|S)Nr/z2Exception caught when parsing error response body:T)exc_info)_parse_error_from_bodyr!r3r4_parse_error_from_http_status)r r=rrr r rr7s zRestXMLParser._do_error_parsecCsHt|dtjjjj|ddd|djdd|djddddS) Nr0r)rBrCr.zx-amz-request-idz x-amz-id-2)rr)rDr2)rErrFrGrHr:)r r=r r rrs   z+RestXMLParser._parse_error_from_http_statuscCs|d}|j|}|j|}|j||jdkr\|j|}|jdd|jdd||dSd|krvd|jdi|d<ddddi}t|||S) Nr/rDrrr)rDr2r2)rCrB)rrnrrirrr)r r=rrr>rdefaultr r rrs         z$RestXMLParser._parse_error_from_bodycstt|j||}|S)N)r]rr)r rr)rJr rrszRestXMLParser._handle_string) rrrrr(rr7rrr rrr r )rJrrzs  r)Zec2queryrz rest-jsonzrest-xml))rZr^r+rxml.etree.cElementTreerxloggingZbotocore.compatrrZbotocore.eventstreamrrZbotocore.utilsrrrr getLoggerrr3r#objectr rr Exceptionr!r"r\rrrrrrrrrrrr r r rtsB   .QN$cN