U Xzf_@sJdZdZddlZddlZddlZdZejr0eZnddl Z e e dZ ddl mZddl mZddl mZdd lmZd Ze ZedZejZd d Zd dZedeZedeZedeZedeZddZddZddZ ddZ!ddZ"ddZ#dKd d!Z$eej%eZ&eej%eZ'eej%eZ(eej%eZ)e ej%eej*Z+e ej%eej*Z,e!ej-d"Z.e!ej/d#Z0e!ej-d$Z1e!ej/d%Z2e"Z3e#Z4e ej%ee5Z6dLd&d'Z7dMd(d)Z8d*d+Z9d,d-Z:e;d.ej<Z=d/d0Z>d1d2Z?d3d4Z@d5d6ZAd7d8ZBd9d:ZCd;d<ZDdNd=d>ZEd?d@ZFdAdBZGdCdDZHdEdFZIdGdHZJdIdJZKeKZLdS)Oa Code for decoding protocol buffer primitives. This code is very similar to encoder.py -- read the docs for that module first. A "decoder" is a function with the signature: Decode(buffer, pos, end, message, field_dict) The arguments are: buffer: The string containing the encoded message. pos: The current position in the string. end: The position in the string where the current message ends. May be less than len(buffer) if we're reading a sub-message. message: The message object into which we're parsing. field_dict: message._fields (avoids a hashtable lookup). The decoder reads the field and stores it into field_dict, returning the new buffer position. A decoder for a repeated field may proactively decode all of the elements of that field, if they appear consecutively. Note that decoders may throw any of the following: IndexError: Indicates a truncated message. struct.error: Unpacking of a fixed-width field failed. message.DecodeError: Other errors. Decoders are expected to raise an exception if they are called with pos > end. This allows callers to be lax about bounds checking: it's fineto read past "end" as long as you are sure that someone else will notice and throw an exception later on. Something up the call stack is expected to catch IndexError and struct.error and convert them to message.DecodeError. Decoders are constructed using decoder constructors with the signature: MakeDecoder(field_number, is_repeated, is_packed, key, new_default) The arguments are: field_number: The field number of the field we want to decode. is_repeated: Is the field a repeated field? (bool) is_packed: Is the field a packed field? (bool) key: The key to use when looking up the field within field_dict. (This is actually the FieldDescriptor but nothing in this file should depend on that.) new_default: A function which takes a message object as a parameter and returns a new instance of the default value for this field. (This is called for repeated fields and sub-messages, when an instance does not already exist.) As with encoders, we define a decoder constructor for every type of field. Then, for every field of every message class we construct an actual decoder. That decoder goes into a dict indexed by tag, so when we decode a message we repeatedly read a tag, look up the corresponding decoder, and invoke it. z kenton@google.com (Kenton Varda)Niz[\ud800-\udfff]) containers)encoder) wire_format)messagegcsfdd}|S)aReturn an encoder for a basic varint value (does not include tag). Decoded values will be bitwise-anded with the given mask before being returned, e.g. to limit them to 32 bits. The returned decoder does not take the usual "end" parameter -- the caller is expected to do bounds checking after the fact (often the caller can defer such checking until later). The decoder returns a (value, new_pos) pair. csjd}d}t||}||d@|>O}|d7}|d@sL|M}|}||fS|d7}|dkrtdqdSNr@z$Too many bytes when decoding varint.six indexbytes _DecodeErrorbufferposresultshiftbmask result_typex/private/var/folders/n9/53xbvtmd7sjg1q1l55xmpg58n4mgng/T/pip-unpacked-wheel-3cr5c46n/google/protobuf/internal/decoder.py DecodeVarintzs z$_VarintDecoder..DecodeVarintr)rrrrrr_VarintDecoderps rcs,d|d>d|>dfdd}|S)z0Like _VarintDecoder() but decodes signed values.rcsvd}d}t||}||d@|>O}|d7}|d@sX|M}|A}|}||fS|d7}|dkrtdqdSrr rrrZsignbitrrrs  z*_SignedVarintDecoder..DecodeVarintr)bitsrrrrr_SignedVarintDecoders  rlr l cCs>|}t||d@r|d7}q|d7}|||}||fS)aRead a tag from the memoryview, and return a (tag_bytes, new_pos) tuple. We return the raw bytes of the tag rather than decoding them. The raw bytes can then be used to look up the proper decoder. This effectively allows us to trade some work that would be done in pure-python (decoding a varint) for work that is done in C (searching for a byte string in a hash table). In a low-level language it would be much cheaper to decode the varint and use that, but not in Python. Args: buffer: memoryview object of the encoded bytes pos: int of the current position to start from Returns: Tuple[bytes, int] of the tag data and new position. r r)r rtobytes)rrstart tag_bytesrrrReadTags  r$csdfdd }|S)zReturn a constructor for a decoder for fields of a particular type. Args: wire_type: The field's wire type. decode_value: A function which decodes an individual value, e.g. _DecodeVarint() Fc sf|rtfdd}|S|rNt|tfdd}|Sfdd}|SdS)Ncs|}|dkr"||}||\}}||7}||krHtd||krj||\}}||qH||kr|d=td|S)NTruncated message.Packed element was truncated.get setdefaultrappend)rrendr field_dictvalueendpointelement) decode_valuekeylocal_DecodeVarint new_defaultrrDecodePackedFields  zB_SimpleDecoder..SpecificDecoder..DecodePackedFieldcst|}|dkr"||}||\}}|||}|||ksZ||kr"||krjtd|Sq"dSNr%)r)r*r+rrrr,rr-r.r0new_pos)r1r2r4r#tag_lenrrDecodeRepeatedFields  zD_SimpleDecoder..SpecificDecoder..DecodeRepeatedFieldcs@||\}}||krtdr4|s4|dn||<|Sr6rpop)rrr,rr- new_value)clear_if_defaultr1r2rr DecodeFieldsz<_SimpleDecoder..SpecificDecoder..DecodeField) _DecodeVarintrTagByteslen field_number is_repeated is_packedr2r4r>r5r:r?r1 wire_typer>r2r3r4r#r9rSpecificDecoders  z'_SimpleDecoder..SpecificDecoder)Fr)rHr1rJrrGr_SimpleDecoders 3rKcsfdd}t||S)zLike SimpleDecoder but additionally invokes modify_value on every value before storing it. Usually modify_value is ZigZagDecode. cs||\}}||fS)Nr)rrrr8r1 modify_valuerr InnerDecodesz%_ModifiedDecoder..InnerDecode)rK)rHr1rMrNrrLr_ModifiedDecoder srOcs*ttjfdd}t||S)zReturn a constructor for a decoder for a fixed-width field. Args: wire_type: The field's wire type. format: The format string to pass to struct.unpack(). cs&|}|||d}||fS)Nrr)rrr8rformat local_unpackZ value_sizerrrN*sz'_StructPackDecoder..InnerDecode)structcalcsizeunpackrK)rHrQrNrrPr_StructPackDecoders  rVcstjfdd}ttj|S)zReturns a decoder for a float field. This code works around a bug in struct.unpack for non-finite 32-bit floating-point values. cs|d}|||}|dddkrp|dddkrp|dddkrPt|fS|dddkrht|fSt|fSd |d}||fS) a#Decode serialized float to a float and new position. Args: buffer: memoryview of the serialized bytes pos: int, position in the memory view to start at. Returns: Tuple[float, int] of the deserialized float value and new position in the serialized data. rsz.InnerDecode)rSrUrKrWIRETYPE_FIXED32rNrr`r _FloatDecoder1s !rccstjfdd}ttj|S)zkReturns a decoder for a double field. This code works around a bug in struct.unpack for not-a-number. csf|d}|||}|dddkrP|dddkrP|dddkrPt|fSd|d}||fS) a"Decode serialized double to a double and new position. Args: buffer: memoryview of the serialized bytes. pos: int, position in the memory view to start at. Returns: Tuple[float, int] of the decoded double value and new position in the serialized data. r rYrsz.InnerDecode)rSrUrKrWIRETYPE_FIXED64rbrr`r_DoubleDecoder^s rhFc stj|r&tfdd}|S|rZttjtfdd}|Sfdd}|SdS)z!Returns a decoder for enum field.c s|}|dkr"||}||\}}||7}||krHtd||kr|}t||\}}|jkrx||qH|jsg|_tt j } |j| ||| f|j dkrt |_ |j t j |qH||kr |jkr|d=n|jd=|j jd=td|S)aDecode serialized packed enum to its value and a new position. Args: buffer: memoryview of the serialized bytes. pos: int, position in the memory view to start at. end: int, end position of serialized data message: Message object to store unknown fields in field_dict: Map[Descriptor, Any] to store decoded values in. Returns: int, new position in serialized data. Nr%r&r')r)r*r_DecodeSignedVarint32values_by_numberr+_unknown_fieldsrrArWIRETYPE_VARINTr!_unknown_field_setrUnknownFieldSet_addZ_values) rrr,rr-r.r/value_start_posr0r#) enum_typerDr2r3r4rrr5sD        z&EnumDecoder..DecodePackedFieldcs|}|dkr"||}t||\}}|jkrF||nN|jsRg|_|j|||f|jdkrt |_|j t j ||}|||ks||kr"||krt d|Sq"dS)Decode serialized repeated enum to its value and a new position. Args: buffer: memoryview of the serialized bytes. pos: int, position in the memory view to start at. end: int, end position of serialized data message: Message object to store unknown fields in field_dict: Map[Descriptor, Any] to store decoded values in. Returns: int, new position in serialized data. Nr%)r)r*rirjr+rkr!rmrrnrorrlrr7)rqrDr2r4r#r9rrr:s.     z(EnumDecoder..DecodeRepeatedFieldcs|}t||\}}||kr"tdr:|s:|d|S|jkrN||<n\|jsZg|_ttj}|j |||| f|j dkrt |_ |j tj||S)rrr%N)rirr<rjrkrrArrlr+r!rmrrnro)rrr,rr-rpZ enum_valuer#)r>rqrDr2rrr?s2      z EnumDecoder..DecodeFieldN)rqr@rrArrlrBrCr)r>rqrDr2r3r4r#r9r EnumDecoders1'$rs._ConvertToUnicodecs|}|dkr"||}||\}}||}||krHtd|||||}|||ks~||kr"|Sq"dSNzTruncated string.r(rrr,rr-r.sizer8)rr2r3r4r#r9rrr:Rs z*StringDecoder..DecodeRepeatedFieldcsT||\}}||}||kr&tdr<|s<|dn||||<|Srr;rrr,rr-rr8)rr>r2r3rrr?csz"StringDecoder..DecodeFieldN) r@r text_typeAssertionErrorrrArWIRETYPE_LENGTH_DELIMITEDrB) rDrErFr2r4rr>r:r?r) rr>rr2r3rr4r#r9r StringDecoder1s rcsVt|r t|r>t|tjtfdd}|Sfdd}|SdS)z$Returns a decoder for a bytes field.cs|}|dkr"||}||\}}||}||krHtd|||||}|||ks~||kr"|Sq"dSr)r)r*rr+r!rr2r3r4r#r9rrr:{s z)BytesDecoder..DecodeRepeatedFieldcsT||\}}||}||kr&tdr<|s<|dn||||<|Sr)rr<r!r)r>r2r3rrr?sz!BytesDecoder..DecodeFieldNr@rrrArrrB)rDrErFr2r4r>r:r?rrIr BytesDecoderps rcslt|tjt|rt|rRt|tjtfdd}|Sfdd}|SdS)z$Returns a decoder for a group field.cs|}|dkr"||}|}|dkrD||}||||}|}|||ksv||kr~td|}|||ks||kr"|Sq"dSNMissing group end tag.)r)r*add_InternalParserrrr,rr-r.r8 end_tag_bytes end_tag_lenr2r4r#r9rrr:s  z)GroupDecoder..DecodeRepeatedFieldcs\|}|dkr"||}||||}|}|||ksP||krXtd|Sr)r)r*rrr)rrr2r4rrr?s z!GroupDecoder..DecodeFieldN)rrArWIRETYPE_END_GROUPrBrWIRETYPE_START_GROUPrDrErFr2r4r:r?rrr GroupDecoders rcsVt|r t|r>t|tjtfdd}|Sfdd}|SdS)z&Returns a decoder for a message field.cs|}|dkr"||}||\}}||}||krHtd|||||krftd|}|||ks||kr"|Sq"dSNr%Unexpected end-group tag.)r)r*rrrrrrrr:s z+MessageDecoder..DecodeRepeatedFieldcsf|}|dkr"||}||\}}||}||krHtd|||||krbtd|Sr)r)r*rrr)r2r3r4rrr?s z#MessageDecoder..DecodeFieldNrrrrrMessageDecodersrrcsNtdtjtdtjtdtjttt}fdd}|S)aReturns a decoder for a MessageSet item. The parameter is the message Descriptor. The message set message looks like this: message MessageSet { repeated group Item = 1 { required int32 type_id = 2; required string message = 3; } } rZrXrcs|}d}d}d}||\} }| kr6||\}}q| krZ||\} }|| }}q| krfqqt|||| }|dkrtdq||krtd|dkrtd|dkrtd|j|} | dk r*|| } | dkr | j} t| ds|j| | | | } | ||||krtdn^|j s8g|_ |j t|||f|jdkrjt|_|j|tj||||S) aDecode serialized message set to its value and new position. Args: buffer: memoryview of the serialized bytes. pos: int, position in the memory view to start at. end: int, end position of serialized data message: Message object to store unknown fields in field_dict: Map[Descriptor, Any] to store decoded values in. Returns: int, new position in serialized data. r&rr%z MessageSet item missing type_id.z MessageSet item missing message.N_concrete_classr) SkipFieldrZ ExtensionsZ_FindExtensionByNumberr) message_typehasattrZ_FACTORYZ GetPrototyper*rrrkr+MESSAGE_SET_ITEM_TAGr!rmrrnrorr)rrr,rr-Zmessage_set_item_startZtype_id message_startZ message_endr#r extensionr.rZitem_end_tag_bytesr3Z local_ReadTagZmessage_tag_bytesZtype_id_tag_bytesrr DecodeItems^           z)MessageSetItemDecoder..DecodeItem) rrArrlrrr$r@r)Z descriptorZlocal_SkipFieldrrrrMessageSetItemDecodersHrcsB|t|jtjtt|jfdd}|S)z"Returns a decoder for a map field.c s}|}|dkr*||}||\}}||}||krPtd||||||krrtdr||j|jn |j||j<|}|||ks||kr*|Sq*dSr) rr)r*rZClearrr2ZCopyFromr.) rrr,rr-Zsubmsgr.rr8is_message_mapr2r3rr4r#r9rr DecodeMapks"  zMapDecoder..DecodeMap)rrAnumberrrrBr@r)Zfield_descriptorr4rrrrr MapDecoder`srcCsBt|||dd@r&|d7}q|d7}||kr>td|S)z/Skip a varint value. Returns the new position.rr r%)ordr!rrrr,rrr _SkipVarints  rcCs|d7}||krtd|S)z0Skip a fixed64 value. Returns the new position.rdr%rrrrr _SkipFixed64srcCs$|d}td|||d|fS)zDecode a fixed64.rdrurrSrUrrr8rrr_DecodeFixed64srcCs*t||\}}||7}||kr&td|S)z9Skip a length-delimited value. Returns the new position.r%)r@r)rrr,rrrr_SkipLengthDelimiteds rcCs2t||\}}t||||}|dkr(|S|}qdS)z*Skip sub-group. Returns the new position.r&N)r$r)rrr,r#r8rrr _SkipGroups rc Csvt}|dks||krnt||\}}t|d\}}t|\}}|tjkrNqnt|||\} }|||| q||fS)zFDecode UnknownFieldSet. Returns the UnknownFieldSet and new position.Nr) rrnr$r@rZ UnpackTagr_DecodeUnknownFieldro) rrZend_posZunknown_field_setr#tag_rDrHdatarrr_DecodeUnknownFieldSets rcCs|tjkrt||\}}n|tjkr4t||\}}n|tjkrNt||\}}nf|tjkrt||\}}||||}||7}n0|tj krt ||\}}n|tj krdSt d||fS)zCDecode a unknown field. Returns the UnknownField and new position.)rr&zWrong wire type in tag.) rrlr@rgrra_DecodeFixed32rr!rrrr)rrrHrrrrrrs        rcCsdS)zFSkipping an END_GROUP tag returns -1 to tell the parent loop to break.r&rrrrr _EndGroupsrcCs|d7}||krtd|S)z0Skip a fixed32 value. Returns the new position.rWr%rrrrr _SkipFixed32srcCs$|d}td|||d|fS)zDecode a fixed32.rWrtrrrrrrrsrcCs tddS)z;Skip function for unknown wire types. Raises an exception.zTag had invalid wire type.Nrrrrr_RaiseInvalidWireTypesrcs,ttttttttgtjfdd}|S)z"Constructs the SkipField function.cs$t|dd@}||||S)aSkips a field with the specified tag. |pos| should point to the byte immediately after the tag. Returns: The new position (after the tag value), or -1 if the tag is an end-group tag (in which case the calling loop should break). rr)r)rrr,r#rHZWIRETYPE_TO_SKIPPERZ wiretype_maskrrrs z _FieldSkipper..SkipField) rrrrrrrrZ TAG_TYPE_MASK)rrrr _FieldSkippers r)F)FF)F)N)M__doc__ __author__rSryr r{PY3intlongrecompileur|Zgoogle.protobuf.internalrrrZgoogle.protobufrr_r^r]r~rrrr@Z_DecodeSignedVarintZ_DecodeVarint32rir$rKrOrVrcrhrsrlZ Int32DecoderZ Int64DecoderZ UInt32DecoderZ UInt64DecoderZ ZigZagDecodeZ SInt32DecoderZ SInt64DecoderraZFixed32DecoderrgZFixed64DecoderZSFixed32DecoderZSFixed64DecoderZ FloatDecoderZ DoubleDecoderboolZ BoolDecoderrrrrrArrrrrrrrrrrrrrrrrrrrrs2        ?-)        @ )/4b.