a _bhH@s|ddlZddlmZddlmZddlmZmZmZddl m Z m Z m Z m Z ddlmZmZmZdd ZGd d d eZdS) N)chain)Urllib3HttpConnection)ConnectionPoolDummyConnectionPoolEmptyConnectionPool)ConnectionErrorConnectionTimeoutSerializationErrorTransportError)DEFAULT_SERIALIZERS DeserializerJSONSerializercCs|dgdgkrdS|S)am Simple callback that takes the node info from `/_cluster/nodes` and a parsed connection information and return the connection information. If `None` is returned this node will be skipped. Useful for filtering nodes (by proximity for example) or if additional information needs to be provided for the :class:`~opensearchpy.Connection` class. By default master only nodes are filtered out since they shouldn't typically be used for API operations. :arg node_info: node information from `/_cluster/nodes` :arg host: connection information (host, port) extracted from the node info ZrolesZmasterN)get) node_infohostr=/tmp/pip-target-4jja1joz/lib/python/opensearchpy/transport.py get_host_info)src@seZdZdZeZdeeddddeddddddfd d Z d d Z d dZ ddZ dddZ ddZd ddZddZd!ddZddZddZdS)" Transportz Encapsulation of transport-related to logic. Handles instantiation of the individual connections as well as creating a connection pool to hold them. Main interface is the `perform_request` method. NFg?zapplication/json)iiiGETcKs|dur|j}t}| || j<| r.|| t|| |_| |_||_| |_ ||_ | |_ ||_ ||_ ||_||_t|_|r||t|jjdd|_ng|_|ddrd}d}||_||_||_t|_||_||_|r|ddS)a :arg hosts: list of dictionaries, each containing keyword arguments to create a `connection_class` instance :arg connection_class: subclass of :class:`~opensearchpy.Connection` to use :arg connection_pool_class: subclass of :class:`~opensearchpy.ConnectionPool` to use :arg host_info_callback: callback responsible for taking the node information from `/_cluster/nodes`, along with already extracted information, and producing a list of arguments (same as `hosts` parameter) :arg sniff_on_start: flag indicating whether to obtain a list of nodes from the cluster at startup time :arg sniffer_timeout: number of seconds between automatic sniffs :arg sniff_on_connection_fail: flag controlling if connection failure triggers a sniff :arg sniff_timeout: timeout used for the sniff request - it should be a fast api call and we are talking potentially to more nodes so we want to fail quickly. Not used during initial sniffing (if ``sniff_on_start`` is on) when the connection still isn't initialized. :arg serializer: serializer instance :arg serializers: optional dict of serializer instances that will be used for deserializing data coming from the server. (key is the mimetype) :arg default_mimetype: when no mimetype is specified by the server response assume this mimetype, defaults to `'application/json'` :arg max_retries: maximum number of retries before an exception is propagated :arg retry_on_status: set of HTTP status codes on which we should retry on a different node. defaults to ``(502, 503, 504)`` :arg retry_on_timeout: should timeout trigger a retry on different node? (default `False`) :arg send_get_body_as: for GET requests with body this option allows you to specify an alternate way of execution for environments that don't support passing bodies with GET requests. If you set this to 'POST' a POST method will be used instead, if to 'source' then the body will be serialized and passed as a query parameter `source`. Any extra keyword arguments will be passed to the `connection_class` when creating and instance unless overridden by that connection's options provided as part of the hosts parameter. NZcloud_idFT)DEFAULT_CONNECTION_CLASSr copymimetypeupdater deserializer max_retriesretry_on_timeoutretry_on_statussend_get_body_as serializerconnection_pool_classconnection_classkwargshostsrconnection_poolset_connectionslist connectionsseed_connectionsrsniffer_timeoutsniff_on_startsniff_on_connection_failtime last_sniff sniff_timeouthost_info_callback sniff_hosts)selfr%r#r"r1r,r+r0r-r!Z serializersZdefault_mimetyperrrr r$Z _serializersrrr__init__Gs@8      zTransport.__init__cCs|j|||jdS)z Create a new :class:`~opensearchpy.Connection` instance and add it to the pool. :arg host: kwargs that will be used to create the instance N)r%appendr')r3rrrradd_connections zTransport.add_connectioncsVfdd}t||}tt||}t|dkr._create_connectionrN)mapr(ziplenrr&r"r$)r3r%r:r)rr9rr's    zTransport.set_connectionscCs,|jr"t|j|jkr"||jS)z Retrieve a :class:`~opensearchpy.Connection` instance from the :class:`~opensearchpy.ConnectionPool` instance. )r+r.r/r2r&get_connectionr9rrrr>szTransport.get_connectionc Cs|j}zt|_t|jj|jD]l}zP|jdd|s:|jndd\}}}dd|D}|j || d}WqWq"t t fyYq"0q"tdd Wnty||_Yn0t|d S) a Perform the request to get sniffing information. Returns a list of dictionaries (one per node) containing all the information from the cluster. It also sets the last_sniff attribute in case of a successful attempt. In rare cases it might be possible to override this method in your custom Transport class to serve data from alternative source like configuration management. rz/_nodes/_all/httpN)timeoutcSsi|]\}}||qSrlower.0headervaluerrr sz-Transport._get_sniff_data.. content-typeN/AzUnable to sniff hosts.nodes)r/r.rr&r)r*perform_requestr0itemsrloadsrrr r Exceptionr(values)r3initialZprevious_sniffc_headersrrrr_get_sniff_datas0      zTransport._get_sniff_datacCsi}|did}|r"d|vr&dSd|vrl|dd\}}||d<|dd\}|d<t|d|d<n(|dd\|d<|d<t|d|d<|||S)NhttpZpublish_address:/rrport)rsplitrsplitintr1)r3Z host_inforaddressZfqdn ipaddressrQrrr_get_host_info s zTransport._get_host_infocsB|}ttdfdd|D}|s4tdd|dS)af Obtain a list of nodes from the cluster and create a new connection pool using the information retrieved. To extract the node connection parameters use the ``nodes_to_host_callback``. :arg initial: flag indicating if this is during startup (``sniff_on_start``), ignore the ``sniff_timeout`` if ``True`` Nc3s|]}|VqdS)N)r])rCnr9rr Az(Transport.sniff_hosts..rHz.Unable to sniff hosts - no viable hosts found.)rSr(filterr r')r3rOrr%rr9rr25s zTransport.sniff_hostscCs|j||jr|dS)z Mark a connection as dead (failed) in the connection pool. If sniffing on failure is enabled this will initiate the sniffing process. :arg connection: instance of :class:`~opensearchpy.Connection` that failed N)r& mark_deadr-r2)r3r8rrrrbLs zTransport.mark_deadc Cs||||\}}}}}t|jdD]T}|} z4| j|||||||d\} } } dd| D} Wnty$} z|dkr| jdkrWYd} ~ dSd}t| t r|j }n t| t rd }n| j|j vrd }|r z| | WntyYn0||jkr| n| WYd} ~ q&d} ~ 00|j| |dkrZd | koPd knS| rt|j| | d } | Sq&dS) a Perform the actual request. Retrieve a connection from the connection pool, pass all the information to it's perform_request method and return the data. If an exception was raised, mark the connection as failed and retry (up to `max_retries` times). If the operation was successful and the connection used was previously marked as dead, mark it as live, resetting it's failure count. :arg method: HTTP method to use :arg url: absolute url (without host) to target :arg headers: dictionary of headers, will be handed over to the underlying :class:`~opensearchpy.Connection` class :arg params: dictionary of query parameters, will be handed over to the underlying :class:`~opensearchpy.Connection` class for serialization :arg body: body of the request, will be serialized using serializer and passed to the connection r)rRignorer?cSsi|]\}}||qSrr@rBrrrrFsz-Transport.perform_request..HEADiNFTi,rG)_resolve_request_argsrangerr>rJrKr status_code isinstancer rrrrbr&Z mark_liverrLr)r3methodurlrRparamsbodyrcr?attemptr8statusZheaders_responsedataeretryrrrrJXsX          zTransport.perform_requestcCs|jdS)z/ Explicitly closes connections N)r&closer9rrrrsszTransport.closec Cs|durX|j|}|dvrX|jdkrX|jdkr6d}n"|jdkrX|durLi}||d<d}|durz|dd}WnttfyYn0d}d}|r|d d}|d d}t|tr|f}|||||fS) z*Resolves parameters for .perform_request()N)rdrrPOSTsourcezutf-8 surrogatepassrrequest_timeoutrc) r!dumpsr encodeUnicodeDecodeErrorAttributeErrorpoprirZ)r3rjrlrmrcr?rrrrfs.      zTransport._resolve_request_args)F)F)NNN)__name__ __module__ __qualname____doc__rrrrrr4r6r'r>rSr]r2rbrJrsrfrrrrr=s6 t " 0  Sr)r. itertoolsrr8rr&rrr exceptionsrr r r r!r r rrobjectrrrrrs