o dJM@s6ddlmZddlZddlZddlZddlmZddlmZm Z m Z ddl m Z m Z mZmZmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZgd ZeeZ dZ!dZ"e#de"Z$dZ%e#de%Z&ddZ'e(e'e$e(e'e$dZ)ee dZ*GdddeZ+Gddde+Z,ddZ-dS))absolute_importN)RecentlyUsedContainer)HTTPConnectionPoolHTTPSConnectionPoolport_by_scheme)LocationValueError MaxRetryErrorProxySchemeUnknownProxySchemeUnsupportedURLSchemeUnknown)six)urljoin)RequestMethods)connection_requires_http_tunnel)Retry) parse_url) PoolManager ProxyManagerproxy_from_url) key_file cert_file cert_reqsca_certs ssl_version ca_cert_dir ssl_context key_passwordserver_hostname) key_schemekey_hostkey_port key_timeout key_retries key_strict key_blockkey_source_address key_key_filekey_key_password key_cert_file key_cert_reqs key_ca_certskey_ssl_versionkey_ca_cert_dirkey_ssl_context key_maxsize key_headers key__proxykey__proxy_headerskey__proxy_configkey_socket_optionskey__socks_optionskey_assert_hostnamekey_assert_fingerprintkey_server_hostnamePoolKey)ruse_forwarding_for_https ProxyConfigcCs|}|d|d<|d|d<dD]}||vr,||dur,t||||<q|d}|dur>> manager = PoolManager(num_pools=2) >>> r = manager.request('GET', 'http://google.com/') >>> r = manager.request('GET', 'http://google.com/mail') >>> r = manager.request('GET', 'http://yahoo.com/') >>> len(manager.pools) 2 N cKs8t||||_t|ddd|_t|_t|_dS)NcSs|SN)close)prCrCrTsz&PoolManager.__init__..) dispose_func)r__init__connection_pool_kwrpoolspool_classes_by_schemekey_fn_by_schemerD)self num_poolsr>r`rCrCrTr_s zPoolManager.__init__cCs|SrZrCrdrCrCrT __enter__szPoolManager.__enter__cCs |dS)NF)clear)rdexc_typeexc_valexc_tbrCrCrT__exit__szPoolManager.__exit__cCsb|j|}|dur|j}dD]}||dq|dkr(tD]}||dq|||fi|S)a Create a new :class:`urllib3.connectionpool.ConnectionPool` based on host, port, scheme, and any additional pool keyword arguments. If ``request_context`` is provided, it is provided as keyword arguments to the pool class used. This method is used to actually create the connection pools handed out by :meth:`connection_from_url` and companion methods. It is intended to be overridden for customization. N)r<r=portrW)rbr`rDrL SSL_KEYWORDS)rdr<r=rmrOpool_clsrQkwrCrCrT _new_pools  zPoolManager._new_poolcCs|jdS)z Empty our store of pools and direct them all to close. This will not affect in-flight connections, but they will not be re-used after completion. N)rarhrfrCrCrTrhszPoolManager.clearrWcCsT|std||}|pd|d<|st|dd}||d<||d<||S)a Get a :class:`urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme. If ``port`` isn't given, it will be derived from the ``scheme`` using ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is provided, it is merged with the instance's ``connection_pool_kw`` variable and used to create the new connection pool, if one is needed. zNo host specified.rWr<Prmr=)r_merge_pool_kwargsrrHrEconnection_from_context)rdr=rmr< pool_kwargsrOrCrCrTconnection_from_hosts    z PoolManager.connection_from_hostcCs:|d}|j|}|st|||}|j||dS)z Get a :class:`urllib3.connectionpool.ConnectionPool` based on the request context. ``request_context`` must at least contain the ``scheme`` key and its value must be a key in ``key_fn_by_scheme`` instance variable. r<rO)rErcrHr connection_from_pool_key)rdrOr<pool_key_constructorpool_keyrCrCrTrts  z#PoolManager.connection_from_contextcCs|jj4|j|}|r|WdS|d}|d}|d}|j||||d}||j|<Wd|S1s;wY|S)a Get a :class:`urllib3.connectionpool.ConnectionPool` based on the provided pool key. ``pool_key`` should be a namedtuple that only contains immutable objects. At a minimum it must have the ``scheme``, ``host``, and ``port`` fields. Nr<r=rmrw)ralockrHrq)rdrzrOpoolr<r=rmrCrCrTrxs    z$PoolManager.connection_from_pool_keycCs t|}|j|j|j|j|dS)a Similar to :func:`urllib3.connectionpool.connection_from_url`. If ``pool_kwargs`` is not provided and a new pool needs to be constructed, ``self.connection_pool_kw`` is used to initialize the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs`` is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided ``pool_kwargs`` are not used. )rmr<ru)rrvr=rmr<)rdurlruurCrCrTconnection_from_urls zPoolManager.connection_from_urlc CsR|j}|r'|D]\}}|dur"z||=Wq ty!Yq w|||<q |S)a Merge a dictionary of override values for self.connection_pool_kw. This does not modify self.connection_pool_kw and returns a new dict. Any keys in the override dictionary with a value of ``None`` are removed from the merged dictionary. N)r`rDrGKeyError)rdoverridebase_pool_kwargsrQvaluerCrCrTrs/s    zPoolManager._merge_pool_kwargscCs"|jdurdSt|j|j|j S)z Indicates if the proxy requires the complete destination URL in the request. Normally this is only needed when not using an HTTP CONNECT tunnel. NF)proxyr proxy_configr<)rd parsed_urlrCrCrT!_proxy_requires_url_absolute_formCs  z-PoolManager._proxy_requires_url_absolute_formcCsD|jdus |dkr dS|jjdkrdStjr|jjs tddSdS)z Validates that were not attempting to do TLS in TLS connections on Python2 or with unsupported SSL implementations. NrXzfContacting HTTPS destinations through HTTPS proxies 'via CONNECT tunnels' is not supported in Python 2)rr<r PY2rr:r )rd url_schemerCrCrT$_validate_proxy_scheme_url_selectionPs z0PoolManager._validate_proxy_scheme_url_selectionTc Kst|}||j|j|j|j|jd}d|d<d|d<d|vr(|j|d<||r8|j ||fi|}n |j ||j fi|}|oH| }|sM|St ||}|j dkrYd}|d} t| tsjtj| |d } | jr||stt|d} | D]} | | jvr|d| d q}z | j||||d } Wnty| jr||YSw| |d<||d<td ||||j ||fi|S) aN Same as :meth:`urllib3.HTTPConnectionPool.urlopen` with custom cross-host redirect logic and only sends the request-uri portion of the ``url``. The given ``url`` parameter must be absolute, such that an appropriate :class:`urllib3.connectionpool.ConnectionPool` can be chosen for it. )rmr<Fassert_same_hostredirectr>i/GETretries)rN)response_poolzRedirecting %s -> %s)rrr<rvr=rmr>rDrurlopen request_uriget_redirect_locationrstatusrH isinstancerfrom_intremove_headers_on_redirect is_same_hostrJr iterkeysrErL incrementr raise_on_redirect drain_connloginfo) rdmethodr}rrpr~connrredirect_locationrr>headerrCrCrTrasP         zPoolManager.urlopen)rYNrZNrWNT)__name__ __module__ __qualname____doc__rrr_rgrlrqrhrvrtrxrrsrrrrCrCrCrTrs"       rcsReZdZdZ     dfdd Zdfdd Zdd d Zdfd d ZZS)ra> Behaves just like :class:`PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs. :param proxy_url: The URL of the proxy to be used. :param proxy_headers: A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication. :param proxy_ssl_context: The proxy SSL context is used to establish the TLS connection to the proxy when using HTTPS proxies. :param use_forwarding_for_https: (Defaults to False) If set to True will forward requests to the HTTPS proxy to be made on behalf of the client instead of creating a TLS tunnel via the CONNECT method. **Enabling this flag means that request and response headers and content will be visible from the HTTPS proxy** whereas tunneling keeps request and response headers and content private. IP address, target hostname, SNI, and port are always visible to an HTTPS proxy even when this flag is disabled. Example: >>> proxy = urllib3.ProxyManager('http://localhost:3128/') >>> r1 = proxy.request('GET', 'http://google.com/') >>> r2 = proxy.request('GET', 'http://httpbin.org/') >>> len(proxy.pools) 1 >>> r3 = proxy.request('GET', 'https://httpbin.org/') >>> r4 = proxy.request('GET', 'https://twitter.com/') >>> len(proxy.pools) 3 rYNFc  st|trd|j|j|jf}t|}|jdvrt|j|js-t|jd} |j | d}||_ |p3i|_ ||_ t |||_|j |d<|j |d<|j|d<tt|j||fi|dS)Nz %s://%s:%irVrr)rm_proxyr? _proxy_config)rrr<r=rmrr rrH_replacer proxy_headersproxy_ssl_contextr;rsuperrr_) rd proxy_urlrer>rrr:r`rrm __class__rCrTr_s(         zProxyManager.__init__rWcsD|dkrtt|j||||dStt|j|jj|jj|jj|dS)NrX)ru)rrrvrr=rmr<)rdr=rmr<rurrCrTrvs  z!ProxyManager.connection_from_hostcCs0ddi}t|j}|r||d<|r|||S)z Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user. Acceptz*/*Host)rnetlocupdate)rdr}r>headers_rrCrCrT_set_proxy_headerss  zProxyManager._set_proxy_headersTc sVt|}t|j|j|js|d|j}||||d<tt |j ||fd|i|S)z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.r>r) rrrrr<rHr>rrrr)rdrr}rrpr~r>rrCrTr s zProxyManager.urlopen)rYNNNFrrZr) rrrrr_rvrr __classcell__rCrCrrTrs*% rcKstdd|i|S)NrrC)r)r}rprCrCrTrsr). __future__r collections functoolslogging _collectionsrconnectionpoolrrr exceptionsrr r r r packagesr Zpackages.six.moves.urllib.parserrequestrZ util.proxyrZ util.retryrZutil.urlr__all__ getLoggerrrrn _key_fields namedtupler9_proxy_config_fieldsr;rUpartialrcrbrrrrCrCrCrTs<            6   s