3 fH^+ @sdZddlmZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z mZddlmZddlmZdd lmZdd lmZd d lmZydd lmZWnek rdZYnXejeZGddde ZGddde Z GdddeZ!ej"Z"ej#Z#ej$Z$ej%Z%ej&Z&dS)aC This module provides a pool manager that uses Google App Engine's `URLFetch Service `_. Example usage:: from urllib3 import PoolManager from urllib3.contrib.appengine import AppEngineManager, is_appengine_sandbox if is_appengine_sandbox(): # AppEngineManager uses AppEngine's URLFetch API behind the scenes http = AppEngineManager() else: # PoolManager uses a socket-level API behind the scenes http = PoolManager() r = http.request('GET', 'https://google.com/') There are `limitations `_ to the URLFetch service and it may not be the best choice for your application. There are three options for using urllib3 on Google App Engine: 1. You can use :class:`AppEngineManager` with URLFetch. URLFetch is cost-effective in many circumstances as long as your usage is within the limitations. 2. You can use a normal :class:`~urllib3.PoolManager` by enabling sockets. Sockets also have `limitations and restrictions `_ and have a lower free quota than URLFetch. To use sockets, be sure to specify the following in your ``app.yaml``:: env_variables: GAE_USE_SOCKETS_HTTPLIB : 'true' 3. If you are using `App Engine Flexible `_, you can use the standard :class:`PoolManager` without any configuration or special environment variables. )absolute_importN)urljoin) HTTPError HTTPWarning MaxRetryError ProtocolError TimeoutErrorSSLError)RequestMethods) HTTPResponse)Timeout)Retry)_appengine_environ)urlfetchc@s eZdZdS)AppEnginePlatformWarningN)__name__ __module__ __qualname__rr`_. Notably it will raise an :class:`AppEnginePlatformError` if: * URLFetch is not available. * If you attempt to use this on App Engine Flexible, as full socket support is available. * If a request size is more than 10 megabytes. * If a response size is more than 32 megabtyes. * If you use an unsupported request method such as OPTIONS. Beyond those cases, it will raise normal urllib3 errors. NTcCs@ts tdtjdttj||||_||_|p8t j |_ dS)Nz.URLFetch is not available in this environment.zurllib3 is using URLFetch on Google App Engine sandbox instead of sockets. To use sockets directly instead of URLFetch see https://urllib3.readthedocs.io/en/latest/reference/urllib3.contrib.html.) rrwarningswarnrr __init__validate_certificateurlfetch_retriesrDEFAULTretries)selfheadersr rrrrrrcs zAppEngineManager.__init__cCs|S)Nr)r!rrr __enter__|szAppEngineManager.__enter__cCsdS)NFr)r!exc_typeexc_valexc_tbrrr__exit__szAppEngineManager.__exit__cKs|j||}yF|o |jdko |j} tj||||p2id|jo<| |j||jd} WnBtjk r} zt || WYdd} ~ Xntj k r} z$dt | krt d| t | WYdd} ~ Xntjk r} z(dt | krt||| dt | WYdd} ~ Xntjk r6} zt d| WYdd} ~ Xn`tjk rb} zt| WYdd} ~ Xn4tjk r} zt d || WYdd} ~ XnX|j| fd |i|} |o| j} | rr|jr|jrt||d n| jd krd }y|j||| |d}Wn*tk r.|jr*t||d | SX|j| tjd|| t|| }|j||||f|||d|St| jd}|j || j|r|j||| |d}tjd||j!| |j||f|||||d|S| S)NrF)payloadmethodr"Zallow_truncatedfollow_redirectsdeadlinerz too largezOURLFetch request too large, URLFetch only supports requests up to 10mb in size.zToo many redirects)reasonzPURLFetch response too large, URLFetch only supportsresponses up to 32mb in size.z$URLFetch does not support method: %sr ztoo many redirectsi/GET)response_poolzRedirecting %s -> %s)r redirecttimeoutz Retry-Afterz Retry: %s)bodyr"r r0r1)" _get_retriesr0totalrfetchr_get_absolute_timeoutrZDeadlineExceededErrorr ZInvalidURLErrorstrrrZ DownloadErrorrZResponseTooLargeErrorZSSLCertificateErrorr ZInvalidMethodError#_urlfetch_response_to_http_responseget_redirect_locationraise_on_redirectstatus incrementsleep_for_retrylogdebugrurlopenbool getheaderis_retrysleep)r!r)urlr2r"r r0r1 response_kwr*r.e http_responseredirect_locationZ redirect_urlhas_retry_afterrrrr@s         zAppEngineManager.urlopencKstr"|jjd}|dkr"|jd=|jjd}|dkrZ|jd}|jddj||jd<tftj|j |j |j|j d|}tftj|j |j|j |d|S)Nzcontent-encodingdeflateztransfer-encodingchunked,)r2msgr"r;)r2r"r;original_response) is_prod_appenginer"getsplitremovejoinr ioBytesIOcontentZ header_msg status_code)r!Z urlfetch_resprFcontent_encodingZtransfer_encoding encodingsrOrrrr8s*      z4AppEngineManager._urlfetch_response_to_http_responsecCsB|tjkrdSt|tr>|jdk s,|jdk r8tjdt|jS|S)NzdURLFetch does not support granular timeout settings, reverting to total or default URLFetch timeout.) r DEFAULT_TIMEOUT isinstance_read_connectrrrr4)r!r1rrrr6s  z&AppEngineManager._get_absolute_timeoutcCs>t|tstj|||jd}|js.|js.|jr:tjdt |S)N)r0defaultzhURLFetch only supports total retries and does not recognize connect, read, or redirect retry parameters.) r\rfrom_intr connectreadr0rrr)r!r r0rrrr3&s zAppEngineManager._get_retries)NNTT) rrr__doc__rr#r'r r[r@r8r6r3rrrrrOs j$ r)'rc __future__rrUloggingrZpackages.six.moves.urllib.parser exceptionsrrrrr r requestr r.r Z util.timeoutr Z util.retryrrZgoogle.appengine.apir ImportError getLoggerrr>rrrZ is_appengineZis_appengine_sandboxZis_local_appenginerPZis_prod_appengine_mvmsrrrr's2         h