Our integration platform now gets couple of "Unable to fetch URL: ..." errors everyday. I don't know what's the cause of this.
我们的集成平台现在每天都会收到几个“无法获取URL:...”错误。我不知道这是什么原因。
First GAE tries to authorize with OAuth2 from within application to BigQuery through the BigQuery API. it tries in every second for more than 30 seconds (that's just 1 of 30 similar lines):
第一个GAE尝试通过BigQuery API从应用程序内部向OAQuery授权OAuth2。它每秒都会尝试超过30秒(这只是30条相似线中的一条):
2015-05-12 05:59:02.727
URL being requested: https://www.googleapis.com/bigquery/v2/projects/XXX/jobs/job_w-z5K2zQObXeSaLy3hx7m4FOMXc?alt=json
Below is trace itself (just after logs presented above):
下面是跟踪本身(就在上面的日志之后):
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/connectors/connector_XXX_v1_0.py", line 189, in send
status = self._verify_status(resp, content)
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/connectors/connector_XXX_v1_0.py", line 250, in _verify_status
jobId=jobReference).execute()
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/modules/oauth2client/util.py", line 128, in positional_wrapper
return wrapped(*args, **kwargs)
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/modules/apiclient/http.py", line 676, in execute
body=self.body, headers=self.headers)
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/modules/oauth2client/util.py", line 128, in positional_wrapper
return wrapped(*args, **kwargs)
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/modules/oauth2client/client.py", line 490, in new_request
redirections, connection_type)
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/modules/httplib2/__init__.py", line 1570, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/modules/httplib2/__init__.py", line 1317, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/base/data/home/apps/s~XXX/processes:uno.383135852015228853/modules/httplib2/__init__.py", line 1286, in _conn_request
response = conn.getresponse()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/gae_override/httplib.py", line 536, in getresponse
'An error occured while connecting to the server: %s' % e)
error: An error occured while connecting to the server: Unable to fetch URL: https://www.googleapis.com/bigquery/v2/projects/XXX/jobs/job_d_ONsjBebYjreJuDD-ZdEqCHYv0?alt=json
These are from processes fetching data from Google Datastore and loading into BigQuery.
这些来自从Google Datastore获取数据并加载到BigQuery的流程。
These errors are undeterministic. Sometimes they happen. Sometimes don't. I can't see any pattern here.
这些错误是不确定的。有时他们会发生。有时不这样做。我在这里看不到任何模式。
One thing to note: up until 2015-05-01 there were no problems with those processes. Data was loaded into BQ without hassle.
有一点需要注意:直到2015-05-01,这些流程都没有问题。数据加载到BQ没有麻烦。
My assumption is that it may be somehow related to OAuth2.0. We use
我的假设是它可能与OAuth2.0有某种关系。我们用
oauth2client v1.1. The newest is 1.4.7. Maybe update would help? Or problem lies somewhere else?
oauth2client v1.1。最新的是1.4.7。也许更新会有帮助吗?或问题出在其他地方?
1 个解决方案
#1
To minimize this type of error implement the retry strategy in your application while inserting the Data into the BigQuery. Go through the document for better explaination
要最小化此类错误,请在将数据插入BigQuery时在应用程序中实施重试策略。浏览文档以获得更好的解释
#1
To minimize this type of error implement the retry strategy in your application while inserting the Data into the BigQuery. Go through the document for better explaination
要最小化此类错误,请在将数据插入BigQuery时在应用程序中实施重试策略。浏览文档以获得更好的解释