Python requests.exceptions.ChunkedEncodingError

Question

我正在编写一个 python 脚本来执行以下这些步骤。

查询一个MongoDB数据库
解析和汇总结果
通过 REST API

这在大部分时间都有效，但偶尔我会看到这个错误：

requests.exceptions.ChunkedEncodingError: ("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer'))

此错误会停止脚本并阻止捕获整个数据集。

我可以做些什么来缓解这个问题？

Python 2.7.5

代码

#!/usr/bin/env python

from config import *

import os, sys

mypath = os.path.dirname(os.path.realpath(__file__))
sys.path.append(os.path.join(mypath, "api-python-client"))

from apiclient.mongo import *

from pymongo import MongoClient

import json

import requests

from bson.json_util import dumps

client = MongoClient(mongo_uri)

#Create ServiceNow URL
svcnow_url = create_svcnow_url('u_imp_cmps')

#BITSDB Nmap Collection
db = client[mongo_db]

#Aggregate - RDBMS equivalent to Alias select x as y
#Rename fields to match ServiceNow field names
computers = db['computer'].aggregate([
        {"$unwind": "$hostnames"},
        {"$project" : {
                "_id":0,
                "u_hostname": "$hostnames.name",
                "u_ipv4": "$addresses.ipv4",
                "u_status": "$status.state",
                "u_updated_timestamp": "$last_seen"
        }}

])

j = dumps({"records":computers})
#print(j)


#Set proper headers
headers = {"Content-Type":"application/json","Accept":"application/json"}

#Build HTTP Request
response = requests.post(url=svcnow_url, auth=(svcnow_user, svcnow_pwd), headers=headers ,data=j)

#Check for HTTP codes other than 200
if response.status_code != 200:
        print('Status:', response.status_code, 'Headers:', response.headers, 'Response Text', response.text, 'Error Response:',response.json())
        exit()

#Decode the JSON response into a dictionary and use the data
print('Status:',response.status_code,'Headers:',response.headers,'Response:',response.json())

错误

Traceback (most recent call last):
  File "/usr/src/computer_pingable_import.py", line 50, in <module>
    response = requests.post(url=svcnow_url, auth=(svcnow_user, svcnow_pwd), headers=headers ,data=j)
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 107, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 608, in send
    r.content
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 737, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 663, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer'))

另一次尝试和略有不同的错误消息：

Traceback (most recent call last):
  File "/usr/src/computer_pingable_import.py", line 50, in <module>
    response = requests.post(url=svcnow_url, auth=(svcnow_user, svcnow_pwd), headers=headers ,data=j)
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 107, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 426, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))

Answer 1

ServiceNow prevents REST transactions 来自运行超过 60 秒。

我不太确定您的数据集有多大，但您会希望将数据分成更小的部分以确保事务始终运行。

Python requests.exceptions.ChunkedEncodingError

Python requests.exceptions.ChunkedEncodingError

python

mongodb

rest

import

servicenow