使用 python-gitlab 列出所有项目时列出超出范围的索引
list index out of range when listing all projects using python-gitlab
我正在尝试使用 python-gitlab 列出我所有的 gitlab 项目。但是当我使用下面的代码列出我在gitlab中的部分项目时，我得到了list index out of range错误。
# file name: poc.py
import gitlab
gl = gitlab.Gitlab(
'http://my-gitlab.com:10080',
private_token='<my_private_token>'
)
projects = gl.projects.list(all=True)
for project in projects:
print(project.attributes['name'])
我的gitlab服务器有300多个项目。
完整的错误信息如下：
Traceback (most recent call last):
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 783, in next
item = self._data[self._current]
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x107784be0>: Failed to establish a new connection: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/MyUserName/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='my-gitlab.com', port=80): Max retries exceeded with url: /api/v4/projects?membership=false&order_by=created_at&owned=false&page=2&per_page=20&simple=false&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x107784be0>: Failed to establish a new connection: [Errno 61] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "poc.py", line 15, in <module>
projects = gl.projects.list(as_list=False, all=True)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/exceptions.py", line 255, in wrapped_f
return f(*args, **kwargs)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/mixins.py", line 133, in list
obj = self.gitlab.http_list(path, **data)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 597, in http_list
return list(GitlabList(self, url, query_data, **kwargs))
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 779, in __next__
return self.next()
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 788, in next
self._query(self._next_url)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 716, in _query
**kwargs)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 498, in http_request
result = self.session.send(prepped, timeout=timeout, **settings)
File "/Users/MyUserName/.local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/Users/MyUserName/.local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='my-gitlab.com', port=80): Max retries exceeded with url: /api/v4/projects?membership=false&order_by=created_at&owned=false&page=2&per_page=20&simple=false&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x107784be0>: Failed to establish a new connection: [Errno 61] Connection refused',))
错误信息requests.exceptions.ConnectionError: HTTPConnectionPool(host='my-gitlab.com', port=80): Max retries exceeded说它使用端口80而不是10080与gitlab服务器通信。这是因为 line 825 of gitlab/init.py 触发了 IndexError。所以 python-gitlab 尝试使用端口 80 看看这个端口是否有效。
我检查过self._data的长度是20。而且我不知道为什么self._current被添加到20。 self._data[20] 触发 IndexError 因为索引的最大值是 19.
现在，我使用以下解决方法列出所有项目：
import gitlab
gl = gitlab.Gitlab(
'http://my-gitlab.com:10080',
private_token='<my_private_token>'
)
current_page = 1
projects = dest_gitlab.projects.list(as_list=False)
total_pages = projects.total_pages
while (current_page <= total_pages):
projects = gl.projects.list(as_list=False, per_page=100, page=current_page)
for project in projects:
print(project.attributes['name'])
current_page += 1
但是这样列出所有项目很不方便。
文档 here 没有提及任何有关此错误的信息。
有人知道如何使用 python-gitlab 列出所有项目吗？
这是 python-gitlab 的错误吗？
调用 gl.projects.list(all=True) 时 len(self._data) == 20 是预期的行为吗？
之所以python-gitlab使用80端口与gitlab服务器通信是因为这个gitlab bug.
目前我只能使用解决方法或者等待gitlab修复这个bug。
import gitlab
gl = gitlab.Gitlab(
'http://my-gitlab.com:10080',
private_token='<my_private_token>'
)
current_page = 1
projects = dest_gitlab.projects.list(as_list=False)
total_pages = projects.total_pages
while (current_page <= total_pages):
projects = gl.projects.list(as_list=False, per_page=100, page=current_page)
for project in projects:
print(project.attributes['name'])
current_page += 1
我正在尝试使用 python-gitlab 列出我所有的 gitlab 项目。但是当我使用下面的代码列出我在gitlab中的部分项目时，我得到了list index out of range错误。
# file name: poc.py
import gitlab
gl = gitlab.Gitlab(
'http://my-gitlab.com:10080',
private_token='<my_private_token>'
)
projects = gl.projects.list(all=True)
for project in projects:
print(project.attributes['name'])
我的gitlab服务器有300多个项目。
完整的错误信息如下：
Traceback (most recent call last):
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 783, in next
item = self._data[self._current]
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x107784be0>: Failed to establish a new connection: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/MyUserName/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='my-gitlab.com', port=80): Max retries exceeded with url: /api/v4/projects?membership=false&order_by=created_at&owned=false&page=2&per_page=20&simple=false&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x107784be0>: Failed to establish a new connection: [Errno 61] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "poc.py", line 15, in <module>
projects = gl.projects.list(as_list=False, all=True)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/exceptions.py", line 255, in wrapped_f
return f(*args, **kwargs)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/mixins.py", line 133, in list
obj = self.gitlab.http_list(path, **data)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 597, in http_list
return list(GitlabList(self, url, query_data, **kwargs))
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 779, in __next__
return self.next()
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 788, in next
self._query(self._next_url)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 716, in _query
**kwargs)
File "/Users/MyUserName/.pyenv/versions/3.6.5/lib/python3.6/site-packages/gitlab/__init__.py", line 498, in http_request
result = self.session.send(prepped, timeout=timeout, **settings)
File "/Users/MyUserName/.local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/Users/MyUserName/.local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='my-gitlab.com', port=80): Max retries exceeded with url: /api/v4/projects?membership=false&order_by=created_at&owned=false&page=2&per_page=20&simple=false&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x107784be0>: Failed to establish a new connection: [Errno 61] Connection refused',))
错误信息requests.exceptions.ConnectionError: HTTPConnectionPool(host='my-gitlab.com', port=80): Max retries exceeded说它使用端口80而不是10080与gitlab服务器通信。这是因为 line 825 of gitlab/init.py 触发了 IndexError。所以 python-gitlab 尝试使用端口 80 看看这个端口是否有效。
我检查过self._data的长度是20。而且我不知道为什么self._current被添加到20。 self._data[20] 触发 IndexError 因为索引的最大值是 19.
现在，我使用以下解决方法列出所有项目：
import gitlab
gl = gitlab.Gitlab(
'http://my-gitlab.com:10080',
private_token='<my_private_token>'
)
current_page = 1
projects = dest_gitlab.projects.list(as_list=False)
total_pages = projects.total_pages
while (current_page <= total_pages):
projects = gl.projects.list(as_list=False, per_page=100, page=current_page)
for project in projects:
print(project.attributes['name'])
current_page += 1
但是这样列出所有项目很不方便。
文档 here 没有提及任何有关此错误的信息。
有人知道如何使用 python-gitlab 列出所有项目吗？
这是 python-gitlab 的错误吗？
调用 gl.projects.list(all=True) 时 len(self._data) == 20 是预期的行为吗？
之所以python-gitlab使用80端口与gitlab服务器通信是因为这个gitlab bug.
目前我只能使用解决方法或者等待gitlab修复这个bug。
import gitlab
gl = gitlab.Gitlab(
'http://my-gitlab.com:10080',
private_token='<my_private_token>'
)
current_page = 1
projects = dest_gitlab.projects.list(as_list=False)
total_pages = projects.total_pages
while (current_page <= total_pages):
projects = gl.projects.list(as_list=False, per_page=100, page=current_page)
for project in projects:
print(project.attributes['name'])
current_page += 1