在 gitlab CI/CD 管道中使用 pytest 测试烧瓶应用程序时如何忽略某些脚本？
How to ignore certain scripts while testing flask app using pytest in gitlab CI/CD pipeline?
我有一个 flask-restx 文件夹，其结构如下
.
├── app
│ ├── extensions.py
│ ├── __init__.py
│ └── pv_dimensioning
│ ├── controller.py
│ ├── __init__.py
│ ├── models
│ │ ├── dto.py
│ │ ├── __init__.py
│ │ ├── input_output_model.py
│ │ └── vendor_models.py
│ ├── services
│ │ ├── calculator.py
│ │ ├── database.py
│ │ ├── db_crud.py
│ │ ├── db_input_output.py
│ │ ├── __init__.py
│ │ └── processor.py
│ └── utils
│ ├── decode_verify_jwt.py
│ ├── decorator.py
│ └── __init__.py
├── config.py
├── main.py
├── package.json
├── package-lock.json
├── Pipfile
├── Pipfile.lock
├── README.md
├── serverless.yml
└── tests
└── test_processor.py
应用程序连接到数据库，这就是应用程序中有许多脚本需要 VPN 连接的原因。
我正在使用 pytest 编写测试，然后我将在 gitlab CI/CD 上 运行 以获得正确的 coverage。我想避免或省略在连接 VPN 时只能 运行 的脚本，并且只为不需要 VPN 的脚本编写测试。 （我对需要 VPN 的脚本进行了测试，但我只是不想 运行 它们在 CI/CD 管道中）
唯一不需要 VPN 的脚本是 processor.py，测试在 test_processor.py.
我想避免的脚本在 .coveragerc:
[run]
omit =
*/site-packages/*
*/distutils/*
tests/*
/usr/*
app/__init__.py
app/extensions.py
app/pv_dimensioning/models/*
app/pv_dimensioning/utils/*
app/pv_dimensioning/controller.py
app/pv_dimensioning/services/calculator.py
app/pv_dimensioning/services/database.py
app/pv_dimensioning/services/db_crud.py
app/pv_dimensioning/services/db_input_output.py
[html]
directory = htmlcov
.gitlab-ci.yml
的覆盖部分
stages:
- coverage
coverage:
image: python:3.7
stage: coverage
artifacts:
paths:
- htmlcov/
before_script:
- apt-get -y update
- apt-get install curl
- pip install pipenv
- pipenv install --dev
script:
- pipenv run python -m coverage run -m pytest
- pipenv run python -m coverage report -m
- pipenv run python -m coverage html
after_script:
- pipenv run bash <(curl -s https://codecov.io/bash)
当我运行管道中的测试时，出现以下错误：
$ pipenv run python -m coverage run -m pytest
============================= test session starts ==============================
platform linux -- Python 3.7.10, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /builds/EC/tool/dt-service
plugins: cov-2.11.0
collected 0 items / 1 error
==================================== ERRORS ====================================
___________________ ERROR collecting tests/test_processor.py ___________________
/usr/local/lib/python3.7/urllib/request.py:1350: in do_open
encode_chunked=req.has_header('Transfer-encoding'))
/usr/local/lib/python3.7/http/client.py:1277: in request
self._send_request(method, url, body, headers, encode_chunked)
/usr/local/lib/python3.7/http/client.py:1323: in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
/usr/local/lib/python3.7/http/client.py:1272: in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
/usr/local/lib/python3.7/http/client.py:1032: in _send_output
self.send(msg)
/usr/local/lib/python3.7/http/client.py:972: in send
self.connect()
/usr/local/lib/python3.7/http/client.py:1439: in connect
super().connect()
/usr/local/lib/python3.7/http/client.py:944: in connect
(self.host,self.port), self.timeout, self.source_address)
/usr/local/lib/python3.7/socket.py:707: in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
/usr/local/lib/python3.7/socket.py:752: in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
E socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
tests/test_processor.py:2: in <module>
from app.pv_dimensioning.services.processor import PreProcessings
app/pv_dimensioning/__init__.py:4: in <module>
from .controller import admin_crud_ns as admin_crud_namespace
app/pv_dimensioning/controller.py:5: in <module>
from .services.calculator import DimensionCalculator
app/pv_dimensioning/services/calculator.py:3: in <module>
from .database import DatabaseService
app/pv_dimensioning/services/database.py:6: in <module>
from ..utils.decode_verify_jwt import verifier
app/pv_dimensioning/utils/decode_verify_jwt.py:12: in <module>
with urllib.request.urlopen(keys_url) as f:
/usr/local/lib/python3.7/urllib/request.py:222: in urlopen
return opener.open(url, data, timeout)
/usr/local/lib/python3.7/urllib/request.py:525: in open
response = self._open(req, data)
/usr/local/lib/python3.7/urllib/request.py:543: in _open
'_open', req)
/usr/local/lib/python3.7/urllib/request.py:503: in _call_chain
result = func(*args)
/usr/local/lib/python3.7/urllib/request.py:1393: in https_open
context=self._context, check_hostname=self._check_hostname)
/usr/local/lib/python3.7/urllib/request.py:1352: in do_open
raise URLError(err)
E urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
=========================== short test summary info ============================
ERROR tests/test_processor.py - urllib.error.URLError: <urlopen error [Errno ...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 1.62s ===============================
在跟踪中，可以看出我试图忽略的脚本并没有被忽略。我做错了什么？
据我了解，覆盖率 是关于报告测试了多少代码库，而不是 运行 测试了哪些代码。您正在做的是从报告中排除内容，而不是停止正在创建的报告的数据。
如果您知道测试会失败（由于外部配置），您应该做的是跳过测试。幸运的是 pytest provides for this 使用了 skipif 装饰器。
我会在 tests/conftest.py 中创建一个函数，如果 VPN 处于活动状态，它会跳过测试。类似于：
import socket
import pytest
def _requires_vpn():
has_vpn = False
try:
socket.gethostbyname("<some address only accessible in the VPN>")
has_vpn = True
except socket.error:
pass
return pytest.mark.skipif(not has_vpn, reason="access to the vpn is required")
requires_vpn = _requires_vpn() # this is like the minversion example on the link
然后你可以忽略整个测试文件，在顶部添加这个：
pytestmark = requires_vpn
并且你可以通过装饰这个来跳过特定的测试。
@requires_vpn
def test_this_will_be_skipped():
pass
我有一个 flask-restx 文件夹，其结构如下
.
├── app
│ ├── extensions.py
│ ├── __init__.py
│ └── pv_dimensioning
│ ├── controller.py
│ ├── __init__.py
│ ├── models
│ │ ├── dto.py
│ │ ├── __init__.py
│ │ ├── input_output_model.py
│ │ └── vendor_models.py
│ ├── services
│ │ ├── calculator.py
│ │ ├── database.py
│ │ ├── db_crud.py
│ │ ├── db_input_output.py
│ │ ├── __init__.py
│ │ └── processor.py
│ └── utils
│ ├── decode_verify_jwt.py
│ ├── decorator.py
│ └── __init__.py
├── config.py
├── main.py
├── package.json
├── package-lock.json
├── Pipfile
├── Pipfile.lock
├── README.md
├── serverless.yml
└── tests
└── test_processor.py
应用程序连接到数据库，这就是应用程序中有许多脚本需要 VPN 连接的原因。
我正在使用 pytest 编写测试，然后我将在 gitlab CI/CD 上 运行 以获得正确的 coverage。我想避免或省略在连接 VPN 时只能 运行 的脚本，并且只为不需要 VPN 的脚本编写测试。 （我对需要 VPN 的脚本进行了测试，但我只是不想 运行 它们在 CI/CD 管道中）
唯一不需要 VPN 的脚本是 processor.py，测试在 test_processor.py.
我想避免的脚本在 .coveragerc:
[run]
omit =
*/site-packages/*
*/distutils/*
tests/*
/usr/*
app/__init__.py
app/extensions.py
app/pv_dimensioning/models/*
app/pv_dimensioning/utils/*
app/pv_dimensioning/controller.py
app/pv_dimensioning/services/calculator.py
app/pv_dimensioning/services/database.py
app/pv_dimensioning/services/db_crud.py
app/pv_dimensioning/services/db_input_output.py
[html]
directory = htmlcov
.gitlab-ci.yml
stages:
- coverage
coverage:
image: python:3.7
stage: coverage
artifacts:
paths:
- htmlcov/
before_script:
- apt-get -y update
- apt-get install curl
- pip install pipenv
- pipenv install --dev
script:
- pipenv run python -m coverage run -m pytest
- pipenv run python -m coverage report -m
- pipenv run python -m coverage html
after_script:
- pipenv run bash <(curl -s https://codecov.io/bash)
当我运行管道中的测试时，出现以下错误：
$ pipenv run python -m coverage run -m pytest
============================= test session starts ==============================
platform linux -- Python 3.7.10, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /builds/EC/tool/dt-service
plugins: cov-2.11.0
collected 0 items / 1 error
==================================== ERRORS ====================================
___________________ ERROR collecting tests/test_processor.py ___________________
/usr/local/lib/python3.7/urllib/request.py:1350: in do_open
encode_chunked=req.has_header('Transfer-encoding'))
/usr/local/lib/python3.7/http/client.py:1277: in request
self._send_request(method, url, body, headers, encode_chunked)
/usr/local/lib/python3.7/http/client.py:1323: in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
/usr/local/lib/python3.7/http/client.py:1272: in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
/usr/local/lib/python3.7/http/client.py:1032: in _send_output
self.send(msg)
/usr/local/lib/python3.7/http/client.py:972: in send
self.connect()
/usr/local/lib/python3.7/http/client.py:1439: in connect
super().connect()
/usr/local/lib/python3.7/http/client.py:944: in connect
(self.host,self.port), self.timeout, self.source_address)
/usr/local/lib/python3.7/socket.py:707: in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
/usr/local/lib/python3.7/socket.py:752: in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
E socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
tests/test_processor.py:2: in <module>
from app.pv_dimensioning.services.processor import PreProcessings
app/pv_dimensioning/__init__.py:4: in <module>
from .controller import admin_crud_ns as admin_crud_namespace
app/pv_dimensioning/controller.py:5: in <module>
from .services.calculator import DimensionCalculator
app/pv_dimensioning/services/calculator.py:3: in <module>
from .database import DatabaseService
app/pv_dimensioning/services/database.py:6: in <module>
from ..utils.decode_verify_jwt import verifier
app/pv_dimensioning/utils/decode_verify_jwt.py:12: in <module>
with urllib.request.urlopen(keys_url) as f:
/usr/local/lib/python3.7/urllib/request.py:222: in urlopen
return opener.open(url, data, timeout)
/usr/local/lib/python3.7/urllib/request.py:525: in open
response = self._open(req, data)
/usr/local/lib/python3.7/urllib/request.py:543: in _open
'_open', req)
/usr/local/lib/python3.7/urllib/request.py:503: in _call_chain
result = func(*args)
/usr/local/lib/python3.7/urllib/request.py:1393: in https_open
context=self._context, check_hostname=self._check_hostname)
/usr/local/lib/python3.7/urllib/request.py:1352: in do_open
raise URLError(err)
E urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
=========================== short test summary info ============================
ERROR tests/test_processor.py - urllib.error.URLError: <urlopen error [Errno ...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 1.62s ===============================
在跟踪中，可以看出我试图忽略的脚本并没有被忽略。我做错了什么？
据我了解，覆盖率 是关于报告测试了多少代码库，而不是 运行 测试了哪些代码。您正在做的是从报告中排除内容，而不是停止正在创建的报告的数据。
如果您知道测试会失败（由于外部配置），您应该做的是跳过测试。幸运的是 pytest provides for this 使用了 skipif 装饰器。
我会在 tests/conftest.py 中创建一个函数，如果 VPN 处于活动状态，它会跳过测试。类似于：
import socket
import pytest
def _requires_vpn():
has_vpn = False
try:
socket.gethostbyname("<some address only accessible in the VPN>")
has_vpn = True
except socket.error:
pass
return pytest.mark.skipif(not has_vpn, reason="access to the vpn is required")
requires_vpn = _requires_vpn() # this is like the minversion example on the link
然后你可以忽略整个测试文件，在顶部添加这个：
pytestmark = requires_vpn
并且你可以通过装饰这个来跳过特定的测试。
@requires_vpn
def test_this_will_be_skipped():
pass