当存在多个场景时 Behat 挂起，但适用于单个场景
Behat hangs when there are multiple scenarios, but works on a single one
我有这样写的 Behat 测试用例：
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
Background:
Given step 1
And step 2
@Ready
Scenario: Deliver now
When step 3
Then step 4
@NoneReady
Scenario: Deliver later
When step a
Then step b
And step c
@AddressNotCovered
Scenario: Address Not Covered
When step i
Then step ii
如果我 运行 behat 在单个标签上，它工作得很好：
$ behat --tags=Ready
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now # tests/features/Checkout/CheckOut.feature:9
step 1
And step 2
..
1 scenario (1 passed)
7 steps (7 passed)
0m3.85s (36.62Mb)
但是如果我 运行 它在多个标签上，它会挂在第一个标签的末尾：
behat --tags=Ready,AddressNotCovered
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now # tests/features/Checkout/CheckOut.feature:9
Given step ..
..
And ..
// hangs here
我做错了什么？
环境
Laravel 5.4
Behat 3.1.0
PHP 7.1.23
PHPUnit 5.7.27
来自我的 composer.json
"require": {
"php": ">=5.5.9",
"laravel/framework": "5.4.*",
..
"behat/behat": "3.1.0",
"laracasts/behat-laravel-extension": "^1.1",
},
"require-dev": {
"phpunit/phpunit": "~5.7",
"phpspec/phpspec": "~2.1",
"johnkary/phpunit-speedtrap": "^1.0",
},
Behat.yml
default:
extensions:
Laracasts\Behat:
env_path: .env.testing
autoload:
- ./tests/features/bootstrap
suites:
Checkout:
paths: [./tests/features/Checkout]
contexts: [CheckoutFeatureContext]
更新
我试图创建示例小黄瓜来说明上述问题。我 运行 在尝试自动附加片段时遇到了同样的问题。附加片段适用于单个场景，但在多个场景中失败：
工作示例：单一场景
# tests/features/Example/Example.feature
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background:
Givens setup condition 1
And setup condition 2
Scenario: scenario one
When I perform first sample trigger point
Then result one must happen
And result two must happen
当我运行下面的命令
behat tests/features/Example/Example.feature --append-snippets
添加片段效果很好
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background: # tests/features/Example/Example.feature:9
Givens setup condition 1
And setup condition 2
Scenario: scenario one # tests/features/Example/Example.feature:13
When I perform first sample trigger point
Then result one must happen
And result two must happen
1 scenario (1 undefined)
4 steps (4 undefined)
0m0.48s (24.63Mb)
u tests/features/bootstrap/FeatureContext.php - `setup condition 2` definition added
u tests/features/bootstrap/FeatureContext.php - `I perform first sample trigger point` definition added
u tests/features/bootstrap/FeatureContext.php - `result one must happen` definition added
u tests/features/bootstrap/FeatureContext.php - `result two must happen` definition added
失败示例：多个场景
当我们有多个场景时
# tests/features/Example/Example.feature
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background:
Givens setup condition 1
And setup condition 2
Scenario: scenario one
When I perform first sample trigger point
Then result one must happen
And result two must happen
Scenario: scenario two
When I perform second sample trigger point
Then result a must happen
And result b must happen
运行使用相同的 --append-snippets 命令阻塞：
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background: # tests/features/Example/Example.feature:9
Givens setup condition 1
And setup condition 2
Scenario: scenario one # tests/features/Example/Example.feature:13
When I perform first sample trigger point
Then result one must happen
And result two must happen
^C // had to abort here
以下步骤适用于 Arch Linux 和 PHP 7.3：
composer global require laravel/installer
laravel new behat-laravel
cd behat-laravel
composer require behat/behat behat/mink behat/mink-extension laracasts/behat-laravel-extension --dev
touch behat.yml
# edit behat.yml
# edit features/bootstrap/FeatureContext.php
vendor/bin/behat --init
# Add steps and tag them
vendor/bin/behat
vendor/bin/behat --tags Ready,NoneReady
# behat.yml
default:
extensions:
Laracasts\Behat:
# env_path: .env.behat
Behat\MinkExtension:
default_session: laravel
laravel: ~
<?php
// features/bootstrap/FeatureContext.php
use Behat\Behat\Hook\Scope\AfterStepScope;
use Behat\Behat\Tester\Exception\PendingException;
use Behat\Behat\Context\Context;
use Behat\Behat\Context\SnippetAcceptingContext;
use Behat\Gherkin\Node\PyStringNode;
use Behat\Gherkin\Node\TableNode;
#This will be needed if you require "behat/mink-selenium2-driver"
#use Behat\Mink\Driver\Selenium2Driver;
use Behat\MinkExtension\Context\MinkContext;
/**
* Defines application features from the specific context.
*/
class FeatureContext extends MinkContext implements Context, SnippetAcceptingContext
{
/**
* Initializes context.
*
* Every scenario gets its own context instance.
* You can also pass arbitrary arguments to the
* context constructor through behat.yml.
*/
public function __construct()
{
}
/**
* @When stepready :arg1
*/
public function stepready($arg1)
{
return true;
}
/**
* @When steplater :arg1
*/
public function steplater($arg1)
{
return true;
}
}
# feature/customer.feature
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now
When stepready 1
Then stepready 2
@NoneReady
Scenario: Deliver later
When steplater 1
Then steplater 2
frosch ➜ behat-laravel vendor/bin/behat --tags Ready,NoneReady
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now # features/customer.feature:7
When stepready 1 # FeatureContext::stepready()
Then stepready 2 # FeatureContext::stepready()
@NoneReady
Scenario: Deliver later # features/customer.feature:12
When steplater 1 # FeatureContext::steplater()
Then steplater 2 # FeatureContext::steplater()
2 Szenarien (2 bestanden)
4 Schritte (4 bestanden)
0m0.02s (18.47Mb)
原来上面的例子太简单了。在做了一些研究之后（特别有用的是 this post），我意识到这个 "stalling" 是由于每次测试后都拆除了数据库。所以这是修复它的方法：
首先，我在我的 FeatureContext class:
中用 DatabaseMigrations 替换了 DatabaseTransactions
class FeatureContext extends TestCase implements Context, SnippetAcceptingContext
{
use DatabaseMigrations, ..
鉴于上述情况，我从我的 bitbucket 管道脚本中删除了手动迁移命令
- php artisan --env=testing config:cache
这是有道理的，因为使用新代码，每次测试前都会刷新和迁移数据库。
然后我将 setUp() 调用添加到 behat hooks:
/** @BeforeScenario */
public function before(BeforeScenarioScope $scope)
{
parent::setUp();
}
就是这样。这个解决方案最好的部分是它使我的本地测试环境与 bitbucket 管道完全一致，因此结果始终相同。
进一步解释：来自我们的 wiki
一般来说，最好重新开始每个测试，不要遗留上一个测试的遗留物（尤其是在涉及数据库时）。 In the words of laravel:
It is often useful to reset your database after each test so that data
from a previous test does not interfere with subsequent tests.
为此，我们使用迁移。也就是说，由于我们实际使用的是 Behat，因此我们需要在每个场景生命周期之前和之后进行此迁移。我们使用 Behat's hooks 来做到这一点。我们在这里这样做：
/** @BeforeScenario */
public function before(BeforeScenarioScope $scope)
{
parent::setUp();
}
parent::setUP() 告诉 Laravel framework 在每个场景前后做必要的工作：
protected function setUp()
{
if (! $this->app) {
$this->refreshApplication();
}
$this->setUpTraits(); <---- here
..
这又会调用设置特征：
protected function setUpTraits()
{
$uses = array_flip(class_uses_recursive(static::class));
if (isset($uses[DatabaseMigrations::class])) {
$this->runDatabaseMigrations();
}
..
这叫这个
public function runDatabaseMigrations()
{
$this->artisan('migrate:fresh');
$this->app[Kernel::class]->setArtisan(null);
$this->beforeApplicationDestroyed(function () {
$this->artisan('migrate:rollback');
RefreshDatabaseState::$migrated = false;
});
}
请注意，一旦应用程序被销毁，Laravel 也会回滚更改。了解这一点非常重要，这样可以防止在存在多个场景且给定场景出现时 Behat 停滞。还要记住，当我们像这样使用 Gherkin 时：
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
Background:
Given step 1
And step 2
@Ready
Scenario: Deliver now
When step 3
Then step 4
@NoneReady
Scenario: Deliver later
When step a
Then step b
And step c
然后每个场景都从后台步骤开始，而不是场景步骤本身
示例：
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
Background:
Given step 1 <-- every scenario starts here, so we call setup before this step
And step 2
@Ready
Scenario: Deliver now
When step 3 <-- not here
Then step 4
@NoneReady
Scenario: Deliver later
When step a
Then step b
And step c
我有这样写的 Behat 测试用例：
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
Background:
Given step 1
And step 2
@Ready
Scenario: Deliver now
When step 3
Then step 4
@NoneReady
Scenario: Deliver later
When step a
Then step b
And step c
@AddressNotCovered
Scenario: Address Not Covered
When step i
Then step ii
如果我 运行 behat 在单个标签上，它工作得很好：
$ behat --tags=Ready
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now # tests/features/Checkout/CheckOut.feature:9
step 1
And step 2
..
1 scenario (1 passed)
7 steps (7 passed)
0m3.85s (36.62Mb)
但是如果我 运行 它在多个标签上，它会挂在第一个标签的末尾：
behat --tags=Ready,AddressNotCovered
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now # tests/features/Checkout/CheckOut.feature:9
Given step ..
..
And ..
// hangs here
我做错了什么？
环境
Laravel 5.4
Behat 3.1.0
PHP 7.1.23
PHPUnit 5.7.27
来自我的 composer.json
"require": {
"php": ">=5.5.9",
"laravel/framework": "5.4.*",
..
"behat/behat": "3.1.0",
"laracasts/behat-laravel-extension": "^1.1",
},
"require-dev": {
"phpunit/phpunit": "~5.7",
"phpspec/phpspec": "~2.1",
"johnkary/phpunit-speedtrap": "^1.0",
},
Behat.yml
default:
extensions:
Laracasts\Behat:
env_path: .env.testing
autoload:
- ./tests/features/bootstrap
suites:
Checkout:
paths: [./tests/features/Checkout]
contexts: [CheckoutFeatureContext]
更新
我试图创建示例小黄瓜来说明上述问题。我 运行 在尝试自动附加片段时遇到了同样的问题。附加片段适用于单个场景，但在多个场景中失败：
工作示例：单一场景
# tests/features/Example/Example.feature
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background:
Givens setup condition 1
And setup condition 2
Scenario: scenario one
When I perform first sample trigger point
Then result one must happen
And result two must happen
当我运行下面的命令
behat tests/features/Example/Example.feature --append-snippets
添加片段效果很好
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background: # tests/features/Example/Example.feature:9
Givens setup condition 1
And setup condition 2
Scenario: scenario one # tests/features/Example/Example.feature:13
When I perform first sample trigger point
Then result one must happen
And result two must happen
1 scenario (1 undefined)
4 steps (4 undefined)
0m0.48s (24.63Mb)
u tests/features/bootstrap/FeatureContext.php - `setup condition 2` definition added
u tests/features/bootstrap/FeatureContext.php - `I perform first sample trigger point` definition added
u tests/features/bootstrap/FeatureContext.php - `result one must happen` definition added
u tests/features/bootstrap/FeatureContext.php - `result two must happen` definition added
失败示例：多个场景
当我们有多个场景时
# tests/features/Example/Example.feature
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background:
Givens setup condition 1
And setup condition 2
Scenario: scenario one
When I perform first sample trigger point
Then result one must happen
And result two must happen
Scenario: scenario two
When I perform second sample trigger point
Then result a must happen
And result b must happen
运行使用相同的 --append-snippets 命令阻塞：
Feature: Example
In order to show dev team how to use behat/gherkin using background
As a developer
I need to be able write gherkin using a background and multiple scenarios
And all scenarios should run
Background: # tests/features/Example/Example.feature:9
Givens setup condition 1
And setup condition 2
Scenario: scenario one # tests/features/Example/Example.feature:13
When I perform first sample trigger point
Then result one must happen
And result two must happen
^C // had to abort here
以下步骤适用于 Arch Linux 和 PHP 7.3：
composer global require laravel/installer
laravel new behat-laravel
cd behat-laravel
composer require behat/behat behat/mink behat/mink-extension laracasts/behat-laravel-extension --dev
touch behat.yml
# edit behat.yml
# edit features/bootstrap/FeatureContext.php
vendor/bin/behat --init
# Add steps and tag them
vendor/bin/behat
vendor/bin/behat --tags Ready,NoneReady
# behat.yml
default:
extensions:
Laracasts\Behat:
# env_path: .env.behat
Behat\MinkExtension:
default_session: laravel
laravel: ~
<?php
// features/bootstrap/FeatureContext.php
use Behat\Behat\Hook\Scope\AfterStepScope;
use Behat\Behat\Tester\Exception\PendingException;
use Behat\Behat\Context\Context;
use Behat\Behat\Context\SnippetAcceptingContext;
use Behat\Gherkin\Node\PyStringNode;
use Behat\Gherkin\Node\TableNode;
#This will be needed if you require "behat/mink-selenium2-driver"
#use Behat\Mink\Driver\Selenium2Driver;
use Behat\MinkExtension\Context\MinkContext;
/**
* Defines application features from the specific context.
*/
class FeatureContext extends MinkContext implements Context, SnippetAcceptingContext
{
/**
* Initializes context.
*
* Every scenario gets its own context instance.
* You can also pass arbitrary arguments to the
* context constructor through behat.yml.
*/
public function __construct()
{
}
/**
* @When stepready :arg1
*/
public function stepready($arg1)
{
return true;
}
/**
* @When steplater :arg1
*/
public function steplater($arg1)
{
return true;
}
}
# feature/customer.feature
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now
When stepready 1
Then stepready 2
@NoneReady
Scenario: Deliver later
When steplater 1
Then steplater 2
frosch ➜ behat-laravel vendor/bin/behat --tags Ready,NoneReady
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
@Ready
Scenario: Deliver now # features/customer.feature:7
When stepready 1 # FeatureContext::stepready()
Then stepready 2 # FeatureContext::stepready()
@NoneReady
Scenario: Deliver later # features/customer.feature:12
When steplater 1 # FeatureContext::steplater()
Then steplater 2 # FeatureContext::steplater()
2 Szenarien (2 bestanden)
4 Schritte (4 bestanden)
0m0.02s (18.47Mb)
原来上面的例子太简单了。在做了一些研究之后（特别有用的是 this post），我意识到这个 "stalling" 是由于每次测试后都拆除了数据库。所以这是修复它的方法：
首先，我在我的 FeatureContext class:
中用DatabaseMigrations 替换了 DatabaseTransactions
class FeatureContext extends TestCase implements Context, SnippetAcceptingContext
{
use DatabaseMigrations, ..
鉴于上述情况，我从我的 bitbucket 管道脚本中删除了手动迁移命令
- php artisan --env=testing config:cache
这是有道理的，因为使用新代码，每次测试前都会刷新和迁移数据库。
然后我将 setUp() 调用添加到 behat hooks:
/** @BeforeScenario */
public function before(BeforeScenarioScope $scope)
{
parent::setUp();
}
就是这样。这个解决方案最好的部分是它使我的本地测试环境与 bitbucket 管道完全一致，因此结果始终相同。
进一步解释：来自我们的 wiki
一般来说，最好重新开始每个测试，不要遗留上一个测试的遗留物（尤其是在涉及数据库时）。 In the words of laravel:
It is often useful to reset your database after each test so that data from a previous test does not interfere with subsequent tests.
为此，我们使用迁移。也就是说，由于我们实际使用的是 Behat，因此我们需要在每个场景生命周期之前和之后进行此迁移。我们使用 Behat's hooks 来做到这一点。我们在这里这样做：
/** @BeforeScenario */
public function before(BeforeScenarioScope $scope)
{
parent::setUp();
}
parent::setUP() 告诉 Laravel framework 在每个场景前后做必要的工作：
protected function setUp()
{
if (! $this->app) {
$this->refreshApplication();
}
$this->setUpTraits(); <---- here
..
这又会调用设置特征：
protected function setUpTraits()
{
$uses = array_flip(class_uses_recursive(static::class));
if (isset($uses[DatabaseMigrations::class])) {
$this->runDatabaseMigrations();
}
..
这叫这个
public function runDatabaseMigrations()
{
$this->artisan('migrate:fresh');
$this->app[Kernel::class]->setArtisan(null);
$this->beforeApplicationDestroyed(function () {
$this->artisan('migrate:rollback');
RefreshDatabaseState::$migrated = false;
});
}
请注意，一旦应用程序被销毁，Laravel 也会回滚更改。了解这一点非常重要，这样可以防止在存在多个场景且给定场景出现时 Behat 停滞。还要记住，当我们像这样使用 Gherkin 时：
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
Background:
Given step 1
And step 2
@Ready
Scenario: Deliver now
When step 3
Then step 4
@NoneReady
Scenario: Deliver later
When step a
Then step b
And step c
然后每个场景都从后台步骤开始，而不是场景步骤本身
示例：
Feature: Checkout
In order to buy products
As a customer
I need to be able to checkout items in the cart
Background:
Given step 1 <-- every scenario starts here, so we call setup before this step
And step 2
@Ready
Scenario: Deliver now
When step 3 <-- not here
Then step 4
@NoneReady
Scenario: Deliver later
When step a
Then step b
And step c