如何在 iOS UI 测试中定位元素以进行 flutter fastlane 截图

Question

我希望能够点击底部导航栏中的按钮之一导航到我的应用程序的每个选项卡，以便截取屏幕截图。我根据 https://docs.fastlane.tools/getting-started/ios/screenshots/ 进行了所有设置，并且截图成功。问题是我无法弄清楚如何解决该按钮。我以为我可以做这样的事情：

print(app.navigationBars)

但是这个returns难以理解：

<XCUIElementQuery: 0x600003b46b20>

然后我想查看 XCode 中应用程序的层次结构视图（根据 How do I inspect the view hierarchy in iOS?），但对于 Flutter 应用程序，它只显示一堆名称无用的黑框.

一般来说，作为这些 UI 截屏测试的一部分，我该如何解决按钮问题？有些是直观的，例如 app.buttons["Search"]，但其他的则不那么容易工作，例如app.navigationBars["Revision"].

我读过的其他资源包括以下内容，但它们并不是非常有用：

How to get index of XCUIElement in XCUIElementQuery?

谢谢！

Answer 1

你想要的是不可能的。最起码到现在。 Flutter UI 渲染为游戏渲染，比常规 iOs UI 更深。此外，flutter 有自己的手势框架——因此您将无法将 iOs 手势正确地转换为 flutter 手势（您可以，但需要付出太多努力）。此外，原生 iOs UI 测试框架（Xcode UI 测试）不支持 flutter，我认为它永远不会。

你可以做的是研究 flutter 集成测试 here and here。因为它们是 flutter 原生的——你将能够通过各种不同的方式（通过键、class 名称、小部件属性等）来处理 UI。您也可以与 UI 互动。

关于屏幕截图 - 官方不支持它们yet but there are other ways基本上你需要做的是：

//1. Replace your test_driver/integration_test.dart by this code(or similar by approach):
import 'dart:io';
import 'package:integration_test/integration_test_driver_extended.dart';

Future<void> main() async {
  try {
    await integrationDriver(
      onScreenshot: (String screenshotName, List<int> screenshotBytes) async {
        final File image = await File('screenshots/$screenshotName.png').create(recursive: true);
        image.writeAsBytesSync(screenshotBytes);
        return true;
      },
    );
  } catch (e) {
    print('Error occured: $e');
  }
}


//2. Go into your integration_test/your_test_file.dart and add:
final binding = IntegrationTestWidgetsFlutterBinding();
IntegrationTestWidgetsFlutterBinding.ensureInitialized();

//3. And add this to your test:
await binding.takeScreenshot('some_screeshot_name_placeholder');
//If you run your tests on Android device, you must also add //convertFlutterSurfaceToImage() function before takeScreenshot() because //there must be converted a Flutter surface into image like this:
await binding.convertFlutterSurfaceToImage();
await tester.pumpAndSettle();
await binding.takeScreenshot('some_screeshot_name_placeholder');
//For iOS or web you don't need this convert function.

代码不是我的，它来自上面的 link，当我在那篇文章之前的一段时间前尝试实现它时 - 它没有工作 - 也许现在是 - 你必须试试。

您还可以查看 flutter 的黄金测试 here or here or just google it). There is even a library for that here。

关于通过 Fastlane 截取屏幕截图 - 也许可以将上述方法与 Fastlane 脚本结合使用 - 我不确定，因为这是非常不寻常的过程，但你可以尝试。

Answer 2

Pavlo's 答案是我最终选择的答案。我想用一个更完整的指南来回应这个问题，解释该怎么做。

首先，将这些 dev_dependencies 添加到 pubspec.yaml:

  flutter_test:
    sdk: flutter
  integration_test:
    sdk: flutter
  flutter_driver:
    sdk: flutter

然后创建一个名为 test_driver/integration_driver.dart 的文件，如下所示：

import 'dart:io';
import 'package:integration_test/integration_test_driver_extended.dart';

Future<void> main() async {
  try {
    await integrationDriver(
      onScreenshot: (String screenshotName, List<int> screenshotBytes) async {
        final File image = await File('screenshots/$screenshotName.png')
            .create(recursive: true);
        image.writeAsBytesSync(screenshotBytes);
        return true;
      },
    );
  } catch (e) {
    print('Error occured taking screenshot: $e');
  }
}

然后创建一个名为 integration_test/screenshot_test.dart:

的文件

import 'dart:io';
import 'dart:ui';

import 'package:device_info/device_info.dart';
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';

import 'package:auslan_dictionary/common.dart';
import 'package:auslan_dictionary/flashcards_landing_page.dart';
import 'package:auslan_dictionary/globals.dart';
import 'package:auslan_dictionary/main.dart';
import 'package:auslan_dictionary/word_list_logic.dart';

// Note, sometimes the test will crash at the end, but the screenshots do
// actually still get taken.

Future<void> takeScreenshot(
    WidgetTester tester,
    IntegrationTestWidgetsFlutterBinding binding,
    ScreenshotNameInfo screenshotNameInfo,
    String name) async {
  if (Platform.isAndroid) {
    await binding.convertFlutterSurfaceToImage();
    await tester.pumpAndSettle();
  }
  await tester.pumpAndSettle();
  await binding.takeScreenshot(
      "${screenshotNameInfo.platformName}/en-AU/${screenshotNameInfo.deviceName}-${screenshotNameInfo.physicalScreenSize}-${screenshotNameInfo.getAndIncrementCounter()}-$name");
}

class ScreenshotNameInfo {
  String platformName;
  String deviceName;
  String physicalScreenSize;
  int counter = 1;

  ScreenshotNameInfo(
      {required this.platformName,
      required this.deviceName,
      required this.physicalScreenSize});

  int getAndIncrementCounter() {
    int out = counter;
    counter += 1;
    return out;
  }

  static Future<ScreenshotNameInfo> buildScreenshotNameInfo() async {
    Size size = window.physicalSize;
    String physicalScreenSize = "${size.width.toInt()}x${size.height.toInt()}";

    DeviceInfoPlugin deviceInfo = DeviceInfoPlugin();

    String platformName;
    String deviceName;
    if (Platform.isAndroid) {
      platformName = "android";
      AndroidDeviceInfo info = await deviceInfo.androidInfo;
      deviceName = info.product;
    } else if (Platform.isIOS) {
      platformName = "ios";
      IosDeviceInfo info = await deviceInfo.iosInfo;
      deviceName = info.name;
    } else {
      throw "Unsupported platform";
    }

    return ScreenshotNameInfo(
        platformName: platformName,
        deviceName: deviceName,
        physicalScreenSize: physicalScreenSize);
  }
}

void main() async {
  final IntegrationTestWidgetsFlutterBinding binding =
      IntegrationTestWidgetsFlutterBinding();
  binding.framePolicy = LiveTestWidgetsFlutterBindingFramePolicy.fullyLive;
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();
  testWidgets("takeScreenshots", (WidgetTester tester) async {

    // Just examples of taking screenshots.
    await takeScreenshot(tester, binding, screenshotNameInfo, "search");

    final Finder searchField = find.byKey(ValueKey("searchPage.searchForm"));
    await tester.tap(searchField);
    await tester.pumpAndSettle();
    await tester.enterText(searchField, "hey");
    await takeScreenshot(tester, binding, screenshotNameInfo, "searchWithText");
  });
}

然后您可以像这样调用它：

flutter drive --driver=test_driver/integration_driver.dart --target=integration_test/screenshot_test.dart -d 'iPhone 13 Pro Max'

确保首先创建适当的目录，如下所示：

mkdir -p screenshots/ios/en-AU
mkdir -p screenshots/android/en-AU

Flutter/它的测试部门目前存在一个问题，从 2.10.4 开始，这意味着您必须更改测试包：https://github.com/flutter/flutter/issues/91668。简而言之，对 packages/integration_test/ios/Classes/IntegrationTestPlugin.m 进行以下更改：

+ (void)registerWithRegistrar:(NSObject<FlutterPluginRegistrar> *)registrar {
    [[IntegrationTestPlugin instance] setupChannels:registrar.messenger];
}

在此之后您可能需要运行 flutter clean。

现在你可以截图了！

作为奖励，这个 Python 脚本将为 iOS 和 Android 启动一堆模拟器/模拟器，并驱动上面的集成测试为所有这些进行截图:

import argparse
import asyncio
import logging
import os
import re


# The list of iOS simulators to run.
# This comes from inspecting `xcrun simctl list`
IOS_SIMULATORS = [
    "iPhone 8",
    "iPhone 8 Plus",
    "iPhone 13 Pro Max",
    "iPad Pro (12.9-inch) (5th generation)",
    "iPad Pro (9.7-inch)",
]

ANDROID_EMULATORS = [
    "Nexus_7_API_32",
    "Nexus_10_API_32",
    "Pixel_5_API_32",
]


LOG = logging.getLogger(__name__)
formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
ch = logging.StreamHandler()
ch.setFormatter(formatter)
LOG.addHandler(ch)


def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("-d", "--debug", action="store_true")
    parser.add_argument("--clear-screenshots", action="store_true", help="Delete all existing screenshots")
    args = parser.parse_args()
    return args


class cd:
    """Context manager for changing the current working directory"""
    def __init__(self, newPath):
        self.newPath = os.path.expanduser(newPath)

    def __enter__(self):
        self.savedPath = os.getcwd()
        os.chdir(self.newPath)

    def __exit__(self, etype, value, traceback):
        os.chdir(self.savedPath)


async def run_command(command):
    proc = await asyncio.create_subprocess_exec(
        *command,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    stdout, stderr = await proc.communicate()
    if stderr:
        LOG.debug(f"stderr of command {command}: {stderr}")
    return stdout.decode("utf-8")


async def get_uuids_of_ios_simulators(simulators):
    command_output = await run_command(["xcrun", "simctl", "list"])

    out = {}
    for s in simulators:
        for line in command_output.splitlines():
            r = "    " + re.escape(s) + r" \((.*)\) \(.*"
            m = re.match(r, line)
            if m is not None:
                out[s] = m[1]

    return out


async def start_ios_simulators(uuids_of_ios_simulators):
    async def start_ios_simulator(uuid):
        await run_command(["xcrun", "simctl", "boot", uuid])

    await asyncio.gather(
        *[start_ios_simulator(uuid) for uuid in uuids_of_ios_simulators.values()]
    )


async def start_android_emulators(android_emulator_names):
    async def start_android_emulator(name):
        await run_command(["flutter", "emulators", "--launch", name])

    await asyncio.gather(
        *[start_android_emulator(name) for name in android_emulator_names]
    )


async def get_all_device_ids():
    raw = await run_command(["flutter", "devices"])
    out = []
    for line in raw.splitlines():
        if "•" not in line:
            continue
        if "Daniel" in line:
            continue
        if "Chrome" in line:
            continue
        device_id = line.split("•")[1].lstrip().rstrip()
        out.append(device_id)

    return out


async def run_tests(device_ids):
    async def run_test(device_id):
        LOG.info(f"Started testing for {device_id}")
        await run_command(
            [
                "flutter",
                "drive",
                "--driver=test_driver/integration_driver.dart",
                "--target=integration_test/screenshot_test.dart",
                "-d",
                device_id,
            ]
        )
        LOG.info(f"Finished testing for {device_id}")

    for device_id in device_ids:
        await run_test(device_id)

    # await asyncio.gather(*[run_test(device_id) for device_id in device_ids])


async def main():
    args = parse_args()

    # chdir to location of python file.
    abspath = os.path.abspath(__file__)
    dname = os.path.dirname(abspath)
    os.chdir(dname)

    if args.debug:
        LOG.setLevel("DEBUG")
    else:
        LOG.setLevel("INFO")

    if args.clear_screenshots:
        await run_command(["rm", "ios/en-AU/*"])
        await run_command(["rm", "android/en-AU/*"])
        LOG.info("Cleared existing screenshots")

    uuids_of_ios_simulators = await get_uuids_of_ios_simulators(IOS_SIMULATORS)
    LOG.info(f"iOS simulatior name to UUID: {uuids_of_ios_simulators}")

    LOG.info("Launching iOS simulators")
    await start_ios_simulators(uuids_of_ios_simulators)
    LOG.info("Launched iOS simulators")

    LOG.info("Launching Android emulators")
    await start_android_emulators(ANDROID_EMULATORS)
    LOG.info("Launched Android emulators")

    await asyncio.sleep(5)

    device_ids = await get_all_device_ids()
    LOG.debug(f"Device IDs: {device_ids}")

    LOG.info("Running tests")
    await run_tests(device_ids)
    LOG.info("Ran tests")

    LOG.info("Done!")


if __name__ == "__main__":
    asyncio.run(main())

请注意，如果您尝试使用这种方法在 Android 上截取多个屏幕截图，您会遇到麻烦。查看 https://github.com/flutter/flutter/issues/92381。修复仍在进行中。

如何在 iOS UI 测试中定位元素以进行 flutter fastlane 截图

How to locate elements in iOS UI test for flutter fastlane screnshots

xcode

flutter

fastlane