如何将初始数据部署到 Flutter App

Question

我遇到了以下问题：我们正在编写一个图书馆应用程序，其中包含一堆内部文档（大约 900 个）。所有文档的总大小约为 2.5+ GB。我必须找到一种方法如何在第一次启动时使用 2.5GB 的 Docs 初始化应用程序。我的想法是创建一个初始加载的 zip 文件，从服务器下载它并在首次启动时的安装过程中解压缩。但是我没有找到解压缩这么大文件的方法，因为所有解决方案都先（完全）将 zip 读取到内存，然后再将其写入存储。我也不想在第一次启动时对我们的网络服务器进行 900 多次调用来下载文档。

部署目标是iOS和Android，稍后可能win+mac。

有什么想法吗？

Answer 1

我在 flutter linux 上测试了它，其中 Uri.base 指向项目的根目录（包含 pubspec.yaml、README.md 等的文件夹）——如果你运行它在 android / iOS 检查 Uri.base 指向的位置，如果位置不好则更改 baseUri:

final debug = true;
final Set<Uri> foldersCreated = {};
final baseUri = Uri.base;
final baseUriLength = baseUri.toString().length;
final zipFileUri = baseUri.resolve('inputFolder/backup.zip');

final outputFolderUri = baseUri.resolve('outputFolder/');
print('0. files will be stored in [$outputFolderUri]');
final list = FileList(zipFileUri, debug: debug);
print('1. reading ZipDirectory...');
final directory = ZipDirectory.read(InputStream(list));
print('2. iterating over ZipDirectory file headers...');
for (final zfh in directory.fileHeaders) {
  final zf = zfh.file;
  final content = zf.content;

  // writing file
  final uri = outputFolderUri.resolve(zf.filename);
  final folderUri = uri.resolve('.');
  if (foldersCreated.add(folderUri)) {
    if (debug) print(' #### creating folder [${folderUri.toString().substring(baseUriLength)}] #### ');
    Directory.fromUri(folderUri).createSync(recursive: true);
  }
  File.fromUri(uri).writeAsBytesSync(content);

  print("file: [${zf.filename}], compressed: ${zf.compressedSize}, uncompressed: ${zf.uncompressedSize}, length: ${content.length}");
}
list.close();
print('3. all done!');

这里是一个 List backed by a LruMap 从你的巨大的 zip 文件中读取数据的块：

class FileList with ListMixin<int> {
  RandomAccessFile _file;
  LruMap<int, List<int>> _cache;
  final int maximumPages;
  final int pageSize;
  final bool debug;

  FileList(Uri uri, {
    this.pageSize = 1024, // 1024 is just for tests: make it bigger (1024 * 1024 for example) for normal use
    this.maximumPages = 4, // maybe even 2 is good enough?
    this.debug = false,
  }) {
    _file = File.fromUri(uri).openSync();
    length = _file.lengthSync();
    _cache = LruMap(maximumSize: maximumPages);
  }

  void close() => _file.closeSync();

  @override
  int length;

  int minIndex = -1;
  int maxIndex = -1;
  List<int> page;
  @override
  int operator [](int index) {
    // print(index);

    // 1st cache level
    if (index >= minIndex && index < maxIndex) {
      return page[index - minIndex];
    }

    // 2nd cache level
    int key = index ~/ pageSize;
    final pagePosition = key * pageSize;
    page = _cache.putIfAbsent(key, () {
      if (debug) print(' #### reading page #$key (position $pagePosition) #### ');
      _file.setPositionSync(pagePosition);
      return _file.readSync(pageSize);
    });
    minIndex = pagePosition;
    maxIndex = pagePosition + pageSize;
    return page[index - pagePosition];
  }

  @override
  void operator []=(int index, int value) => null;
}

您可以使用 pageSize 和 maximumPages 来找到最佳解决方案 - 我认为您可以使用 pageSize: 1024 * 1024 和 maximumPages: 4 来解决 tar 但是你得自己检查一下

当然所有这些代码都应该是运行在一些 Isolate 中，因为解压缩几 GB 需要很多时间然后你的 UI 会冻结，但首先运行照原样查看日志

编辑

似乎 ZipFile.content 有一些内存泄漏，所以替代方案可能是基于“tar 文件”的解决方案，它使用 tar package and since it reads a Stream as an input you can use compressed *.tar.gz files (your Documents.tar had 17408 bytes while Documents.tar.gz has 993 bytes), notice that you can even read your data directly from the socket's stream 所以不需要任何中间 .tar.gz 文件:

final baseUri = Uri.base;
final tarFileUri = baseUri.resolve('inputFolder/Documents.tar.gz');

final outputFolderUri = baseUri.resolve('outputFolder/');
print('0. files will be stored in [$outputFolderUri]');
final stream = File.fromUri(tarFileUri)
  .openRead()
  .transform(gzip.decoder);
final reader = TarReader(stream);
print('1. iterating over tar stream...');

while (await reader.moveNext()) {
  final entry = reader.current;
  if (entry.type == TypeFlag.dir) {
    print("dir: [${entry.name}]");
    final folderUri = outputFolderUri.resolve(entry.name);
    await Directory.fromUri(folderUri).create(recursive: true);
  }
  if (entry.type == TypeFlag.reg) {
    print("file: [${entry.name}], size: ${entry.size}");
    final uri = outputFolderUri.resolve(entry.name);
    await entry.contents.pipe(File.fromUri(uri).openWrite());
  }
}
print('2. all done!');

如何将初始数据部署到 Flutter App

How to deploy initial Data to Flutter App

installation

zip

archive

flutter