Google 导入数据时超出 Fusion Tables 配额

Question

我补google融合tables。我有大约 2mio 条目应该添加到融合 tables.

我用Google.Apis.Fusiontables.v2 1.10.0.50

这工作了几个月。但现在我得到了超出配额的异常。

Google.Apis.Requests.RequestError Quota Exceeded [403] Errors [ Message[Quota Exceeded] Location[ - ] Reason[quotaExceeded] Domain[usageLimits] ]

因为有更多的数据，这并不奇怪。

但是当我查看 https://console.developers.google.com/apis/api/fusiontables/quotas 上的配额时，我看不到任何限制。

同样在全局配额仪表板中，我看不到我达到的任何限制。

我想我可能会达到 250MB table。但是当我下载数据源时，csv 只有 40 MB 但是我觉得这只是读配额没有写？

        private static void WriteItem()
        {
            await WriteAsync("map_data", index.AllEdges<BusEdge<TId>>().Where(x => x.MinFrequentPriceInEur > 0), (busEdge, csvWriter) =>
            {
                WriteItem(busEdge, csvWriter);

                itemHandled();
            });
        }

    private static void WriteItem(BusEdge<TId> busEdge, CsvWriter csvWriter)
    {
        csvWriter.WriteField(Source.Id.ToString());
        csvWriter.WriteField(sink.Id.ToString());

        csvWriter.WriteField(string.Format(CultureInfo.InvariantCulture, "{0} {1}",
            sink.Geolocation.Latitude,
            sink.Geolocation.Longitude));

        csvWriter.WriteField(sink.Population);

        foreach (Language language in Languages.AllLanguages)
        {
            csvWriter.WriteField(sink.DisplayName[language], true);
        }

        var priceWithDate =
            busEdge.Prices.GroupBy(x => x.Value.MinimumFrequentInEur)
                .OrderByDescending(x => x.Count()).SelectMany(x => x)
                .OrderBy(x => x.Key)
                .FirstOrDefault();

        double price = priceWithDate.Value.MinimumFrequentInEur;

        csvWriter.WriteField(Math.Round(price, 2));
        csvWriter.WriteField(Math.Round(CurrencyInfo.EUR.ConvertMoney(price, CurrencyInfo.GBP), 2));
        csvWriter.WriteField(Math.Round(CurrencyInfo.EUR.ConvertMoney(price, CurrencyInfo.HRK), 2));
        csvWriter.WriteField(Math.Round(CurrencyInfo.EUR.ConvertMoney(price, CurrencyInfo.PLN), 2));
        csvWriter.WriteField(priceWithDate.Key.ToString("s"));

  public static async Task WriteAsync<T>(string tableName, IEnumerable<T> list, Action<T, CsvWriter> writerAction)
        {
            using (FusiontablesService service = Connect())
            {
                var tables = service.Table.List().Execute();
                var tableId = tables.Items.Where(x => x.Name == tableName).Select(x => x.TableId).First();

                MemoryStream memoryStream = new MemoryStream();

                bool hasWritten = false;

                Func<Task> uploadAsync = async () =>
                {
                    if (memoryStream.Length > 0)
                    {
                        memoryStream.Position = 0;

                        ResumableUpload<string> request =
                            hasWritten ?
                                (ResumableUpload<string>)service.Table.ImportRows(tableId, memoryStream, "application/octet-stream") :
                                (ResumableUpload<string>)service.Table.ReplaceRows(tableId, memoryStream, "application/octet-stream");

                        var response = await request.UploadAsync();

                        if (response.Exception != null)
                        {
                            throw response.Exception;
                        }

                        memoryStream = new MemoryStream();

                        hasWritten = true;
                    }
                };

                using (StreamWriter writer = new StreamWriter(memoryStream, Encoding.UTF8, 1024, true))
                {
                    using (CsvWriter csvWriter = new CsvWriter(writer, new CsvConfiguration { Delimiter = "," }))
                    {
                        foreach (T item in list)
                        {
                            writerAction(item, csvWriter);

                            csvWriter.NextRecord();

                            writer.Flush();

                            if (memoryStream.Length >= Limit)
                            {
                                await uploadAsync();
                            }
                        }
                    }
                }

                await uploadAsync();
            }

有什么建议我可以找出具体的限制吗？以及对代码改进的任何建议？

Answer 1

我给客服发了邮件。他们告诉我他们会看看。我没有听到任何新消息，但第二天它没有任何变化地工作。

现在我的数据增加了大约 20%，而且它仍然有效。所以它看起来像是一个 google 错误

Google 导入数据时超出 Fusion Tables 配额

Google Fusion Tables Quota Exceeded when importing data

.net

c#

google-fusion-tables