FoxPro 6.0 错误消息 - 临时文件太大
FoxPro 6.0 error message - temp file is too large
这是我的 FoxPro 程序。 Select 子句导致错误。
PROCEDURE IFRS_Split_Cashflows_2_v7_4_CSV
PARAMETERS model_dir, cube2_output_path, output_path, scenario, version
temp_dir = GETENV("TEMP")
**** Load up utility function and Create an output folder ****
SET PROCEDURE TO model_dir + "\CUBE2_" + version + "\PRGS\Utilities_" + version + ".prg"
lcTextFile = cube2_output_path + "\IFRS_Split_Cashflows_2\run_" + scenario + ".csv"
Do delete_file WITH lcTextFile
Do create_folder WITH cube2_output_path + "\IFRS_Split_Cashflows_2"
**** Assign variables to the input files ****
moses_output_file = output_path + "\annuities~main10.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output1
moses_output_file = output_path + "\annuities~main10~reinsured.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output2
moses_output_file = output_path + "\annuities~main10~reinsurance_premium.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output3
**** Split the group string and select relevant data ****
SELECT ;
c1.t_from AS t_from, ;
c1.cal_mth as calendar_month, ;
c1.cal_yr as calendar_year, ;
substr(c1.group,1,3) AS BUS_LINE, ;
padr( IIF( isnumeric( strextract(c1.group, "|", "|", 1, 1), ""), ;
substr(c1.group,1,4) + strextract(c1.group, "|", "|", 1, 1), ;
strextract( c1.group,"|", "|",1, 1)), 10) AS reinsurance_book,;
padr( strextract(c1.group, "|","|", 3,1),10) AS Esc_Index,;
padr( IIF( left(c1.group,3) = "LRT", ;
strextract(c1.group, "|", "|",5,1), ;
strextract(c1.group,"|","|",4,1)) ,10) AS MAP_ELIG,;
padr( IIF( left(c1.group,3) = "LRT", ;
strextract( trim(c1.group - "|"),"|","|", 6, 1), ;
strextract( trim(c1.group - "|"),"|","|", 5,1)), 10) AS MAP_STAT,;
SUM(c1.ann_ben_if) AS ann_ben_if, ;
SUM(c2.dth_out) AS c2death_outgo, ;
SUM(c3.dth_out) AS c3death_outgo;
FROM ;
cube_output1 c1, ;
cube_output2 c2, ;
cube_output3 c3;
INTO ;
CURSOR temp READWRITE;
WHERE ;
c1.t_from = 0 ;
AND ( ( c1.t_from = c2.t_from AND c1.group = c2.group) ;
OR ( c1.t_from = c3.t_from AND c1.group = c3.group)) ;
group by ;
c1.t_from, ;
c1.cal_mth, ;
c1.cal_yr, ;
BUS_LINE, ;
reinsurance_book, ;
Esc_Index, ;
MAP_ELIG, ;
MAP_STAT
**** Copy to a CSV file ****
COPY TO cube2_output_path + "\IFRS_Split_Cashflows_2\run_" + scenario + ".csv" TYPE CSV
生成错误，"File c:\users\jc\appdata\local\temp[=21=]002jo9000h.tmp is too large. Other task failed."
我实际上从 select 语句中删除了几个与 c2 和 c3 有关的字段。我还将 where 子句的第一部分从 c1.t_from > 0 更改为 c1.t_from = 0，以减少观察。
错误仍然出现。我了解临时文件的文件大小上限为 2GB。在大小方面，输入文件如下：c1 为 275MB，C2 为 275MB，c3 为 191MB。鉴于此，令我惊讶的是临时文件太大了。为什么太大了？有什么方法可以改进上面的 Select 语句，以正确 return 请求的字段？
我认为您在查询中得到了笛卡尔结果，这让您丧命。
您的查询未使用 C2 和 C3 别名引用中的任何实际列值，只是确保在 C2 或 C3 table 中的任一个中找到给定的 T_From 和组。
为了消除您的临时文件崩溃，我会确保您在每个 C2 和 C3 table 上都有一个基于 T_From 和组（例如
）的索引
moses_output_file = output_path + "\annuities~main10~reinsured.dbf"
Use (moses_output_file) ALIAS cube_output2 EXCLUSIVE
index on str( T_From ) + Group tag TFromGroup
moses_output_file = output_path + "\annuities~main10~reinsurance_premium.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output3 EXCLUSIVE
index on str( T_From ) + Group tag TFromGroup
索引可以在此函数之外创建一次，并且将始终保持可用。如果索引不存在，可以在这里使用。
现在，查询修复。将 FROM/WHERE 子句更改为以下内容。我正在做的是从您的 C1 table 到子查询结果的 SQL-JOIN。由于您只关心在其他 table 的 EITHER 中有记录的记录，因此我正在执行 PreCheck 查询以获取不同的 T_From / Group 记录。然后只拉那些 T_From = 0 的 C1 记录。
请注意，PreCheck 查询还在 T_From = 0 上应用 where 子句，所以我不能加入那些发现 T_From = 0 的组。您将完全消除任何可能的笛卡尔结果，因为内部查询只会产生 DISTINCT "Group" 条记录。
select（所有其他字段相同...最后一个字段添加逗号）
C2Sum.C2Death_Outgo,;
C3Sum.C3Death_Outgo;
FROM ;
cube_output1 c1;
LEFT JOIN ( select c2.Group,;
SUM(c2.dth_out) AS c2death_outgo ;
FROM cube_output2 c2;
WHERE c2.t_from = 0;
GROUP BY c2.Group ) C2Sum ;
on c1.t_group = C2Sum.Group; ;
LEFT JOIN ( select c3.Group, ;
SUM(c3.dth_out) AS c3death_outgo;
FROM cube_output3 c3 ;
WHERE c3.t_from = 0 ;
GROUP BY c3.Group ) C3Sum ;
ON c1.t_group = C3Sum.group;
WHERE ;
c1.t_from = 0 ;
AND ( NOT ISNULL( c2Sum.Group ) OR NOT ISNULL( c3Sum.Group ))
我在这里做的是为每个人分组 table 以防止他们之间出现笛卡尔坐标。我正在对它们进行 LEFT-JOIN，因为不知道你的数据，每个 C2 / C3 中可能都有一条记录 table 并且不想扭曲求和。
通过完成 where 子句，我正在寻找 EITHER LEFT JOIN to NOT BE NULL，这意味着给定组有一条记录。同样，任何一方或双方都可以有一个记录，所以它会被计算一次用于连接，而不是导致重复。
现在，由于分组依据的金额仅在组级别，并且您的 OUTER 查询按公交线路、保险簿、索引等的更多部分分组，无论它们在哪个子分类中， GROUP 金额将对所有人显示相同，这可能不是您要找的。
最后，您的大量解析、填充、字符串提取似乎不是分类事物的好方法，但使用单独的列可能会更有效……只是一个想法。
正如 Drapp 所说，您得到的是笛卡尔坐标（不是真的但接近它）结果，因此您的临时文件变得越来越大。如果你做了一个小模型:
立方体1
t_from cal_mth cal_yr ann_be_if group
------ ------- ------ --------- -----
0 1 2016 100 1
0 2 2016 800 2
1 3 2016 500 3
魔方2
t_from fieldX group
------ ------ -----
0 1 1
0 2 1
0 3 1
立方体3
t_from fieldY group
------ ------ -----
0 1 2
0 2 2
0 3 2
0 4 2
使用您的 SQL，这将导致从 cube1 中选择 2 行 3*4 次（您的条件匹配 cube2 中的 3 行和 cube3 中的 4 行），这将是 24 行。由于您重复选择相同的行 3*4 次，因此您也将 ann_be_if 的值重复 3*4 次并将它们相加为 (800+100)*12 = 10800。这不太可能是您想。
You shouldn't do aggregation when you have joins (either implicit or
explicit) with other tables (unless you are sure you wouldn't have
this duplication - i.e. joining sample customer, orders,orditems
tables wouldn't exhibit this duplication).
再次阅读您的 SQL，第二个和第三个表对结果没有任何作用，只是您只是对它们进行 "exists" 检查。所以您可以将代码重写为：
**** Assign variables to the input files ****
*!* moses_output_file = output_path + "\annuities~main10.dbf"
*!* SELECT 0
*!* Use (moses_output_file) ALIAS cube_output1
*!* moses_output_file = output_path + "\annuities~main10~reinsured.dbf"
*!* SELECT 0
*!* Use (moses_output_file) ALIAS cube_output2
*!* moses_output_file = output_path + "\annuities~main10~reinsurance_premium.dbf"
*!* SELECT 0
*!* Use (moses_output_file) ALIAS cube_output3
cube_output1 = m.output_path + "\annuities~main10.dbf"
cube_output2 = m.output_path + "\annuities~main10~reinsured.dbf"
cube_output3 = m.output_path + "\annuities~main10~reinsurance_premium.dbf"
Select t_from, ;
cal_mth As calendar_month, ;
cal_yr As calendar_year, ;
substr(c1.Group,1,3) As BUS_LINE, ;
padr(Iif(isnumeric(Strextract(c1.Group,"|","|",1,1),""), ;
substr(c1.Group,1,4) + Strextract(c1.Group,"|","|",1,1), ;
strextract(c1.Group,"|","|",1,1)),10) As reinsurance_book, ;
padr(Strextract(c1.Group,"|","|",3,1),10) As Esc_Index, ;
padr(Iif(Left(c1.Group,3)="LRT",;
strextract(c1.Group,"|","|",5,1),;
strextract(c1.Group,"|","|",4,1)),10) As MAP_ELIG, ;
padr(Iif(Left(c1.Group,3)="LRT",;
strextract(Trim(c1.Group - "|"),"|","|",6,1), ;
strextract(Trim(c1.Group - "|"),"|","|",5,1)),10) As MAP_STAT, ;
SUM(ann_ben_if) As ann_ben_if ;
FROM (m.cube_output1) c1 ;
WHERE c1.t_from = 0 And ;
exists ( Select * From (m.cube_output2) c2 ;
Where c2.t_from = 0 And c1.Group = c2.Group ) ;
OR ;
exists ( Select * From (m.cube_output3) c3 ;
Where c3.t_from = 0 And c1.Group = c3.Group ) ;
group By t_from, calendar_month, calendar_year, ;
BUS_LINE, reinsurance_book, Esc_Index, MAP_ELIG, MAP_STAT ;
Into Cursor temp ;
Readwrite
**** Copy to a CSV file ****
COPY TO (m.cube2_output_path + "\IFRS_Split_Cashflows_2\run_" + m.scenario + ".csv") TYPE CSV
这是我的 FoxPro 程序。 Select 子句导致错误。
PROCEDURE IFRS_Split_Cashflows_2_v7_4_CSV
PARAMETERS model_dir, cube2_output_path, output_path, scenario, version
temp_dir = GETENV("TEMP")
**** Load up utility function and Create an output folder ****
SET PROCEDURE TO model_dir + "\CUBE2_" + version + "\PRGS\Utilities_" + version + ".prg"
lcTextFile = cube2_output_path + "\IFRS_Split_Cashflows_2\run_" + scenario + ".csv"
Do delete_file WITH lcTextFile
Do create_folder WITH cube2_output_path + "\IFRS_Split_Cashflows_2"
**** Assign variables to the input files ****
moses_output_file = output_path + "\annuities~main10.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output1
moses_output_file = output_path + "\annuities~main10~reinsured.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output2
moses_output_file = output_path + "\annuities~main10~reinsurance_premium.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output3
**** Split the group string and select relevant data ****
SELECT ;
c1.t_from AS t_from, ;
c1.cal_mth as calendar_month, ;
c1.cal_yr as calendar_year, ;
substr(c1.group,1,3) AS BUS_LINE, ;
padr( IIF( isnumeric( strextract(c1.group, "|", "|", 1, 1), ""), ;
substr(c1.group,1,4) + strextract(c1.group, "|", "|", 1, 1), ;
strextract( c1.group,"|", "|",1, 1)), 10) AS reinsurance_book,;
padr( strextract(c1.group, "|","|", 3,1),10) AS Esc_Index,;
padr( IIF( left(c1.group,3) = "LRT", ;
strextract(c1.group, "|", "|",5,1), ;
strextract(c1.group,"|","|",4,1)) ,10) AS MAP_ELIG,;
padr( IIF( left(c1.group,3) = "LRT", ;
strextract( trim(c1.group - "|"),"|","|", 6, 1), ;
strextract( trim(c1.group - "|"),"|","|", 5,1)), 10) AS MAP_STAT,;
SUM(c1.ann_ben_if) AS ann_ben_if, ;
SUM(c2.dth_out) AS c2death_outgo, ;
SUM(c3.dth_out) AS c3death_outgo;
FROM ;
cube_output1 c1, ;
cube_output2 c2, ;
cube_output3 c3;
INTO ;
CURSOR temp READWRITE;
WHERE ;
c1.t_from = 0 ;
AND ( ( c1.t_from = c2.t_from AND c1.group = c2.group) ;
OR ( c1.t_from = c3.t_from AND c1.group = c3.group)) ;
group by ;
c1.t_from, ;
c1.cal_mth, ;
c1.cal_yr, ;
BUS_LINE, ;
reinsurance_book, ;
Esc_Index, ;
MAP_ELIG, ;
MAP_STAT
**** Copy to a CSV file ****
COPY TO cube2_output_path + "\IFRS_Split_Cashflows_2\run_" + scenario + ".csv" TYPE CSV
生成错误，"File c:\users\jc\appdata\local\temp[=21=]002jo9000h.tmp is too large. Other task failed."
我实际上从 select 语句中删除了几个与 c2 和 c3 有关的字段。我还将 where 子句的第一部分从 c1.t_from > 0 更改为 c1.t_from = 0，以减少观察。
错误仍然出现。我了解临时文件的文件大小上限为 2GB。在大小方面，输入文件如下：c1 为 275MB，C2 为 275MB，c3 为 191MB。鉴于此，令我惊讶的是临时文件太大了。为什么太大了？有什么方法可以改进上面的 Select 语句，以正确 return 请求的字段？
我认为您在查询中得到了笛卡尔结果，这让您丧命。
您的查询未使用 C2 和 C3 别名引用中的任何实际列值，只是确保在 C2 或 C3 table 中的任一个中找到给定的 T_From 和组。
为了消除您的临时文件崩溃，我会确保您在每个 C2 和 C3 table 上都有一个基于 T_From 和组（例如
）的索引moses_output_file = output_path + "\annuities~main10~reinsured.dbf"
Use (moses_output_file) ALIAS cube_output2 EXCLUSIVE
index on str( T_From ) + Group tag TFromGroup
moses_output_file = output_path + "\annuities~main10~reinsurance_premium.dbf"
SELECT 0
Use (moses_output_file) ALIAS cube_output3 EXCLUSIVE
index on str( T_From ) + Group tag TFromGroup
索引可以在此函数之外创建一次，并且将始终保持可用。如果索引不存在，可以在这里使用。
现在，查询修复。将 FROM/WHERE 子句更改为以下内容。我正在做的是从您的 C1 table 到子查询结果的 SQL-JOIN。由于您只关心在其他 table 的 EITHER 中有记录的记录，因此我正在执行 PreCheck 查询以获取不同的 T_From / Group 记录。然后只拉那些 T_From = 0 的 C1 记录。
请注意，PreCheck 查询还在 T_From = 0 上应用 where 子句，所以我不能加入那些发现 T_From = 0 的组。您将完全消除任何可能的笛卡尔结果，因为内部查询只会产生 DISTINCT "Group" 条记录。
select（所有其他字段相同...最后一个字段添加逗号）
C2Sum.C2Death_Outgo,;
C3Sum.C3Death_Outgo;
FROM ;
cube_output1 c1;
LEFT JOIN ( select c2.Group,;
SUM(c2.dth_out) AS c2death_outgo ;
FROM cube_output2 c2;
WHERE c2.t_from = 0;
GROUP BY c2.Group ) C2Sum ;
on c1.t_group = C2Sum.Group; ;
LEFT JOIN ( select c3.Group, ;
SUM(c3.dth_out) AS c3death_outgo;
FROM cube_output3 c3 ;
WHERE c3.t_from = 0 ;
GROUP BY c3.Group ) C3Sum ;
ON c1.t_group = C3Sum.group;
WHERE ;
c1.t_from = 0 ;
AND ( NOT ISNULL( c2Sum.Group ) OR NOT ISNULL( c3Sum.Group ))
我在这里做的是为每个人分组 table 以防止他们之间出现笛卡尔坐标。我正在对它们进行 LEFT-JOIN，因为不知道你的数据，每个 C2 / C3 中可能都有一条记录 table 并且不想扭曲求和。
通过完成 where 子句，我正在寻找 EITHER LEFT JOIN to NOT BE NULL，这意味着给定组有一条记录。同样，任何一方或双方都可以有一个记录，所以它会被计算一次用于连接，而不是导致重复。
现在，由于分组依据的金额仅在组级别，并且您的 OUTER 查询按公交线路、保险簿、索引等的更多部分分组，无论它们在哪个子分类中， GROUP 金额将对所有人显示相同，这可能不是您要找的。
最后，您的大量解析、填充、字符串提取似乎不是分类事物的好方法，但使用单独的列可能会更有效……只是一个想法。
正如 Drapp 所说，您得到的是笛卡尔坐标（不是真的但接近它）结果，因此您的临时文件变得越来越大。如果你做了一个小模型:
立方体1
t_from cal_mth cal_yr ann_be_if group
------ ------- ------ --------- -----
0 1 2016 100 1
0 2 2016 800 2
1 3 2016 500 3
魔方2
t_from fieldX group
------ ------ -----
0 1 1
0 2 1
0 3 1
立方体3
t_from fieldY group
------ ------ -----
0 1 2
0 2 2
0 3 2
0 4 2
使用您的 SQL，这将导致从 cube1 中选择 2 行 3*4 次（您的条件匹配 cube2 中的 3 行和 cube3 中的 4 行），这将是 24 行。由于您重复选择相同的行 3*4 次，因此您也将 ann_be_if 的值重复 3*4 次并将它们相加为 (800+100)*12 = 10800。这不太可能是您想。
You shouldn't do aggregation when you have joins (either implicit or explicit) with other tables (unless you are sure you wouldn't have this duplication - i.e. joining sample customer, orders,orditems tables wouldn't exhibit this duplication).
再次阅读您的 SQL，第二个和第三个表对结果没有任何作用，只是您只是对它们进行 "exists" 检查。所以您可以将代码重写为：
**** Assign variables to the input files ****
*!* moses_output_file = output_path + "\annuities~main10.dbf"
*!* SELECT 0
*!* Use (moses_output_file) ALIAS cube_output1
*!* moses_output_file = output_path + "\annuities~main10~reinsured.dbf"
*!* SELECT 0
*!* Use (moses_output_file) ALIAS cube_output2
*!* moses_output_file = output_path + "\annuities~main10~reinsurance_premium.dbf"
*!* SELECT 0
*!* Use (moses_output_file) ALIAS cube_output3
cube_output1 = m.output_path + "\annuities~main10.dbf"
cube_output2 = m.output_path + "\annuities~main10~reinsured.dbf"
cube_output3 = m.output_path + "\annuities~main10~reinsurance_premium.dbf"
Select t_from, ;
cal_mth As calendar_month, ;
cal_yr As calendar_year, ;
substr(c1.Group,1,3) As BUS_LINE, ;
padr(Iif(isnumeric(Strextract(c1.Group,"|","|",1,1),""), ;
substr(c1.Group,1,4) + Strextract(c1.Group,"|","|",1,1), ;
strextract(c1.Group,"|","|",1,1)),10) As reinsurance_book, ;
padr(Strextract(c1.Group,"|","|",3,1),10) As Esc_Index, ;
padr(Iif(Left(c1.Group,3)="LRT",;
strextract(c1.Group,"|","|",5,1),;
strextract(c1.Group,"|","|",4,1)),10) As MAP_ELIG, ;
padr(Iif(Left(c1.Group,3)="LRT",;
strextract(Trim(c1.Group - "|"),"|","|",6,1), ;
strextract(Trim(c1.Group - "|"),"|","|",5,1)),10) As MAP_STAT, ;
SUM(ann_ben_if) As ann_ben_if ;
FROM (m.cube_output1) c1 ;
WHERE c1.t_from = 0 And ;
exists ( Select * From (m.cube_output2) c2 ;
Where c2.t_from = 0 And c1.Group = c2.Group ) ;
OR ;
exists ( Select * From (m.cube_output3) c3 ;
Where c3.t_from = 0 And c1.Group = c3.Group ) ;
group By t_from, calendar_month, calendar_year, ;
BUS_LINE, reinsurance_book, Esc_Index, MAP_ELIG, MAP_STAT ;
Into Cursor temp ;
Readwrite
**** Copy to a CSV file ****
COPY TO (m.cube2_output_path + "\IFRS_Split_Cashflows_2\run_" + m.scenario + ".csv") TYPE CSV