在嵌套的 foreach 循环中使用 MySQL 查询的缓慢 Excel 报告 [Laravel 5.3]
Slow Excel report with MySQL queries inside nested foreach loops [Laravel 5.3]
我在使用 Laravel 的 Maatwebsite 插件创建 XLS/XLSX 报告时遇到问题(大约 5 - 30、30 - 60 分钟,具体取决于报告)这是很多时间!!
网上搜索发现Maatwebsite使用的是PHPExcel,生成XLS/XLSX文件时如果有很多or register,速度慢,耗内存。
检查时间,我遇到过由于嵌套的 foreach 循环,我的查询几乎呈指数增长,报告的基本生成是这样的:
$clients = Client::all(); // can change depending on conditions, worse case with all clients
Excel::create('Client Statement', function($excel) use ($clients) {
$excel->sheet('Sheet 1', function($sheet) {
$row = 1; // row number
foreach($clients as $client) {
$sheet->row($row++, [$client->data1,...,$client->dataN]);
$charges = Charge::get_ByClient($client->id);
foreach($charges as $charge) {
$sheet->row($row++, [$charge->data1,...,$charge->dataN]);
$payments = Payment::get_ByCharge($charge->id);
foreach($payments as $payment) {
$sheet->row($row++, [$payment->data1,...,$payment->dataN]);
}
}
}
});
});
基本解释是:
- 获取所有客户
- 对于每个客户,获得所有费用
- 对于每笔费用,获得所有付款
问题是,我有大约 1500 个客户,每个客户可以收取 10 - 100 笔费用,每笔费用可以有 1 - 5 笔付款,这导致性能缓慢。为此,将库生成 XLS/XLSX 文件所需的时间相加。
有什么建议吗?提前致谢。
如果我没理解错的话。
您可以使用以下示例。
效果很好。
摘自
http://www.websleson.info/2016/02/ex
创建 MYSQL TABLE
CREATE TABLE IF NOT EXISTS `tbl_customer` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`CustomerName` varchar(50) NOT NULL,
`Address` varchar(50) NOT NULL,
`City` varchar(50) NOT NULL,
`PostalCode` int(12) NOT NULL,
`Country` varchar(50) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=3 ;
INSERT INTO `tbl_customer` (`id`, `CustomerName`, `Address`, `City`, `PostalCode`, `Country`) VALUES
(1, 'Maria Anders', 'Obere Str. 57', 'Berlin', 12209, 'Germany'),
(2, 'Ana Trujillo', 'Avda. de la Construction 2222', 'Mexico D.F.', 5021, 'Mexico');
INDEX.PHP
<?php
$connect = mysqli_connect("localhost", "root", "", "testing");
$sql = "SELECT * FROM tbl_customer";
$result = mysqli_query($connect, $sql);
?>
<html>
<head>
<title>Export MySQL data to Excel in PHP</title>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" />
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script>
</head>
<body>
<div class="container">
<br />
<br />
<br />
<div class="table-responsive">
<h2 align="center">Export MySQL data to Excel in PHP</h2><br />
<table class="table table-bordered">
<tr>
<th>Name</th>
<th>Address</th>
<th>City</th>
<th>Postal Code</th>
<th>Country</th>
</tr>
<?php
while($row = mysqli_fetch_array($result))
{
echo '
<tr>
<td>'.$row["CustomerName"].'</td>
<td>'.$row["Address"].'</td>
<td>'.$row["City"].'</td>
<td>'.$row["PostalCode"].'</td>
<td>'.$row["Country"].'</td>
</tr>
';
}
?>
</table>
<br />
<form method="post" action="export.php">
<input type="submit" name="export" class="btn btn-success" value="Export" />
</form>
</div>
</div>
</body>
</html>
EXSPORT.PHP
<?php
//export.php
$connect = mysqli_connect("localhost", "root", "", "testing");
$output = '';
if(isset($_POST["export"]))
{
$query = "SELECT * FROM tbl_customer";
$result = mysqli_query($connect, $query);
if(mysqli_num_rows($result) > 0)
{
$output .= '
<table class="table" bordered="1">
<tr>
<th>Name</th>
<th>Address</th>
<th>City</th>
<th>Postal Code</th>
<th>Country</th>
</tr>
';
while($row = mysqli_fetch_array($result))
{
$output .= '
<tr>
<td>'.$row["CustomerName"].'</td>
<td>'.$row["Address"].'</td>
<td>'.$row["City"].'</td>
<td>'.$row["PostalCode"].'</td>
<td>'.$row["Country"].'</td>
</tr>
';
}
$output .= '</table>';
header('Content-Type: application/xls');
header('Content-Disposition: attachment; filename=download.xls');
echo $output;
}
}
?>
为了澄清这一点,我确实按照@Mark Baker 建议的那样使用 Eloquent 的关系来加快速度,我的问题是典型的 N + 1 问题。为了解决这个问题,我使用了 Laravel's Eager Loading
我在使用 Laravel 的 Maatwebsite 插件创建 XLS/XLSX 报告时遇到问题(大约 5 - 30、30 - 60 分钟,具体取决于报告)这是很多时间!!
网上搜索发现Maatwebsite使用的是PHPExcel,生成XLS/XLSX文件时如果有很多or register,速度慢,耗内存。
检查时间,我遇到过由于嵌套的 foreach 循环,我的查询几乎呈指数增长,报告的基本生成是这样的:
$clients = Client::all(); // can change depending on conditions, worse case with all clients
Excel::create('Client Statement', function($excel) use ($clients) {
$excel->sheet('Sheet 1', function($sheet) {
$row = 1; // row number
foreach($clients as $client) {
$sheet->row($row++, [$client->data1,...,$client->dataN]);
$charges = Charge::get_ByClient($client->id);
foreach($charges as $charge) {
$sheet->row($row++, [$charge->data1,...,$charge->dataN]);
$payments = Payment::get_ByCharge($charge->id);
foreach($payments as $payment) {
$sheet->row($row++, [$payment->data1,...,$payment->dataN]);
}
}
}
});
});
基本解释是:
- 获取所有客户
- 对于每个客户,获得所有费用
- 对于每笔费用,获得所有付款
问题是,我有大约 1500 个客户,每个客户可以收取 10 - 100 笔费用,每笔费用可以有 1 - 5 笔付款,这导致性能缓慢。为此,将库生成 XLS/XLSX 文件所需的时间相加。
有什么建议吗?提前致谢。
如果我没理解错的话。 您可以使用以下示例。 效果很好。 摘自 http://www.websleson.info/2016/02/ex
创建 MYSQL TABLE
CREATE TABLE IF NOT EXISTS `tbl_customer` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`CustomerName` varchar(50) NOT NULL,
`Address` varchar(50) NOT NULL,
`City` varchar(50) NOT NULL,
`PostalCode` int(12) NOT NULL,
`Country` varchar(50) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=3 ;
INSERT INTO `tbl_customer` (`id`, `CustomerName`, `Address`, `City`, `PostalCode`, `Country`) VALUES
(1, 'Maria Anders', 'Obere Str. 57', 'Berlin', 12209, 'Germany'),
(2, 'Ana Trujillo', 'Avda. de la Construction 2222', 'Mexico D.F.', 5021, 'Mexico');
INDEX.PHP
<?php
$connect = mysqli_connect("localhost", "root", "", "testing");
$sql = "SELECT * FROM tbl_customer";
$result = mysqli_query($connect, $sql);
?>
<html>
<head>
<title>Export MySQL data to Excel in PHP</title>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" />
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script>
</head>
<body>
<div class="container">
<br />
<br />
<br />
<div class="table-responsive">
<h2 align="center">Export MySQL data to Excel in PHP</h2><br />
<table class="table table-bordered">
<tr>
<th>Name</th>
<th>Address</th>
<th>City</th>
<th>Postal Code</th>
<th>Country</th>
</tr>
<?php
while($row = mysqli_fetch_array($result))
{
echo '
<tr>
<td>'.$row["CustomerName"].'</td>
<td>'.$row["Address"].'</td>
<td>'.$row["City"].'</td>
<td>'.$row["PostalCode"].'</td>
<td>'.$row["Country"].'</td>
</tr>
';
}
?>
</table>
<br />
<form method="post" action="export.php">
<input type="submit" name="export" class="btn btn-success" value="Export" />
</form>
</div>
</div>
</body>
</html>
EXSPORT.PHP
<?php
//export.php
$connect = mysqli_connect("localhost", "root", "", "testing");
$output = '';
if(isset($_POST["export"]))
{
$query = "SELECT * FROM tbl_customer";
$result = mysqli_query($connect, $query);
if(mysqli_num_rows($result) > 0)
{
$output .= '
<table class="table" bordered="1">
<tr>
<th>Name</th>
<th>Address</th>
<th>City</th>
<th>Postal Code</th>
<th>Country</th>
</tr>
';
while($row = mysqli_fetch_array($result))
{
$output .= '
<tr>
<td>'.$row["CustomerName"].'</td>
<td>'.$row["Address"].'</td>
<td>'.$row["City"].'</td>
<td>'.$row["PostalCode"].'</td>
<td>'.$row["Country"].'</td>
</tr>
';
}
$output .= '</table>';
header('Content-Type: application/xls');
header('Content-Disposition: attachment; filename=download.xls');
echo $output;
}
}
?>
为了澄清这一点,我确实按照@Mark Baker 建议的那样使用 Eloquent 的关系来加快速度,我的问题是典型的 N + 1 问题。为了解决这个问题,我使用了 Laravel's Eager Loading