提高更新 big table、Laravel 的性能

Question

我有一个函数应该运行每周一次（cron 作业），现在我尝试进行压力测试。

在我的请求中我得到：

Maximum execution time of 60 seconds exceeded

protected function updateAnswerHistory(){
    $answer_statistics = AnswerStatistic::select('question_id','paragraph_id','lkp_answer_id')->get(); //about 500row

    $history = AnswerHistory::select('id', 'question_id','paragraph_id','lkp_answer_id','correct_answer_score')->get(); //about 40k rows

    foreach ($history as $row) {

        if($row->question_id){
            $lkp_answer_id = $answer_statistics->where('question_id', $row->question_id)->pluck('lkp_answer_id')->first();
            if($row->lkp_answer_id === $lkp_answer_id){
                $row->update(['correct_answer_score' => 7]);
            }else{
                $row->update(['correct_answer_score' => 4]);
            }
        }

        if($row->paragraph_id){
            $lkp_answer_id = $answer_statistics->where('paragraph_id', $row->paragraph_id)->pluck('lkp_answer_id')->first();
            if($row->lkp_answer_id === $lkp_answer_id){
                $row->update(['correct_answer_score' => 7]);
            }else{
                $row->update(['correct_answer_score' => 4]);
            }
        }
    }
}

一件坏事是 query 来自 foreach 这需要时间，但我不确定如何改进它。

Answer 1

我不确定我是否正确理解了数据库 table 的结构，

但是从数据库获取数据并更新它们的成本很高

你应该以任何方式在数据库中进行更新过程...

这个代码思路是加入两个基于question_id列的table然后制作'wheres'然后更新，我没有机会测试它......

AnswerHistory::join('answer_statistics','answer_statistics.question_id','answer_histories.question_id')-> where('answer_histories.question_id','!=',null)->
    where('answer_histories.lkp_answer_id','=',DB::raw('answer_statistics.lkp_answer_id'))
        ->update(['correct_answer_score' => 3]);

AnswerHistory::join('answer_statistics','answer_statistics.question_id','answer_histories.question_id')-> where('answer_histories.question_id','!=',null)->
    where('answer_histories.lkp_answer_id','!=',DB::raw('answer_statistics.lkp_answer_id'))
        ->update(['correct_answer_score' => 0]);

如果有帮助请告诉我

Answer 2

检索大数据集时，使用 DB facade 比使用 Eloquent Model 更快因为它将减少将数据库记录转换为 eloquent 模型

的时间

而且由于批量更新比单行更新快，你可以这样做

$questionList = DB::table('answer_history')
 ->join('answer_statistic')
 ->on('answer_history.question_id', 'answer_statistic.question_id')
 ->on('answer_history.lkp_answer_id', 'answer_statistic.lkp_answer_id')
 ->whereNull('answer_statistic.deleted_at')
 ->pluck('answer_history.id')
 ->toArray();


$paraList = DB::table('answer_history')
 ->join('answer_statistic')
 ->on('answer_history.paragraph_id', 'answer_statistic.paragraph_id')
 ->on('answer_history.lkp_answer_id', 'answer_statistic.lkp_answer_id')
 ->whereNull('answer_statistic.deleted_at')
 ->pluck('answer_history.id')
 ->toArray();

$ids7 = array_merge($questionList,$paraList); 

AnswerHistory::wherein('id', $ids7) >update(['correct_answer_score' => 7]);
AnswerHistory::whereNotin('id',$ids7 )->update(['correct_answer_score' => 4]);

已编辑：innerjoin -> join

提高更新 big table、Laravel 的性能

Improve performance on updating big table, Laravel

laravel

laravel-7