Long-运行具有响应式表单的流程 - 性能改进

Question

所以，我正在为我的内部应用程序开发一个库，它与我们的 PostgreSQL 数据库（以及许多其他东西）进行交互。目前的一个要求是该库可以将数据从数据库转储到文件中。我有一些工作，但我一直在努力尽可能地提高它的性能。这是我目前正在查看的内容：

Using COPYReader As NpgsqlCopyTextReader = CType(CIADB.DBConnection.BeginTextExport(COPYSQL), NpgsqlCopyTextReader)
    With COPYReader
        Dim stopWatch As New Stopwatch
        Dim ts As TimeSpan
        Dim elapsedTime As String

        ' ** FIRST ATTEMPT
        stopWatch.Start()
        Dim BufferText As String = .ReadLine

        Do While Not BufferText Is Nothing
            CurrentPosition += 1
            OutputFile.WriteLine(BufferText)

            If Not UpdateForm Is Nothing Then
                UpdateForm.UpdateProgress(Convert.ToInt32((CurrentPosition / MaxRecords) * 100))
            End If

            BufferText = .ReadLine
        Loop

        OutputFile.Flush()
        OutputFile.Close()

        stopWatch.Stop()
        ts = stopWatch.Elapsed
        elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)

        ' ** FIRST ATTEMPT RESULTS
        ' ** Records Retrieved: 65358
        ' ** Time To Complete: 2:12.07
        ' ** Lines Written: 65358
        ' ** File Size: 8,166 KB

        ' ** SECOND ATTEMPT
        stopWatch.Start()

        Using TestOutputFile As New IO.StreamWriter(DestinationFile.FullName.Replace(".TXT", "_TEST.TXT"), False)
            TestOutputFile.Write(.ReadToEndAsync.Result)
        End Using

        stopWatch.Stop()
        ts = stopWatch.Elapsed
        elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)

        ' ** SECOND ATTEMPT
        ' ** Records Retrieved: 65358
        ' ** Time To Complete: 1:04.01
        ' ** Lines Written: 65358
        ' ** File Size: 8,102 KB
    End With
End Using

我已经运行对每种方法进行了多次测试，得出了几乎相同的结果。 第一次尝试 花费的时间大约是 第二次尝试

的两倍

显然，FIRST ATTEMPT中使用的UpdateForm.UpdateProgress方法（用于保持表单响应并显示当前导出进度）将导致进程由于表单更新等原因需要更长的时间，更不用说逐行写入文件了。这几乎就是为什么我希望通过在一行代码中执行完整转储来减少额外调用的次数。问题是，如果我使用“单线”，在流程完成之前，表格完全没有反应。

我已经尝试将 SECOND ATTEMPT 中的“一次性”转储代码移动到单独的 Async 方法中，但是我我非常不熟悉一般的异步方法，所以我（显然）做得不对：

Private Async Sub OutputToFile(ByVal COPYReader As NpgsqlCopyTextReader, ByVal DestinationFile As IO.FileInfo)
    ' ** METHOD 3
    Using TestOutputFile As New IO.StreamWriter(DestinationFile.FullName.Replace(".TXT", "_TEST.TXT"), False)
        Await TestOutputFile.WriteAsync(COPYReader.ReadToEndAsync.Result)
    End Using

    ' ** METHOD 3 RESULTS
    ' ** Records Retrieved: 65358
    ' ** Time To Complete: 0:15.07
    ' ** Lines Written: 34
    ' ** File Size: 4 KB
End Sub

还有一件事要提：我尝试将所有这些移动到 BackgroundWorker，但是当我尝试调用我的 UpdateForm.UpdateProgress 方法时我遇到了一些奇怪的行为，这导致应用程序完全跳过实际的转储过程。我目前已经放弃尝试将它放到一个单独的线程中，但我仍然愿意接受其他建议。这实际上是我要丢弃的较小的桌子之一，所以我不期待其中一张较大的桌子会做什么。

为了完整起见，这里是我在我的库中实现的 UpdateForm class 以实现跨其他应用程序的可重用性：

Imports System.Windows.Forms

Namespace Common
    Public Class FormHandler
        Implements IDisposable

        Public Property ApplicationForm As Form
        Public Property ApplicationStatusLabel As Label
        Public Property ApplicationToolStripLabel As ToolStripStatusLabel
        Public Property ApplicationProgressBar As ProgressBar

        Private LabelVisibleState As Boolean = True
        Private ProgressBarVisibleState As Boolean = True
        Private CurrentStatusText As String
        Private CurrentProgress As Integer

        Public Sub New(ByVal AppForm As Form)
            ApplicationForm = AppForm
        End Sub

        Public Sub New(ByVal StatusLabel As Label, ByVal Progress As ProgressBar)
            ApplicationStatusLabel = StatusLabel
            ApplicationToolStripLabel = Nothing
            ApplicationProgressBar = Progress
            ApplicationForm = ApplicationProgressBar.Parent.FindForm

            LabelVisibleState = StatusLabel.Visible
            ProgressBarVisibleState = Progress.Visible

            With ApplicationProgressBar
                .Minimum = 0
                .Maximum = 100
                .Value = 0
                .Visible = True
            End With

            With ApplicationStatusLabel
                .Visible = True
                .Text = ""
            End With
        End Sub

        Public Sub New(ByVal StatusLabel As ToolStripStatusLabel, ByVal Progress As ProgressBar)
            ApplicationToolStripLabel = StatusLabel
            ApplicationStatusLabel = Nothing
            ApplicationProgressBar = Progress
            ApplicationForm = ApplicationProgressBar.Parent.FindForm

            LabelVisibleState = StatusLabel.Visible
            ProgressBarVisibleState = Progress.Visible

            With ApplicationProgressBar
                .Minimum = 0
                .Maximum = 100
                .Value = 0
                .Visible = True
            End With

            With ApplicationToolStripLabel
                .Visible = True
                .Text = ""
            End With
        End Sub

        Public Sub New(ByVal AppForm As Form, ByVal StatusLabel As Label, ByVal Progress As ProgressBar)
            ApplicationForm = AppForm
            ApplicationStatusLabel = StatusLabel
            ApplicationToolStripLabel = Nothing
            ApplicationProgressBar = Progress

            LabelVisibleState = StatusLabel.Visible
            ProgressBarVisibleState = Progress.Visible

            With ApplicationProgressBar
                .Minimum = 0
                .Maximum = 100
                .Value = 0
                .Visible = True
            End With

            With ApplicationStatusLabel
                .Visible = True
                .Text = ""
            End With
        End Sub

        Public Sub New(ByVal AppForm As Form, ByVal StatusLabel As ToolStripStatusLabel, ByVal Progress As ProgressBar)
            ApplicationForm = AppForm
            ApplicationToolStripLabel = StatusLabel
            ApplicationStatusLabel = Nothing
            ApplicationProgressBar = Progress

            LabelVisibleState = StatusLabel.Visible
            ProgressBarVisibleState = Progress.Visible

            With ApplicationProgressBar
                .Minimum = 0
                .Maximum = 100
                .Value = 0
                .Visible = True
            End With

            With ApplicationToolStripLabel
                .Visible = True
                .Text = ""
            End With
        End Sub

        Friend Sub UpdateProgress(ByVal StatusText As String, ByVal CurrentPosition As Integer, ByVal MaxValue As Integer)
            CurrentStatusText = StatusText
            CurrentProgress = Convert.ToInt32((CurrentPosition / MaxValue) * 100)
            UpdateStatus()
        End Sub

        Friend Sub UpdateProgress(ByVal StatusText As String, ByVal PercentComplete As Decimal)
            CurrentStatusText = StatusText
            CurrentProgress = Convert.ToInt32(PercentComplete)
            UpdateStatus()
        End Sub

        Friend Sub UpdateProgress(ByVal StatusText As String)
            CurrentStatusText = StatusText
            CurrentProgress = 0
            UpdateStatus()
        End Sub

        Friend Sub UpdateProgress(ByVal PercentComplete As Decimal)
            CurrentProgress = Convert.ToInt32(PercentComplete)
            UpdateStatus()
        End Sub

        Friend Sub UpdateProgress(ByVal CurrentPosition As Integer, ByVal MaxValue As Integer)
            CurrentProgress = Convert.ToInt32((CurrentPosition / MaxValue) * 100)
            UpdateStatus()
        End Sub

        Friend Sub ResetProgressUpdate()
            CurrentStatusText = ""
            CurrentProgress = 0
            UpdateStatus()
        End Sub

        Private Sub UpdateStatus()
            If Not ApplicationForm Is Nothing Then
                If ApplicationForm.InvokeRequired Then
                    Dim UpdateInvoker As New MethodInvoker(AddressOf UpdateStatus)

                    Try
                        ApplicationForm.Invoke(UpdateInvoker)
                    Catch ex As Exception
                        Dim InvokeError As New ErrorHandler(ex)

                        InvokeError.LogException()
                    End Try
                Else
                    UpdateApplicationProgress(CurrentStatusText)
                End If
            End If
        End Sub

        Friend Sub UpdateApplicationProgress(ByVal ProgressText As String)
            If Not ApplicationForm Is Nothing Then
                With ApplicationForm
                    If Not ProgressText Is Nothing Then
                        If Not ApplicationStatusLabel Is Nothing Then
                            ApplicationStatusLabel.Text = ProgressText
                        End If

                        If Not ApplicationToolStripLabel Is Nothing Then
                            ApplicationToolStripLabel.Text = ProgressText
                        End If
                    End If

                    If Not ApplicationProgressBar Is Nothing Then
                        ApplicationProgressBar.Value = CurrentProgress
                    End If
                End With

                ApplicationForm.Refresh()
                Application.DoEvents()
            End If
        End Sub

        Public Sub Dispose() Implements IDisposable.Dispose
            If Not ApplicationForm Is Nothing Then
                ApplicationForm.Dispose()
            End If

            If Not ApplicationStatusLabel Is Nothing Then
                ApplicationStatusLabel.Visible = LabelVisibleState
                ApplicationStatusLabel.Dispose()
            End If

            If Not ApplicationToolStripLabel Is Nothing Then
                ApplicationToolStripLabel.Visible = LabelVisibleState
                ApplicationToolStripLabel.Dispose()
            End If

            If Not ApplicationProgressBar Is Nothing Then
                ApplicationProgressBar.Visible = ProgressBarVisibleState
                ApplicationProgressBar.Dispose()
            End If
        End Sub
    End Class
End Namespace

编辑

根据@the_lotus 评论中的建议，我稍微修改了 FIRST ATTEMPT 以检查当前进度的值（我声明了 CurrentProgress 变量作为 Integer)，它 显着地 改进了所用时间：

' ** FOURTH ATTEMPT
Using COPYReader As NpgsqlCopyTextReader = CType(CIADB.DBConnection.BeginTextExport(COPYSQL), NpgsqlCopyTextReader)
    With COPYReader
        Dim stopWatch As New Stopwatch
        Dim ts As TimeSpan
        Dim elapsedTime As String
        Dim CurrentProgress As Integer = 0

        stopWatch.Start()
        Dim BufferText As String = .ReadLine

        Do While Not BufferText Is Nothing
            CurrentPosition += 1
            OutputFile.WriteLine(BufferText)

            ' ** Checks to see if the value of the ProgressBar will actually
            ' ** be changed by the CurrentPosition before making a call to
            ' ** UpdateProgress.  If the value doesn't change, don't waste
            ' ** the call
            If Convert.ToInt32((CurrentPosition / MaxRecords) * 100) <> CurrentProgress Then
                CurrentProgress = Convert.ToInt32((CurrentPosition / MaxRecords) * 100)

                If Not UpdateForm Is Nothing Then
                    UpdateForm.UpdateProgress(CurrentProgress)
                End If
            End If

            BufferText = .ReadLine
        Loop

        OutputFile.Flush()
        OutputFile.Close()

        stopWatch.Stop()
        ts = stopWatch.Elapsed
        elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)
    End With
End Using
' ** FOURTH ATTEMPT RESULTS
' ** Records Retrieved: 65358
' ** Time To Complete: 0:47.45
' ** Lines Written: 65358
' ** File Size: 8,166 KB

当然，与我在每条记录上调用时相比，表单的响应“稍微”慢一些，但我认为这是值得的权衡。

编辑#2

这样我就可以最大限度地减少每次使用 UpdateProgress 方法，我已经将更改值的测试移到那里，它似乎以相同的性能改进运行。同样，为了完整起见，这里是执行实际 progress/status 更新所涉及的两个私有方法的代码：

Private Sub UpdateStatus()
    If Not ApplicationForm Is Nothing Then
        If ApplicationForm.InvokeRequired Then
            Dim UpdateInvoker As New MethodInvoker(AddressOf UpdateStatus)

            Try
                ApplicationForm.Invoke(UpdateInvoker)
            Catch ex As Exception
                Dim InvokeError As New ErrorHandler(ex)

                InvokeError.LogException()
            End Try
        Else
            UpdateApplicationProgress()
        End If
    End If
End Sub

Private Sub UpdateApplicationProgress()
    Dim Changed As Boolean = False

    If Not ApplicationForm Is Nothing Then
        With ApplicationForm
            If Not CurrentStatusText Is Nothing Then
                If Not ApplicationStatusLabel Is Nothing Then
                    If ApplicationStatusLabel.Text <> CurrentStatusText Then
                        Changed = True
                        ApplicationStatusLabel.Text = CurrentStatusText
                    End If
                End If

                If Not ApplicationToolStripLabel Is Nothing Then
                    If ApplicationToolStripLabel.Text <> CurrentStatusText Then
                        Changed = True
                        ApplicationToolStripLabel.Text = CurrentStatusText
                    End If
                End If
            End If

            If Not ApplicationProgressBar Is Nothing Then
                If ApplicationProgressBar.Value <> CurrentProgress Then
                    Changed = True
                    ApplicationProgressBar.Value = CurrentProgress
                End If
            End If
        End With

        If Changed Then
            ApplicationForm.Refresh()
        End If

        Application.DoEvents()
    End If
End Sub

这样做还有一个额外的好处，即可以将一些响应性返回到以前丢失的表单。我希望至少有一些代码和信息对外面的人有帮助。

Answer 1

您不需要在每次调用时都调用 UpdateProgress。当百分比甚至没有移动时就没有必要了。尝试做一个小检查，只在需要时更新百分比。

也有可能第二次尝试更快，因为它没有进入数据库。可以缓存数据。

Long-运行具有响应式表单的流程 - 性能改进

Long-running Process with Responsive Form - Performance Improvements

vb.net

npgsql

编辑

编辑#2

Long-运行 具有响应式表单的流程 - 性能改进

Long-running Process with Responsive Form - Performance Improvements

vb.net

npgsql

编辑

编辑#2

Long-运行具有响应式表单的流程 - 性能改进