来自 MySQL 数据库 reader 的 Powershell Export-CSV 导出失败

Question

我对 PowerShell 有点陌生，我有一个新的要求，要将数据从 MySQL 数据库中取出并放入 Oracle 数据库中。我选择的策略是输出为CSV，然后将CSV导入Oracle。

我想获得从 MySQL 导出到 CSV 的进度条，所以我使用数据 reader 来实现这一点。它可以工作，并开始导出，但在导出过程中的某个地方（大约 450 万中的 5,000 记录 -- 不一致）它会抛出错误：

Exception calling "Read" with "0" argument(s): "Fatal error encountered during data read." Exception calling "Close" with "0" argument(s): "Timeout in IO operation" Method invocation failed because [System.Management.Automation.PSObject] does not contain a method named 'op_Addition'. Exception calling "ExecuteReader" with "0" argument(s): "The CommandText property has not been properly initialized."

适用的代码块如下。我不确定我在这里做错了什么，如果有任何反馈，我将不胜感激。几天来我一直在为此烦恼。

注意事项： $tableObj 是一个自定义对象，带有一些字符串字段以保存 table 名称和 SQL 值。此处未显示那些 SQL 查询，但它们有效。

Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()

#
# Get Count of records in table
#
$countCmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlCount, $conn)
$recordCount = 0
try{
  $recordCount = $countCmd.ExecuteScalar()
} Catch { 
  Write-Host "[ERROR]: (" $tableObj.Table ") Error getting Count."
  Write-Host "---" $_.Exception.Message
  Exit
} 
$recordCountString = $recordCount.ToString('N0')
Write-Host "[INFO]: Count for table '" $tableObj.Table "' is " $recordCountString

#
# Compose the command
#
$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlExportInit, $conn)

#
# Write to CSV using DataReader
#
Write-Host "[INFO]: Data gathered into memory. Writing data to CSV file '" $tableObj.OutFile "'"
$counter = 0 # Tracks items selected
$reader=$cmd.ExecuteReader()
$dataRows = @()
# Read all rows into a hash table
while ($reader.Read())
{
    $counter++
    $percent = ($counter/$recordCount)*100
    $percentString = [math]::Round($percent,3)
    $counterString = $counter.ToString('N0')
    Write-Progress -Activity '[INFO]: CSV Export In Progress' -Status "$percentString% Complete" -CurrentOperation "($($counterString) of $($recordCountString))" -PercentComplete $percent 
    $row = @{}
    for ($i = 0; $i -lt $reader.FieldCount; $i++)
    {
        $row[$reader.GetName($i)] = $reader.GetValue($i)
    }
    # Convert hashtable into an array of PSObjects
    $dataRows += New-Object psobject -Property $row            
}
$conn.Close()
$dataRows | Export-Csv $tableObj.OutFile -NoTypeInformation

EDIT：没有用，但我也将此行添加到我的连接字符串中：defaultcommandtimeout=600;connectiontimeout=25 per

Answer 1

使用@Carl Ardiente 的想法，查询超时，您必须将超时设置为疯狂的值才能完全执行。在开始获取数据之前，您只需为会话设置超时值。

Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()

# Set timeout on MySql

$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand("set net_write_timeout=99999; set net_read_timeout=99999", $conn) 
$cmd.ExecuteNonQuery()

#
# Get Count of records in table
#
...Etc....

Answer 2

并不是说我找到了解决方案，但是 none 的连接字符串更改起作用了。手动设置超时似乎也无济于事。好像是返回的行数太多造成的，所以我分批将函数分解为运行，然后附加到CSV。这消除了 IO / 超时错误。

来自 MySQL 数据库 reader 的 Powershell Export-CSV 导出失败

Powershell Export-CSV from MySQL database reader fails mid-export

mysql

powershell

csv

mysqldatareader