通过 Powershell 将大型 XML 文件导入 SQL Express

Question

如标题所示，我必须将大型 (6GB) XML 文件导入 SQLExpress 服务器。由于这必须每天在 VServer 本身上发生，并且我不允许安装更多软件，所以我仅限于 powershell（尽管已升级到版本 4）。我的灵感来源是 Use PowerShell to Load XML Data into SQL Server

我当前的脚本如下所示：

#database variables
$serverInstance="QLIKVIEWSQL\QLIKV_SQLEXPRESS";
$database = "Xml-Import_TEST"; 
$table = "dbo.vkjournal";

#opening the sql connection
$cn = new-object System.Data.SqlClient.SqlConnection;
$user="user";
$password="password"
$cn.ConnectionString="Server=$serverInstance;Database=$database;user   Id=$user;password=$password;Integrated Security=SSPI;";
$cn.Open();


#Semi-automatic creation of the XmlSchema
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.CommandText = "SELECT * FROM "+$table+" WHERE 1 = 2";
$cmd.Connection = $cn
$da = New-Object System.Data.SqlClient.SqlDataAdapter;
$da.SelectCommand = $cmd
$ds = New-Object System.Data.DataSet
$da.Fill($ds, $table) | Out-Null
$ds.WriteXmlSchema('C:\Users\qlikview\Desktop\SQLTEST\'+$table+'.xsd');


#Reading the xml-schema and loading bw1.xml into an DataSet object. Then bulkcopying to sql.
$ds = new-object "System.Data.DataSet"
$ds.ReadXmlSchema("C:\Users\qlikview\Desktop\SQLTEST\dbo.vkjournal.xsd");
$ds.ReadXml("C:\Users\qlikview\Desktop\SQLTEST\bw1.xml")
$dtProd = $ds.Tables[0]
$bc = new-object ("System.Data.SqlClient.SqlBulkCopy") $cn
$bc.DestinationTableName = $table;
$bc.WriteToServer($dtProd)

$cn.Close()

像往常一样，powershell 不会抛出任何错误，只会抛出 IgnoreSchema（如果我注释掉 ReadXmlSchema 行）或 InferSchema（如果我不注释）这些词，并且在 SQLServer 端没有任何反应。

创建的XML架构（缩短）如下所示：

<?xml version="1.0" standalone="yes"?>
<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
  <xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
    <xs:complexType>
      <xs:choice minOccurs="0" maxOccurs="unbounded">
        <xs:element name="dbo.vkjournal">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="KASS_NR" type="xs:short" minOccurs="0" />
              <xs:element name="VKJO_DATUM" type="xs:dateTime" minOccurs="0" />
              <xs:element name="VKJO_ZEIT" type="xs:duration" minOccurs="0" />
              <xs:element name="VKJO_BESTE" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:choice>
    </xs:complexType>
  </xs:element>
</xs:schema>

虽然我尝试按照 1 中的建议更改一些元素名称，但结果没有改变。

非常感谢您提供一些提示，谢谢。

Answer 1

即使这个问题在某种程度上不受欢迎，我还是找到了一种使用 this microsoft link. The full script throw some errors so i ditched the validation and copied only the bulk load. I had to install msxml 6.0 的工作方法，如果这很重要，我确实安装了 SQLXML 4.0。对于那些像我一样拥有大 table 定义的人，上面的脚本（请参阅#Semi-automatic creation of the XmlSchema 部分）有助于极大地减少所需的手写。

set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkload.4.0")
objBL.ConnectionString = "provider=SQLOLEDB;data source=SERVER\SQLEXPRESS_INSTANCE;database=YOURDB;integrated security=SSPI"
objBL.ErrorLogFile = "C:\error.log"

' Check constraints and initiate transaction (if needed)
   ' objBL.CheckConstraints = True
   ' objBL.Transaction=True
  'Execute XML bulkload using file.
  objBL.Execute "C:\XSDfile.xml", "C:\File.xml"
  set objBL=Nothing

通过 Powershell 将大型 XML 文件导入 SQL Express

Importing large XML File into MSSQLExpress through Powerhsell

sql-server

xml

powershell

sqlbulkcopy