在平面文件连接器上设置行分隔符属性

Question

我正在创建一个允许我导入 CSV 的 BIML 脚本。

但是，平面文件连接管理器上的属性 Row Delimiter 以某种方式设置为 Vertical Bar，就像列分隔符一样。这使我无法导入文件。

列分隔符作为竖线对吧，我只是不明白为什么它也将行分隔符设置为竖线...

每当我手动将行定界符设置为 CRLF 时，它就像一个魅力。但在 BIML 中，它始终设置为与列分隔符相同的值。

我找不到发生这种情况的地方....

我尝试以多种方式使用表达式来设置行定界符，但是......

有人知道吗？

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
    <#
    string Applicatie = "BIML";
    string Prefix = "import";
     
    string fileName;
    string path = @"c:\temp";
    string[] myFiles = Directory.GetFiles(path, "*.csv");
     
    string[] myColumns;
    #>
    <FileFormats>
    <#
    foreach (string filePath in myFiles)
    {
 
    #>
    <FlatFileFormat Name="FlatFileFormat<#=Path.GetFileNameWithoutExtension(filePath)#>" RowDelimiter="CRLF" ColumnNamesInFirstDataRow="true" IsUnicode="false">
        <Columns>
            <# 
                 
                StreamReader myFile = new StreamReader(filePath);
                myColumns = myFile.ReadLine().Replace("\"","").Split('|');
                 myFile.Close();
                 
                // to determine the column delimiter 
                int columnCount = 0;
                string columnDelimiter = "|";
 
                 foreach (string myColumn in myColumns) 
                {
                    columnCount++;
                    if (columnCount == myColumns.Length)
                    {
                        columnDelimiter = "CRLF";
                    }
                    {
                        columnDelimiter = "|";
                    }
                #>
                <Column Name="<#=myColumn#>" DataType = "String" Length="250" Delimiter="<#=columnDelimiter#>"></Column>
                <# } #>
            </Columns>
        </FlatFileFormat>
            <#}#>
    </FileFormats>
    <Connections>
            <#
            foreach (string filePath in myFiles)
            {
             
            #>
            <FlatFileConnection Name="FF_CSV-<#=Path.GetFileNameWithoutExtension(filePath)#>" 
                                FilePath="<#=filePath#>" 
                                FileFormat="FlatFileFormat<#=Path.GetFileNameWithoutExtension(filePath)#>" 
             >
                 <Expressions>
                    <Expression ExternalProperty="TextQualifier">&quot;\&quot;&quot;</Expression>
                    <Expression ExternalProperty="HeaderRowDelimiter">&quot;CRLF&quot;</Expression>
                    <Expression ExternalProperty="RowDelimiter">&quot;CRLF&quot;</Expression>
                </Expressions>

            </FlatFileConnection>
            <# } #>
            <OleDbConnection
                Name="STG_<#=Applicatie#>" 
                ConnectionString="Data Source=SQLSERVER;Initial Catalog=TEST;Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;">
            </OleDbConnection>
                 <Connection ConnectionName="FF_CSV - <#=Path.GetFileNameWithoutExtension(filePath)#>">
                    <Expressions>
                        <Expression ExternalProperty="TextQualifier">&quot;\&quot;&quot;</Expression>
                        <Expression ExternalProperty="HeaderRowDelimiter">&quot;CRLF&quot;</Expression>
                        <Expression ExternalProperty="RowDelimiter">&quot;CRLF&quot;</Expression>
                    </Expressions>
                </Connection>
    </Connections> 
        <Packages>
 
    <#       // Loop trough the files
            int TableCount = 0;
            foreach (string filePath in myFiles)
            { 
                TableCount++;
                fileName = Path.GetFileNameWithoutExtension(filePath);
                #>
                <Package Name="stg_<#=Prefix#>_<#=TableCount.ToString()#>_<#=fileName#>" ConstraintMode="Linear" AutoCreateConfigurationsType="None" ProtectionLevel="EncryptSensitiveWithPassword" PackagePassword="secret">
                    <Variables>
                        <Variable Name="CountStage" DataType="Int32" Namespace="User">0</Variable>
                    </Variables>               
                    <Tasks>
                        <ExecuteSQL ConnectionName="STG_<#=Applicatie#>" Name="SQL - Truncate <#=fileName#>">
                            <DirectInput>TRUNCATE TABLE dbo.<#=Prefix#>_<#=fileName#></DirectInput>
                        </ExecuteSQL>
                         
                        <Dataflow Name="DFT-Transport CSV_<#=fileName#>">
                            <Transformations>
                                <FlatFileSource Name="SRC_FF-<#=fileName#> " ConnectionName="FF_CSV - <#=Path.GetFileNameWithoutExtension(filePath)#>">
                                </FlatFileSource>
                                 
                                <OleDbDestination ConnectionName="STG_<#=Applicatie#>" Name="OLE_DST - <#=fileName#>" >
                                <ExternalTableOutput Table="dbo.<#=Prefix#>_<#=fileName#>"/>
                                </OleDbDestination>
                            </Transformations>
                        </Dataflow>
                    </Tasks>
                </Package>
    <#    }    #>
 
                <!-- Create Master Package -->
                <Package Name="stg_<#=Prefix#>_0_Master" ConstraintMode="Parallel" AutoCreateConfigurationsType="None" ProtectionLevel="EncryptSensitiveWithPassword" PackagePassword="secret">
                    <Tasks>
                    <#  int TableCount2 = 0;
                        foreach (string filePath in myFiles)
                        { 
                                TableCount2++;
                                fileName = Path.GetFileNameWithoutExtension(filePath); #>
                 
                            <ExecutePackage Name="stg_<#=Prefix#>_<#=TableCount2.ToString()#>_<#=fileName#>">
                                <ExternalProjectPackage  Package="stg_<#=Prefix#>_<#=TableCount2.ToString()#>_<#=fileName#>.dtsx" />
                            </ExecutePackage>
                        <#
                        }
                        #>    
                    </Tasks>
                </Package>
 
    </Packages>
</Biml>
 
<!--Includes/Imports for C#-->
<#@ template language="C#" hostspecific="true"#>
<#@ import namespace="System.Data"#>
<#@ import namespace="System.IO"#>

这是实际输出的 BIML：

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
    <Connections>
        <FlatFileConnection Name="FF_CSV - AB" FilePath="c:\temp\AB.csv" FileFormat="FlatFileFormatAB">
            <Expressions>
                <Expression ExternalProperty="TextQualifier">&quot;\&quot;&quot;</Expression>
                <Expression ExternalProperty="RowDelimiter">&quot;CRLF&quot;</Expression>
            </Expressions>
        </FlatFileConnection>
        <Connection Name="STG_BIML" ConnectionString="Data Source=SQLSERVER;Initial Catalog=TEST;Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;" />
    </Connections>
    <Packages>
        <Package Name="stg_import_1_AB" ConstraintMode="Linear" ProtectionLevel="EncryptSensitiveWithPassword" PackagePassword="secret">
            <Variables>
                <Variable Name="CountStage" DataType="Int32">0</Variable>
            </Variables>
            <Tasks>
                <ExecuteSQL Name="SQL - Truncate AB" ConnectionName="STG_BIML">
                    <DirectInput>TRUNCATE TABLE dbo.import_AB</DirectInput>
                </ExecuteSQL>
                <Dataflow Name="DFT - Transport CSV_AB">
                    <Transformations>
                        <FlatFileSource Name="SRC_FF - AB " ConnectionName="FF_CSV - AB" />
                        <OleDbDestination Name="OLE_DST - AB" ConnectionName="STG_BIML">
                            <ExternalTableOutput Table="dbo.import_AB" />
                        </OleDbDestination>
                    </Transformations>
                </Dataflow>
            </Tasks>
        </Package>
        <Package Name="stg_import_0_Master" ProtectionLevel="EncryptSensitiveWithPassword" PackagePassword="secret">
            <Tasks>
                <ExecutePackage Name="stg_import_1_AB">
                    <ExternalProjectPackage Package="stg_import_1_AB.dtsx" />
                </ExecutePackage>
            </Tasks>
        </Package>
    </Packages>
    <FileFormats>
        <FlatFileFormat Name="FlatFileFormatAB" ColumnNamesInFirstDataRow="true" RowDelimiter="CRLF" IsUnicode="false">
            <Columns>
                <Column Name="Relatienummer" Length="250" DataType="String" Delimiter="|" />
                <Column Name="Tussenpersoon" Length="250" DataType="String" Delimiter="|" />
                <Column Name="Polisnummer" Length="250" DataType="String" Delimiter="|" />
                <Column Name="Subpakketpolisnummer" Length="250" DataType="String" Delimiter="|" />
                <Column Name="Product" Length="250" DataType="String" Delimiter="|" />
                <Column Name="Maatschappij" Length="250" DataType="String" Delimiter="|" />
                <Column Name="CodeVerzekering" Length="250" DataType="String" Delimiter="|" />
                <Column Name="Branche" Length="250" DataType="String" Delimiter="|" />
                <Column Name="Termijn" Length="250" DataType="String" Delimiter="|" />
                <Column Name="SoortBedrijf" Length="250" DataType="String" Delimiter="|" />
                <Column Name="SBIcode" Length="250" DataType="String" Delimiter="|" />
                <Column Name="VerzekerdBedragA" Length="250" DataType="String" Delimiter="|" />
                <Column Name="VerzekerdBedragB" Length="250" DataType="String" Delimiter="|" />
            </Columns>
        </FlatFileFormat>
    </FileFormats>
</Biml>

Answer 1

找到了

                if (columnCount == myColumns.Length)
                {
                    columnDelimiter = "CRLF";
                }
                {
                    columnDelimiter = "|";
                }

你那里缺少 else。应该是

                if (columnCount == myColumns.Length)
                {
                    columnDelimiter = "CRLF";
                }
                else
                {
                    columnDelimiter = "|";
                }

最小重复

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<#
List<string> myColumns = new List<string>{"A", "B", "C"};

#>
<FileFormats>
    <FlatFileFormat Name="SO_66970366" RowDelimiter="CRLF" ColumnNamesInFirstDataRow="true" IsUnicode="false">
        <Columns>
<#
int columnCount = 0;
string columnDelimiter = "";
WriteLine($"<!-- ref count {myColumns.Count} -->");
    foreach(string myColumn in myColumns)
    {
    
        columnCount++;
        bool finalColumn = columnCount == myColumns.Count;
        WriteLine($"<!-- actual count {columnCount} -->");
        WriteLine($"<!-- what {columnCount == myColumns.Count} {finalColumn} -->");
        
        if (finalColumn)
        {
            columnDelimiter = "CRLF";
        }
        else
        {
            columnDelimiter = "|";
        }
        WriteLine($"<!-- delimiter {columnDelimiter} -->");
        
#>
    <Column Name="<#=myColumn#>" DataType = "String" Length="250" Delimiter="<#=columnDelimiter#>" />
<#
    }
#>            
        </Columns>
    </FlatFileFormat>
</FileFormats>    
        
</Biml>

发出的 Biml 看起来像

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
  <FileFormats>
    <FlatFileFormat Name="SO_66970366" RowDelimiter="CRLF" ColumnNamesInFirstDataRow="true" IsUnicode="false">
      <Columns>
        <!-- ref count 3 -->
        <!-- act count 1 -->
        <!-- what False False -->
        <!-- delimiter | -->
        <Column Name="A" DataType = "String" Length="250" Delimiter="|" />
        <!-- act count 2 -->
        <!-- what False False -->
        <!-- delimiter | -->
        <Column Name="B" DataType = "String" Length="250" Delimiter="|" />
        <!-- act count 3 -->
        <!-- what True True -->
        <!-- delimiter CRLF -->
        <Column Name="C" DataType = "String" Length="250" Delimiter="|" />
      </Columns>
    </FlatFileFormat>
  </FileFormats>
</Biml>

在平面文件连接器上设置行分隔符属性

Set row delimiter property on Flat File Connector

c#

sql-server

visual-studio

ssis

biml

在平面文件连接器上设置行分隔符 属性

Set row delimiter property on Flat File Connector

c#

sql-server

visual-studio

ssis

biml

在平面文件连接器上设置行分隔符属性