如何为 XML 编写 "filter" 流包装器？

Question

我有一些大的 XML 提要文件，其中包含非法字符（0x1 等）。这些文件是第三方的，我无法更改编写它们的过程。

我想使用 XmlReader 处理这些文件，但它会在这些非法字符上爆炸。

我可以读取文件，过滤掉坏字符，保存它们，然后处理它们......但是这是很多I/O，看起来应该是不必要的。

我想做的是这样的：

using(var origStream = File.OpenRead(fileName))
using(var cleanStream = new CleansedXmlStream(origStream))
using(var streamReader = new StreamReader(cleanStream))
using(var xmlReader = XmlReader.Create(streamReader))
{
    //do stuff with reader
}

我尝试从 Stream 继承，但是当我开始实施 Read(byte[] buffer, int offset, int count) 时，我失去了一些信心。毕竟，我正计划删除字符，所以看起来计数会关闭，而且我必须将每个字节转换为 char 这看起来很昂贵（尤其是在大文件上）而且我不清楚这是怎么回事可以使用 Unicode 编码，但我的问题的答案并不直观。

谷歌搜索 "c# stream wrapper" 或 "c# filter stream" 时，我没有得到满意的结果。可能我使用了错误的词或描述了错误的概念，所以我希望 SO 社区可以解决我的问题。

使用上面的示例，CleansedXmlStream 会是什么样子？

这是我的第一次尝试：

public class CleansedXmlStream : Stream
{
    private readonly Stream _baseStream;

    public CleansedXmlStream(Stream stream)
    {
        this._baseStream = stream;
    }

    public new void Dispose()
    {
        if (this._baseStream != null)
        {
            this._baseStream.Dispose();
        }
        base.Dispose();
    }

    public override bool CanRead
    {
        get { return this._baseStream.CanRead; }
    }

    public override bool CanSeek
    {
        get { return this._baseStream.CanSeek; }
    }

    public override bool CanWrite
    {
        get { return this._baseStream.CanWrite; }
    }

    public override long Length
    {
        get { return this._baseStream.Length; }
    }

    public override long Position
    {
        get { return this._baseStream.Position; }
        set { this._baseStream.Position = value; }
    }

    public override void Flush()
    {
        this._baseStream.Flush();
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        //what does this look like?

        throw new NotImplementedException();
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        return this._baseStream.Seek(offset, origin);
    }

    public override void SetLength(long value)
    {
        this._baseStream.SetLength(value);
    }

    public override void Write(byte[] buffer, int offset, int count)
    {
        throw new NotSupportedException();
    }
}

Answer 1

受@CharlesMager 评论的启发，我最终没有制作 Stream，而是 StreamReader 像这样：

public class CleanTextReader : StreamReader
{
    private readonly ILog _logger;

    public CleanTextReader(Stream stream, ILog logger) : base(stream)
    {
        this._logger = logger;
    }

    public CleanTextReader(Stream stream) : this(stream, LogManager.GetLogger<CleanTextReader>())
    {
        //nothing to do here.
    }

    /// <summary>
    ///     Reads a specified maximum of characters from the current stream into a buffer, beginning at the specified index.
    /// </summary>
    /// <returns>
    ///     The number of characters that have been read, or 0 if at the end of the stream and no data was read. The number
    ///     will be less than or equal to the <paramref name="count" /> parameter, depending on whether the data is available
    ///     within the stream.
    /// </returns>
    /// <param name="buffer">
    ///     When this method returns, contains the specified character array with the values between
    ///     <paramref name="index" /> and (<paramref name="index + count - 1" />) replaced by the characters read from the
    ///     current source.
    /// </param>
    /// <param name="index">The index of <paramref name="buffer" /> at which to begin writing. </param>
    /// <param name="count">The maximum number of characters to read. </param>
    /// <exception cref="T:System.ArgumentException">
    ///     The buffer length minus <paramref name="index" /> is less than
    ///     <paramref name="count" />.
    /// </exception>
    /// <exception cref="T:System.ArgumentNullException"><paramref name="buffer" /> is null. </exception>
    /// <exception cref="T:System.ArgumentOutOfRangeException">
    ///     <paramref name="index" /> or <paramref name="count" /> is
    ///     negative.
    /// </exception>
    /// <exception cref="T:System.IO.IOException">An I/O error occurs, such as the stream is closed. </exception>
    public override int Read(char[] buffer, int index, int count)
    {
        try
        {
            var rVal = base.Read(buffer, index, count);
            var filteredBuffer = buffer.Select(x => XmlConvert.IsXmlChar(x) ? x : ' ').ToArray();
            Buffer.BlockCopy(filteredBuffer, 0, buffer, 0, count);
            return rVal;
        }
        catch (Exception ex)
        {
            this._logger.Error("Read(char[], int, int)", ex);
            throw;
        }
    }

    /// <summary>
    ///     Reads a maximum of <paramref name="count" /> characters from the current stream, and writes the data to
    ///     <paramref name="buffer" />, beginning at <paramref name="index" />.
    /// </summary>
    /// <returns>
    ///     The position of the underlying stream is advanced by the number of characters that were read into
    ///     <paramref name="buffer" />.The number of characters that have been read. The number will be less than or equal to
    ///     <paramref name="count" />, depending on whether all input characters have been read.
    /// </returns>
    /// <param name="buffer">
    ///     When this method returns, this parameter contains the specified character array with the values
    ///     between <paramref name="index" /> and (<paramref name="index" /> + <paramref name="count" /> -1) replaced by the
    ///     characters read from the current source.
    /// </param>
    /// <param name="index">The position in <paramref name="buffer" /> at which to begin writing.</param>
    /// <param name="count">The maximum number of characters to read. </param>
    /// <exception cref="T:System.ArgumentNullException"><paramref name="buffer" /> is null. </exception>
    /// <exception cref="T:System.ArgumentException">
    ///     The buffer length minus <paramref name="index" /> is less than
    ///     <paramref name="count" />.
    /// </exception>
    /// <exception cref="T:System.ArgumentOutOfRangeException">
    ///     <paramref name="index" /> or <paramref name="count" /> is
    ///     negative.
    /// </exception>
    /// <exception cref="T:System.ObjectDisposedException">The <see cref="T:System.IO.TextReader" /> is closed. </exception>
    /// <exception cref="T:System.IO.IOException">An I/O error occurs. </exception>
    public override int ReadBlock(char[] buffer, int index, int count)
    {
        try
        {
            var rVal = base.ReadBlock(buffer, index, count);
            var filteredBuffer = buffer.Select(x => XmlConvert.IsXmlChar(x) ? x : ' ').ToArray();
            Buffer.BlockCopy(filteredBuffer, 0, buffer, 0, count);
            return rVal;
        }
        catch (Exception ex)
        {
            this._logger.Error("ReadBlock(char[], in, int)", ex);
            throw;
        }
    }

    /// <summary>
    ///     Reads the stream from the current position to the end of the stream.
    /// </summary>
    /// <returns>
    ///     The rest of the stream as a string, from the current position to the end. If the current position is at the end of
    ///     the stream, returns an empty string ("").
    /// </returns>
    /// <exception cref="T:System.OutOfMemoryException">
    ///     There is insufficient memory to allocate a buffer for the returned
    ///     string.
    /// </exception>
    /// <exception cref="T:System.IO.IOException">An I/O error occurs. </exception>
    public override string ReadToEnd()
    {
        var chars = new char[4096];
        int len;
        var sb = new StringBuilder(4096);
        while ((len = Read(chars, 0, chars.Length)) != 0)
        {
            sb.Append(chars, 0, len);
        }
        return sb.ToString();
    }
}

我的单元测试是这样的：

[TestMethod]
public void CleanTextReaderCleans()
{
    //arrange
    var originalString = "The quick brown fox jumped over the lazy dog.";
    var badChars = new string(new[] {(char) 0x1});
    var concatenated = string.Concat(badChars, originalString);

    //act
    using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(concatenated)))
    {
        using (var reader = new CleanTextReader(stream))
        {
            var newString = reader.ReadToEnd().Trim();
            //assert
            Assert.IsTrue(originalString.Equals(newString));
        }
    }
}

... 用法如下所示：

using(var origStream = File.OpenRead(fileName))
using(var streamReader = new CleanTextReader(origStream))
using(var xmlReader = XmlReader.Create(streamReader))
{
    //do stuff with reader
}

如果有人有改进建议，我很乐意听取。

Answer 2

我尝试了@JeremyHolovacs 流实现，但它仍然不足以满足我的用例：

using (var fstream = File.OpenRead(dlpath))
{
    using (var zstream = new GZipStream(fstream, CompressionMode.Decompress))
    {
        using (var xstream = new CleanTextReader(zstream))
        {
            var ser = new XmlSerializer(typeof(MyType));
            prods = ser.Deserialize(XmlReader.Create(xstream, new XmlReaderSettings() { CheckCharacters = false })) as MyType;
        }
    }
}

不知何故，并不是所有相关的重载都必须实现。我按如下方式调整了 class，它按预期工作：

public class CleanTextReader : StreamReader
{
    public CleanTextReader(Stream stream) : base(stream)
    {
    }

    public override int Read()
    {
        var val = base.Read();
        return XmlConvert.IsXmlChar((char)val) ? val : (char)' ';
    }

    public override int Read(char[] buffer, int index, int count)
    {
        var ret = base.Read(buffer, index, count);

        for (int i=0; i<ret; i++)
        {
            int idx = index + i;
            if (!XmlConvert.IsXmlChar(buffer[idx]))
                buffer[idx] = ' ';
        }

        return ret;
    }

    public override int ReadBlock(char[] buffer, int index, int count)
    {
        var ret = base.ReadBlock(buffer, index, count);

        for (int i = 0; i < ret; i++)
        {
            int idx = index + i;
            if (!XmlConvert.IsXmlChar(buffer[idx]))
                buffer[idx] = ' ';
        }

        return ret;
    }
}

如何为 XML 编写 "filter" 流包装器？

How to write a "filter" stream wrapper for XML?

c#

xml

filter

stream

wrapper