在对非托管资源执行 P/Invoke 时，何时需要 GC.KeepAlive(this)？

Question

我有一个 TestNet 本机组件的包装器。本机组件公开了一个 blocking TestNative::Foo()，它通过调用托管回调与托管部分进行通信，以及一个用于检索对 .NET 包装器的引用的弱 GCHandle并提供上下文。 GCHandle 很弱，因为 .NET 包装器旨在隐藏正在向用户处理非托管资源的事实，并且故意不实现 IDisposable 接口：不弱会阻止 TestNet 实例完全不被收集，造成内存泄漏。发生的事情是，在 Release 构建中，只有垃圾收集器将在执行托管回调时收集对 .NET 包装器的引用，甚至在 TestNative::Foo() 和令人惊讶的 TestNet::Foo() 解锁之前.我了解我自己的问题，我可以通过在 P/Invoke 调用后发出 GC.KeepAlive(this) 来解决它，但由于这方面的知识不是很普遍，似乎很多人都做错了。我有几个问题：

如果最后一条指令是对非托管资源的 P/Invoke 调用，托管方法中是否总是需要 GC.KeepAlive(this) 或者在这种特殊情况下只需要它，即在编组时切换到托管执行上下文来自本机代码的托管回调？问题可能是：我应该把 GC.KeepAlive(this) 放在任何地方吗？这个老微软blog (original link is 404, here is cached) 好像是这么建议的！但这将改变游戏规则，基本上这意味着大多数人从未正确地 P/Invoke，因为这将需要审查包装器中的大多数 P/Invoke 调用。例如，是否有一条规则说垃圾收集器（EDIT：或更好的终结器）不能运行对于属于当前线程的对象，而执行上下文是不受管理的（母语）？
在哪里可以找到合适的文档？我可以找到 CodeAnalysis 策略 CA2115 pointing to generically use GC.KeepAlive(this) any time a unmanaged resource is accessed with P/Invoke. In general GC.KeepAlive(this) seems to be very rarely needed when dealing with finalizers.
为什么这只发生在发布版本中？它看起来像是一个优化，但在调试构建中根本不需要隐藏垃圾收集器的一个重要行为。

注意：我对收集代表没有任何问题，这是一个不同的问题，我知道如何正确处理。这里的问题是当 P/Invoke 调用尚未完成时收集非托管资源的对象。

它遵循的代码清楚地表明了问题。创建一个 C# 控制台应用程序和一个 C++ Dll1 项目并在 Release 模式下构建它们：

Program.cs:

using System;
using System.Runtime.InteropServices;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            var test = new TestNet();
            try
            {
                test.Foo();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }

    class TestNet
    {
        [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
        delegate void Callback(IntPtr data);

        static Callback _callback;

        IntPtr _nativeHandle;
        GCHandle _thisHandle;

        static TestNet()
        {
            // NOTE: Keep delegates references so they can be
            // stored persistently in unmanaged resources
            _callback = callback;
        }

        public TestNet()
        {
            _nativeHandle = CreateTestNative();

            // Keep a weak handle to self. Weak is necessary
            // to not prevent garbage collection of TestNet instances
            _thisHandle = GCHandle.Alloc(this, GCHandleType.Weak);

            TestNativeSetCallback(_nativeHandle, _callback, GCHandle.ToIntPtr(_thisHandle));
        }

        ~TestNet()
        {
            Console.WriteLine("this.~TestNet()");
            FreeTestNative(_nativeHandle);
            _thisHandle.Free();
        }

        public void Foo()
        {
            Console.WriteLine("this.Foo() begins");
            TestNativeFoo(_nativeHandle);

            // This is never printed when the object is collected!
            Console.WriteLine("this.Foo() ends");

            // Without the following GC.KeepAlive(this) call
            // in Release build the program will consistently collect
            // the object in callback() and crash on next iteration 
            //GC.KeepAlive(this);
        }

        static void callback(IntPtr data)
        {
            Console.WriteLine("TestNet.callback() begins");
            // Retrieve the weak reference to self. As soon as the istance
            // of TestNet exists. 
            var self = (TestNet)GCHandle.FromIntPtr(data).Target;
            self.callback();

            // Enforce garbage collection. On release build
            self = null;
            GC.Collect();
            GC.WaitForPendingFinalizers();
            Console.WriteLine("TestNet.callback() ends");
        }

        void callback()
        {
            Console.WriteLine("this.callback()");
        }

        [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
        static extern IntPtr CreateTestNative();

        [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
        static extern void FreeTestNative(IntPtr obj);

        [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
        static extern void TestNativeSetCallback(IntPtr obj, Callback callback, IntPtr data);

        [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
        static extern void TestNativeFoo(IntPtr obj);
    }
}

Dll1.cpp:

#include <iostream>

extern "C" typedef void (*Callback)(void *data);

class TestNative
{
public:
    void SetCallback(Callback callback1, void *data);
    void Foo();
private:
    Callback m_callback;
    void *m_data;
};

void TestNative::SetCallback(Callback callback, void * data)
{
    m_callback = callback;
    m_data = data;
}

void TestNative::Foo()
{
    // Foo() will never end
    while (true)
    {
        m_callback(m_data);
    }
}

extern "C"
{
    __declspec(dllexport) TestNative * CreateTestNative()
    {
        return new TestNative();
    }

    __declspec(dllexport) void FreeTestNative(TestNative *obj)
    {
        delete obj;
    }

    __declspec(dllexport) void TestNativeSetCallback(TestNative *obj, Callback callback1, void * data)
    {
        obj->SetCallback(callback1, data);
    }

    __declspec(dllexport) void TestNativeFoo(TestNative *obj)
    {
        obj->Foo();
    }
}

输出始终如一：

this.Foo() begins
TestNet.callback() begins
this.callback()
this.~TestNet()
TestNet.callback() ends
TestNet.callback() begins
System.NullReferenceException: Object reference not set to an instance of an object.

如果取消注释 TestNet.Foo() 中的 GC.KeepAlive(this) 调用，程序将永远不会结束。

Answer 1

总结非常有用的评论和所做的研究：

1) 如果最后一条指令是使用实例持有的非托管资源的 P/Invoke 调用，托管实例方法中是否总是需要 GC.KeepAlive(this)？

是的，如果您不希望 API 的用户在病态情况下最后负责持有托管对象实例的 non-collectible 引用，请查看下面的示例.但这不是唯一的方法：HandleRef or SafeHandle 技术也可用于在执行 P/Invoke Interop 时延长托管对象的生命周期。

该示例随后将通过持有本机资源的托管实例调用本机方法：

using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            new Thread(delegate()
            {
                // Run a separate thread enforcing GC collections every second
                while(true)
                {
                    GC.Collect();
                    Thread.Sleep(1000);
                }
            }).Start();

            while (true)
            {
                var test = new TestNet();
                test.Foo();
                TestNet.Dump();
            }
        }
    }

    class TestNet
    {
        static ManualResetEvent _closed;
        static long _closeTime;
        static long _fooEndTime;

        IntPtr _nativeHandle;

        public TestNet()
        {
            _closed = new ManualResetEvent(false);
            _closeTime = -1;
            _fooEndTime = -1;
            _nativeHandle = CreateTestNative();
        }

        public static void Dump()
        {
            // Ensure the now the object will now be garbage collected
            GC.Collect();
            GC.WaitForPendingFinalizers();

            // Wait for current object to be garbage collected
            _closed.WaitOne();
            Trace.Assert(_closeTime != -1);
            Trace.Assert(_fooEndTime != -1);
            if (_closeTime <= _fooEndTime)
                Console.WriteLine("WARN: Finalize() commenced before Foo() return");
            else
                Console.WriteLine("Finalize() commenced after Foo() return");
        }

        ~TestNet()
        {
            _closeTime = Stopwatch.GetTimestamp();
            FreeTestNative(_nativeHandle);
            _closed.Set();
        }

        public void Foo()
        {
            // The native implementation just sleeps for 250ms
            TestNativeFoo(_nativeHandle);

            // Uncomment to have all Finalize() to commence after Foo()
            //GC.KeepAlive(this);
            _fooEndTime = Stopwatch.GetTimestamp();
        }

        [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
        static extern IntPtr CreateTestNative();

        [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
        static extern void FreeTestNative(IntPtr obj);

        [DllImport("Dll1", CallingConvention = CallingConvention.Cdecl)]
        static extern void TestNativeFoo(IntPtr obj);
    }
}

为了使本机调用始终安全，我们希望终结器仅在 Foo() return 之后被调用。相反，我们可以通过在后台线程中手动调用垃圾收集来轻松强制执行违规。输出如下：

Finalize() commenced after Foo() return
WARN: Finalize() commenced before Foo() return
Finalize() commenced after Foo() return
Finalize() commenced after Foo() return
Finalize() commenced after Foo() return
WARN: Finalize() commenced before Foo() return
Finalize() commenced after Foo() return

2) 我在哪里可以找到文档？

GC.KeepAlive() provides an example very similar to the managed callback in the original question. HandleRef 的文档对托管对象和互操作的生命周期也有非常有趣的考虑：

If you use platform invoke to call a managed object, and the object is not referenced elsewhere after the platform invoke call, it is possible for the garbage collector to finalize the managed object. This action releases the resource and invalidates the handle, causing the platform invoke call to fail. Wrapping a handle with HandleRef guarantees that the managed object is not garbage collected until the platform invoke call completes.

@GSerg 还发现 link[1] 解释了对象何时符合收集条件，指出 this 引用不在根集中，允许在实例时收集它方法尚未 returned.

3) 为什么这只发生在发布版本中？

正如@SimonMourier 所指出的那样，这是一种优化，并且在启用优化的情况下也可能发生在调试版本中。默认情况下，在 Debug 中也未启用它，因为它可能会阻止调试当前方法范围内的变量，如这些 answers.

中所述

[1] https://devblogs.microsoft.com/oldnewthing/20100810-00/?p=13193?

在对非托管资源执行 P/Invoke 时，何时需要 GC.KeepAlive(this)？

When GC.KeepAlive(this) is needed when doing P/Invoke on unmanaged resources?

c#

callback

garbage-collection

pinvoke

finalizer