如何正确取消初始化 OpenSSL

How to properly uninitialize OpenSSL

在我的 OpenSSL 客户端中,我遇到了一个问题,即在我选择静态而不是动态地 link libeay32 和 ssleay32 的那一刻,我从 Visual Leak Detector 收到了大量内存泄漏错误。我从 this thread 中的 OP 复制了命令,但我还剩下 6 个。然后我在同一个线程中按照 4LegsDrivenCat 的建议添加了 sk_SSL_COMP_free(SSL_COMP_get_compression_methods());,只剩下 4 个,所有这些显然都与加载我用来与服务器证书进行比较的可信证书有关。

我使用 Visual Studio 2013 Express、OpenSSL 1.0.1L(32 位和 64 位)、VLD 2.4RC2,我的电脑是 Windows 7 64 位。

下面的调用堆栈是安全模式下来自 VLD 的 64 位。在 32 位 VLD 中在安全模式下崩溃(虽然它在快速模式下工作但不会产生像样的调用堆栈)。我删除了调用堆栈中引用我自己的函数以及十六进制数据的部分。

Visual Leak Detector Version 2.4RC2 installed.
WARNING: Visual Leak Detector detected memory leaks!
---------- Block 5671 at 0x000000000097E9B0: 180 bytes ----------
  Leak Hash: 0xA14DA3AA, Count: 1, Total 180 bytes
  Call Stack (TID 7088):
    0x000000007746FAC0 (File and line number not available): ntdll.dll!RtlAllocateHeap
    f:\dd\vctools\crt\crtw32\heap\malloc.c (58): MyLib.dll!_heap_alloc_base
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (431): MyLib.dll!_heap_alloc_dbg_impl + 0xA bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (239): MyLib.dll!_nh_malloc_dbg_impl + 0x22 bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (302): MyLib.dll!_nh_malloc_dbg + 0x2A bytes
    f:\dd\vctools\crt\crtw32\misc\dbgmalloc.c (56): MyLib.dll!malloc + 0x21 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\mem.c (312): MyLib.dll!CRYPTO_malloc + 0xF bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\lhash\lhash.c (121): MyLib.dll!lh_new + 0x16 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (450): MyLib.dll!int_thread_get + 0x13 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (509): MyLib.dll!int_thread_set_item + 0xF bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (1031): MyLib.dll!ERR_get_state
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (730): MyLib.dll!ERR_put_error + 0x5 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_lib.c (703): MyLib.dll!PEM_read_bio + 0x20 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_info.c (280): MyLib.dll!PEM_X509_INFO_read_bio + 0x10 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (278): MyLib.dll!X509_load_cert_crl_file
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (123): MyLib.dll!by_file_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_lu.c (120): MyLib.dll!X509_LOOKUP_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_d2.c (92): MyLib.dll!X509_STORE_load_locations + 0x1D bytes

---------- Block 5670 at 0x000000001AC815C0: 164 bytes ----------
  Leak Hash: 0x38C8916E, Count: 1, Total 164 bytes
  Call Stack (TID 7088):
    0x000000007746FAC0 (File and line number not available): ntdll.dll!RtlAllocateHeap
    f:\dd\vctools\crt\crtw32\heap\malloc.c (58): MyLib.dll!_heap_alloc_base
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (431): MyLib.dll!_heap_alloc_dbg_impl + 0xA bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (239): MyLib.dll!_nh_malloc_dbg_impl + 0x22 bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (302): MyLib.dll!_nh_malloc_dbg + 0x2A bytes
    f:\dd\vctools\crt\crtw32\misc\dbgmalloc.c (56): MyLib.dll!malloc + 0x21 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\mem.c (312): MyLib.dll!CRYPTO_malloc + 0xF bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\lhash\lhash.c (119): MyLib.dll!lh_new + 0x13 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (450): MyLib.dll!int_thread_get + 0x13 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (509): MyLib.dll!int_thread_set_item + 0xF bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (1031): MyLib.dll!ERR_get_state
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (730): MyLib.dll!ERR_put_error + 0x5 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_lib.c (703): MyLib.dll!PEM_read_bio + 0x20 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_info.c (280): MyLib.dll!PEM_X509_INFO_read_bio + 0x10 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (278): MyLib.dll!X509_load_cert_crl_file
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (123): MyLib.dll!by_file_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_lu.c (120): MyLib.dll!X509_LOOKUP_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_d2.c (92): MyLib.dll!X509_STORE_load_locations + 0x1D bytes

---------- Block 5669 at 0x000000001ADABE80: 588 bytes ----------
  Leak Hash: 0xC3E47B0F, Count: 1, Total 588 bytes
  Call Stack (TID 7088):
    0x000000007746FAC0 (File and line number not available): ntdll.dll!RtlAllocateHeap
    f:\dd\vctools\crt\crtw32\heap\malloc.c (58): MyLib.dll!_heap_alloc_base
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (431): MyLib.dll!_heap_alloc_dbg_impl + 0xA bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (239): MyLib.dll!_nh_malloc_dbg_impl + 0x22 bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (302): MyLib.dll!_nh_malloc_dbg + 0x2A bytes
    f:\dd\vctools\crt\crtw32\misc\dbgmalloc.c (56): MyLib.dll!malloc + 0x21 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\mem.c (312): MyLib.dll!CRYPTO_malloc + 0xF bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (1019): MyLib.dll!ERR_get_state + 0x17 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (730): MyLib.dll!ERR_put_error + 0x5 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_lib.c (703): MyLib.dll!PEM_read_bio + 0x20 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_info.c (280): MyLib.dll!PEM_X509_INFO_read_bio + 0x10 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (278): MyLib.dll!X509_load_cert_crl_file
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (123): MyLib.dll!by_file_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_lu.c (120): MyLib.dll!X509_LOOKUP_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_d2.c (92): MyLib.dll!X509_STORE_load_locations + 0x1D bytes

---------- Block 5672 at 0x000000001ADC4180: 76 bytes ----------
  Leak Hash: 0x02B2EA5E, Count: 1, Total 76 bytes
  Call Stack (TID 7088):
    0x000000007746FAC0 (File and line number not available): ntdll.dll!RtlAllocateHeap
    f:\dd\vctools\crt\crtw32\heap\malloc.c (58): MyLib.dll!_heap_alloc_base
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (431): MyLib.dll!_heap_alloc_dbg_impl + 0xA bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (239): MyLib.dll!_nh_malloc_dbg_impl + 0x22 bytes
    f:\dd\vctools\crt\crtw32\misc\dbgheap.c (302): MyLib.dll!_nh_malloc_dbg + 0x2A bytes
    f:\dd\vctools\crt\crtw32\misc\dbgmalloc.c (56): MyLib.dll!malloc + 0x21 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\mem.c (312): MyLib.dll!CRYPTO_malloc + 0xF bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\lhash\lhash.c (193): MyLib.dll!lh_insert + 0x15 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (515): MyLib.dll!int_thread_set_item
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (1031): MyLib.dll!ERR_get_state
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c (730): MyLib.dll!ERR_put_error + 0x5 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_lib.c (703): MyLib.dll!PEM_read_bio + 0x20 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\pem\pem_info.c (280): MyLib.dll!PEM_X509_INFO_read_bio + 0x10 bytes
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (278): MyLib.dll!X509_load_cert_crl_file
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\by_file.c (123): MyLib.dll!by_file_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_lu.c (120): MyLib.dll!X509_LOOKUP_ctrl
    d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\x509\x509_d2.c (92): MyLib.dll!X509_STORE_load_locations + 0x1D bytes


Visual Leak Detector detected 4 memory leaks (1008 bytes).
Largest number used: 529114 bytes.
Total allocations: 1070421 bytes.
Visual Leak Detector is now exiting.

编辑: 我将泄漏归因于对 SSL_CTX_load_verify_locations 的调用。有谁知道使用此功能时我需要取消分配什么?只是注释这个函数会导致泄漏消失,所以这不是因为我传递给它的参数。

d:\cfiles\projects\winssl\openssl-1.0.1l\crypto\err\err.c

抓住这一点,似乎需要释放一些错误状态(或字符串)。


How to properly uninitialize OpenSSL

启动和关闭的代码如下所示(包括FIPS)。如果您执行加载 DH 参数之类的操作,那么您也需要清理它们。

博士。 Henson 对他为每个线程调用 ERR_remove_state 的建议很有帮助。请参阅 OpenSSL 邮件列表中的 Order of Cleanup to avoid memory leaks?

启动

  • SSL_library_init();
  • SSL_load_error_strings();
  • FIPS_mode_set(1);
  • CRYPTO_set_id_callback(<fn>);
  • CRYPTO_set_locking_callback(<fn>);

关机

  • FIPS_mode_set(0);
  • CRYPTO_set_locking_callback(NULL);
  • CRYPTO_set_id_callback(NULL);
  • ENGINE_cleanup();
  • CONF_modules_unload();
  • ERR_free_strings();
  • EVP_cleanup();
  • CRYPTO_cleanup_all_ex_data();

并且,对于每个线程:

  • ERR_remove_state();

如果您的程序是多线程的,您只需要 CRYPTO_set_id_callbackCRYPTO_set_locking_callback。请参阅 Openssl threads(3) man page.

我相信你可以在 OpenSSL 1.0.2 及更高版本中调用 SSL_COMP_free_compression_methods。这解决了下面的一些投诉。但它在 OpenSSL 1.0.1 及以下版本中不可用。


SSL_COMP_get_compression_methods()...

是的,这会导致泄漏。它是众所周知的,并且由于 ssl_comp_methods 被延迟分配但从未被释放。参见 OpenSSL Issue 2561: Memory leak with SSL built-in compressions

一些 OpenSSL 开发人员认为不值得花时间修复它。请参阅 OpenSSL 邮件列表中的 Small memory leak on multithreaded server

以下是开发者对它的立场之一:

A fixed amount of memory that is not deallocated and is independent of the number of operations performed, is NOT a memory leak. Libraries to allocate memory for the lifetime of the process during one time initialization or first use of a function. This is normal.

Tracking this down is a waste of time IMHO.

这是关于该特定泄漏的另一个线程:Preferred way to free ssl_comp_methods?。这是同一个开发者的回应:

Why is anyone obsessed about freeing memory that is assigned to static pointers at most once. There's no "memory leak" associated with such allocations because the amount of extra memory used is fixed.

他没有意识到的是 Java 而 .Net 会在程序的生命周期中多次 load/unload 库,因此少量的 "who cares" 可以无限增长。

当他被告知替代用例时,这是他的回复。我猜,他是在建议 Oracle 和 Java 重新设计他们的语言。或者不要在其中使用 OpenSSL。

Unloading of shared libraries is generally unsafe.

这是维护 Java VM 的人之一的回应:

As the maintainer of an "alternative" JavaVM I can confirm that we absolutely had to support library unloading because one customer was using it heavily - and that was quite a few years ago. Early Sun VMs didn't support library unloading, but then those VMs also did not garbage-collect obsolete classes either.


这里是修复 ssl_comp_methods 漏洞的部分。

在所有情况下,您都需要添加下述补丁。

对于手动执行此操作的程序,只需添加一个名为 free_compressions(或类似名称)的函数,并像上面列出的所有其他方法一样在关机时调用它。函数需要导出。

要在 Linux 下自动执行它需要一点技巧。您必须使用 GCC 扩展:__attribute__ ((destructor)).

/* Add to end of <openssl dir>/ssl/ssl_ciph.c */
#if !defined(OPENSSL_NO_COMP) && defined(__GNU_C__)
void free_compressions(void) __attribute__ ((destructor));
void free_compressions(void)
{
    if (ssl_comp_methods != NULL)
        {
        sk_SSL_COMP_free(ssl_comp_methods);
        ssl_comp_methods = NULL;
        }
}
#endif 

要在 Windows 下自动完成,您必须在 DllMain 下完成。但是你必须小心你在DllMain中做了什么,以及你是怎么做的。所以也许是这样的:

/* Add to end of <openssl dir>/ssl/ssl.h*/
#if !defined(OPENSSL_NO_COMP) && defined(WIN32)
__declspec(dllexport)
    void free_compressions(void);
#endof

/* Add to end of <openssl dir>/ssl/ssl_ciph.c */
#if !defined(OPENSSL_NO_COMP) && defined(WIN32)
void free_compressions(void)
{
    if (ssl_comp_methods != NULL)
        {
        sk_SSL_COMP_free(ssl_comp_methods);
        ssl_comp_methods = NULL;
        }
}
#endif 

-----

Why is this thread being downvoted? The thread I linked is a lot less detailed and it got 10 upvotes (plus one from me). Did you guys become a lot more strict in the last few years?

查看接近原因(which you can't do at the moment),最后投票的原因是:

Questions seeking debugging help ("why isn't this code working?") must include the desired behavior, a specific problem or error and the shortest code necessary to reproduce it in the question itself.

通常适用。但在你的情况下它没有;对于不熟悉该问题的人来说,这并不明显。事实上,您可以编写一个简单的程序来初始化然后初始化库,它可能会泄漏...

根据政策,网站不能制定规则 "Always provide relevant code except for some OpenSSL memory leaks"(这实际上是我们处理您的情况所需要的)。

jww post 的小补充,如果您的 OpenSSL 版本是使用 zlib 库构建的,那么您应该添加 COMP_zlib_cleanup();到关机部分。因为它的DSO模块默认是没有释放的。 所以,完整的关机代码应该是:

FIPS_mode_set(0);
CRYPTO_set_locking_callback(nullptr);
CRYPTO_set_id_callback(nullptr);

ERR_remove_state(0);

SSL_COMP_free_compression_methods();

ENGINE_cleanup();

CONF_modules_free();
CONF_modules_unload(1);

COMP_zlib_cleanup();

ERR_free_strings();
EVP_cleanup();

CRYPTO_cleanup_all_ex_data();