Caffe：libcaffe.a 在尝试整个存档时并未真正链接

Question

问题总结

我正在尝试 link Caffe 静态库 (libcaffe.a) 到我的项目，但我想 link 使用命令 -Wl,--whole-archive。问题是在 ldd -d mylib.so 之后它显示了很多来自 Caffe 的 undefined symbol，好像 libcaffe.a 并不是真的 linked 到 mylib.so。

重现步骤

创建一个 docker 容器，最小 Ubuntu 18.04
apt-get 更新
apt-get install libhdf5-serial-dev protobuf-compiler libprotobuf-dev libgflags-dev libgoogle-glog-dev libopenblas-dev libboost-dev libboost-all-dev
git 克隆 — 分支 1.0 https://github.com/BVLC/caffe.git
cd 咖啡
mkdir 构建
光盘构建
cmake -DBLAS=打开..
全部
复制libcaffe.a到我项目的lib目录
转到我的项目构建目录
cmake ..
制作
ldd -d mylib.so

尝试过的解决方案

我尝试使用另一个版本的 gcc/g++，但我发现了另一个我报告过的问题 here

系统配置

操作系统：Ubuntu18.04
编译器：gcc/g++ 7
CUDA 版本（如果适用）：关闭
CUDNN 版本（如果适用）：关闭
BLAS：OpensBLAS 0.2.20
Python 版本（如果使用 pycaffe）：OFF
MATLAB 版本（如果使用 matcaffe）：关闭

我的CMakeLists.txt生成`mylib.so`

EDIT 01 : 另外 whole-archive libproto.a 我之前没有尝试过，因为以前从来没有必要包含这个库。也许与我当前的 ubuntu 配置、gcc/g++ 版本和库版本

有关

编辑 02： 更改了库 linking 顺序，linking boost_thread 在 libcaffe.a 和 libproto.a[=27= 之后]

cmake_minimum_required(VERSION 3.0)
project(mylib)

SET(CMAKE_BUILD_TYPE Release)
SET(CMAKE_VERBOSE_MAKEFILE OFF)

add_definitions(-DCPU_ONLY=1 -std=c++11 -fPIC -m64)

SET (CAFFE_INCLUDE  /mypath/caffe/include/)
SET (PROTO_INCLUDE  /mypath/caffe/build/include/)

SET (DEP_LIBS ${CMAKE_SOURCE_DIR}/../libs)
SET (CMAKE_SHARED_LINKER_FLAGS "-Wl,-rpath,$ORIGIN/../libs")

include_directories(${CAFFE_INCLUDE} ${PROTO_INCLUDE})

add_library(
    mylib SHARED
    ${CMAKE_SOURCE_DIR}/src/mycode.cpp)

target_link_libraries(
    mylib
    -Wl,--whole-archive ${DEP_LIBS}/libcaffe.a -Wl,--no-whole-archive
    -Wl,--whole-archive ${DEP_LIBS}/libproto.a -Wl,--no-whole-archive
    ${DEP_LIBS}/libboost_thread.so.1.65.1
    ${DEP_LIBS}/libprotobuf.so.10
    ${DEP_LIBS}/libglog.so.0
    )

第 8 步后的终端输出

The C compiler identification is GNU 7.4.0
The CXX compiler identification is GNU 7.4.0
Check for working C compiler: /usr/bin/cc
Check for working C compiler: /usr/bin/cc -- works
Detecting C compiler ABI info
Detecting C compiler ABI info - done
Detecting C compile features
Detecting C compile features - done
Check for working CXX compiler: /usr/bin/c++
Check for working CXX compiler: /usr/bin/c++ -- works
Detecting CXX compiler ABI info
Detecting CXX compiler ABI info - done
Detecting CXX compile features
Detecting CXX compile features - done
Looking for pthread.h
Looking for pthread.h - found
Looking for pthread_create
Looking for pthread_create - not found
Looking for pthread_create in pthreads
Looking for pthread_create in pthreads - not found
Looking for pthread_create in pthread
Looking for pthread_create in pthread - found
Found Threads: TRUE  
Boost version: 1.65.1
Found the following Boost libraries:
  system
  thread
  filesystem
  chrono
  date_time
  atomic
Found GFlags: /usr/include  
Found gflags  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
Found Glog: /usr/include  
Found glog    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread (found version "3.0.0") 
Found PROTOBUF Compiler: /usr/bin/protoc
HDF5: Using hdf5 compiler wrapper to determine C configuration
HDF5: Using hdf5 compiler wrapper to determine CXX configuration
Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_cpp.so;/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libsz.so;/usr/lib/x86_64-linux-gnu/libz.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so (found version "1.10.0.1") found components:  HL 
CUDA is disabled. Building without it...
Found OpenBLAS libraries: /usr/lib/x86_64-linux-gnu/libopenblas.so
Found OpenBLAS include: /usr/include/x86_64-linux-gnu
Python interface is disabled or not all required dependencies found. Building without it...
Found Git: /usr/bin/git (found version "2.17.1") 

******************* Caffe Configuration Summary *******************
General:
  Version           :   1.0.0
  Git               :   1.0-dirty
  System            :   Linux
  C++ compiler      :   /usr/bin/c++
  Release CXX flags :   -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
  Debug CXX flags   :   -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
  Build type        :   Release

  BUILD_SHARED_LIBS :   OFF
  BUILD_python      :   OFF
  BUILD_matlab      :   OFF
  BUILD_docs        :   OFF
  CPU_ONLY          :   ON
  USE_OPENCV        :   OFF
  USE_LEVELDB       :   OFF
  USE_LMDB          :   OFF
  USE_NCCL          :   OFF
  ALLOW_LMDB_NOLOCK :   OFF

Dependencies:
  BLAS              :   Yes (open)
  Boost             :   Yes (ver. 1.65)
  glog              :   Yes
  gflags            :   Yes
  protobuf          :   Yes (ver. 3.0.0)
  CUDA              :   No

Install:
  Install path      :   /mypath/caffe/build/install

Configuring done
Generating done
Build files have been written to: /mypath/caffe/build

第 14 步后的终端输出

linux-vdso.so.1 (0x00007fff869d5000)
libstdc++.so.6 => /home/shared-folder/dev/cppsaffe/../libs/libstdc++.so.6 (0x00007f7d10862000)
libm.so.6 => /home/shared-folder/dev/cppsaffe/../libs/libm.so.6 (0x00007f7d104c4000)
libgcc_s.so.1 => /home/shared-folder/dev/cppsaffe/../libs/libgcc_s.so.1 (0x00007f7d102ac000)
libc.so.6 => /home/shared-folder/dev/cppsaffe/../libs/libc.so.6 (0x00007f7d0febb000)
libboost_thread.so.1.65.1 => /home/shared-folder/dev/cppsaffe/../libs/libboost_thread.so.1.65.1 (0x00007f7d0fc8d000)
libboost_system.so.1.65.1 => /home/shared-folder/dev/cppsaffe/../libs/libboost_system.so.1.65.1 (0x00007f7d0fa88000)
libprotobuf.so.10 => /home/shared-folder/dev/cppsaffe/../libs/libprotobuf.so.10 (0x00007f7d0f5c5000)
libglog.so.0 => /home/shared-folder/dev/cppsaffe/../libs/libglog.so.0 (0x00007f7d0f390000)
libgflags.so.2.2 => /home/shared-folder/dev/cppsaffe/../libs/libgflags.so.2.2 (0x00007f7d0f16b000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7d1110f000)
librt.so.1 => /home/shared-folder/dev/cppsaffe/../libs/../libs/librt.so.1 (0x00007f7d0ef63000)
libpthread.so.0 => /home/shared-folder/dev/cppsaffe/../libs/../libs/libpthread.so.0 (0x00007f7d0ed44000)
libz.so.1 => /home/shared-folder/dev/cppsaffe/../libs/../libs/libz.so.1 (0x00007f7d0eb27000)
libunwind.so.8 => /home/shared-folder/dev/cppsaffe/../libs/../libs/libunwind.so.8 (0x00007f7d0e90b000)
liblzma.so.5 => /home/shared-folder/dev/cppsaffe/../libs/../libs/../libs/liblzma.so.5 (0x00007f7d0e6e4000)
libdl.so.2 => /home/shared-folder/dev/cppsaffe/../libs/../libs/../libs/../libs/libdl.so.2 (0x00007f7d0e4e0000)
undefined symbol: _ZN5caffe14ScaleParameter17default_instance_E (./libcppsaffe.so)
undefined symbol: _ZTIN5caffe16V1LayerParameterE    (./libcppsaffe.so)
undefined symbol: _ZTIN5caffe9BlobProtoE    (./libcppsaffe.so)
undefined symbol: _ZTIN5caffe9BlobShapeE    (./libcppsaffe.so)
undefined symbol: _ZN5caffe18ParameterParameter17default_instance_E (./libcppsaffe.so)
undefined symbol: _ZN5caffe16ReshapeParameter17default_instance_E   (./libcppsaffe.so)
undefined symbol: _ZN5caffe19WindowDataParameter19_default_crop_mode_B5cxx11E   (./libcppsaffe.so)
undefined symbol: _ZN5caffe18RecurrentParameter17default_instance_E (./libcppsaffe.so)
undefined symbol: _ZN5caffe20ConvolutionParameter17default_instance_E   (./libcppsaffe.so)
undefined symbol: _ZN5caffe14PReLUParameter17default_instance_E (./libcppsaffe.so)
undefined symbol: _ZN5caffe16V1LayerParameter17default_instance_E   (./libcppsaffe.so)
undefined symbol: _ZN5caffe13BiasParameter17default_instance_E  (./libcppsaffe.so)
undefined symbol: _ZTIN5caffe12NetStateRuleE    (./libcppsaffe.so)
undefined symbol: _ZN5caffe15SolverParameter14_default_type_B5cxx11E    (./libcppsaffe.so)
undefined symbol: _ZN5caffe14EmbedParameter17default_instance_E (./libcppsaffe.so)
undefined symbol: _ZN5caffe21InnerProductParameter17default_instance_E  (./libcppsaffe.so)
undefined symbol: _ZN5caffe15FillerParameter14_default_type_B5cxx11E    (./libcppsaffe.so)
undefined symbol: _ZN5caffe14LayerParameter17default_instance_E (./libcppsaffe.so)
undefined symbol: _ZTIN5caffe9ParamSpecE    (./libcppsaffe.so)
undefined symbol: _ZN5caffe12NetParameter17default_instance_E   (./libcppsaffe.so)
undefined symbol: _ZN5caffe16V0LayerParameter17default_instance_E   (./libcppsaffe.so)
undefined symbol: _ZN5caffe9BlobProto17default_instance_E   (./libcppsaffe.so)
undefined symbol: _ZN5caffe15SolverParameter17default_instance_E    (./libcppsaffe.so)
undefined symbol: _ZTIN5caffe14LayerParameterE  (./libcppsaffe.so)

Answer 1

我已经用解决方案编辑了我的问题 EDIT 01 和 EDIT 02。

Caffe：libcaffe.a 在尝试整个存档时并未真正链接

Caffe: libcaffe.a is not really linked when trying whole-archive

linker

gcc

g++

caffe

问题总结

重现步骤

尝试过的解决方案

系统配置

我的CMakeLists.txt生成`mylib.so`

第 8 步后的终端输出

第 14 步后的终端输出

Caffe：libcaffe.a 在尝试整个存档时并未真正链接

Caffe: libcaffe.a is not really linked when trying whole-archive

linker

gcc

g++

caffe

问题总结

重现步骤

尝试过的解决方案

系统配置

我的CMakeLists.txt生成mylib.so

第 8 步后的终端输出

第 14 步后的终端输出

我的CMakeLists.txt生成`mylib.so`