Gem Red-Arrow 安装到 Rails 项目上的 Ruby 时遇到问题
Having problems with Gem Red-Arrow installing to Ruby on Rails project
当我尝试 gem install red-arrow -v 3.0.0 in docker image based on LinuxKit with 'rails', '~> 6.0.3', 我得到以下错误：
# gem install red-arrow -v 3.0.0
Building native extensions. This could take a while...
ERROR: Error installing red-arrow:
ERROR: Failed to build gem native extension.
current directory: /usr/local/bundle/gems/red-arrow-3.0.0/ext/arrow
/usr/local/bin/ruby -I /usr/local/lib/ruby/2.6.0 -r ./siteconf20211103-1947-7io2n7.rb extconf.rb
checking --enable-debug-build option... no
checking C++ compiler... g++
checking g++ version... 8.3 (gnu++14)
checking for --enable-debug-build option... no
checking for -Wall option to compiler... yes
checking for -Waggregate-return option to compiler... yes
checking for -Wcast-align option to compiler... yes
checking for -Wextra option to compiler... yes
checking for -Wformat=2 option to compiler... yes
checking for -Winit-self option to compiler... yes
checking for -Wlarger-than-65500 option to compiler... yes
checking for -Wmissing-declarations option to compiler... yes
checking for -Wmissing-format-attribute option to compiler... yes
checking for -Wmissing-include-dirs option to compiler... yes
checking for -Wmissing-noreturn option to compiler... yes
checking for -Wmissing-prototypes option to compiler... yes
checking for -Wnested-externs option to compiler... no
checking for -Wold-style-definition option to compiler... yes
checking for -Wpacked option to compiler... yes
checking for -Wp,-D_FORTIFY_SOURCE=2 option to compiler... yes
checking for -Wpointer-arith option to compiler... yes
checking for -Wundef option to compiler... yes
checking for -Wout-of-line-declaration option to compiler... no
checking for -Wunsafe-loop-optimizations option to compiler... yes
checking for -Wwrite-strings option to compiler... yes
checking for Homebrew... no
checking for arrow version (>= 3.0.0)... yes
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/local/bin/$(RUBY_BASE_NAME)
--enable-debug-build
--disable-debug-build
--enable-debug-build
--disable-debug-build
--with-pkg-config
--without-pkg-config
--with-override-variables
--without-override-variables
/usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:389:in `parse_pc': .pc doesn't exist: <thrift> (PackageConfig::NotFoundError)
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:233:in `declaration'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:178:in `requires_private'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:509:in `block in all_required_packages'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:259:in `collect_requires'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:265:in `block in collect_requires'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:260:in `each'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:260:in `collect_requires'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:508:in `all_required_packages'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:283:in `collect_cflags'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:191:in `cflags_only_other'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:569:in `cflags_only_other'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:616:in `have_package'
from /usr/local/bundle/gems/glib2-3.4.9/lib/mkmf-gnome.rb:458:in `required_pkg_config_package'
from extconf.rb:28:in `<main>'
To see why this extension failed to compile, please check the mkmf.log which can be found here:
/usr/local/bundle/extensions/x86_64-linux/2.6.0/red-arrow-3.0.0/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /usr/local/bundle/gems/red-arrow-3.0.0 for inspection.
Results logged to /usr/local/bundle/extensions/x86_64-linux/2.6.0/red-arrow-3.0.0/gem_make.out
#
我的 docker 文件：
FROM ruby:2.6.6
ENV RAILS_ENV=production
ENV NODE_VERSION 10.24.1
ENV SPA_ORIGIN our.url.com
ENV SECRET_KEY_BASE 1
RUN mkdir -p /app
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends \
&& apt-get -y autoremove \
&& rm -rf /var/cache/apt
RUN \
curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz" \
&& sha256sum node-v$NODE_VERSION-linux-x64.tar.xz \
&& echo "somekey node-v$NODE_VERSION-linux-x64.tar.xz" | sha256sum -c - \
&& tar -xJf "node-v$NODE_VERSION-linux-x64.tar.xz" -C /usr/local --strip-components=1 \
&& ln -s /usr/local/bin/node /usr/local/bin/nodejs \
&& rm node-v$NODE_VERSION-linux-x64.tar.xz
WORKDIR /app
COPY config/puma.rb config/puma.rb
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN apt-get update -qq \
&& apt install -y -V ca-certificates lsb-release \
&& wget https://packages.red-data-tools.org/$(lsb_release --id --short \
| tr 'A-Z' 'a-z')/red-data-tools-apt-source-latest-$(lsb_release --codename --short).deb \
&& apt install -y -V ./red-data-tools-apt-source-latest-$(lsb_release --codename --short).deb
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends ca-certificates lsb-release \
&& curl -o /tmp/apache-arrow.deb -SL https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb \
&& apt-get install -y --no-install-recommends /tmp/apache-arrow.deb \
&& rm /tmp/apache-arrow.deb
RUN npm install -g yarn
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends build-essential libparquet-glib300\
&& bundle install --without development test \
&& apt-get purge -y build-essential \
&& apt-get -y autoremove \
&& rm -rf /var/cache/apt
RUN yarn install
COPY . .
RUN bin/rake assets:precompile
RUN rails webpacker:compile
EXPOSE 3000
CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]
您需要设置 apache thrift。
https://thrift-tutorial.readthedocs.io/en/latest/installation.html
将此代码段添加到您的 dockerfile。
RUN \
apt-get install -y ant libboost-dev libboost-test-dev libboost-program-options-dev libboost-filesystem-dev libboost-thread-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev && \
wget --no-check-certificate http://dlcdn.apache.org/thrift/0.15.0/thrift-0.15.0.tar.gz && \
tar -xvf thrift-0.15.0.tar.gz
WORKDIR ./thrift-0.15.0
RUN \
./bootstrap.sh && \
./configure && \
make && make install
Apache Arrow 6.0.0 版本（当前最新版本）没有列出 caused this missing pkg-config error.
的 libthrift-dev 依赖项
它已经 fixed 并将很快在 Apache Arrow 6.0.1 中发布
同时，您可以在安装 arrow 之前安装 libthrift-dev（无需从源代码构建）：
apt-get install -y -V libthrift-dev
当我尝试 gem install red-arrow -v 3.0.0 in docker image based on LinuxKit with 'rails', '~> 6.0.3', 我得到以下错误：
# gem install red-arrow -v 3.0.0
Building native extensions. This could take a while...
ERROR: Error installing red-arrow:
ERROR: Failed to build gem native extension.
current directory: /usr/local/bundle/gems/red-arrow-3.0.0/ext/arrow
/usr/local/bin/ruby -I /usr/local/lib/ruby/2.6.0 -r ./siteconf20211103-1947-7io2n7.rb extconf.rb
checking --enable-debug-build option... no
checking C++ compiler... g++
checking g++ version... 8.3 (gnu++14)
checking for --enable-debug-build option... no
checking for -Wall option to compiler... yes
checking for -Waggregate-return option to compiler... yes
checking for -Wcast-align option to compiler... yes
checking for -Wextra option to compiler... yes
checking for -Wformat=2 option to compiler... yes
checking for -Winit-self option to compiler... yes
checking for -Wlarger-than-65500 option to compiler... yes
checking for -Wmissing-declarations option to compiler... yes
checking for -Wmissing-format-attribute option to compiler... yes
checking for -Wmissing-include-dirs option to compiler... yes
checking for -Wmissing-noreturn option to compiler... yes
checking for -Wmissing-prototypes option to compiler... yes
checking for -Wnested-externs option to compiler... no
checking for -Wold-style-definition option to compiler... yes
checking for -Wpacked option to compiler... yes
checking for -Wp,-D_FORTIFY_SOURCE=2 option to compiler... yes
checking for -Wpointer-arith option to compiler... yes
checking for -Wundef option to compiler... yes
checking for -Wout-of-line-declaration option to compiler... no
checking for -Wunsafe-loop-optimizations option to compiler... yes
checking for -Wwrite-strings option to compiler... yes
checking for Homebrew... no
checking for arrow version (>= 3.0.0)... yes
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/local/bin/$(RUBY_BASE_NAME)
--enable-debug-build
--disable-debug-build
--enable-debug-build
--disable-debug-build
--with-pkg-config
--without-pkg-config
--with-override-variables
--without-override-variables
/usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:389:in `parse_pc': .pc doesn't exist: <thrift> (PackageConfig::NotFoundError)
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:233:in `declaration'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:178:in `requires_private'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:509:in `block in all_required_packages'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:259:in `collect_requires'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:265:in `block in collect_requires'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:260:in `each'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:260:in `collect_requires'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:508:in `all_required_packages'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:283:in `collect_cflags'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:191:in `cflags_only_other'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:569:in `cflags_only_other'
from /usr/local/bundle/gems/pkg-config-1.4.6/lib/pkg-config.rb:616:in `have_package'
from /usr/local/bundle/gems/glib2-3.4.9/lib/mkmf-gnome.rb:458:in `required_pkg_config_package'
from extconf.rb:28:in `<main>'
To see why this extension failed to compile, please check the mkmf.log which can be found here:
/usr/local/bundle/extensions/x86_64-linux/2.6.0/red-arrow-3.0.0/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /usr/local/bundle/gems/red-arrow-3.0.0 for inspection.
Results logged to /usr/local/bundle/extensions/x86_64-linux/2.6.0/red-arrow-3.0.0/gem_make.out
#
我的 docker 文件：
FROM ruby:2.6.6
ENV RAILS_ENV=production
ENV NODE_VERSION 10.24.1
ENV SPA_ORIGIN our.url.com
ENV SECRET_KEY_BASE 1
RUN mkdir -p /app
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends \
&& apt-get -y autoremove \
&& rm -rf /var/cache/apt
RUN \
curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz" \
&& sha256sum node-v$NODE_VERSION-linux-x64.tar.xz \
&& echo "somekey node-v$NODE_VERSION-linux-x64.tar.xz" | sha256sum -c - \
&& tar -xJf "node-v$NODE_VERSION-linux-x64.tar.xz" -C /usr/local --strip-components=1 \
&& ln -s /usr/local/bin/node /usr/local/bin/nodejs \
&& rm node-v$NODE_VERSION-linux-x64.tar.xz
WORKDIR /app
COPY config/puma.rb config/puma.rb
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN apt-get update -qq \
&& apt install -y -V ca-certificates lsb-release \
&& wget https://packages.red-data-tools.org/$(lsb_release --id --short \
| tr 'A-Z' 'a-z')/red-data-tools-apt-source-latest-$(lsb_release --codename --short).deb \
&& apt install -y -V ./red-data-tools-apt-source-latest-$(lsb_release --codename --short).deb
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends ca-certificates lsb-release \
&& curl -o /tmp/apache-arrow.deb -SL https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb \
&& apt-get install -y --no-install-recommends /tmp/apache-arrow.deb \
&& rm /tmp/apache-arrow.deb
RUN npm install -g yarn
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends build-essential libparquet-glib300\
&& bundle install --without development test \
&& apt-get purge -y build-essential \
&& apt-get -y autoremove \
&& rm -rf /var/cache/apt
RUN yarn install
COPY . .
RUN bin/rake assets:precompile
RUN rails webpacker:compile
EXPOSE 3000
CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]
您需要设置 apache thrift。
https://thrift-tutorial.readthedocs.io/en/latest/installation.html
将此代码段添加到您的 dockerfile。
RUN \
apt-get install -y ant libboost-dev libboost-test-dev libboost-program-options-dev libboost-filesystem-dev libboost-thread-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev && \
wget --no-check-certificate http://dlcdn.apache.org/thrift/0.15.0/thrift-0.15.0.tar.gz && \
tar -xvf thrift-0.15.0.tar.gz
WORKDIR ./thrift-0.15.0
RUN \
./bootstrap.sh && \
./configure && \
make && make install
Apache Arrow 6.0.0 版本（当前最新版本）没有列出 caused this missing pkg-config error.
的 libthrift-dev 依赖项它已经 fixed 并将很快在 Apache Arrow 6.0.1 中发布
同时，您可以在安装 arrow 之前安装 libthrift-dev（无需从源代码构建）：
apt-get install -y -V libthrift-dev