Nifi 图像不是 运行 over openshift
Nifi image Not Running over openshift
在尝试 运行 docker apache nifi 图像时 docker hub 在 open shift 中,它给了我docker 图像的权限问题是 运行 用户 nifi 不允许通过 openshft。所以我使用下面的 docker 文件构建 docker 图像,但现在我什至无法 运行 在我的本地 docker 容器中构建图像。
FROM openjdk:8-jre
ARG NIFI_VERSION=1.12.1
ARG BASE_URL=https://archive.apache.org/dist
ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
ARG NIFI_BINARY_PATH=${NIFI_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip}
ARG NIFI_TOOLKIT_BINARY_PATH=${NIFI_TOOLKIT_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-toolkit-${NIFI_VERSION}-bin.zip}
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_HOME ${NIFI_BASE_DIR}/nifi-current
ENV NIFI_TOOLKIT_HOME ${NIFI_BASE_DIR}/nifi-toolkit-current
ENV NIFI_PID_DIR=${NIFI_HOME}/run
ENV NIFI_LOG_DIR=${NIFI_HOME}/logs
USER root
ADD sh/ ${NIFI_BASE_DIR}/scripts/
# Setup NiFi user and create necessary directories
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_TOOLKIT_HOME}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} \
&& chgrp -R 0 ${NIFI_BASE_DIR} \
&& chmod -R g+rwX ${NIFI_BASE_DIR} \
&& chmod -R g=u ${NIFI_BASE_DIR}/ \
&& chmod -R g=u /etc/passwd
#ADD bootstrap.conf ${NIFI_HOME}/conf/bootstrap.conf
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > ${NIFI_HOME}/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000
WORKDIR ${NIFI_HOME}
USER 1001
# Apply configuration and start NiFi
#
# We need to use the exec form to avoid running our command in a subshell and omitting signals,
# thus being unable to shut down gracefully:
# https://docs.docker.com/engine/reference/builder/#entrypoint
#
# Also we need to use relative path, because the exec form does not invoke a command shell,
# thus normal shell processing does not happen:
# https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
ENTRYPOINT ["../scripts/start.sh"]
在 docker 容器中 运行 时出现此错误。
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
/opt/nifi/scripts/toolkit.sh: 18: /opt/nifi/scripts/toolkit.sh: cannot create //.nifi-cli.nifi.properties: Permission denied
此构建适用于 open shift,因为 apache nifi 用户未在 openshift 中工作,并且在启动本地 docker
时出现权限问题
我在将自定义 nifi 容器部署到 Openshift 时遇到了类似的问题。将此添加到 spec:
下的 deployment.yaml 有帮助:
securityContext:
runAsUser: 1000
fsGroup: 1000
所以我在 Openshift 上尝试 运行 NIFI 时遇到了同样的问题,希望对您有所帮助。
我使用的步骤是:
如@JuanD 所示,我在 openshift 上添加了配置:
securityContext:
runAsUser: 1000
此外我还做了:
RUN chmod -R g+rw ${NIFI_BASE_DIR} \
&& chmod -R g+rwX ${NIFI_BASE_DIR}/scripts \
&& useradd --shell /bin/bash -u ${UID} -g ${GID} -m nifi
我所做的另一个重新安排是将要执行的复制文件移动到此命令之前。
为了避免任何不必要的问题,我还添加了 uid-entrypoint.sh
#!/bin/bash
if ! whoami &> /dev/null; then
if [ -w /etc/passwd ]; then
echo "${USER_NAME:-nifi}:x:$(id -u):0:${USER_NAME:-nifi} user:${HOME}:/sbin/nologin" >> /etc/passwd
fi
fi
exec "$@"
整个dockerfile:
ARG IMAGE_NAME=openjdk
ARG IMAGE_TAG=8-jre
FROM ${IMAGE_NAME}:${IMAGE_TAG}
ARG MAINTAINER="Apache NiFi <dev@nifi.apache.org>"
LABEL maintainer="${MAINTAINER}"
LABEL site="https://nifi.apache.org"
ARG UID=1000
ARG GID=0
ARG NIFI_VERSION=1.14.0
ARG BASE_URL=https://archive.apache.org/dist
ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
ARG NIFI_BINARY_PATH=${NIFI_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip}
ARG NIFI_TOOLKIT_BINARY_PATH=${NIFI_TOOLKIT_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-toolkit-${NIFI_VERSION}-bin.zip}
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_HOME ${NIFI_BASE_DIR}/nifi-current
ENV NIFI_TOOLKIT_HOME ${NIFI_BASE_DIR}/nifi-toolkit-current
ENV NIFI_PID_DIR=${NIFI_HOME}/run
ENV NIFI_LOG_DIR=${NIFI_HOME}/logs
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps \
&& curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}
COPY scripts/ ${NIFI_BASE_DIR}/scripts/
RUN chmod -R g+rw ${NIFI_BASE_DIR} \
&& chmod -R g+rwX ${NIFI_BASE_DIR}/scripts \
&& useradd --shell /bin/bash -u ${UID} -g ${GID} -m nifi
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > $NIFI_HOME/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000 8000
WORKDIR ${NIFI_HOME}
USER ${UID}
ENTRYPOINT [ "../scripts/uid-entrypoint.sh" ]
CMD [ "../scripts/start.sh" ]
希望对您有所帮助。
在尝试 运行 docker apache nifi 图像时 docker hub 在 open shift 中,它给了我docker 图像的权限问题是 运行 用户 nifi 不允许通过 openshft。所以我使用下面的 docker 文件构建 docker 图像,但现在我什至无法 运行 在我的本地 docker 容器中构建图像。
FROM openjdk:8-jre
ARG NIFI_VERSION=1.12.1
ARG BASE_URL=https://archive.apache.org/dist
ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
ARG NIFI_BINARY_PATH=${NIFI_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip}
ARG NIFI_TOOLKIT_BINARY_PATH=${NIFI_TOOLKIT_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-toolkit-${NIFI_VERSION}-bin.zip}
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_HOME ${NIFI_BASE_DIR}/nifi-current
ENV NIFI_TOOLKIT_HOME ${NIFI_BASE_DIR}/nifi-toolkit-current
ENV NIFI_PID_DIR=${NIFI_HOME}/run
ENV NIFI_LOG_DIR=${NIFI_HOME}/logs
USER root
ADD sh/ ${NIFI_BASE_DIR}/scripts/
# Setup NiFi user and create necessary directories
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_TOOLKIT_HOME}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} \
&& chgrp -R 0 ${NIFI_BASE_DIR} \
&& chmod -R g+rwX ${NIFI_BASE_DIR} \
&& chmod -R g=u ${NIFI_BASE_DIR}/ \
&& chmod -R g=u /etc/passwd
#ADD bootstrap.conf ${NIFI_HOME}/conf/bootstrap.conf
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > ${NIFI_HOME}/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000
WORKDIR ${NIFI_HOME}
USER 1001
# Apply configuration and start NiFi
#
# We need to use the exec form to avoid running our command in a subshell and omitting signals,
# thus being unable to shut down gracefully:
# https://docs.docker.com/engine/reference/builder/#entrypoint
#
# Also we need to use relative path, because the exec form does not invoke a command shell,
# thus normal shell processing does not happen:
# https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
ENTRYPOINT ["../scripts/start.sh"]
在 docker 容器中 运行 时出现此错误。
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
/opt/nifi/scripts/toolkit.sh: 18: /opt/nifi/scripts/toolkit.sh: cannot create //.nifi-cli.nifi.properties: Permission denied
此构建适用于 open shift,因为 apache nifi 用户未在 openshift 中工作,并且在启动本地 docker
时出现权限问题我在将自定义 nifi 容器部署到 Openshift 时遇到了类似的问题。将此添加到 spec:
下的 deployment.yaml 有帮助:
securityContext:
runAsUser: 1000
fsGroup: 1000
所以我在 Openshift 上尝试 运行 NIFI 时遇到了同样的问题,希望对您有所帮助。 我使用的步骤是:
如@JuanD 所示,我在 openshift 上添加了配置:
securityContext:
runAsUser: 1000
此外我还做了:
RUN chmod -R g+rw ${NIFI_BASE_DIR} \
&& chmod -R g+rwX ${NIFI_BASE_DIR}/scripts \
&& useradd --shell /bin/bash -u ${UID} -g ${GID} -m nifi
我所做的另一个重新安排是将要执行的复制文件移动到此命令之前。
为了避免任何不必要的问题,我还添加了 uid-entrypoint.sh
#!/bin/bash
if ! whoami &> /dev/null; then
if [ -w /etc/passwd ]; then
echo "${USER_NAME:-nifi}:x:$(id -u):0:${USER_NAME:-nifi} user:${HOME}:/sbin/nologin" >> /etc/passwd
fi
fi
exec "$@"
整个dockerfile:
ARG IMAGE_NAME=openjdk
ARG IMAGE_TAG=8-jre
FROM ${IMAGE_NAME}:${IMAGE_TAG}
ARG MAINTAINER="Apache NiFi <dev@nifi.apache.org>"
LABEL maintainer="${MAINTAINER}"
LABEL site="https://nifi.apache.org"
ARG UID=1000
ARG GID=0
ARG NIFI_VERSION=1.14.0
ARG BASE_URL=https://archive.apache.org/dist
ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
ARG NIFI_BINARY_PATH=${NIFI_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip}
ARG NIFI_TOOLKIT_BINARY_PATH=${NIFI_TOOLKIT_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-toolkit-${NIFI_VERSION}-bin.zip}
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_HOME ${NIFI_BASE_DIR}/nifi-current
ENV NIFI_TOOLKIT_HOME ${NIFI_BASE_DIR}/nifi-toolkit-current
ENV NIFI_PID_DIR=${NIFI_HOME}/run
ENV NIFI_LOG_DIR=${NIFI_HOME}/logs
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps \
&& curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}
COPY scripts/ ${NIFI_BASE_DIR}/scripts/
RUN chmod -R g+rw ${NIFI_BASE_DIR} \
&& chmod -R g+rwX ${NIFI_BASE_DIR}/scripts \
&& useradd --shell /bin/bash -u ${UID} -g ${GID} -m nifi
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > $NIFI_HOME/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000 8000
WORKDIR ${NIFI_HOME}
USER ${UID}
ENTRYPOINT [ "../scripts/uid-entrypoint.sh" ]
CMD [ "../scripts/start.sh" ]
希望对您有所帮助。