使用 Pyro4 使用 docker-compose 连接单独容器中的 python 脚本
Using Pyro4 to connect python scripts in separate containers using docker-compose
问题
我想使用 Pyro4 通过 docker-compose 跨多个容器进行远程过程调用。目前，我只是想在不同的机器上实现 Pyro4 warehouse example 的简化版本，而不是默认的本地主机，因为我使用的是多个容器。
我可以在自己的容器中成功启动 Pyro 名称服务器，但在另一个容器中我无法发布仓库 class 并启动 Pyro 的请求循环。我收到错误 OSError: [Errno 99] Cannot assign requested address.
我的尝试和其他信息
我正在使用balena将其部署到Raspberry Pi 3 B+，并且我有一个环境变量（balena cloud中的设备变量）"PYRO_HOST=pyro-ns"来设置地址pyro 名称服务器。
我看到 pyro 名称服务器已创建
05.02.20 15:27:33 (-0500) pyro-ns Broadcast server running on 0.0.0.0:9091
05.02.20 15:27:33 (-0500) pyro-ns NS running on pyro-ns:9090 (172.17.0.3)
05.02.20 15:27:33 (-0500) pyro-ns Warning: HMAC key not set. Anyone can connect to this server!
05.02.20 15:27:33 (-0500) pyro-ns URI = PYRO:Pyro.NameServer@pyro-ns:9090
但是，当我尝试发布仓库 class 并使用
启动 Pyro 的请求循环时出现错误 OSError: [Errno 99] Cannot assign requested address
Pyro4.Daemon.serveSimple(
{
Warehouse: "example.warehouse"
},
ns=True, verbose=True)
我得到以下内容
05.02.20 16:52:00 (-0500) container_B Traceback (most recent call last):
05.02.20 16:52:00 (-0500) container_B File "src/container_B_main.py", line 33, in <module>
05.02.20 16:52:00 (-0500) container_B main()
05.02.20 16:52:00 (-0500) container_B File "src/container_B_main.py", line 30, in main
05.02.20 16:52:00 (-0500) container_B ns=True, verbose=True)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/core.py", line 1204, in serveSimple
05.02.20 16:52:00 (-0500) container_B daemon = Daemon(host, port)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/core.py", line 1141, in __init__
05.02.20 16:52:00 (-0500) container_B self.transportServer.init(self, host, port, unixsocket)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/socketserver/threadpoolserver.py", line 134, in init
05.02.20 16:52:00 (-0500) container_B sslContext=sslContext)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/socketutil.py", line 298, in createSocket
05.02.20 16:52:00 (-0500) container_B bindOnUnusedPort(sock, bind[0])
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/socketutil.py", line 542, in bindOnUnusedPort
05.02.20 16:52:00 (-0500) container_B sock.bind((host, 0))
05.02.20 16:52:00 (-0500) container_B OSError: [Errno 99] Cannot assign requested address
我错过了什么能让 Pyro 使用 docker-compose 跨多个容器工作？
以下是我的代码：
docker-compose.yml
version: '2'
services:
pyro-ns:
privileged: true
restart: always
build: ./pyro-ns
ports:
- "9090:9090"
container_A:
privileged: true
restart: always
build: ./container_A
depends_on:
- pyro-ns
- container_B
container_B:
privileged: true
restart: always
build: ./container_B
depends_on:
- pyro-ns
pyro-ns
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
ENV PYRO_SERIALIZERS_ACCEPTED=serpent,json,marshal,pickle,dill
ENV PYTHONUNBUFFERED=0
CMD ["python", "-m", "Pyro4.naming"]
EXPOSE 9090
container_A
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
# Set our working directory
WORKDIR /usr/src/container_A
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# main.py will run when container starts up on the device
CMD ["python","-u","src/container_A_main.py"]
container_A_main.py
import Pyro4
import Pyro4.util
import sys
sys.excepthook = Pyro4.util.excepthook
try:
print('Top of container A')
warehouse = Pyro4.Proxy("PYRONAME:example.warehouse")
print('The warehouse contains: ', warehouse.list_contents())
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print(message)
container_B
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
# Set our working directory
WORKDIR /usr/src/container_B
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# container_B_main.py will run when container starts up on the device
CMD ["python","-u","src/container_B_main.py"]
container_B_main.py
from __future__ import print_function
import Pyro4
@Pyro4.expose
@Pyro4.behavior(instance_mode="single")
class Warehouse(object):
def __init__(self):
self.contents = ["chair", "bike", "flashlight", "laptop", "couch"]
def list_contents(self):
return self.contents
def take(self, name, item):
self.contents.remove(item)
print("{0} took the {1}.".format(name, item))
def store(self, name, item):
self.contents.append(item)
print("{0} stored the {1}.".format(name, item))
def main():
#Pyro4.config.HOST = "pyro-ns"
Pyro4.Daemon.serveSimple(
{
Warehouse: "example.warehouse"
},
ns=True, verbose=True)
if __name__ == "__main__":
main()
对于container_A和container_B，requirements.txt文件是相同的。
requirements.txt
Pyro4
在 balena 论坛的一些帮助下，我能够成功地将我的 Pyro 示例添加到 运行。
以下是我更新后的工作文件供参考。
---
docker-compose.yml
version: '2'
services:
pyro-ns:
privileged: true
restart: always
build: ./pyro-ns
command: ["--host=pyro-ns"]
container_A:
privileged: true
restart: always
build: ./container_A
depends_on:
- pyro-ns
- container_B
container_B:
privileged: true
restart: always
build: ./container_B
depends_on:
- pyro-ns
pyro-ns
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
ENV PYRO_SERIALIZERS_ACCEPTED=serpent,json,marshal,pickle,dill
ENV PYTHONUNBUFFERED=0
ENTRYPOINT ["pyro4-ns"]
EXPOSE 9090
container_A
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
# Set our working directory
WORKDIR /usr/src/container_A
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# main.py will run when container starts up on the device
CMD ["python","-u","src/container_A_main.py"]
container_A_main.py
import Pyro4
import Pyro4.util
import sys
sys.excepthook = Pyro4.util.excepthook
try:
print('Top of container A')
warehouse = Pyro4.Proxy("PYRONAME:example.warehouse")
print('The warehouse contains: ', warehouse.list_contents())
print('Inifite loop after running the Warehouse class via Pyro4')
while True:
# Infinite loop.
pass
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print(message)
container_B
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
# Set our working directory
WORKDIR /usr/src/container_B
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# main.py will run when container starts up on the device
CMD ["python","-u","src/container_B_main.py"]
EXPOSE 9100
container_B_main.py
from __future__ import print_function
import Pyro4
@Pyro4.expose
@Pyro4.behavior(instance_mode="single")
class Warehouse(object):
def __init__(self):
self.contents = ["chair", "bike", "flashlight", "laptop", "couch"]
def list_contents(self):
return self.contents
def take(self, name, item):
self.contents.remove(item)
print("{0} took the {1}.".format(name, item))
def store(self, name, item):
self.contents.append(item)
print("{0} stored the {1}.".format(name, item))
def main():
Pyro4.config.SERIALIZER = 'pickle'
daemon = Pyro4.Daemon(host="container_B", port=9100)
uri = daemon.register(Warehouse)
print('The uri of the registered Warehouse class')
print(uri)
try:
ns = Pyro4.locateNS(host="pyro-ns", port=9090)
print('The located name server')
print(ns)
ns.register("example.warehouse", uri)
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print(message)
daemon.requestLoop()
if __name__ == "__main__":
main()
对于container_A和container_B，requirements.txt文件是相同的。
requirements.txt
`Pyro4`
问题
我想使用 Pyro4 通过 docker-compose 跨多个容器进行远程过程调用。目前，我只是想在不同的机器上实现 Pyro4 warehouse example 的简化版本，而不是默认的本地主机，因为我使用的是多个容器。
我可以在自己的容器中成功启动 Pyro 名称服务器，但在另一个容器中我无法发布仓库 class 并启动 Pyro 的请求循环。我收到错误 OSError: [Errno 99] Cannot assign requested address.
我的尝试和其他信息
我正在使用balena将其部署到Raspberry Pi 3 B+，并且我有一个环境变量（balena cloud中的设备变量）"PYRO_HOST=pyro-ns"来设置地址pyro 名称服务器。
我看到 pyro 名称服务器已创建
05.02.20 15:27:33 (-0500) pyro-ns Broadcast server running on 0.0.0.0:9091
05.02.20 15:27:33 (-0500) pyro-ns NS running on pyro-ns:9090 (172.17.0.3)
05.02.20 15:27:33 (-0500) pyro-ns Warning: HMAC key not set. Anyone can connect to this server!
05.02.20 15:27:33 (-0500) pyro-ns URI = PYRO:Pyro.NameServer@pyro-ns:9090
但是，当我尝试发布仓库 class 并使用
启动 Pyro 的请求循环时出现错误OSError: [Errno 99] Cannot assign requested address
Pyro4.Daemon.serveSimple(
{
Warehouse: "example.warehouse"
},
ns=True, verbose=True)
我得到以下内容
05.02.20 16:52:00 (-0500) container_B Traceback (most recent call last):
05.02.20 16:52:00 (-0500) container_B File "src/container_B_main.py", line 33, in <module>
05.02.20 16:52:00 (-0500) container_B main()
05.02.20 16:52:00 (-0500) container_B File "src/container_B_main.py", line 30, in main
05.02.20 16:52:00 (-0500) container_B ns=True, verbose=True)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/core.py", line 1204, in serveSimple
05.02.20 16:52:00 (-0500) container_B daemon = Daemon(host, port)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/core.py", line 1141, in __init__
05.02.20 16:52:00 (-0500) container_B self.transportServer.init(self, host, port, unixsocket)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/socketserver/threadpoolserver.py", line 134, in init
05.02.20 16:52:00 (-0500) container_B sslContext=sslContext)
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/socketutil.py", line 298, in createSocket
05.02.20 16:52:00 (-0500) container_B bindOnUnusedPort(sock, bind[0])
05.02.20 16:52:00 (-0500) container_B File "/usr/local/lib/python3.5/site-packages/Pyro4/socketutil.py", line 542, in bindOnUnusedPort
05.02.20 16:52:00 (-0500) container_B sock.bind((host, 0))
05.02.20 16:52:00 (-0500) container_B OSError: [Errno 99] Cannot assign requested address
我错过了什么能让 Pyro 使用 docker-compose 跨多个容器工作？
以下是我的代码：
docker-compose.yml
version: '2'
services:
pyro-ns:
privileged: true
restart: always
build: ./pyro-ns
ports:
- "9090:9090"
container_A:
privileged: true
restart: always
build: ./container_A
depends_on:
- pyro-ns
- container_B
container_B:
privileged: true
restart: always
build: ./container_B
depends_on:
- pyro-ns
pyro-ns
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
ENV PYRO_SERIALIZERS_ACCEPTED=serpent,json,marshal,pickle,dill
ENV PYTHONUNBUFFERED=0
CMD ["python", "-m", "Pyro4.naming"]
EXPOSE 9090
container_A
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
# Set our working directory
WORKDIR /usr/src/container_A
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# main.py will run when container starts up on the device
CMD ["python","-u","src/container_A_main.py"]
container_A_main.py
import Pyro4
import Pyro4.util
import sys
sys.excepthook = Pyro4.util.excepthook
try:
print('Top of container A')
warehouse = Pyro4.Proxy("PYRONAME:example.warehouse")
print('The warehouse contains: ', warehouse.list_contents())
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print(message)
container_B
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
# Set our working directory
WORKDIR /usr/src/container_B
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# container_B_main.py will run when container starts up on the device
CMD ["python","-u","src/container_B_main.py"]
container_B_main.py
from __future__ import print_function
import Pyro4
@Pyro4.expose
@Pyro4.behavior(instance_mode="single")
class Warehouse(object):
def __init__(self):
self.contents = ["chair", "bike", "flashlight", "laptop", "couch"]
def list_contents(self):
return self.contents
def take(self, name, item):
self.contents.remove(item)
print("{0} took the {1}.".format(name, item))
def store(self, name, item):
self.contents.append(item)
print("{0} stored the {1}.".format(name, item))
def main():
#Pyro4.config.HOST = "pyro-ns"
Pyro4.Daemon.serveSimple(
{
Warehouse: "example.warehouse"
},
ns=True, verbose=True)
if __name__ == "__main__":
main()
对于container_A和container_B，requirements.txt文件是相同的。
requirements.txt
Pyro4
在 balena 论坛的一些帮助下，我能够成功地将我的 Pyro 示例添加到 运行。
以下是我更新后的工作文件供参考。
---
docker-compose.yml
version: '2'
services:
pyro-ns:
privileged: true
restart: always
build: ./pyro-ns
command: ["--host=pyro-ns"]
container_A:
privileged: true
restart: always
build: ./container_A
depends_on:
- pyro-ns
- container_B
container_B:
privileged: true
restart: always
build: ./container_B
depends_on:
- pyro-ns
pyro-ns
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
ENV PYRO_SERIALIZERS_ACCEPTED=serpent,json,marshal,pickle,dill
ENV PYTHONUNBUFFERED=0
ENTRYPOINT ["pyro4-ns"]
EXPOSE 9090
container_A
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
RUN pip install Pyro4 dill
# Set our working directory
WORKDIR /usr/src/container_A
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# main.py will run when container starts up on the device
CMD ["python","-u","src/container_A_main.py"]
container_A_main.py
import Pyro4
import Pyro4.util
import sys
sys.excepthook = Pyro4.util.excepthook
try:
print('Top of container A')
warehouse = Pyro4.Proxy("PYRONAME:example.warehouse")
print('The warehouse contains: ', warehouse.list_contents())
print('Inifite loop after running the Warehouse class via Pyro4')
while True:
# Infinite loop.
pass
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print(message)
container_B
Dockerfile.template
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run
# enable container init system.
ENV INITSYSTEM on
# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git
RUN pip install --upgrade pip
# Set our working directory
WORKDIR /usr/src/container_B
# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt
# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt
# This will copy all files in our root to the working directory in the container
COPY . ./
# main.py will run when container starts up on the device
CMD ["python","-u","src/container_B_main.py"]
EXPOSE 9100
container_B_main.py
from __future__ import print_function
import Pyro4
@Pyro4.expose
@Pyro4.behavior(instance_mode="single")
class Warehouse(object):
def __init__(self):
self.contents = ["chair", "bike", "flashlight", "laptop", "couch"]
def list_contents(self):
return self.contents
def take(self, name, item):
self.contents.remove(item)
print("{0} took the {1}.".format(name, item))
def store(self, name, item):
self.contents.append(item)
print("{0} stored the {1}.".format(name, item))
def main():
Pyro4.config.SERIALIZER = 'pickle'
daemon = Pyro4.Daemon(host="container_B", port=9100)
uri = daemon.register(Warehouse)
print('The uri of the registered Warehouse class')
print(uri)
try:
ns = Pyro4.locateNS(host="pyro-ns", port=9090)
print('The located name server')
print(ns)
ns.register("example.warehouse", uri)
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print(message)
daemon.requestLoop()
if __name__ == "__main__":
main()
对于container_A和container_B，requirements.txt文件是相同的。
requirements.txt
`Pyro4`