第一次使用next获取迭代器中的元素,抛出异常
The first time I use next to get the elements in the iterator, an exception is thrown
有两个.py文件
test.py
# coding=utf-8
from itertools import chain
def task():
c = [1, 21, 31, 4, 51, 61, 71]
d = ['a1', 'b1', 'c', 'd', 'e1', 'f', 'g1']
e = chain(c, d)
return id(e)
test1.py
# coding=utf-8
from test import task
import _ctypes
obj_id = task()
tk = _ctypes.PyObj_FromPtr(int(obj_id))
next(tk)
test1 脚本 运行 时出现异常,如下所示:
StopIteration
我需要在一个脚本中return一个对象地址,在另一个脚本中通过对象地址获取对象。
备注:这个想法可行吗?
非常感谢!
您正在尝试的是 Undefined Behavior。这是一个更简单的例子:
code00.py:
#!/usr/bin/env python3
import sys
import _ctypes
def func():
custom_object = [1, 2]
return id(custom_object)
def main():
addr = func()
print("Address: {0:d} (0x{1:016X})".format(addr, addr))
o = _ctypes.PyObj_FromPtr(addr)
print(type(o))
print(o)
print(dir(o))
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main()
print("\nDone.")
输出:
[cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Address: 2221013745352 (0x000002051EBC3EC8)
<class 'list'>
[cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> echo %errorlevel%
-1073741819
[cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> "e:\Work\Dev\VEnvs\py_064_02.07.15_test0\Scripts\python.exe" code00.py
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] 64bit on win32
Address: 57931144 (0x000000000373F588)
<type 'list'>
[[...]]
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']
Done.
可见,程序:
- Python 3: 崩溃
- Python 2:没有崩溃但是 "reconstructed" 对象是垃圾
为什么?
- custom_object是在func
中创建的
- 其地址正在returned
- 但是当函数returns时,对象也计划删除(这可能会或可能不会发生在下一步之前,取决于gc)
- 使用对象地址(悬挂指针)
解决这个问题的最明显方法是将对象(您想要 return 的地址)放在函数之外,这样当它的地址将被使用时它仍然有效。这是您的示例(我将其全部保存在一个文件中,但您可以将其拆分为 2 个)。
code01.py:
#!/usr/bin/env python3
import sys
import _ctypes
import itertools
c = [1, 21, 31, 4, 51, 61, 71]
d = ["a1", "b1", "c", "d", "e1", "f", "g1"]
custom_object = itertools.chain(c, d)
def func():
global custom_object
return id(custom_object)
def main():
addr = func()
print("Address: {0:d} (0x{1:016X})".format(addr, addr))
o = _ctypes.PyObj_FromPtr(addr)
print(type(o))
print(o)
print(dir(o))
try:
while True:
print(next(o))
except StopIteration:
pass
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main()
print("\nDone.")
输出:
[cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code01.py
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Address: 1519278021096 (0x00000161BC06D9E8)
<class 'itertools.chain'>
<itertools.chain object at 0x000002B3795337F0>
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', 'from_iterable']
1
21
31
4
51
61
71
a1
b1
c
d
e1
f
g1
Done.
备注:
- 这解决了您当前的问题。但无论如何,这个问题有点不寻常。你为什么需要它?应该有其他(更清洁的)方法来实现您的目标。这是错误设计或 XY 问题 的信号。检查 [SO]: Is it possible to dereference variable id's?
- 你不依赖 [Python 3.Docs]: ctypes - A foreign function library for Python public API,这意味着
_ctypes.PyObj_FromPtr
可能在未来的 Python 版本中不可用,或者其行为可能会更改,恕不另行通知
- 通常 ctypes 函数仅适用于 ctypes 对象。有点奇怪的是,这个对象可以处理任何类型的对象(尽管它 return 是一个 PyObject(在 C[ 中是 "the father of everything" =116=]实现)指针)
有两个.py文件
test.py
# coding=utf-8
from itertools import chain
def task():
c = [1, 21, 31, 4, 51, 61, 71]
d = ['a1', 'b1', 'c', 'd', 'e1', 'f', 'g1']
e = chain(c, d)
return id(e)
test1.py
# coding=utf-8
from test import task
import _ctypes
obj_id = task()
tk = _ctypes.PyObj_FromPtr(int(obj_id))
next(tk)
test1 脚本 运行 时出现异常,如下所示:
StopIteration
我需要在一个脚本中return一个对象地址,在另一个脚本中通过对象地址获取对象。
备注:这个想法可行吗?
非常感谢!
您正在尝试的是 Undefined Behavior。这是一个更简单的例子:
code00.py:
#!/usr/bin/env python3
import sys
import _ctypes
def func():
custom_object = [1, 2]
return id(custom_object)
def main():
addr = func()
print("Address: {0:d} (0x{1:016X})".format(addr, addr))
o = _ctypes.PyObj_FromPtr(addr)
print(type(o))
print(o)
print(dir(o))
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main()
print("\nDone.")
输出:
[cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 Address: 2221013745352 (0x000002051EBC3EC8) <class 'list'> [cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> echo %errorlevel% -1073741819 [cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> "e:\Work\Dev\VEnvs\py_064_02.07.15_test0\Scripts\python.exe" code00.py Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] 64bit on win32 Address: 57931144 (0x000000000373F588) <type 'list'> [[...]] ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] Done.
可见,程序:
- Python 3: 崩溃
- Python 2:没有崩溃但是 "reconstructed" 对象是垃圾
为什么?
- custom_object是在func 中创建的
- 其地址正在returned
- 但是当函数returns时,对象也计划删除(这可能会或可能不会发生在下一步之前,取决于gc)
- 使用对象地址(悬挂指针)
解决这个问题的最明显方法是将对象(您想要 return 的地址)放在函数之外,这样当它的地址将被使用时它仍然有效。这是您的示例(我将其全部保存在一个文件中,但您可以将其拆分为 2 个)。
code01.py:
#!/usr/bin/env python3
import sys
import _ctypes
import itertools
c = [1, 21, 31, 4, 51, 61, 71]
d = ["a1", "b1", "c", "d", "e1", "f", "g1"]
custom_object = itertools.chain(c, d)
def func():
global custom_object
return id(custom_object)
def main():
addr = func()
print("Address: {0:d} (0x{1:016X})".format(addr, addr))
o = _ctypes.PyObj_FromPtr(addr)
print(type(o))
print(o)
print(dir(o))
try:
while True:
print(next(o))
except StopIteration:
pass
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main()
print("\nDone.")
输出:
[cfati@CFATI-5510-0:e:\Work\Dev\Whosebug\q057805717]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code01.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 Address: 1519278021096 (0x00000161BC06D9E8) <class 'itertools.chain'> <itertools.chain object at 0x000002B3795337F0> ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', 'from_iterable'] 1 21 31 4 51 61 71 a1 b1 c d e1 f g1 Done.
备注:
- 这解决了您当前的问题。但无论如何,这个问题有点不寻常。你为什么需要它?应该有其他(更清洁的)方法来实现您的目标。这是错误设计或 XY 问题 的信号。检查 [SO]: Is it possible to dereference variable id's?
- 你不依赖 [Python 3.Docs]: ctypes - A foreign function library for Python public API,这意味着
_ctypes.PyObj_FromPtr
可能在未来的 Python 版本中不可用,或者其行为可能会更改,恕不另行通知- 通常 ctypes 函数仅适用于 ctypes 对象。有点奇怪的是,这个对象可以处理任何类型的对象(尽管它 return 是一个 PyObject(在 C[ 中是 "the father of everything" =116=]实现)指针)