有没有办法声明一个函数应该使用调用者的范围?
Is there a way to declare that a function should use the scope of the caller?
是否有类似于 C 宏的功能,可以让您以内联方式重用代码,而无需为该段代码创建单独的作用域?
例如:
a=3
def foo():
a=4
foo()
print a
将打印 3,但我希望它打印 4。
我知道涉及 类 或全局字典等对象的解决方案,但是我正在寻找更原始的解决方案(例如函数装饰器),它只会让我在内部进行更改调用者的范围。
非常感谢
编辑:任何需要声明我将使用哪些变量或事先声明 "namespace" 之类的可变对象的解决方案都不是我正在寻找的解决方案。
我自己尝试过:
def pgame():
a=3
c=5
print locals()
game(a)
print locals()
class inline_func(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
#to be @inline_func
def game(b, a=4):
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1] [0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\ninspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))")
try:
print "your code here"
finally:
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))")
@inline_func
def strip_game(b, a=4):
print "your code here"
但是我 运行 遇到了一个严重的问题,即如何在不破坏程序的可调试性的情况下将代码注入 strip_game
,因为我只想创建一个新的代码对象或使用 exec , 都遇到了一些严重的问题。
主要编辑:
好的,所以我有一些接近可行的解决方案,但是我遇到了一个非常奇怪的问题:
import inspect
import ctypes
import struct
import dis
import types
def cgame():
a=3
c=5
print locals()
strip_game(a)
print locals()
def pgame():
a=3
c=5
print locals()
game(a)
print locals()
class empty_deco(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
debug_func = None
class inline_func(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
init_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\n" + \
"inspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))"
fini_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))"
co_stacksize = max(6, self.f.func_code.co_stacksize) # make sure we have enough space on the stack for everything
co_consts = self.f.func_code.co_consts +(init_exec_string, fini_exec_string)
init = "d" + struct.pack("H", len(strip_game.f.func_code.co_consts)) #LOAD_CONST init_exec_string
init += "d\x00\x00\x04U" # LOAD_CONST None, DUP_TOP, EXEC_STMT
init += "z" + struct.pack("H", len(self.f.func_code.co_code) + 4) #SETUP_FINALLY
fini = "Wd\x00\x00" # POP_BLOCK, LOAD_CONST None
fini += "d" + struct.pack("H", len(strip_game.f.func_code.co_consts) + 1) #LOAD_CONST fini_exec_string
fini += "d\x00\x00\x04UXd\x00\x00S" # LOAD_CONST None, DUP_TOP, EXEC_STMT, END_FINALLY, LOAD_CONST None, RETURN
co_code = init + self.f.func_code.co_code + fini
co_lnotab = "\x00\x00\x0b" + self.f.func_code.co_lnotab[1:] # every error in init will be attributed to @inline_func, errors in the function will be treated as expected, errors in fini will be attributed to the last line probably.
new_code = types.CodeType(
self.f.func_code.co_argcount,
self.f.func_code.co_nlocals,
co_stacksize,
self.f.func_code.co_flags & ~(1), # optimized functions are problematic for us
co_code,
co_consts,
self.f.func_code.co_names,
self.f.func_code.co_varnames,
self.f.func_code.co_filename,
self.f.func_code.co_name,
self.f.func_code.co_firstlineno,
co_lnotab,
self.f.func_code.co_freevars,
self.f.func_code.co_cellvars,)
self.inline_f = types.FunctionType(new_code, self.f.func_globals, self.f.func_name, self.f.func_defaults, self.f.func_closure)
#dis.dis(self.inline_f)
global debug_func
debug_func = self.inline_f
return self.inline_f(*args, **kwargs)
@empty_deco
def game(b, a=4):
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\ninspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))")
try:
print "inner locals:"
print locals()
print c
return None
finally:
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))")
@inline_func
def strip_game(b, a=4):
print "inner locals:"
print locals()
print c
return None
def stupid():
exec("print 'hello'")
try:
a=1
b=2
c=3
d=4
finally:
exec("print 'goodbye'")
现在这个 似乎 可以工作但是,我得到以下信息:
>>>cgame()
{'a': 3, 'c': 5}
{'a': 4, 'c': 5, 'b': 3}
your code here
Traceback (most recent call last):
File "<pyshell#43>", line 1, in <module>
cgame()
File "C:\Python27\somefile.py", line 14, in cgame
strip_game(a)
File "C:\Python27\somefile.py", line 78, in __call__
return self.inline_f(*args, **kwargs)
File "C:\Python27\somefile.py", line 94, in strip_game
z = c
NameError: global name 'c' is not defined
现在,当我反汇编函数时,我得到以下 game
和 strip_game
之间非常奇怪的编译差异:
在游戏中:
86 16 LOAD_NAME 0 (locals)
19 CALL_FUNCTION 0
22 PRINT_ITEM
23 PRINT_NEWLINE
87 24 **LOAD_NAME** 1 (c)
27 PRINT_ITEM
28 PRINT_NEWLINE
在脱衣游戏中:
95 16 LOAD_GLOBAL 0 (locals)
19 CALL_FUNCTION 0
22 PRINT_ITEM
23 PRINT_NEWLINE
96 24 LOAD_GLOBAL 1 (c)
27 PRINT_ITEM
28 PRINT_NEWLINE
为什么会出现这种差异?
在这种情况下,只需使用global
关键字:
a=3
def foo():
global a
a=4
foo()
print (a)
修改外部作用域,如果它是全局的。
如果外部作用域是一个函数,则使用 nonlocal
关键字代替 - 这是在 Python 3.0 中引入的。
动态范围
然而,改变调用者函数的范围并不是Python的前提,而是一种语言特性。
可以完成。但是仅仅通过调用私有 C api(将 'locals' 值返回到快速局部变量中)绝对不是一个好习惯。
也可以通过魔术装饰器来实现,但是装饰器必须重写内部函数中的字节码——通过检索和更新caler locals
,并且,在函数的末尾 - https://programtalk.com/python-examples/ctypes.pythonapi.PyFrame_LocalsToFast/
例子
也就是说,这是一个概念证明。当然,它是线程和异步 非常不安全 - 但如果代理中的属性 class
被提升为 threadlocals 或 context-local (pep 555),它应该可以工作。
应该很容易调整它以搜索 local-variables 以在调用堆栈上进行更改(以便在 sub-sub-call 中所做的更改可以更改祖父母本地人,就像在动态范围语言中一样)
如问题中所述,无需将调用方的变量声明为任何内容 - 它们必须是普通的局部变量。但是,这需要在装饰函数上声明我想在调用者范围内更改为 'global' 的变量,以便更改然后将通过我可以自定义的 object。如果你连这个都做不到,你确实不得不求助于重写装饰函数上的字节码,或者使用用于编写调试器的挂钩(在代码上设置 "trace on")。
nb 最近对语言指定了更改 locals() 的确切行为 - 在 3.8 之前,IIRC, - "locals_to_fast" 似乎是
足够稳定 API - 但它可能会在未来发生变化。
# Tested in Python 3.8.0
import ctypes
from functools import wraps
from sys import _getframe as getframe
from types import FunctionType
class GlobalProxy(dict):
__slots__ = ("parent", "frame", "mode")
def __init__(self, parent):
self.parent = parent
self.frame = None
self.mode = None
def __getitem__(self, name):
if self.mode == "target":
if name in self.frame.f_locals:
return self.frame.f_locals[name]
if name in self.parent:
return self.parent[name]
return getattr(self.parent["__builtins__"], name)
return super().__getitem__(name)
"""
# This is not run - Python's VM STORE_GLOBAL bypasses the custom __setitem__ (although __getitem__ above runs)
def __setitem__(self, name, value):
if name in self.frame.f_locals:
self.frame.f_locals[name] = value
bake_locals(self.frame)
self.parent[name] = value
"""
def bake_locals(self):
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(self.frame), ctypes.c_int(1))
def save_changes(self):
self.mode = "inner"
target = self.frame.f_locals
target_names = set(target.keys())
for key in self:
if key in target_names:
target[key] = self[key]
else:
self.parent[key] = self[key]
self.bake_locals()
def caller_changer(func):
"""Makes all global variable changes on the decorated function affect _local_ variables on the callee function instead.
"""
code = func.__code__
# NB: for Python 2, these dunder-attributes for functions have other names.
# this is for Python 3
proxy = GlobalProxy(func.__globals__)
new_function = FunctionType(code, proxy, func.__name__, func.__defaults__, func.__closure__)
@wraps(func)
def wrapper(*args, **kw):
proxy.frame = getframe().f_back
proxy.mode = "target"
result = new_function(*args, **kw)
proxy.save_changes()
return result
wrapper.proxy = proxy
return wrapper
### Example and testing code:
@caller_changer
def blah():
global iwillchange
iwillchange = "new value"
def bleh():
iwillchange = "original value"
print(iwillchange)
blah()
print(iwillchange)
然后,将所有内容粘贴到 IPython shell:
In [121]: bleh()
original value
new value
(我可能会补充说,测试它感觉很奇怪,因为
改变局部变量不需要任何装饰器,
或对变量的任何特殊声明)
好吧,经过几个小时的研究,我终于写出了一个解决方案,在解决这个问题时有一些主要的陷阱,我会在下面指出它们
import inspect
import ctypes
import struct
import dis
import types
def dump(obj):
for attr in dir(obj):
print("obj.%s = %r" % (attr, getattr(obj, attr)))
def cgame():
a=3
c=5
print locals()
strip_game(a)
print locals()
def pgame():
a=3
c=5
print locals()
game(a)
print locals()
class empty_deco(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
debug_func = None
class inline_func(object):
def __init__(self, f):
self.f = f
# this is the price we pay for using 2.7
# also, there is a huge glraing issue here, which is what happens if the user TRIES to access a global variable?
@staticmethod
def replace_globals_with_name_lookups(co):
res = ""
code = list(co)
n = len(code)
i = 0
while i < n:
c = code[i]
op = ord(c)
if dis.opname[op] == "STORE_GLOBAL":
code[i] = chr(dis.opmap['STORE_NAME'])
elif dis.opname[op] == "DELETE_GLOBAL":
code[i] = chr(dis.opmap['DELETE_NAME'])
elif dis.opname[op] == "LOAD_GLOBAL":
code[i] = chr(dis.opmap['LOAD_NAME'])
i = i+1
if op >= dis.HAVE_ARGUMENT:
i = i+2
return "".join(code)
def __call__(self, *args, **kwargs):
init_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\n" + \
"inspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))"
fini_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))"
co_stacksize = max(6, self.f.func_code.co_stacksize) # make sure we have enough space on the stack for everything
co_consts = self.f.func_code.co_consts +(init_exec_string, fini_exec_string)
init = "d" + struct.pack("H", len(strip_game.f.func_code.co_consts)) #LOAD_CONST init_exec_string
init += "d\x00\x00\x04U" # LOAD_CONST None, DUP_TOP, EXEC_STMT
init += "z" + struct.pack("H", len(self.f.func_code.co_code) + 4) #SETUP_FINALLY
fini = "Wd\x00\x00" # POP_BLOCK, LOAD_CONST None
fini += "d" + struct.pack("H", len(strip_game.f.func_code.co_consts) + 1) #LOAD_CONST fini_exec_string
fini += "d\x00\x00\x04UXd\x00\x00S" # LOAD_CONST None, DUP_TOP, EXEC_STMT, END_FINALLY, LOAD_CONST None, RETURN
co_code = init + self.replace_globals_with_name_lookups(self.f.func_code.co_code) + fini
co_lnotab = "\x00\x00\x0b" + self.f.func_code.co_lnotab[1:] # every error in init will be attributed to @inline_func, errors in the function will be treated as expected, errors in fini will be attributed to the last line probably.
new_code = types.CodeType(
self.f.func_code.co_argcount,
self.f.func_code.co_nlocals,
co_stacksize,
self.f.func_code.co_flags & ~(1), # optimized functions are problematic for us
co_code,
co_consts,
self.f.func_code.co_names,
self.f.func_code.co_varnames,
self.f.func_code.co_filename,
self.f.func_code.co_name,
self.f.func_code.co_firstlineno,
co_lnotab,
self.f.func_code.co_freevars,
self.f.func_code.co_cellvars,)
self.inline_f = types.FunctionType(new_code, self.f.func_globals, self.f.func_name, self.f.func_defaults, self.f.func_closure)
#dis.dis(self.inline_f)
global debug_func
debug_func = self.inline_f
return self.inline_f(*args, **kwargs)
@empty_deco
def game(b, a=4):
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\ninspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))")
try:
print "inner locals:"
print locals()
print c
return None
finally:
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))")
@inline_func
def strip_game(b, a=4):
print "inner locals:"
print locals()
print c
return None
所需的实际代码位于 class inline_func
和一些导入(也许您可以将它们置于 class 内部?我真的不确定)
那么这整件事是做什么的呢?好吧,这使得 strip_game
和 game
的代码(几乎)相同,即:
- 它插入一个函数序言,更新调用者的局部变量,然后将调用者的局部变量添加到被调用者。
- 在函数周围插入一个 try finally 块
- 将每个符号查找从全局查找更改为普通(名称)查找,经过一番思考后我意识到这实际上没有任何效果
- 进入 finally 块后,更新调用方本地信息。
做这些事情有一些主要的陷阱,我将列出一些我遇到的问题:
- cpython
compiler_nameop
函数根据给定函数的简单性优化命名空间查找,这意味着如果可以的话它将把名称查找优化为全局查找
- 改变字节码意味着影响程序的调试能力,我已经在
co_lnotab
变量 中解决了这个问题
- 对于大型函数,此解决方案将不起作用,因为某些操作码必须使用 extended_args:即变量的加载和 try-finally 块(这一点可通过使用 extended_args 无论如何...)
感谢@jsbueno 抽出时间指点我 PyFrame_LocalsToFast。
P.S。此解决方案适用于 python 2.7.6,python 在 API 的稳定性方面存在一些问题,因此对于较新的版本,这可能需要修复。
是否有类似于 C 宏的功能,可以让您以内联方式重用代码,而无需为该段代码创建单独的作用域?
例如:
a=3
def foo():
a=4
foo()
print a
将打印 3,但我希望它打印 4。
我知道涉及 类 或全局字典等对象的解决方案,但是我正在寻找更原始的解决方案(例如函数装饰器),它只会让我在内部进行更改调用者的范围。
非常感谢
编辑:任何需要声明我将使用哪些变量或事先声明 "namespace" 之类的可变对象的解决方案都不是我正在寻找的解决方案。
我自己尝试过:
def pgame():
a=3
c=5
print locals()
game(a)
print locals()
class inline_func(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
#to be @inline_func
def game(b, a=4):
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1] [0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\ninspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))")
try:
print "your code here"
finally:
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))")
@inline_func
def strip_game(b, a=4):
print "your code here"
但是我 运行 遇到了一个严重的问题,即如何在不破坏程序的可调试性的情况下将代码注入 strip_game
,因为我只想创建一个新的代码对象或使用 exec , 都遇到了一些严重的问题。
主要编辑:
好的,所以我有一些接近可行的解决方案,但是我遇到了一个非常奇怪的问题:
import inspect
import ctypes
import struct
import dis
import types
def cgame():
a=3
c=5
print locals()
strip_game(a)
print locals()
def pgame():
a=3
c=5
print locals()
game(a)
print locals()
class empty_deco(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
debug_func = None
class inline_func(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
init_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\n" + \
"inspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))"
fini_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))"
co_stacksize = max(6, self.f.func_code.co_stacksize) # make sure we have enough space on the stack for everything
co_consts = self.f.func_code.co_consts +(init_exec_string, fini_exec_string)
init = "d" + struct.pack("H", len(strip_game.f.func_code.co_consts)) #LOAD_CONST init_exec_string
init += "d\x00\x00\x04U" # LOAD_CONST None, DUP_TOP, EXEC_STMT
init += "z" + struct.pack("H", len(self.f.func_code.co_code) + 4) #SETUP_FINALLY
fini = "Wd\x00\x00" # POP_BLOCK, LOAD_CONST None
fini += "d" + struct.pack("H", len(strip_game.f.func_code.co_consts) + 1) #LOAD_CONST fini_exec_string
fini += "d\x00\x00\x04UXd\x00\x00S" # LOAD_CONST None, DUP_TOP, EXEC_STMT, END_FINALLY, LOAD_CONST None, RETURN
co_code = init + self.f.func_code.co_code + fini
co_lnotab = "\x00\x00\x0b" + self.f.func_code.co_lnotab[1:] # every error in init will be attributed to @inline_func, errors in the function will be treated as expected, errors in fini will be attributed to the last line probably.
new_code = types.CodeType(
self.f.func_code.co_argcount,
self.f.func_code.co_nlocals,
co_stacksize,
self.f.func_code.co_flags & ~(1), # optimized functions are problematic for us
co_code,
co_consts,
self.f.func_code.co_names,
self.f.func_code.co_varnames,
self.f.func_code.co_filename,
self.f.func_code.co_name,
self.f.func_code.co_firstlineno,
co_lnotab,
self.f.func_code.co_freevars,
self.f.func_code.co_cellvars,)
self.inline_f = types.FunctionType(new_code, self.f.func_globals, self.f.func_name, self.f.func_defaults, self.f.func_closure)
#dis.dis(self.inline_f)
global debug_func
debug_func = self.inline_f
return self.inline_f(*args, **kwargs)
@empty_deco
def game(b, a=4):
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\ninspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))")
try:
print "inner locals:"
print locals()
print c
return None
finally:
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))")
@inline_func
def strip_game(b, a=4):
print "inner locals:"
print locals()
print c
return None
def stupid():
exec("print 'hello'")
try:
a=1
b=2
c=3
d=4
finally:
exec("print 'goodbye'")
现在这个 似乎 可以工作但是,我得到以下信息:
>>>cgame()
{'a': 3, 'c': 5}
{'a': 4, 'c': 5, 'b': 3}
your code here
Traceback (most recent call last):
File "<pyshell#43>", line 1, in <module>
cgame()
File "C:\Python27\somefile.py", line 14, in cgame
strip_game(a)
File "C:\Python27\somefile.py", line 78, in __call__
return self.inline_f(*args, **kwargs)
File "C:\Python27\somefile.py", line 94, in strip_game
z = c
NameError: global name 'c' is not defined
现在,当我反汇编函数时,我得到以下 game
和 strip_game
之间非常奇怪的编译差异:
在游戏中:
86 16 LOAD_NAME 0 (locals)
19 CALL_FUNCTION 0
22 PRINT_ITEM
23 PRINT_NEWLINE
87 24 **LOAD_NAME** 1 (c)
27 PRINT_ITEM
28 PRINT_NEWLINE
在脱衣游戏中:
95 16 LOAD_GLOBAL 0 (locals)
19 CALL_FUNCTION 0
22 PRINT_ITEM
23 PRINT_NEWLINE
96 24 LOAD_GLOBAL 1 (c)
27 PRINT_ITEM
28 PRINT_NEWLINE
为什么会出现这种差异?
在这种情况下,只需使用global
关键字:
a=3
def foo():
global a
a=4
foo()
print (a)
修改外部作用域,如果它是全局的。
如果外部作用域是一个函数,则使用 nonlocal
关键字代替 - 这是在 Python 3.0 中引入的。
动态范围
然而,改变调用者函数的范围并不是Python的前提,而是一种语言特性。
可以完成。但是仅仅通过调用私有 C api(将 'locals' 值返回到快速局部变量中)绝对不是一个好习惯。
也可以通过魔术装饰器来实现,但是装饰器必须重写内部函数中的字节码——通过检索和更新caler locals
,并且,在函数的末尾 - https://programtalk.com/python-examples/ctypes.pythonapi.PyFrame_LocalsToFast/
例子
也就是说,这是一个概念证明。当然,它是线程和异步 非常不安全 - 但如果代理中的属性 class 被提升为 threadlocals 或 context-local (pep 555),它应该可以工作。 应该很容易调整它以搜索 local-variables 以在调用堆栈上进行更改(以便在 sub-sub-call 中所做的更改可以更改祖父母本地人,就像在动态范围语言中一样)
如问题中所述,无需将调用方的变量声明为任何内容 - 它们必须是普通的局部变量。但是,这需要在装饰函数上声明我想在调用者范围内更改为 'global' 的变量,以便更改然后将通过我可以自定义的 object。如果你连这个都做不到,你确实不得不求助于重写装饰函数上的字节码,或者使用用于编写调试器的挂钩(在代码上设置 "trace on")。
nb 最近对语言指定了更改 locals() 的确切行为 - 在 3.8 之前,IIRC, - "locals_to_fast" 似乎是 足够稳定 API - 但它可能会在未来发生变化。
# Tested in Python 3.8.0
import ctypes
from functools import wraps
from sys import _getframe as getframe
from types import FunctionType
class GlobalProxy(dict):
__slots__ = ("parent", "frame", "mode")
def __init__(self, parent):
self.parent = parent
self.frame = None
self.mode = None
def __getitem__(self, name):
if self.mode == "target":
if name in self.frame.f_locals:
return self.frame.f_locals[name]
if name in self.parent:
return self.parent[name]
return getattr(self.parent["__builtins__"], name)
return super().__getitem__(name)
"""
# This is not run - Python's VM STORE_GLOBAL bypasses the custom __setitem__ (although __getitem__ above runs)
def __setitem__(self, name, value):
if name in self.frame.f_locals:
self.frame.f_locals[name] = value
bake_locals(self.frame)
self.parent[name] = value
"""
def bake_locals(self):
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(self.frame), ctypes.c_int(1))
def save_changes(self):
self.mode = "inner"
target = self.frame.f_locals
target_names = set(target.keys())
for key in self:
if key in target_names:
target[key] = self[key]
else:
self.parent[key] = self[key]
self.bake_locals()
def caller_changer(func):
"""Makes all global variable changes on the decorated function affect _local_ variables on the callee function instead.
"""
code = func.__code__
# NB: for Python 2, these dunder-attributes for functions have other names.
# this is for Python 3
proxy = GlobalProxy(func.__globals__)
new_function = FunctionType(code, proxy, func.__name__, func.__defaults__, func.__closure__)
@wraps(func)
def wrapper(*args, **kw):
proxy.frame = getframe().f_back
proxy.mode = "target"
result = new_function(*args, **kw)
proxy.save_changes()
return result
wrapper.proxy = proxy
return wrapper
### Example and testing code:
@caller_changer
def blah():
global iwillchange
iwillchange = "new value"
def bleh():
iwillchange = "original value"
print(iwillchange)
blah()
print(iwillchange)
然后,将所有内容粘贴到 IPython shell:
In [121]: bleh()
original value
new value
(我可能会补充说,测试它感觉很奇怪,因为 改变局部变量不需要任何装饰器, 或对变量的任何特殊声明)
好吧,经过几个小时的研究,我终于写出了一个解决方案,在解决这个问题时有一些主要的陷阱,我会在下面指出它们
import inspect
import ctypes
import struct
import dis
import types
def dump(obj):
for attr in dir(obj):
print("obj.%s = %r" % (attr, getattr(obj, attr)))
def cgame():
a=3
c=5
print locals()
strip_game(a)
print locals()
def pgame():
a=3
c=5
print locals()
game(a)
print locals()
class empty_deco(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
debug_func = None
class inline_func(object):
def __init__(self, f):
self.f = f
# this is the price we pay for using 2.7
# also, there is a huge glraing issue here, which is what happens if the user TRIES to access a global variable?
@staticmethod
def replace_globals_with_name_lookups(co):
res = ""
code = list(co)
n = len(code)
i = 0
while i < n:
c = code[i]
op = ord(c)
if dis.opname[op] == "STORE_GLOBAL":
code[i] = chr(dis.opmap['STORE_NAME'])
elif dis.opname[op] == "DELETE_GLOBAL":
code[i] = chr(dis.opmap['DELETE_NAME'])
elif dis.opname[op] == "LOAD_GLOBAL":
code[i] = chr(dis.opmap['LOAD_NAME'])
i = i+1
if op >= dis.HAVE_ARGUMENT:
i = i+2
return "".join(code)
def __call__(self, *args, **kwargs):
init_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\n" + \
"inspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))"
fini_exec_string = "inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\n" + \
"ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))"
co_stacksize = max(6, self.f.func_code.co_stacksize) # make sure we have enough space on the stack for everything
co_consts = self.f.func_code.co_consts +(init_exec_string, fini_exec_string)
init = "d" + struct.pack("H", len(strip_game.f.func_code.co_consts)) #LOAD_CONST init_exec_string
init += "d\x00\x00\x04U" # LOAD_CONST None, DUP_TOP, EXEC_STMT
init += "z" + struct.pack("H", len(self.f.func_code.co_code) + 4) #SETUP_FINALLY
fini = "Wd\x00\x00" # POP_BLOCK, LOAD_CONST None
fini += "d" + struct.pack("H", len(strip_game.f.func_code.co_consts) + 1) #LOAD_CONST fini_exec_string
fini += "d\x00\x00\x04UXd\x00\x00S" # LOAD_CONST None, DUP_TOP, EXEC_STMT, END_FINALLY, LOAD_CONST None, RETURN
co_code = init + self.replace_globals_with_name_lookups(self.f.func_code.co_code) + fini
co_lnotab = "\x00\x00\x0b" + self.f.func_code.co_lnotab[1:] # every error in init will be attributed to @inline_func, errors in the function will be treated as expected, errors in fini will be attributed to the last line probably.
new_code = types.CodeType(
self.f.func_code.co_argcount,
self.f.func_code.co_nlocals,
co_stacksize,
self.f.func_code.co_flags & ~(1), # optimized functions are problematic for us
co_code,
co_consts,
self.f.func_code.co_names,
self.f.func_code.co_varnames,
self.f.func_code.co_filename,
self.f.func_code.co_name,
self.f.func_code.co_firstlineno,
co_lnotab,
self.f.func_code.co_freevars,
self.f.func_code.co_cellvars,)
self.inline_f = types.FunctionType(new_code, self.f.func_globals, self.f.func_name, self.f.func_defaults, self.f.func_closure)
#dis.dis(self.inline_f)
global debug_func
debug_func = self.inline_f
return self.inline_f(*args, **kwargs)
@empty_deco
def game(b, a=4):
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))\ninspect.stack()[1][0].f_locals.update(inspect.stack()[3][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[1][0]),ctypes.c_int(0))")
try:
print "inner locals:"
print locals()
print c
return None
finally:
exec("inspect.stack()[3][0].f_locals.update(inspect.stack()[1][0].f_locals)\nctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(inspect.stack()[3][0]),ctypes.c_int(0))")
@inline_func
def strip_game(b, a=4):
print "inner locals:"
print locals()
print c
return None
所需的实际代码位于 class inline_func
和一些导入(也许您可以将它们置于 class 内部?我真的不确定)
那么这整件事是做什么的呢?好吧,这使得 strip_game
和 game
的代码(几乎)相同,即:
- 它插入一个函数序言,更新调用者的局部变量,然后将调用者的局部变量添加到被调用者。
- 在函数周围插入一个 try finally 块
- 将每个符号查找从全局查找更改为普通(名称)查找,经过一番思考后我意识到这实际上没有任何效果
- 进入 finally 块后,更新调用方本地信息。
做这些事情有一些主要的陷阱,我将列出一些我遇到的问题:
- cpython
compiler_nameop
函数根据给定函数的简单性优化命名空间查找,这意味着如果可以的话它将把名称查找优化为全局查找 - 改变字节码意味着影响程序的调试能力,我已经在
co_lnotab
变量 中解决了这个问题
- 对于大型函数,此解决方案将不起作用,因为某些操作码必须使用 extended_args:即变量的加载和 try-finally 块(这一点可通过使用 extended_args 无论如何...)
感谢@jsbueno 抽出时间指点我 PyFrame_LocalsToFast。
P.S。此解决方案适用于 python 2.7.6,python 在 API 的稳定性方面存在一些问题,因此对于较新的版本,这可能需要修复。