'Attempting to capture an EagerTensor without building a function' Error: While building Federated Averaging Process

Question

我在尝试构建联合平均过程时遇到 'Attempting to capture an EagerTensor without building a function' 错误。我已经尝试了其他类似堆栈溢出问题中给出的 v1 和 v2 兼容性的所有补救措施，即使用 tf.compat.v1.enable_eager_execution() 、 tf.disable_v2_behaviour() 等。但是，没有任何效果。我的相关代码摘录如下。 Python 笔记本中的完整代码在此处 https://gist.github.com/aksingh2411/60796ee58c88e0c3f074c8909b17b5a1.

#Making a Tensorflow Model
from tensorflow import keras

def create_keras_model():
 return tf.keras.models.Sequential([
  hub.KerasLayer(encoder, input_shape=[],dtype=tf.string,trainable=True),
  keras.layers.Dense(32, activation='relu'),
  keras.layers.Dense(16, activation='relu'),
  keras.layers.Dense(1, activation='sigmoid'),
])

def model_fn():
# We _must_ create a new model here, and _not_ capture it from an external
# scope. TFF will call this within different graph contexts.
keras_model = create_keras_model()
return tff.learning.from_keras_model(
  keras_model,
  input_spec=preprocessed_example_dataset.element_spec,
  loss=tf.keras.losses.BinaryCrossentropy(),
  metrics=[tf.keras.metrics.Accuracy()])

# Building the Federated Averaging Process
iterative_process = tff.learning.build_federated_averaging_process(
model_fn,
client_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=0.02),
server_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=1.0))

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-23-68fa27e65b7e> in <module>()
  3     model_fn,
  4     client_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=0.02),
  -->5     server_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=1.0))

 9 frames
 /usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py in 
wrapper(*args, **kwargs)
263       except Exception as e:  # pylint:disable=broad-except
264         if hasattr(e, 'ag_error_metadata'):
--> 265           raise e.ag_error_metadata.to_exception(e)
266         else:
267           raise

RuntimeError: in user code:

/usr/local/lib/python3.6/dist-packages/tensorflow_hub/keras_layer.py:222 call  *
    result = f()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py:486 _call_attribute  **
    return instance.__call__(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py:580 __call__
    result = self._call(*args, **kwds)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py:618 _call
    results = self._stateful_fn(*args, **kwds)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py:2420 __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py:1665 _filtered_call
    self.captured_inputs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py:1760 _call_flat
    flat_outputs = forward_function.call(ctx, args_with_tangents)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py:627 call
    executor_type=executor_type)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/functional_ops.py:1148 partitioned_call
    args = [ops.convert_to_tensor(x) for x in args]
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/functional_ops.py:1148 <listcomp>
    args = [ops.convert_to_tensor(x) for x in args]
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1307 convert_to_tensor
    raise RuntimeError("Attempting to capture an EagerTensor without "

RuntimeError: Attempting to capture an EagerTensor without building a function.

Answer 1

这看起来像是在外部创建张量，然后被 model_fn 捕获。 model_fn()里面的评论相关这里：

# We _must_ create a new model here, and _not_ capture it from an external scope. TFF 
# will call this within different graph contexts.

TensorFlow 不允许引用在不同图形（或 tf.function）中创建的张量，因此我们必须构建将在 model_fn()（或内部 create_keras_model() 中引用的所有内容).

要找到错误张量的创建位置，检查堆栈跟踪可能很有用。堆栈跟踪的第一行似乎表明 tensorflow_hub:

/usr/local/lib/python3.6/dist-packages/tensorflow_hub/keras_layer.py:222 call  *
    result = f()

源码中马上出现使用TF Hub的地方是tf.kears.Sequential构造的第一层：

def create_keras_model():
  return tf.keras.models.Sequential([
    hub.KerasLayer(encoder, input_shape=[],dtype=tf.string,trainable=True),
    …

似乎此函数可能正在“关闭”或“捕获”encoder 的值，这反过来可能具有在不同上下文中创建的张量。是否可以将encoder的构造移到里面 create_keras_model()?

'Attempting to capture an EagerTensor without building a function' Error: While building Federated Averaging Process

'Attempting to capture an EagerTensor without building a function' Error: While building Federated Averaging Process

tensorflow

tf.keras

tensorflow2.0

tensorflow-federated