如何实现自定义编码 Tensorflow Federated

Question

我创建了一个自定义 encoder/decoder，如下所示：

import tensorflow as tf

from tensorflow_model_optimization.python.core.internal import tensor_encoding as te


# noinspection PyUnresolvedReferences
class SparseTernaryCompressionEncodingStage(te.core.EncodingStageInterface):
    AVERAGE = 'average'
    NEGATIVES = 'negatives'
    POSITIVES = 'positives'
    TESTING = 'testing'
    NEW_SHAPE = 'new_shape'
    ORIGINAL_SHAPE = 'original_shape'

    def name(self):
        pass

    def compressible_tensors_keys(self):
        pass

    def commutes_with_sum(self):
        pass

    def decode_needs_input_shape(self):
        pass

    def get_params(self):
        pass

    def encode(self, original_tensor, encode_params):
        original_shape = tf.shape(original_tensor)
        tensor = tf.reshape(original_tensor, [-1])
        sparsification_rate = int(len(tensor) / 100 * 1)
        new_shape = tensor.get_shape().as_list()
        if sparsification_rate == 0:
            sparsification_rate = 1
        mask = tf.cast(tf.abs(tensor) >= tf.math.top_k(tf.abs(tensor), sparsification_rate)[0][-1], tf.float32)
        inv_mask = tf.cast(tf.abs(tensor) < tf.math.top_k(tf.abs(tensor), sparsification_rate)[0][-1], tf.float32)
        tensor_masked = tf.multiply(tensor, mask)
        average = tf.reduce_sum(tf.abs(tensor_masked)) / sparsification_rate
        compressed_tensor = tf.add(tf.multiply(average, mask) * tf.sign(tensor), tf.multiply(tensor_masked, inv_mask))
        negatives = tf.where(compressed_tensor < 0)
        positives = tf.where(compressed_tensor > 0)

        encoded_x = {self.AVERAGE: average, self.NEGATIVES: negatives, self.POSITIVES: positives,
                     self.NEW_SHAPE: new_shape, self.ORIGINAL_SHAPE: original_shape}

        return encoded_x

    def decode(self, encoded_tensors, decode_params, num_summands=None, shape=None):
        decompressed_tensor = tf.zeros(self.NEW_SHAPE, tf.float32)
        average_values_negative = tf.fill([len(self.NEGATIVES), ], -self.AVERAGE)
        average_values_positive = tf.fill([len(self.POSITIVES), ], self.AVERAGE)
        decompressed_tensor = tf.tensor_scatter_nd_update(decompressed_tensor, self.NEGATIVES, average_values_negative)
        decompressed_tensor = tf.tensor_scatter_nd_update(decompressed_tensor, self.POSITIVES, average_values_positive)
        decompressed_tensor = tf.reshape(decompressed_tensor, self.ORIGINAL_SHAPE)
        return decompressed_tensor

现在，我想使用编码函数对客户端发送到服务器的所有权重进行编码，然后在服务器上使用解码函数来获取所有权重。基本上，我不想将所有权重从客户端发送到服务器，而是只想发送一些必要的信息，这样我就可以仅从 5 个信息中创建权重。

问题是我不明白如何告诉客户端使用此编码器发送信息并告诉服务器在尝试之前使用解码器： round_model_delta = tff.federated_mean(client_outputs.weights_delta, weight=weight_denom)

我正在使用 Tensorflow Federated simple_fedavg 作为基本项目。

Answer 1

如果您只想修改聚合，您可以更轻松地使用 tff.learning API 和您拥有的东西，使用 tff.aggregators 对象参数化聚合。例如：

te.core.EncoderComposer(te.testing.PlusOneOverNEncodingStage()).make()

def encoder_fn(value_spec):
  return te.encoders.as_gather_encoder(
     te.core.EncoderComposer(SparseTernaryCompressionEncodingStage()).make(),
     value_spec)

tff.learning.build_federated_averaging_process(
    ...,  # Other args.
    model_update_aggregation_factory=tff.aggregators.EncodedSumFactory(
        encoder_fn))

您可能还会发现这些教程很有帮助：

如何实现自定义编码 Tensorflow Federated

How to implement custom encode Tensorflow Federated

python

tensorflow

tensorflow-federated