model.eval() 和 model.train() 影响哪些 PyTorch 模块？

Question

model.eval() 方法修改了某些模块（层），这些模块需要在训练和推理期间表现不同。 the docs:

中列出了一些示例

This has [an] effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout, BatchNorm, etc.

是否有受影响模块的详尽列表？

Answer 1

在 google 上搜索 site:https://pytorch.org/docs/stable/generated/torch.nn. "during evaluation"，似乎以下模块受到影响：

Base class	Modules	Criteria
_InstanceNorm	InstanceNorm1d InstanceNorm2d InstanceNorm3d	track_running_stats=True
_BatchNorm	BatchNorm1d BatchNorm2d BatchNorm3d SyncBatchNorm
_DropoutNd	Dropout Dropout2d Dropout3d AlphaDropout FeatureAlphaDropout

Answer 2

除了 提供的信息外：

Base class	Module	Criteria
RNNBase	RNN LSTM GRU	dropout > 0 (default: 0)
Transformer layers	Transformer TransformerEncoder TransformerDecoder	dropout > 0 (Transformer default: 0.1)
Lazy variants	LazyBatchNorm currently nightly merged PR	track_running_stats=True