使用 PyTorch 实现一个简单的 ResNet 块
Implementing a simple ResNet block with PyTorch
我正在尝试实现以下 ResNet 块,该 ResNet 由具有两个卷积层和一个跳跃连接的块组成。出于某种原因,它不会添加跳过连接的输出(如果应用)或输入到卷积层的输出。
ResNet 块具有:
两个卷积层:
- 3x3 内核
- 无偏见条款
- 两侧各填充一个像素
- 每个卷积层后的 2d 批量归一化
跳过连接:
- 如果分辨率和通道数不变,则简单地复制输入。
- 如果分辨率或通道数发生变化,跳过连接应该有一个卷积层:
- 无偏置的 1x1 卷积
- 分辨率随步幅变化(可选)
- 不同数量的输入通道和输出通道(可选)
- 1x1 卷积层之后是 2d 批量归一化。
ReLU 非线性应用于第一个卷积层之后和块的末尾。
我的代码:
class Block(nn.Module):
def __init__(self, in_channels, out_channels, stride=1):
"""
Args:
in_channels (int): Number of input channels.
out_channels (int): Number of output channels.
stride (int): Controls the stride.
"""
super(Block, self).__init__()
self.skip = nn.Sequential()
if stride != 1 or in_channels != out_channels:
self.skip = nn.Sequential(
nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(out_channels))
else:
self.skip = None
self.block = nn.Sequential(
nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, padding=1, stride=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(),
nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, padding=1, stride=1, bias=False),
nn.BatchNorm2d(out_channels))
def forward(self, x):
out = self.block(x)
if self.skip is not None:
out = self.skip(x)
else:
out = x
out += x
out = F.relu(out)
return out
问题在于 out
变量的重用。通常,你会像 this:
def forward(self, x):
identity = x
out = self.block(x)
if self.skip is not None:
identity = self.skip(x)
out += identity
out = F.relu(out)
return out
如果你喜欢"one-liners":
def forward(self, x):
out = self.block(x)
out += (x if self.skip is None else self.skip(x))
out = F.relu(out)
return out
如果你真的喜欢单行(拜托,那太多了,不要选择这个选项:))
def forward(self, x):
return F.relu(self.block(x) + (x if self.skip is None else self.skip(x)))
我正在尝试实现以下 ResNet 块,该 ResNet 由具有两个卷积层和一个跳跃连接的块组成。出于某种原因,它不会添加跳过连接的输出(如果应用)或输入到卷积层的输出。
ResNet 块具有:
两个卷积层:
- 3x3 内核
- 无偏见条款
- 两侧各填充一个像素
- 每个卷积层后的 2d 批量归一化
跳过连接:
- 如果分辨率和通道数不变,则简单地复制输入。
- 如果分辨率或通道数发生变化,跳过连接应该有一个卷积层:
- 无偏置的 1x1 卷积
- 分辨率随步幅变化(可选)
- 不同数量的输入通道和输出通道(可选)
- 1x1 卷积层之后是 2d 批量归一化。
ReLU 非线性应用于第一个卷积层之后和块的末尾。
我的代码:
class Block(nn.Module):
def __init__(self, in_channels, out_channels, stride=1):
"""
Args:
in_channels (int): Number of input channels.
out_channels (int): Number of output channels.
stride (int): Controls the stride.
"""
super(Block, self).__init__()
self.skip = nn.Sequential()
if stride != 1 or in_channels != out_channels:
self.skip = nn.Sequential(
nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(out_channels))
else:
self.skip = None
self.block = nn.Sequential(
nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, padding=1, stride=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(),
nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, padding=1, stride=1, bias=False),
nn.BatchNorm2d(out_channels))
def forward(self, x):
out = self.block(x)
if self.skip is not None:
out = self.skip(x)
else:
out = x
out += x
out = F.relu(out)
return out
问题在于 out
变量的重用。通常,你会像 this:
def forward(self, x):
identity = x
out = self.block(x)
if self.skip is not None:
identity = self.skip(x)
out += identity
out = F.relu(out)
return out
如果你喜欢"one-liners":
def forward(self, x):
out = self.block(x)
out += (x if self.skip is None else self.skip(x))
out = F.relu(out)
return out
如果你真的喜欢单行(拜托,那太多了,不要选择这个选项:))
def forward(self, x):
return F.relu(self.block(x) + (x if self.skip is None else self.skip(x)))