MLP的神经网络如何从tensorflow翻译到pytorch
How to translate the neural network of MLP from tensorflow to pytorch
我使用'Tensorflow'构建了一个MLP神经网络,如下所示:
model_mlp=Sequential()
model_mlp.add(Dense(units=35, input_dim=train_X.shape[1], kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=86, kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=86, kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=10, kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=1))
我想用pytorch转换上面的MLP代码。怎么做?我尝试按如下方式进行:
class MLP(nn.Module):
def __init__(self):
super(MLP, self).__init__()
self.fc1 = nn.Linear(train_X.shape[1],35)
self.fc2 = nn.Linear(35, 86)
self.fc3 = nn.Linear(86, 86)
self.fc4 = nn.Linear(86, 10)
self.fc5 = nn.Linear(10, 1)
def forward(self, x):
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = F.relu(self.fc3(x))
x = F.relu(self.fc4(x))
x = self.fc5(x)
return x
def predict(self, x_test):
x_test = torch.from_numpy(x_test).float()
x_test = self.forward(x_test)
return x_test.view(-1).data.numpy()
model = MLP()
我使用相同的数据集,但两个代码给出了两个不同的答案。使用 Tensorflow 编写的代码总是比使用 Pytorch 编写的代码产生更好的结果。我想知道我在pytorch中的代码是否不正确。
如果我在 PyTorch 中编写的代码是正确的,我想知道如何解释这些差异。我期待着任何回复。
欢迎使用 pytorch!
我想问题出在您的网络初始化上。我就是这样做的:
def init_weights(m):
if type(m) == nn.Linear:
torch.nn.init.xavier_normal(m.weight) # initialize with xaver normal (called gorot in tensorflow)
m.bias.data.fill_(0.01) # initialize bias with a constant
class MLP(nn.Module):
def __init__(self, input_dim):
super(MLP, self).__init__()
self.mlp = nn.Sequential(nn.Linear(input_dim ,35), nn.ReLU(),
nn.Linear(35, 86), nn.ReLU(),
nn.Linear(86, 86), nn.ReLU(),
nn.Linear(86, 10), nn.ReLU(),
nn.Linear(10, 1), nn.ReLU())
def forward(self, x):
y =self.mlp(x)
return y
model = MLP(input_dim)
model.apply(init_weights)
optimizer = Adam(model.parameters())
loss_func = BCEWithLogistLoss()
# training loop
for data, label in dataloader:
optimizer.zero_grad()
pred = model(data)
loss = loss_func(pred, lable)
loss.backward()
optimizer.step()
注意在pytorch中我们不调用model.forward(x)
,而是调用model(x)
。这是因为 nn.Module
在 .__call__()
中应用了在反向传递中使用的钩子。
你可以在这里查看权重初始化的文档:https://pytorch.org/docs/stable/nn.init.html
我使用'Tensorflow'构建了一个MLP神经网络,如下所示:
model_mlp=Sequential()
model_mlp.add(Dense(units=35, input_dim=train_X.shape[1], kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=86, kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=86, kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=10, kernel_initializer='normal', activation='relu'))
model_mlp.add(Dense(units=1))
我想用pytorch转换上面的MLP代码。怎么做?我尝试按如下方式进行:
class MLP(nn.Module):
def __init__(self):
super(MLP, self).__init__()
self.fc1 = nn.Linear(train_X.shape[1],35)
self.fc2 = nn.Linear(35, 86)
self.fc3 = nn.Linear(86, 86)
self.fc4 = nn.Linear(86, 10)
self.fc5 = nn.Linear(10, 1)
def forward(self, x):
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = F.relu(self.fc3(x))
x = F.relu(self.fc4(x))
x = self.fc5(x)
return x
def predict(self, x_test):
x_test = torch.from_numpy(x_test).float()
x_test = self.forward(x_test)
return x_test.view(-1).data.numpy()
model = MLP()
我使用相同的数据集,但两个代码给出了两个不同的答案。使用 Tensorflow 编写的代码总是比使用 Pytorch 编写的代码产生更好的结果。我想知道我在pytorch中的代码是否不正确。 如果我在 PyTorch 中编写的代码是正确的,我想知道如何解释这些差异。我期待着任何回复。
欢迎使用 pytorch!
我想问题出在您的网络初始化上。我就是这样做的:
def init_weights(m):
if type(m) == nn.Linear:
torch.nn.init.xavier_normal(m.weight) # initialize with xaver normal (called gorot in tensorflow)
m.bias.data.fill_(0.01) # initialize bias with a constant
class MLP(nn.Module):
def __init__(self, input_dim):
super(MLP, self).__init__()
self.mlp = nn.Sequential(nn.Linear(input_dim ,35), nn.ReLU(),
nn.Linear(35, 86), nn.ReLU(),
nn.Linear(86, 86), nn.ReLU(),
nn.Linear(86, 10), nn.ReLU(),
nn.Linear(10, 1), nn.ReLU())
def forward(self, x):
y =self.mlp(x)
return y
model = MLP(input_dim)
model.apply(init_weights)
optimizer = Adam(model.parameters())
loss_func = BCEWithLogistLoss()
# training loop
for data, label in dataloader:
optimizer.zero_grad()
pred = model(data)
loss = loss_func(pred, lable)
loss.backward()
optimizer.step()
注意在pytorch中我们不调用model.forward(x)
,而是调用model(x)
。这是因为 nn.Module
在 .__call__()
中应用了在反向传递中使用的钩子。
你可以在这里查看权重初始化的文档:https://pytorch.org/docs/stable/nn.init.html