分割任务中的掩码值不是 0 就是 255。我该如何解决这个问题？
Mask values in segmentation task are either 0 or 255. How do I resolve this?
我一定是在使用 fast-ai 库时遇到了严重的问题，因为我似乎是唯一一个遇到这个问题的人。每次我尝试学习率查找器或训练网络时，它都会给我一个错误。我花了一周的时间才产生这个特定的错误消息，这让我检查了掩码值。事实证明，背景像素为 0，前景像素为 255。这是个问题，因为我只有两个 类。如何在我的 Databunch 对象中将 255 个值更改为 1？有没有办法 divide 每个掩码值增加 255 还是我需要以某种方式事先做？我有点迷失在这个过程中。
这是我收到的错误消息：
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-20-c7a9c29f9dd1> in <module>()
----> 1 learn.lr_find()
2 learn.recorder.plot()
8 frames
/usr/local/lib/python3.6/dist-packages/fastai/train.py in lr_find(learn, start_lr, end_lr, num_it, stop_div, wd)
39 cb = LRFinder(learn, start_lr, end_lr, num_it, stop_div)
40 epochs = int(np.ceil(num_it/len(learn.data.train_dl)))
---> 41 learn.fit(epochs, start_lr, callbacks=[cb], wd=wd)
42
43 def to_fp16(learn:Learner, loss_scale:float=None, max_noskip:int=1000, dynamic:bool=True, clip:float=None,
/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
198 else: self.opt.lr,self.opt.wd = lr,wd
199 callbacks = [cb(self) for cb in self.callback_fns + listify(defaults.extra_callback_fns)] + listify(callbacks)
--> 200 fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
201
202 def create_opt(self, lr:Floats, wd:Floats=0.)->None:
/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py in fit(epochs, learn, callbacks, metrics)
99 for xb,yb in progress_bar(learn.data.train_dl, parent=pbar):
100 xb, yb = cb_handler.on_batch_begin(xb, yb)
--> 101 loss = loss_batch(learn.model, xb, yb, learn.loss_func, learn.opt, cb_handler)
102 if cb_handler.on_batch_end(loss): break
103
/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
28
29 if not loss_func: return to_detach(out), to_detach(yb[0])
---> 30 loss = loss_func(out, *yb)
31
32 if opt is not None:
/usr/local/lib/python3.6/dist-packages/fastai/layers.py in __call__(self, input, target, **kwargs)
241 if self.floatify: target = target.float()
242 input = input.view(-1,input.shape[-1]) if self.is_2d else input.view(-1)
--> 243 return self.func.__call__(input, target.view(-1), **kwargs)
244
245 def CrossEntropyFlat(*args, axis:int=-1, **kwargs):
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
539 result = self._slow_forward(*input, **kwargs)
540 else:
--> 541 result = self.forward(*input, **kwargs)
542 for hook in self._forward_hooks.values():
543 hook_result = hook(self, input, result)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py in forward(self, input, target)
914 def forward(self, input, target):
915 return F.cross_entropy(input, target, weight=self.weight,
--> 916 ignore_index=self.ignore_index, reduction=self.reduction)
917
918
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
2007 if size_average is not None or reduce is not None:
2008 reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2009 return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
2010
2011
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
1836 .format(input.size(0), target.size(0)))
1837 if dim == 2:
-> 1838 ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
1839 elif dim == 4:
1840 ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed. at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:97
这也是我设置数据的方式：
data = (SegmentationItemList.from_df(img_df,IMAGE_PATH)
# import from df in greyscale ('L')
.split_by_rand_pct(valid_pct=0.15)
# 1/15 train/validation split
.label_from_func(get_mask, classes = array(['background','cell']))
# segmentation mask and classes
.transform(tfms, tfm_y=True, size=TILE_SHAPE)
# apply data augmentation
.databunch(bs=BATCH_SIZE)
# set batchsize
.normalize()
)
如果您需要更多信息，请告诉我。我已经尝试添加一个 'after_open' 函数，它应该将 divided 全部乘以 255 到 'label_from_func' 部分。我也知道在 fast-ai 的 'open_image' 函数中有一个 div 属性，它应该将 RGB 值归一化在 0 和 1 之间，但我找不到 'label_from_func' .
编辑：
我在 fastai 社区找到了 this post。然而，即使有了这些答案，我也无法解决我的问题。我尝试添加此代码段以将 div=True 传递给 open_mask 函数，但它不起作用：
src.train.y.create_func = partial(open_mask, div=True)
src.valid.y.create_func = partial(open_mask, div=True)
我也在 .label_from_func() 之后尝试了 .set_attr(mask_opener=partial(open_mask, div=True))，但是它抛出了这个属性错误：AttributeError: setattr
仍然需要帮助
下面的自定义 类 是处理使用 0 和 255 编码掩码的二进制图像分割数据集所必需的
class SegLabelListCustom(SegmentationLabelList):
def open(self, fn): return open_mask(fn, div=True)
class SegItemListCustom(SegmentationItemList):
_label_cls = SegLabelListCustom
Link供参考：https://github.com/fastai/fastai/issues/1540
下面是使用这些自定义 类 为数据束创建源的示例。
src = (SegItemListCustom.from_folder('/home/jupyter/AerialImageDataset/train/')
.split_by_folder(train='images', valid='validate')
.label_from_func(get_y_fn, classes=labels))
我真的希望这对你有所帮助，因为不久前我自己也在努力处理这个问题，这就是我的解决方案。这很困难，因为我找到的许多答案都是针对以前版本的，不再有效。
如果您需要更多说明或帮助，请告诉我，因为我知道早期被卡住是多么令人沮丧。
我一定是在使用 fast-ai 库时遇到了严重的问题，因为我似乎是唯一一个遇到这个问题的人。每次我尝试学习率查找器或训练网络时，它都会给我一个错误。我花了一周的时间才产生这个特定的错误消息，这让我检查了掩码值。事实证明，背景像素为 0，前景像素为 255。这是个问题，因为我只有两个 类。如何在我的 Databunch 对象中将 255 个值更改为 1？有没有办法 divide 每个掩码值增加 255 还是我需要以某种方式事先做？我有点迷失在这个过程中。
这是我收到的错误消息：
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-20-c7a9c29f9dd1> in <module>()
----> 1 learn.lr_find()
2 learn.recorder.plot()
8 frames
/usr/local/lib/python3.6/dist-packages/fastai/train.py in lr_find(learn, start_lr, end_lr, num_it, stop_div, wd)
39 cb = LRFinder(learn, start_lr, end_lr, num_it, stop_div)
40 epochs = int(np.ceil(num_it/len(learn.data.train_dl)))
---> 41 learn.fit(epochs, start_lr, callbacks=[cb], wd=wd)
42
43 def to_fp16(learn:Learner, loss_scale:float=None, max_noskip:int=1000, dynamic:bool=True, clip:float=None,
/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
198 else: self.opt.lr,self.opt.wd = lr,wd
199 callbacks = [cb(self) for cb in self.callback_fns + listify(defaults.extra_callback_fns)] + listify(callbacks)
--> 200 fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
201
202 def create_opt(self, lr:Floats, wd:Floats=0.)->None:
/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py in fit(epochs, learn, callbacks, metrics)
99 for xb,yb in progress_bar(learn.data.train_dl, parent=pbar):
100 xb, yb = cb_handler.on_batch_begin(xb, yb)
--> 101 loss = loss_batch(learn.model, xb, yb, learn.loss_func, learn.opt, cb_handler)
102 if cb_handler.on_batch_end(loss): break
103
/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
28
29 if not loss_func: return to_detach(out), to_detach(yb[0])
---> 30 loss = loss_func(out, *yb)
31
32 if opt is not None:
/usr/local/lib/python3.6/dist-packages/fastai/layers.py in __call__(self, input, target, **kwargs)
241 if self.floatify: target = target.float()
242 input = input.view(-1,input.shape[-1]) if self.is_2d else input.view(-1)
--> 243 return self.func.__call__(input, target.view(-1), **kwargs)
244
245 def CrossEntropyFlat(*args, axis:int=-1, **kwargs):
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
539 result = self._slow_forward(*input, **kwargs)
540 else:
--> 541 result = self.forward(*input, **kwargs)
542 for hook in self._forward_hooks.values():
543 hook_result = hook(self, input, result)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py in forward(self, input, target)
914 def forward(self, input, target):
915 return F.cross_entropy(input, target, weight=self.weight,
--> 916 ignore_index=self.ignore_index, reduction=self.reduction)
917
918
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
2007 if size_average is not None or reduce is not None:
2008 reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2009 return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
2010
2011
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
1836 .format(input.size(0), target.size(0)))
1837 if dim == 2:
-> 1838 ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
1839 elif dim == 4:
1840 ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed. at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:97
这也是我设置数据的方式：
data = (SegmentationItemList.from_df(img_df,IMAGE_PATH)
# import from df in greyscale ('L')
.split_by_rand_pct(valid_pct=0.15)
# 1/15 train/validation split
.label_from_func(get_mask, classes = array(['background','cell']))
# segmentation mask and classes
.transform(tfms, tfm_y=True, size=TILE_SHAPE)
# apply data augmentation
.databunch(bs=BATCH_SIZE)
# set batchsize
.normalize()
)
如果您需要更多信息，请告诉我。我已经尝试添加一个 'after_open' 函数，它应该将 divided 全部乘以 255 到 'label_from_func' 部分。我也知道在 fast-ai 的 'open_image' 函数中有一个 div 属性，它应该将 RGB 值归一化在 0 和 1 之间，但我找不到 'label_from_func' .
编辑：
我在 fastai 社区找到了 this post。然而，即使有了这些答案，我也无法解决我的问题。我尝试添加此代码段以将 div=True 传递给 open_mask 函数，但它不起作用：
src.train.y.create_func = partial(open_mask, div=True)
src.valid.y.create_func = partial(open_mask, div=True)
我也在 .label_from_func() 之后尝试了 .set_attr(mask_opener=partial(open_mask, div=True))，但是它抛出了这个属性错误：AttributeError: setattr
仍然需要帮助
下面的自定义 类 是处理使用 0 和 255 编码掩码的二进制图像分割数据集所必需的
class SegLabelListCustom(SegmentationLabelList):
def open(self, fn): return open_mask(fn, div=True)
class SegItemListCustom(SegmentationItemList):
_label_cls = SegLabelListCustom
Link供参考：https://github.com/fastai/fastai/issues/1540
下面是使用这些自定义 类 为数据束创建源的示例。
src = (SegItemListCustom.from_folder('/home/jupyter/AerialImageDataset/train/')
.split_by_folder(train='images', valid='validate')
.label_from_func(get_y_fn, classes=labels))
我真的希望这对你有所帮助，因为不久前我自己也在努力处理这个问题，这就是我的解决方案。这很困难，因为我找到的许多答案都是针对以前版本的，不再有效。
如果您需要更多说明或帮助，请告诉我，因为我知道早期被卡住是多么令人沮丧。