如何使用 ktrain 包中的 learner.lr_plot 理解损失学习率(对数刻度)图?
How to understand loss-learning rate (log scale) plot using learner.lr_plot in ktrain package?
我正在使用 ktrain 包对文本进行分类。我的实验显示为:
lr_find 和 lr_plot 是 ktrain 中的函数。它们可用于突出显示最佳学习率,在图中显示为红点。
我不明白这个剧情怎么看:
- 如何将对数刻度转换为正常的线性刻度?
- 为什么最好的刻度是红点?
正如 lr_find
方法中的文字所说,您可以目视检查该图并在损失在发散之前下降的范围内选择学习率。在这个范围内学习率越高,收敛得越快。这是来自 Leslie Smith's paper that became popular through the fastai library and was later adopted by other libraries like ktrain and Amazon's Gluon library. The red dot in this plot is just a numerical approximation of where the loss is dramatically falling that may be useful for automated scenarios, but not necessarily the best. In this plot, the red dot represents the steepest part of the curve, which is one strategy to automatically select a learning rate from the plot (without visual inspection). Other automated strategies include taking the learning rate associated with the minimum loss and dividing by 10, and finding the learning rate associated with the longest valley.
的一个称为“LR 范围测试”的想法
我正在使用 ktrain 包对文本进行分类。我的实验显示为:
lr_find 和 lr_plot 是 ktrain 中的函数。它们可用于突出显示最佳学习率,在图中显示为红点。
我不明白这个剧情怎么看:
- 如何将对数刻度转换为正常的线性刻度?
- 为什么最好的刻度是红点?
正如 lr_find
方法中的文字所说,您可以目视检查该图并在损失在发散之前下降的范围内选择学习率。在这个范围内学习率越高,收敛得越快。这是来自 Leslie Smith's paper that became popular through the fastai library and was later adopted by other libraries like ktrain and Amazon's Gluon library. The red dot in this plot is just a numerical approximation of where the loss is dramatically falling that may be useful for automated scenarios, but not necessarily the best. In this plot, the red dot represents the steepest part of the curve, which is one strategy to automatically select a learning rate from the plot (without visual inspection). Other automated strategies include taking the learning rate associated with the minimum loss and dividing by 10, and finding the learning rate associated with the longest valley.