Gensim fasttext 无法获得最新的训练损失

Question

问题描述

好像fasttext中的get_latest_training_loss函数returns只有0个。gensim 4.1.0和都是4.0.0 不工作。

from gensim.models.callbacks import CallbackAny2Vec
from pprint import pprint as print
from gensim.models.fasttext import FastText
from gensim.test.utils import datapath

class callback(CallbackAny2Vec):
    '''Callback to print loss after each epoch.'''

    def __init__(self):
        self.epoch = 0

    def on_epoch_end(self, model):
        loss = model.get_latest_training_loss()
        print('Loss after epoch {}: {}'.format(self.epoch, loss))
        self.epoch += 1

# Set file names for train and test data
corpus_file = datapath('lee_background.cor')

model = FastText(vector_size=100, callbacks=[callback()])

# build the vocabulary
model.build_vocab(corpus_file=corpus_file)

# train the model
model.train(
    corpus_file=corpus_file, epochs=model.epochs,
    total_examples=model.corpus_count, total_words=model.corpus_total_words,
    callbacks=model.callbacks, compute_loss=True,
)

print(model)

'Loss after epoch 0: 0.0'
'Loss after epoch 1: 0.0'
'Loss after epoch 2: 0.0'
'Loss after epoch 3: 0.0'
'Loss after epoch 4: 0.0'

如果目前FastText不支持get_latest_training_loss，需要去掉这里的文档：

https://radimrehurek.com/gensim/models/fasttext.html#gensim.models.fasttext.FastText.get_latest_training_loss

版本

我已经在三个不同的环境中尝试过，但都不起作用。

第一环境：

[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform; print(platform.platform())
Linux-3.10.0-1160.36.2.el7.x86_64-x86_64-with-glibc2.17
>>> import sys; print("Python", sys.version)
Python 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:39:48)
[GCC 9.3.0]
>>> import struct; print("Bits", 8 * struct.calcsize("P"))
Bits 64
>>> import numpy; print("NumPy", numpy.__version__)
NumPy 1.21.2
>>> import scipy; print("SciPy", scipy.__version__)
SciPy 1.7.1
>>> import gensim; print("gensim", gensim.__version__)
gensim 4.1.0
>>> from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)
FAST_VERSION 0

第二环境：

Python 3.9.5 (default, May 18 2021, 12:31:01)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform; print(platform.platform())
macOS-10.16-x86_64-i386-64bit
>>> import sys; print("Python", sys.version)
Python 3.9.5 (default, May 18 2021, 12:31:01)
[Clang 10.0.0 ]
>>> import struct; print("Bits", 8 * struct.calcsize("P"))
Bits 64
>>> import numpy; print("NumPy", numpy.__version__)
NumPy 1.20.3
>>> import scipy; print("SciPy", scipy.__version__)
SciPy 1.7.1
>>> import gensim; print("gensim", gensim.__version__)
gensim 4.1.0
>>> from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)
FAST_VERSION 0

第三环境：

Python 3.9.5 (default, May 18 2021, 12:31:01)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform; print(platform.platform())
macOS-10.16-x86_64-i386-64bit
>>> import sys; print("Python", sys.version)
Python 3.9.5 (default, May 18 2021, 12:31:01)
[Clang 10.0.0 ]
>>> import struct; print("Bits", 8 * struct.calcsize("P"))
Bits 64
>>> import numpy; print("NumPy", numpy.__version__)
NumPy 1.20.3
>>> import scipy; print("SciPy", scipy.__version__)
SciPy 1.7.1
>>> import gensim; print("gensim", gensim.__version__)
/Users/jinhuawang/miniconda3/lib/python3.9/site-packages/gensim/similarities/__init__.py:15: UserWarning: The gensim.similarities.levenshtein submodule is disabled, because the optional Levenshtein package <https://pypi.org/project/python-Levenshtein/> is unavailable. Install Levenhstein (e.g. `pip install python-Levenshtein`) to suppress this warning.
  warnings.warn(msg)
gensim 4.0.0
>>> from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)
FAST_VERSION 0

Answer 1

事实上，损失跟踪从未在 Gensim 的 FastText 模型中实现，至少在 4.1.0 版（2021 年 8 月）之前是这样。

该方法的文档出现错误，因为从 Word2Vec 超类继承的方法没有被覆盖以防止超类方法工作的默认假设。

有一个 long-open issue 可以填补空白并修复 Gensim 损失跟踪中的问题（Word2Vec 也有一些错误和不完整）。但是，目前我认为没有任何贡献者在为此工作，而且它还没有被优先用于任何即将发布的版本。它可能需要有人自愿挺身而出并解决问题。

Gensim fasttext 无法获得最新的训练损失

Gensim fasttext cannot get latest training loss

python

nlp

gensim

word2vec

fasttext

问题描述

版本