如何使用 TF Hub 模块找到最接近的单词?
How find closest word using TF Hub module?
如果 np.inner 与嵌入之间 return 它们的相似性。
如何从一个词嵌入中找到最接近的词?
我正在使用 Wiki 模块。
您可以这样做:
# Download 10k most popular english words.
import urllib2
response = urllib2.urlopen('https://raw.githubusercontent.com/first20hours/google-10000-english/master/google-10000-english-no-swears.txt')
words = response.read().split("\n")
# Compute the embedding for 10k words.
with tf.Graph().as_default():
embed = hub.Module("https://tfhub.dev/google/Wiki-words-250/1")
embeddings = embed(words)
with tf.train.MonitoredSession() as sess:
values = sess.run(embeddings)
data = dict(zip(words, values))
def get_neighbors(all_data, target, count=3):
# Compute points in all_data that are the closest to "target".
# Sort the words based on the length of the vector between them.
# Then pick "count" closest ones.
return sorted(all_data, key=lambda emb_key: np.linalg.norm(all_data[emb_key] - target))[:count]
# Then make queries of your choice, e.g.
print(get_neighbors(data, data["first"]))
# Prints ['first', 'second', 'third']
print(get_neighbors(data, data["woman"] - data["man"] + data["father"], count=10))
# Prints ['mother', 'father', 'wife', 'daughter', 'wed', 'husband', 'uncle', 'son', 'child', 'mistress']
如果 np.inner 与嵌入之间 return 它们的相似性。 如何从一个词嵌入中找到最接近的词? 我正在使用 Wiki 模块。
您可以这样做:
# Download 10k most popular english words.
import urllib2
response = urllib2.urlopen('https://raw.githubusercontent.com/first20hours/google-10000-english/master/google-10000-english-no-swears.txt')
words = response.read().split("\n")
# Compute the embedding for 10k words.
with tf.Graph().as_default():
embed = hub.Module("https://tfhub.dev/google/Wiki-words-250/1")
embeddings = embed(words)
with tf.train.MonitoredSession() as sess:
values = sess.run(embeddings)
data = dict(zip(words, values))
def get_neighbors(all_data, target, count=3):
# Compute points in all_data that are the closest to "target".
# Sort the words based on the length of the vector between them.
# Then pick "count" closest ones.
return sorted(all_data, key=lambda emb_key: np.linalg.norm(all_data[emb_key] - target))[:count]
# Then make queries of your choice, e.g.
print(get_neighbors(data, data["first"]))
# Prints ['first', 'second', 'third']
print(get_neighbors(data, data["woman"] - data["man"] + data["father"], count=10))
# Prints ['mother', 'father', 'wife', 'daughter', 'wed', 'husband', 'uncle', 'son', 'child', 'mistress']