僧伽罗语实体提取失败

Question

尝试使用 RASA NLU 为僧伽罗语开发聊天机器人。

我的config.yml

pipeline:
- name: "WhitespaceTokenizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "CountVectorsFeaturizer"
- name: "EmbeddingIntentClassifier"

并且在 data.json 中，我添加了如下示例数据。当我训练 nlu 模型并尝试提取样本输入时，“සිංහලෙන්”作为媒介，它只输出意图和实体值，而不输出实体。我做错了什么？

{
          "text": "සිංහලෙන් දේශන පවත්වන්නේ නැද්ද?",
          "intent": "ask_medium",
          "entities": [{
                "start":0,
                "end":8,
                "value": "සිංහලෙන්",
                "entity": "medium"
          }]
        },
        {
          "text": "සිංහලෙන් lectures කරන්නේ නැද්ද?",
          "intent": "ask_medium",
          "entities": [{
                "start":0,
                "end":8,
                "value": "සිංහලෙන්",
                "entity": "medium"
          }]
        }

我在测试 nlu 模型时得到的响应是

{'intent': 
{'name': 'ask_langmedium', 'confidence': 0.9747527837753296}, 'entities': 
[{'start': 10, 
'end': 18, 
'value': 'සිංහලෙන්',
'entity': '-', 
'confidence': 0.5970129041418675,
'extractor': 'CRFEntityExtractor'}], 
'intent_ranking': [
{'name': 'ask_langmedium', 'confidence': 0.9747527837753296}, 
{'name': 'ask_langmedium_request_possibility', 'confidence': 
0.07433460652828217}],
'text': 'උගන්නන්නේ සිංහලෙන් ද ?'}

Answer 1

如果这是您完成的数据集，那么我不确定您是如何生成模型的，因为 rasa 至少需要两个意图。我用 hello 和其余数据添加了另一个意图我只是在我自己的代码中复制了你的数据并且效果很好，这就是我得到的输出。

Enter a message: උගන්නන්නේ සිංහලෙන් ද?
{
  "intent": {
    "name": "ask_medium",
    "confidence": 0.9638749361038208
  },
  "entities": [
    {
      "start": 10,
      "end": 18,
      "value": "\u0dc3\u0dd2\u0d82\u0dc4\u0dbd\u0dd9\u0db1\u0dca",
      "entity": "medium",
      "confidence": 0.7177257810884379,
      "extractor": "CRFEntityExtractor"
    }
  ]
}

这是我的完整代码

DataSet.json

{
    "rasa_nlu_data": {
        "common_examples": [
            {
                "text": "හෙලෝ",
                "intent": "hello",
                "entities": []
            },
            {
                "text": "සිංහලෙන් දේශන පවත්වන්නේ නැද්ද?",
                "intent": "ask_medium",
                "entities": [{
                      "start":0,
                      "end":8,
                      "value": "සිංහලෙන්",
                      "entity": "medium"
                }]
            },
            {
                "text": "සිංහලෙන් lectures කරන්නේ නැද්ද?",
                "intent": "ask_medium",
                "entities": [{
                      "start":0,
                      "end":8,
                      "value": "සිංහලෙන්",
                      "entity": "medium"
                }]
            }
        ],
        "regex_features" : [],
        "lookup_tables"  : [],
        "entity_synonyms": []
    }
}

nlu_config.yml

pipeline: "supervised_embeddings"

训练指挥

python -m rasa_nlu.train -c ./config/nlu_config.yml --data ./data/sh_data.json -o models --fixed_model_name nlu --project current --verbose

&testing.py

from rasa_nlu.model import Interpreter
import json

interpreter = Interpreter.load('./models/current/nlu')


def predict_intent(text):
    results = interpreter.parse(text)
    print(json.dumps({
        "intent": results["intent"],
        "entities": results["entities"]
    }, indent=2))


keep_asking = True
while(keep_asking):
    text = input('Enter a message: ')
    if (text == 'exit'):
        keep_asking = False
        break
    else:
        predict_intent(text)

僧伽罗语实体提取失败

Entity Extraction fails for Sinhala Language

rasa-nlu