在 stanford-nlp 中训练 NER 模型
Training NER model in stanford-nlp
我一直在尝试使用 stanford Core NLP。我希望训练我自己的 NER 模型。来自 SO 的论坛和官方网站描述了使用 属性 文件来执行此操作。我将如何通过 API 来完成?
Properties props = new Properties();
props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner, parse, sentiment, regexner");
props.setProperty("regexner.mapping", "resources/customRegexNER.txt");
StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
String processedQuestion = "Who is the prime minister of Australia?"
//Annotation annotation = pipeline.process(processedQuestion);
Annotation document = new Annotation(processedQuestion);
pipeline.annotate(document);
List<CoreMap> sentences = document.get(SentencesAnnotation.class);
for (CoreMap sentence : sentences) {
// To get the tokens for the parsed sentence
for (CoreMap tokens : sentence.get(TokensAnnotation.class)) {
String token = tokens.get(TextAnnotation.class);
String POS = tokens.get(PartOfSpeechAnnotation.class);
String NER = tokens.get(NamedEntityTagAnnotation.class);
String Sentiment = tokens.get(SentimentClass.class);
String lemma = tokens.get(LemmaAnnotation.class);
- 如何以及在何处添加 Prop 文件?
- N-gram 标记化(例如,总理被视为单个标记,稍后将此标记传递给 POS,NER 而不是传递两个标记(总理和部长))?
我认为它可以使用该代码:
val props = new Properties()
props.put("annotators", "tokenize, ssplit, pos, lemma, ner, regexner")
props.put("ner.model", "/your/path/ner-model.ser.gz");
val pipeline = new StanfordCoreNLP(props)
我一直在尝试使用 stanford Core NLP。我希望训练我自己的 NER 模型。来自 SO 的论坛和官方网站描述了使用 属性 文件来执行此操作。我将如何通过 API 来完成?
Properties props = new Properties();
props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner, parse, sentiment, regexner");
props.setProperty("regexner.mapping", "resources/customRegexNER.txt");
StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
String processedQuestion = "Who is the prime minister of Australia?"
//Annotation annotation = pipeline.process(processedQuestion);
Annotation document = new Annotation(processedQuestion);
pipeline.annotate(document);
List<CoreMap> sentences = document.get(SentencesAnnotation.class);
for (CoreMap sentence : sentences) {
// To get the tokens for the parsed sentence
for (CoreMap tokens : sentence.get(TokensAnnotation.class)) {
String token = tokens.get(TextAnnotation.class);
String POS = tokens.get(PartOfSpeechAnnotation.class);
String NER = tokens.get(NamedEntityTagAnnotation.class);
String Sentiment = tokens.get(SentimentClass.class);
String lemma = tokens.get(LemmaAnnotation.class);
- 如何以及在何处添加 Prop 文件?
- N-gram 标记化(例如,总理被视为单个标记,稍后将此标记传递给 POS,NER 而不是传递两个标记(总理和部长))?
我认为它可以使用该代码:
val props = new Properties()
props.put("annotators", "tokenize, ssplit, pos, lemma, ner, regexner")
props.put("ner.model", "/your/path/ner-model.ser.gz");
val pipeline = new StanfordCoreNLP(props)