Code Monkey home page Code Monkey logo

Comments (3)

nikitakit avatar nikitakit commented on June 3, 2024 1

This is a known limitation with how the parser is currently formulated: each tree must contain a non-terminal node at the root. In your example above, the "part-of-speech" tag PU is the root node, which breaks that assumption.

The way I usually work around this is by wrapping all trees with a "TOP" symbol for the root node, and then disabling the "strip_top" flag when the treebank is known to contain examples that don't have a root non-terminal label.

from self-attentive-parser.

nikitakit avatar nikitakit commented on June 3, 2024 1

It looks like you're using the English ELMo embeddings to learn a parser for Chinese, which works poorly due to the language mismatch. I recommend that you use the Chinese version of BERT instead. Here is the command needed for training the model: https://github.com/nikitakit/self-attentive-parser/blob/master/EXPERIMENTS.md#chinese-models

It also looks like our dataset sizes don't match. See here for CTB data preparation scripts. My data sizes are 17544 train / 352 dev / 348 test.

from self-attentive-parser.

xiaoxiaoAurora avatar xiaoxiaoAurora commented on June 3, 2024

This is a known limitation with how the parser is currently formulated: each tree must contain a non-terminal node at the root. In your example above, the "part-of-speech" tag PU is the root node, which breaks that assumption.

The way I usually work around this is by wrapping all trees with a "TOP" symbol for the root node, and then disabling the "strip_top" flag when the treebank is known to contain examples that don't have a root non-terminal label.

Great! Thank you very much! That is very helpful.
However, When I tried to train the model on CTB data as the following comand:

python src/main.py train --use-elmo --model-path-base models/ctb_elmo --num-layers 4 --epochs 500

I got a result:

epoch 134 batch 112/112 processed 2,985,118 batch-loss 6.4597 grad-norm 17.0195 epoch-elapsed 0h05m52s total-elapsed 13h54m52s  
dev-fscore (Recall=84.26, Precision=85.77, FScore=85.01, CompleteMatch=19.32) dev-elapsed 0h00m15s total-elapsed 13h55m07s  
Terminating due to lack of improvement in dev fscore.

and this score is less than your "91.69 F1 on CTB 5.1 test set in benepar_zh"
my CTB data size:

  • train data: 22277
  • dev data: 1761

can you provide parameter settings in detail?

from self-attentive-parser.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.