compound-pcfg's Issues
guide papers to understanding
Hi @yoonkim
Could you introduce me to the guide papers to understand the paper (not all the references)? I have been tried several times to understand your PCFG paper, but it is not easy.
(I know studying all the references of this paper is the best way to understand that.. However, I'm just a beginner who studies the CS224N which is an NLP course at Stanford University. Reading all the references for each reference paper is almost impossible for me.)
If you don't mind, could you share the experiences about OpenNMT to me? I hope to know what am I have to do if I want to make Speech recognition and NMT in the future.
Thank you
No root symbol in MAP parse trees
The root symbol is not considered when getting the MAP parse tree both in the paper and in the code:
Lines 176 to 189 in 1c0078c
But according to the Viterbi algorithm and the majority of golden parse trees in the treebank, there should be a root symbol (although I haven't looked at the Viterbi implementation here in PCFG.py
). Why don't we have root symbol in MAP trees?
a fast implementation of C-PCFGs
Hi Yoon! Thank you for sharing the amazing code. I re-implemented C-PCFGs based on Torch-Struct. It is faster (~25min / epoch) and slightly more accurate (~55.7% F1). I am happy to share it. It may help people explore the potential of C-PCFGs more easily. Here is my implementation.
-Yanpeng
Inquiry on some minor implementation details
Hello, thanks for sharing your code, first of all, which is well-structured and easy to understand.
I have a few questions about the details of your implementation.
- In your paper, you said
We employ a curriculum learning strategy (Bengio et al., 2009) where we train only on sentences of length up to 30 in the first epoch, and increase this length limit by 1 each epoch.
But in the code, I found that the maximum of the lengths of sentences increases as
30 -> 40 -> 41 -> ... even though I expected this should change like 30 -> 31 -> 32 -> ...
Maybe this undesirable? working seems coming from the below line in train.py.
Line 130 in 43359b8
Is there any reason you used max instead of min?
It seems right to change the max function into the min function on the basis of your paper.
- And, I'm just curious about why we should consider the EOS token ('</s>') in this model by explicitly adding 1 to the number of words in a sentence as below.
Line 105 in 43359b8
Thanks again for opening your code!
Extra indents in PCFG.py?
Are these extra indents in line 70-71 in PCFG.py?
Lines 70 to 72 in 47a7b16
Should it be:
log_Z = self.beta[:, 0, n-1, :self.nt_states] + root_scores
log_Z = self.logsumexp(log_Z, 1)
return log_Z
How to get PCFG rules and rule probabilities?
Is there a way to get the whole set of PCFG rules and rule probabilities while training the model? I found there's a tensor called rule_scores
with shape b x NT x (NT+T) X (NT+T)
, but I don't see how to get rules and rule probabilities from there, and where are rules going from pre-terminals to terminals.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.