Comments (6)
Hi, can you copy paste your yaml file here? I can try to take a look
from xnmt.
@philip30 I'm using the defalut configuration file provided by the article
http://www.msperber.com/research/self-att/index.html
but I changed tag TransformerSeqTransducer
to SAAMSeqTransducer
, and ContVecReader
to NpzReader
# purely self-att model, using concatenated 'trigonometric' positional encodings
exp029a.1: !Experiment
exp_global: !ExpGlobal
dropout: 0.2
default_layer_dim: 512
placeholders:
DATA_DIR: (removed)
model: !DefaultTranslator
src_embedder: !NoopEmbedder
emb_dim: 40
encoder: !ModularSeqTransducer
modules:
- !SAAMSeqTransducer
layers: 2
input_dim: 40
hidden_dim: 512
downsample_factor: 2
ff_hidden_dim: 512
pos_encoding_type: trigonometric
pos_encoding_combine: concat
pos_encoding_size: 40
max_len: 1500
- !SAAMSeqTransducer
layers: 1
input_dim: 512
hidden_dim: 512
downsample_factor: 1
ff_hidden_dim: 512
attender: !MlpAttender
hidden_dim: 128
trg_embedder: !SimpleWordEmbedder
emb_dim: 64
word_dropout: 0.1
fix_norm: 1
decoder: !MlpSoftmaxDecoder
layers: 1
lstm_dim: 512
input_feeding: True
bridge: !NoBridge {}
src_reader: !NpzReader
transpose: true
feat_from: 0
feat_to: 40
trg_reader: !PlainTextReader
vocab: !Vocab
vocab_file: '{DATA_DIR}/train/src.char.vocab.manual'
train: !SimpleTrainingRegimen
src_file: '{DATA_DIR}/train/src.contvec.npz'
trg_file: '{DATA_DIR}/train/src.char'
max_src_len: &max_src_len 1500
max_trg_len: 350
run_for_epochs: 500
batcher: !WordSrcBatcher
avg_batch_size: 18
pad_src_to_multiple: 4
src_pad_token: ~
trainer: !AdamTrainer
alpha: 0.0003
lr_decay: 0.5
lr_decay_times: 3
patience: 5
initial_patience: 10
dev_every: 0
restart_trainer: True
dev_tasks:
- !AccuracyEvalTask
eval_metrics: wer,cer
src_file: &dev_src '{DATA_DIR}/dev/src.contvec.npz'
ref_file: '{DATA_DIR}/dev/src.txt'
hyp_file: '{EXP_DIR}/models/{EXP}.dev_hyp'
inference: !SimpleInference
post_process: join-char
search_strategy: !BeamSearch
beam_size: 20
max_len: 500
len_norm: !PolynomialNormalization
apply_during_search: true
m: 1.5
- !LossEvalTask
src_file: *dev_src
ref_file: '{DATA_DIR}/dev/src.char'
evaluate:
- !AccuracyEvalTask
eval_metrics: wer,cer
src_file: &test_src '{DATA_DIR}/test/src.contvec.npz'
ref_file: '{DATA_DIR}/test/src.txt'
hyp_file: '{EXP_DIR}/models/{EXP}.test_hyp'
inference: !SimpleInference
post_process: join-char
search_strategy: !BeamSearch
beam_size: 20
max_len: 500
len_norm: !PolynomialNormalization
apply_during_search: true
m: 1.5
- !LossEvalTask
src_file: *test_src
ref_file: '{DATA_DIR}/test/src.char'
from xnmt.
Hmm this is very strange. Which revision of the xnmt are you using? I will try to reproduce the issue
from xnmt.
XNMT: 0.1.0 release (revision 0d094f5)
(I just tried newst revision a87e7b9 and the error is the same)
Python: 3.6.4 (I tried 3.6.8, same error - - )
OS: Ubuntu 16.04
from xnmt.
@philip30 I think I found the key.
When I set argument 'ff_lstm' to 'true', it works normally.
I'm trying to find out the reason.
from xnmt.
Thanks! I am closing this issue.
from xnmt.
Related Issues (20)
- Search Strategies assume decoder supports calc_log_probs HOT 2
- Overabundance of forced decoding
- Stative Attenders HOT 3
- Post Processes on Dev/Test References HOT 1
- Recipe las-pyramidal.yaml does not work HOT 13
- @register_xnmt_event problems
- Confusing error message when YAML file doesn't exist
- Output silently fails when path contains ~
- How to load an existing trained model and make predictions on the unseen data? HOT 1
- dy.sigmoid doesn't exist HOT 1
- Pyyaml Error when running example 19, subword sample
- how can I use for character-level sequences?
- Running 16_transformer.yaml: AssertionError: cannot pass dy.ParameterCollection to a Serializable class. HOT 1
- Segment fatal (core dump) using 3.8.0 HOT 1
- Problem running GPU version on RTX 2070 with 8GB Memory HOT 1
- How to train XNMT with datasets that do not fit in RAM? HOT 1
- How to use already tokenized BPE text and BPE vocab with xnmt ?
- Error when running experiments
- Can't run test/config/transformer.yaml
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xnmt.