Comments (5)
I also think changing parts of a pretrained model would be nice to have! One suggestions would be to make the config file for finetuning look something like:
exp1:
train:
model: !PretrainedModel
location: /path/to/pretrained.mod
overwrite:
- name: translator
value: !TranslatorReinforceLoss {}
- name: encoder.modules.0.dropout
value: 0.1
So we would need to load the serialized YAML file (without initializing components), adjust the parts that are overwritten, and then initialize everything. This will get more complicated once we change components that have DyNet parameters, but I guess we could ignore that case for the first attempt..
from xnmt.
I think these are two different things. I totally agree with @psjanani 's decoupling of the training strategy and the model. @msperber 's overwriting part of the model also sounds like it might be useful, but it'd be more work. I say we do the former immediately, and think about how to do the latter for the future (when it becomes necessary).
from xnmt.
If there is a way to overwrite or initialize the loaded model, we won't have to change the design as much. I actually have the same problem but in other things.
from xnmt.
Sure, I think it would be useful to be able to do this. But it's also important for the training strategy and model to not be conflated.
from xnmt.
Awesome, I can work on decoupling training strategy and model if it is alright.
And overwriting the model would be really useful. I have been using not so pretty hacks to achieve that till now.
from xnmt.
Related Issues (20)
- Search Strategies assume decoder supports calc_log_probs HOT 2
- Overabundance of forced decoding
- Stative Attenders HOT 3
- Post Processes on Dev/Test References HOT 1
- Recipe las-pyramidal.yaml does not work HOT 13
- @register_xnmt_event problems
- Problem while using SAAMSeqTransducer HOT 6
- Confusing error message when YAML file doesn't exist
- Output silently fails when path contains ~
- How to load an existing trained model and make predictions on the unseen data? HOT 1
- dy.sigmoid doesn't exist HOT 1
- Pyyaml Error when running example 19, subword sample
- how can I use for character-level sequences?
- Running 16_transformer.yaml: AssertionError: cannot pass dy.ParameterCollection to a Serializable class. HOT 1
- Segment fatal (core dump) using 3.8.0 HOT 1
- Problem running GPU version on RTX 2070 with 8GB Memory HOT 1
- How to train XNMT with datasets that do not fit in RAM? HOT 1
- How to use already tokenized BPE text and BPE vocab with xnmt ?
- Error when running experiments
- Can't run test/config/transformer.yaml
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xnmt.