Comments (5)
HHousen
commented on May 25, 2024
Discussion began in #25.
from transformersum.
moyid
commented on May 25, 2024
Interesting. So do you plan on implementing it in the future?
from transformersum.
HHousen
commented on May 25, 2024
Interesting. So do you plan on implementing it in the future?
@moyid Yes I will implement this. I'm just not sure when I'll have the time.
from transformersum.
HHousen
commented on May 25, 2024
Profiler reports of the new vs old dataloading methods:
Profiler Report - New Dataloading
Action | Mean duration (s) | Total time (s)
-----------------------------------------------------------------
on_fit_start | 3.182e-05 | 3.182e-05
on_validation_start | 0.00015438 | 0.00030876
on_validation_epoch_start | 1.2273e-05 | 2.4545e-05
on_validation_batch_start | 1.8387e-05 | 0.01653
validation_step_end | 1.8699e-05 | 0.01681
on_validation_batch_end | 2.6562e-05 | 0.02388
on_validation_epoch_end | 1.8851e-05 | 3.7701e-05
on_validation_end | 2.4432 | 4.8864
on_train_start | 0.0011054 | 0.0011054
on_epoch_start | 0.00034487 | 0.00034487
on_train_epoch_start | 4.2872e-05 | 4.2872e-05
get_train_batch | 0.0061234 | 5.4927
on_batch_start | 2.6541e-05 | 0.023807
on_train_batch_start | 2.0627e-05 | 0.018503
training_step_end | 1.9472e-05 | 0.017466
model_forward | 0.29406 | 263.77
model_backward | 0.52279 | 468.94
on_after_backward | 1.7245e-05 | 0.015468
on_batch_end | 2.3942e-05 | 0.021476
on_train_batch_end | 2.8652e-05 | 0.025701
optimizer_step | 0.82151 | 368.86
on_epoch_end | 1.8765e-05 | 1.8765e-05
on_train_epoch_end | 1.0901e-05 | 1.0901e-05
on_train_end | 0.00030913 | 0.00030913
Profiler Report - New Dataloading with 4 Workers
Action | Mean duration (s) | Total time (s)
-----------------------------------------------------------------
on_fit_start | 4.8096e-05 | 4.8096e-05
on_validation_start | 0.00016712 | 0.00033424
on_validation_epoch_start | 2.1737e-05 | 4.3473e-05
on_validation_batch_start | 1.8466e-05 | 0.016601
validation_step_end | 2.1195e-05 | 0.019055
on_validation_batch_end | 2.8794e-05 | 0.025886
on_validation_epoch_end | 2.676e-05 | 5.352e-05
on_validation_end | 2.5799 | 5.1598
on_train_start | 0.000615 | 0.000615
on_epoch_start | 0.00031976 | 0.00031976
on_train_epoch_start | 2.731e-05 | 2.731e-05
get_train_batch | 0.0030533 | 2.7388
on_batch_start | 2.4564e-05 | 0.022034
on_train_batch_start | 2.0619e-05 | 0.018496
training_step_end | 1.7099e-05 | 0.015337
model_forward | 0.294 | 263.71
model_backward | 0.52549 | 471.36
on_after_backward | 2.0427e-05 | 0.018323
on_batch_end | 2.2888e-05 | 0.02053
on_train_batch_end | 2.7878e-05 | 0.025006
optimizer_step | 0.82418 | 370.06
on_epoch_end | 1.8855e-05 | 1.8855e-05
on_train_epoch_end | 1.3511e-05 | 1.3511e-05
on_train_end | 0.00064615 | 0.00064615
Profiler Report - Old Dataloading
Action | Mean duration (s) | Total time (s)
-----------------------------------------------------------------
on_fit_start | 3.0753e-05 | 3.0753e-05
on_validation_start | 0.00019413 | 0.00038826
on_validation_epoch_start | 1.2077e-05 | 2.4155e-05
on_validation_batch_start | 1.6663e-05 | 0.01498
validation_step_end | 1.6222e-05 | 0.014584
on_validation_batch_end | 2.3677e-05 | 0.021285
on_validation_epoch_end | 2.2077e-05 | 4.4154e-05
on_validation_end | 2.4987 | 4.9973
on_train_start | 0.00050591 | 0.00050591
on_epoch_start | 0.00025153 | 0.00025153
on_train_epoch_start | 1.6057e-05 | 1.6057e-05
get_train_batch | 0.0053287 | 4.7799
on_batch_start | 2.6401e-05 | 0.023682
on_train_batch_start | 2.0031e-05 | 0.017968
training_step_end | 1.6952e-05 | 0.015206
model_forward | 0.29384 | 263.57
model_backward | 0.5215 | 467.79
on_after_backward | 2.1365e-05 | 0.019164
on_batch_end | 2.3954e-05 | 0.021487
on_train_batch_end | 2.6314e-05 | 0.023603
optimizer_step | 0.82007 | 368.21
on_epoch_end | 3.2557e-05 | 3.2557e-05
on_train_epoch_end | 1.1635e-05 | 1.1635e-05
on_train_end | 0.00037779 | 0.00037779
from transformersum.
HHousen
commented on May 25, 2024
Added in commit f383286. Closing.
from transformersum.
Related Issues (20)
- TypeError: __init__() got an unexpected keyword argument 'gradient_checkpointing' HOT 1
- predictions_website.py raises AttributeError: '_LazyAutoMapping' object has no attribute '_mapping' HOT 6
- ModuleNotFoundError: No module named 'extractive' HOT 1
- AttributeError: '_LazyAutoMapping' object has no attribute '_mapping' HOT 1
- Abstractive BART Model , RuntimeError: The size of tensor a (64000) must match the size of tensor b (64001) at non-singleton dimension 1
- ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on. HOT 3
- error when training an extractive summarization model HOT 2
- Found keys that are in the model state dict but not in the checkpoint HOT 3
- Suggest about the index order of extractive results
- A Chinese solution for TransformerSum-extractive, and I've implemented your work in my project HOT 1
- After extractive training, a process on one GPU won't terminate automatically.
- Fine-tuning/Inference commands for "roberta-base-ext-sum"
- '--data_type' is not accepted when running main.py (extractive mode)
- Why tokenize twice?
- TypeError: forward() got an unexpected keyword argument 'source'
- Instruction for fine tune
- Installation via Pip
- Some versioning problems when installing the environment HOT 2
- pytorch_lightning.callbacks update HOT 1
- RoBERTa & Longformer extractive model checkpoints availability
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from transformersum.