Comments (3)
This problem is similar to #63, please try that solutions. You can set pool_size
as follows in dataprovider.py
to reduce data cache memory.
@provider(init_hook=hook, pool_size=1024)
from paddle.
I did what you advised ,but it still has problems
lbj@ubuntu:~/work/Paddle-master/demo/sentiment$ ./train.sh
I0912 07:26:11.295408 4219 Util.cpp:144] commandline: /usr/local/bin/../opt/paddle/bin/paddle_trainer --config=trainer_config.py --save_dir=./model_output --job=train --use_gpu=false --trainer_count=4 --num_passes=10 --log_period=10 --dot_period=20 --show_parameter_stats_period=100 --test_all_data_in_one_period=1
I0912 07:26:11.296186 4219 Util.cpp:113] Calling runInitFunctions
I0912 07:26:11.296751 4219 Util.cpp:126] Call runInitFunctions done.
[INFO 2016-09-12 07:26:12,275 networks.py:1122] The input order is [word, label]
[INFO 2016-09-12 07:26:12,275 networks.py:1129] The output order is [cost_0]
I0912 07:26:12.353705 4219 Trainer.cpp:169] trainer mode: Normal
I0912 07:26:13.966311 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,002 dataprovider.py:22] dict len : 101745
I0912 07:26:14.200013 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,218 dataprovider.py:22] dict len : 101745
I0912 07:26:14.218703 4219 GradientMachine.cpp:134] Initing parameters..
I0912 07:26:15.289685 4219 GradientMachine.cpp:141] Init parameters done.
I0912 07:26:19.343603 4220 ThreadLocal.cpp:39] thread use undeterministic rand seed:4221
I0912 07:26:19.514636 4221 ThreadLocal.cpp:39] thread use undeterministic rand seed:4222
I0912 07:26:19.536335 4222 ThreadLocal.cpp:39] thread use undeterministic rand seed:4223
I0912 07:26:19.704776 4223 ThreadLocal.cpp:39] thread use undeterministic rand seed:4224
I0912 07:29:23.252763 4219 TrainerInternal.cpp:162] Batch=10 samples=1280 AvgCost=0.692175 CurrentCost=0.692175 Eval: classification_error_evaluator=0.485156 CurrentEval: classification_error_evaluator=0.485156
I0912 07:32:37.542174 4219 TrainerInternal.cpp:162] Batch=20 samples=2560 AvgCost=0.666661 CurrentCost=0.641147 Eval: classification_error_evaluator=0.401172 CurrentEval: classification_error_evaluator=0.317188
I0912 07:35:50.129344 4219 TrainerInternal.cpp:162] Batch=30 samples=3840 AvgCost=0.626617 CurrentCost=0.54653 Eval: classification_error_evaluator=0.342448 CurrentEval: classification_error_evaluator=0.225
I0912 07:38:51.197893 4219 TrainerInternal.cpp:162] Batch=40 samples=5120 AvgCost=0.585538 CurrentCost=0.462299 Eval: classification_error_evaluator=0.307422 CurrentEval: classification_error_evaluator=0.202344
I0912 07:41:56.019923 4219 TrainerInternal.cpp:162] Batch=50 samples=6400 AvgCost=0.551691 CurrentCost=0.416303 Eval: classification_error_evaluator=0.282031 CurrentEval: classification_error_evaluator=0.180469
I0912 07:44:51.390555 4219 TrainerInternal.cpp:162] Batch=60 samples=7680 AvgCost=0.525094 CurrentCost=0.392109 Eval: classification_error_evaluator=0.264844 CurrentEval: classification_error_evaluator=0.178906
I0912 07:47:44.667140 4219 TrainerInternal.cpp:162] Batch=70 samples=8960 AvgCost=0.502319 CurrentCost=0.365668 Eval: classification_error_evaluator=0.250112 CurrentEval: classification_error_evaluator=0.161719
/usr/local/bin/paddle: line 81: 4219 Killed ${DEBUGGER}
from paddle.
@hdulbj please check your available memory size. If it is not enough, you can try to reduce dictionary size or batch size.
from paddle.
Related Issues (20)
- 发票关键信息抽取.md HOT 4
- 使用cpu没有问题,使用GPU报错 HOT 1
- fluid.contrib.layers.scaled_fc() 函数文档 HOT 1
- 昇腾910B源码编译安装失败 HOT 3
- 支持macos 14.x HOT 3
- 使用pyinstaller将Paddle依赖跟.py文件打包成.exe后,win10系统可以正常运行,但是win7系统或者cup不支持AVX命令集的运行就会报错 HOT 2
- 飞桨AI Studio平台的GPU算力不能使用了吗? HOT 1
- Installation with ROCM HOT 1
- xpu paddle.tile函数功能异常问题 HOT 4
- aistudio可以安装csh吗?c compiler需要管理员权限才能运行吗? HOT 2
- paddle docker 编译cinn无法通过 HOT 1
- [Bug Report] sigmoid_cross_entropy_with_logits 算子的小算子自动微分与调用反向kernel的计算结果不一致 HOT 3
- 我使用多线程启动ppstructure+ocr处理任务(具体起了两个线程),运行时报错PreconditionNotMetError: Tensor's dimension is out of bound.,单线程运行没问题 HOT 2
- PaddleSeg-release-2.9/contrib/MedicalSeg执行模型转换报错,synapse HOT 6
- sparse_embedding使用 HOT 3
- PaddleSeg-release-2.9/contrib/MedicalSeg执行模型转换报错,MSD-brain HOT 3
- fluid.program_guard报错 HOT 1
- 这里维度好像不匹配,请问有大佬知道原因吗? HOT 2
- How to disable .paddleocr directory creation?
- RSSegBenchmark-c2fnet复现出现bug HOT 12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from paddle.