Comments (3)
This problem is similar to #63, please try that solutions. You can set pool_size
as follows in dataprovider.py
to reduce data cache memory.
@provider(init_hook=hook, pool_size=1024)
from paddle.
I did what you advised ,but it still has problems
lbj@ubuntu:~/work/Paddle-master/demo/sentiment$ ./train.sh
I0912 07:26:11.295408 4219 Util.cpp:144] commandline: /usr/local/bin/../opt/paddle/bin/paddle_trainer --config=trainer_config.py --save_dir=./model_output --job=train --use_gpu=false --trainer_count=4 --num_passes=10 --log_period=10 --dot_period=20 --show_parameter_stats_period=100 --test_all_data_in_one_period=1
I0912 07:26:11.296186 4219 Util.cpp:113] Calling runInitFunctions
I0912 07:26:11.296751 4219 Util.cpp:126] Call runInitFunctions done.
[INFO 2016-09-12 07:26:12,275 networks.py:1122] The input order is [word, label]
[INFO 2016-09-12 07:26:12,275 networks.py:1129] The output order is [cost_0]
I0912 07:26:12.353705 4219 Trainer.cpp:169] trainer mode: Normal
I0912 07:26:13.966311 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,002 dataprovider.py:22] dict len : 101745
I0912 07:26:14.200013 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,218 dataprovider.py:22] dict len : 101745
I0912 07:26:14.218703 4219 GradientMachine.cpp:134] Initing parameters..
I0912 07:26:15.289685 4219 GradientMachine.cpp:141] Init parameters done.
I0912 07:26:19.343603 4220 ThreadLocal.cpp:39] thread use undeterministic rand seed:4221
I0912 07:26:19.514636 4221 ThreadLocal.cpp:39] thread use undeterministic rand seed:4222
I0912 07:26:19.536335 4222 ThreadLocal.cpp:39] thread use undeterministic rand seed:4223
I0912 07:26:19.704776 4223 ThreadLocal.cpp:39] thread use undeterministic rand seed:4224
I0912 07:29:23.252763 4219 TrainerInternal.cpp:162] Batch=10 samples=1280 AvgCost=0.692175 CurrentCost=0.692175 Eval: classification_error_evaluator=0.485156 CurrentEval: classification_error_evaluator=0.485156
I0912 07:32:37.542174 4219 TrainerInternal.cpp:162] Batch=20 samples=2560 AvgCost=0.666661 CurrentCost=0.641147 Eval: classification_error_evaluator=0.401172 CurrentEval: classification_error_evaluator=0.317188
I0912 07:35:50.129344 4219 TrainerInternal.cpp:162] Batch=30 samples=3840 AvgCost=0.626617 CurrentCost=0.54653 Eval: classification_error_evaluator=0.342448 CurrentEval: classification_error_evaluator=0.225
I0912 07:38:51.197893 4219 TrainerInternal.cpp:162] Batch=40 samples=5120 AvgCost=0.585538 CurrentCost=0.462299 Eval: classification_error_evaluator=0.307422 CurrentEval: classification_error_evaluator=0.202344
I0912 07:41:56.019923 4219 TrainerInternal.cpp:162] Batch=50 samples=6400 AvgCost=0.551691 CurrentCost=0.416303 Eval: classification_error_evaluator=0.282031 CurrentEval: classification_error_evaluator=0.180469
I0912 07:44:51.390555 4219 TrainerInternal.cpp:162] Batch=60 samples=7680 AvgCost=0.525094 CurrentCost=0.392109 Eval: classification_error_evaluator=0.264844 CurrentEval: classification_error_evaluator=0.178906
I0912 07:47:44.667140 4219 TrainerInternal.cpp:162] Batch=70 samples=8960 AvgCost=0.502319 CurrentCost=0.365668 Eval: classification_error_evaluator=0.250112 CurrentEval: classification_error_evaluator=0.161719
/usr/local/bin/paddle: line 81: 4219 Killed ${DEBUGGER}
from paddle.
@hdulbj please check your available memory size. If it is not enough, you can try to reduce dictionary size or batch size.
from paddle.
Related Issues (20)
- arm源码安装paddle2.6报错void* memcpy(void*, const void*, size_t) HOT 4
- 如果我想跳过单元测试中,关于fp64的所有测试,应该如何做 HOT 6
- paddle进行开启分布式训练时如何进行性能分析? HOT 4
- paddle-gpu总是报找不到类似libnccl.so链接库的错误 HOT 4
- paddle.utils.run_check()报错
- Assertion `p_in_data[idx] >= 0 && p_in_data[idx] < depth` failed. HOT 2
- 对于Custom Device如何注册fake_quantize_range_abs_max HOT 9
- Miscellaneous issues on CUDA HOT 1
- 使用其他op实现2.3中的cumprod算子 HOT 1
- 静态图高阶微分计算结果不符合预期 HOT 1
- jetson nano jetpack4.6 有cp38的paddlepaddle gpu whl吗 HOT 1
- paddle.einsum is broken in dev version, and breaks tests in einops
- 静态图模式下通过nccl等方式完成流水线并行通信是否行得通 HOT 1
- index_add反向传播报错
- 缺少SequentialSampler HOT 2
- Visualdl输入命令后卡住没有反应 HOT 2
- Different result on CPU and GPU HOT 2
- 【Hackathon 6th Fundable Projects 2】修复记录
- paddlepaddle 2.6.1 paddleocr 2.7.3 lang='ch'与ocr_version=‘PP-OCRv4’ 报错FatalError: `Illegal instruction` is detected by the operating system Illegal instruction (core dumped) HOT 1
- PGL(Paddle Graph Learning)模块是否有ARM架构的版本呢? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from paddle.