Comments (7)
这个错误能够复现么?
错误看起来是创建线程时资源不够导致的,跟你机器当时的运行状态或者配置比较相关,我本地重复调用25w次未复现。
可以考虑修改本地进程允许的最大线程数避免一下。
from ddparser.
这个错误能够复现么? 错误看起来是创建线程时资源不够导致的,跟你机器当时的运行状态或者配置比较相关,我本地重复调用25w次未复现。 可以考虑修改本地进程允许的最大线程数避免一下。
可以重现,每次都会导致进程崩掉,已经这个样子一周情况了。我用的是 mac monterey 12.0.1。
这个解释中我理解的是paddle会不断创建线程,导致线程资源耗尽,我如果调高本地最大线程数,会不会在更高短句数据量下也崩溃掉?
你这边追踪bug的话还需要哪些数据,我这边尽量提供给你
from ddparser.
我如果调高本地最大线程数,会不会在更高短句数据量下也崩溃掉?
存在这种可能,因为这涉及到paddle的dataloader内部对资源的调度。
但目前未见过用户报类似的case,paddle下也未见相关issue,建议你先换台空闲机器或者调大最大线程数看是否可以稳定复现。
如果还不行,可以提供下可以复现的数据及脚本。
from ddparser.
我也遇到了同样的问题, 会不停的创建线程, 使用的api 是 ddp.parse_seg
paddlepaddle==2.1.0
ddparser==1.0.6
Resource temporarily unavailable
from ddparser.
我用这个脚本本地运行,大概在160000+次的时候会报新的错误,无法创建线程
# 运行脚本
from ddparser import DDParser
ddp = DDParser()
count = 0
for i in range(500000):
text = "两侧额顶叶深部白质区及侧脑室周围多发点状、小斑片及片状异常信号"
ddp_res = ddp.parse(text)
count += 1
if i % 10000 == 0:
print("==>", i)
# 错误信息
==> 170000
Traceback (most recent call last):
File "/Users/wangy/PycharmProjects/test/tests.py", line 40, in <module>
ddp_res = ddp.parse(text)
File "/Users/wangy/anaconda3/lib/python3.7/site-packages/ddparser/run.py", line 399, in parse
pred_arcs, pred_rels, pred_probs = epoch_predict(self.env, self.args, self.model, dataset.loader)
File "<decorator-gen-270>", line 2, in epoch_predict
File "/Users/wangy/anaconda3/lib/python3.7/site-packages/paddle/fluid/dygraph/base.py", line 276, in __impl__
return func(*args, **kwargs)
File "/Users/wangy/anaconda3/lib/python3.7/site-packages/ddparser/parser/model.py", line 185, in epoch_predict
for batch, inputs in enumerate(loader(), start=1):
File "/Users/wangy/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 961, in __iter__
self._start()
File "/Users/wangy/anaconda3/lib/python3.7/site-packages/paddle/fluid/reader.py", line 947, in _start
self._thread.start()
File "/Users/wangy/anaconda3/lib/python3.7/threading.py", line 852, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
@zhangyimi 辛苦帮忙看看是什么原因,这里的 ddparser==1.0.6,paddlepaddle==2.1.3
非常感谢!
from ddparser.
这个问题会在近期修复。
from ddparser.
这个问题会在近期修复。
辛苦,非常期待!
from ddparser.
Related Issues (20)
- 关于利用ernie HOT 2
- GPU版本的ddp.parse报错 HOT 2
- m1 illegal hardware instruction python HOT 1
- KeyError: 'transformer' HOT 1
- 关于模型输入之前的一些处理 HOT 1
- host='ddparser.bj.bcebos.com', DDParser-ernie-lstm-1.0.6.tar.gz 权限限制,无法下载呀? HOT 2
- 一个明显的依存解析错误,想问下有什么方法应用时规避?
- 如何用paddlehub serving对ddparser部署
- 跑测试case的时候有bug,切词切不开
- 求教:依存句法数据 标注用的是什么工具?
- segmentation fault HOT 1
- bash run_predict.sh
- 预测速度
- 切词模式 HOT 2
- 一个句法分析的bug
- 请问支持成分句法分析吗?
- 使用CTB5数据集训练时除了修改数据集还需要修改什么,预训练模型需要改变吗
- 训练自己的模型时发生异常
- AttributeError: module 'paddle.fluid.dygraph' has no attribute 'Layer' HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ddparser.