Comments (1)
Distribute data to cluster is not added in PaddlePaddle now. You can read data directly from a HDFS file path by PyDataProvider2.
PaddlePaddle not handle how to get data file remotely, just pass the file path into a Python function. It is user's job to OPEN the file (or SQL connection string, or HDFS path), and get each
sample one by one from it.
It is welcome to contribute a script to distribute data to cluster. Or we may add it soon if this feature is very necessary.
from paddle.
Related Issues (20)
- ParamAttr的参数learning_rate取值非1时,paddle是否会为learning_rate创建一个变量,如何给这个变量指定名字? HOT 1
- paddle/phi/kernels/impl/matmul_kernel_impl.h中无法对DenseTensor进行cout HOT 2
- 补全rms_norm算子CPU实现 HOT 2
- `masked_fill_`對int64處理異常,塞入paddle.iinfo(paddle.int64).max會被當作min HOT 11
- cmake编译 GPU 版本的 Paddle 出现错误 LINK : fatal error LNK1248: 映像大小(8046A000)超过允许的最大大小(80000000) HOT 2
- 【求助】关于paddledetection 中 mot_ppyoloe_l_36e_pipeline model的推理过程 HOT 3
- paddle.grad无法计算高阶微分 HOT 2
- paddle的nansum不支持empty的求和 HOT 2
- 调用paddle.grad函数计算指数函数和三角函数的高阶微分,报错如何解决 HOT 1
- 提供详细的模型结构打印信息 HOT 7
- paddle.nn.AdaptiveAvgPool2D没有对输入的维度进行检查 HOT 1
- paddle.nn.MaxPool2D缺乏对输入的形状检查
- 在华为Atlas加速卡服务器,银河麒麟V10系统的服务器上编译paddle,cmake执行完毕但是没有生成makefile HOT 1
- 【PIR】Sparse 算子 Python API 适配 HOT 1
- 2.6.0在windows上编译cpu版本失败 HOT 3
- help paddle-fluid-v1.7.1 textcnn running error HOT 1
- 同一份模型训练代码在CUDA11.8和CUDA11.6上的运行结果差异较大 HOT 4
- 运行预测 self.predictor.run() 后,引擎内存释放不掉 HOT 8
- 【bug】paddle多机多卡训练的时候master指定为ipv6地址时,代码报错 HOT 2
- paddle.histogram遇到特定整數pattern發生illegal memory access,導致進程必須重啟 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from paddle.