Comments (7)
dongeforever
commented on May 31, 2024
from aliyun-emapreduce-datasources.
uncleGen
commented on May 31, 2024
@dongeforever E-MapReduce支持创建Kafka集群。你可以看下这个文档https://help.aliyun.com/document_detail/56932.html
from aliyun-emapreduce-datasources.
dongeforever
commented on May 31, 2024
@uncleGen 我们现在有业务已经接入阿里云Kafka了，希望有一个快速的办法把阿里云Kafka的数据导入EMR集群内；
我看EMR是可以和ONS对接的；为啥不能和Kafka对接呢；
from aliyun-emapreduce-datasources.
uncleGen
commented on May 31, 2024
@dongeforever 你说的对接是啥意思？
from aliyun-emapreduce-datasources.
dongeforever
commented on May 31, 2024
就是我开通了阿里云Kafka，现在想把数据接入EMR集群里面跑，类似于https://help.aliyun.com/document_detail/28120.html?spm=5176.doc56932.6.630.ntErfw
from aliyun-emapreduce-datasources.
uncleGen
commented on May 31, 2024
现在还没有demo，理论上自定义一个receiver是很方便实现的。
from aliyun-emapreduce-datasources.
uncleGen
commented on May 31, 2024
直接使用spark提供的kafka接口也是可以的
from aliyun-emapreduce-datasources.
Related Issues (20)
- add kudu support HOT 1
- We can only read data in at most 5 minutes in each batch.
- when initialPartitionOffsets is out of date, getLatestHistograms returns a empty list, which may cause a Exception.
- batch datasource for datahub
- Sink data to specific shard of datahub by key
- Failed with missing config 'tunnel.id' when batch read tablestore data. HOT 1
- emr-oss get OSSClient exception HOT 4
- Tablestore datasource support both UPDATE and INSERT HOT 2
- Streaming+SLS may failed in some corner case
- jdbc source：remove query name requirement
- remove fastjson dependency
- Can't load this .dll (machine code=0xe800) on a AMD 64-bit platform HOT 5
- tablestore binlog parser udf HOT 1
- Timeout error occurred when use hadoop FileSystem to rename big file with JindoFS SDK HOT 6
- NotSerializableException when read from kudu datasource
- `GLIBC_2.16' not found HOT 10
- support for Windows HOT 1
- SLS structured streaming 消费加上错误重试
- use hadoop fs -ls oss://<ak>:<secret>@<bucket>.<endpoint>/ meet exception
- Any Spark 3 support for log service
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aliyun-emapreduce-datasources.