Comments (2)
Steps to Reproduce
- Upgrade to Flink version 1.19 or later where full YAML 1.2 support is introduced.
- Attempt to run a Flink CDC job using the latest version of Flink CDC, which still references the
flink-conf.yaml
file. - Using flink-cdc.sh script :
bin/flink-cdc.sh mysql2doris.yaml
exception:
Exception in thread "main" java.io.FileNotFoundException: Cannot find configuration file at "/opt/flink/conf/flink-conf.yaml"
Environment Information
- Flink Version: 1.19.0
- Flink CDC Version: flink-cdc-3.0.1
Suggested Solution
To enable Flink CDC to simultaneously support both new and old Flink configuration file standards, it is recommended to implement a simple logic check within the Flink CDC codebase to determine whether to use config.yaml
or flink-conf.yaml
. This can be achieved by utilizing the Files.exists
method to check for the presence of the config.yaml
file in the system. If it exists, Flink CDC should use config.yaml
as the configuration file; if it does not, then it should fall back to using flink-conf.yaml
. Such an implementation will ensure that Flink CDC can seamlessly transition between different versions of Flink while maintaining backward compatibility.
from flink-cdc.
If the issue still exists, please report it on Apache Jira under the project Flink
, using the component tag Flink CDC
. Thank you!
from flink-cdc.
Related Issues (20)
- FlinkCDC3.0的任务cancel之后,怎样启动指定checkpoint HOT 1
- ChunkKeyColumns that synchronize multiple tables without a primary key need to be concatenated a lot. Why not simply pass Map<ObjectPath, String> chunkKeys HOT 2
- flinksql设置并发后,会报错Database connection failed when starting copy(postgres-cdc) HOT 2
- The XML dependency of Flink Oracle CDC conflicts with the XML related classes of Java8:ERROR StatusLogger Caught javax.xml.parsers.ParserConfigurationException HOT 3
- 如何通过yarn-session运行flink cdc作业?
- Oracle problem,sync fail HOT 1
- flink mongodb cdc 方式同步数据,mongodb 4.2版本的不支持,update的更新数据同步吗 HOT 4
- I found flink-cdc supported connectors using YAML definition,its very convenient,i love it.so my question is when other pipeline is supported?for example,i want sync data form oracle to pg or pg to mysql.Thanks! HOT 1
- Flink Mongodb CDC在遇到DDL操作(eg: db.c.drop()),程序报错, 期待相应功能的支持 HOT 1
- SQL Server does not support scan. startup. mode: initial only HOT 2
- flink cdc 2.4.2版本关于报错io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4 的问题 HOT 1
- Exception in thread "main" java.lang.NoSuchMethodError: 'void org.apache.flink.streaming.runtime.operators.sink.CommitterOperatorFactory.<init>(org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink, boolean, boolean)' HOT 3
- Information that has disappeared from github appears on the web page. HOT 1
- DataStreamSource oracle ID HOT 1
- Information that has disappeared from github appears on the web page. HOT 2
- supported mysql8.4? HOT 1
- Flink 1.18.1 and 1.19.0 having build jdk 1.8 and causing incompatibilities with Java 17 HOT 3
- update operation does not take effect HOT 5
- mysql pipeline connector lost table which database and table with the same name
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flink-cdc.