Code Monkey home page Code Monkey logo

know_how_know_why's Introduction

项目简介

本项目会覆盖公众号【孙金城】中的文章和《Apache Flink 知其然，知其所以然》视频课程的所有示例代码。特别说明，示例代码仅用于原理和API使用的说明，由于某些知识点是局部的，示例代码不会全盘考虑，进而所有示例代码不保证达到投产的质量。

课程综述

《Apache Flink 知其然，知其所以然》课程，在内容上会先对Flink整体架构和所适用的场景做一个基础介绍，让你对Flink有一个整体的认识！然后对核心概念进行详细介绍，让你深入了解流计算中一些核心术语的含义，然后对Flink 各个层面的API，如 SQL/Table&DataStreamAPI/PythonAPI 进行详细的介绍,以及底层的实现原理进行剖析和具体场景的最佳实践分析，让你对Flink所提供的功能做到 知其然，知其所以然。整体课程的三个部分定位如下：

  • Flink知其然 - 注重Flink的使用，面向初级人群；

  • Flink知其所以然 - 注重原理分析和生产经验分享，面向高级进阶人群；

  • Flink开发者 - 重点为大家分享作者在社区的开发经验，意在辅助大家更好的参与社区开发；

本课程将会让一线开发能具备使用Apache Flink进行线上业务快速开发的能力，帮助架构师或者团队Team leader和Tech lead对实际工作中构建流计算平台做更好的技术选型决策和规划。

关于作者

孙金城（金竹），2011年加入阿里，9年的阿里工作中，主导过很多内部核心系统，如，阿里集团行为日志，阿里郎，云转码，文档转换等云服务！在2016年初开始了解Apache Flink社区，由初期的参与社区开发到后来逐渐主导具体模块的开发，到负责Apache Flink Python API(PyFlink) 的建设。 目前是 阿里巴巴 Tech Lead，ASF Member，PMC Member of Apache Flink and Apache IoTDB 以及 Committer for Apache Flink, Apache Beam and Apache IoTDB 。

我坚信: 才，德之资也，德，才之帅也！

know_how_know_why's People

Contributors

sunjincheng121 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

alivablededode superainbower jfly0902 mofeipf czwanglei spiderle heipark latincross folgerhuang stevenlol jbshen tjx2014 liangzaiatgithub lixianxun fishcus yiyijin wangshumin xrosliang tengcong5210 alexwangx daqianliao maikouliujian luoqifei yangguang001 dengmingjie ddddddexter banposeu jimmy777 zgq25302111 vivian1020 fredcrawley chengaoying david931229 jinshuangxian yungthuis arb1992 chunji96 dudulook27 huangxiaofeng10047 colonelhou mibil bigdataedison viking714 jiamx easonyi davidcheon lzshlzsh sanwen1221 aceelong baosince youxuehu bigdata-caowei taomingming konglunbo zihaodeng 8847141 kangweihao0321 timgrk lin9long houmaozheng fast-jack wild-river2016 felixzh2020 zhupengkai baronnd leasy2022 chengwei-li tiny123 forus0322 sishen123258 danotes coderlinj5945 zjw-zjw pengcaifang qinjxqjx anyingbinglong zhouji1993 gujiacun china-bear nickscut sunshineit shawnyang90 ljc520313 maaaayp chujian3yi gcczuis 0xqq thorntree 1987v5 chen-bj yunluwen liujingsshine topersuit royal007a fastpassion liyubin117 jsqf toratto michaeljian daipeng1104

know_how_know_why's Issues

flink-1.10.1 webui 查看metric问题

`CREATE TABLE source_table (
bulletin STRING
) WITH (
'connector.type' = 'kafka',
'connector.version' = 'universal',
'connector.startup-mode' = 'latest-offset',
'connector.topic' = 'webids-ids_dolog',
'connector.properties.group.id' = 'vul_testGroup',
'connector.properties.zookeeper.connect' = '',
'connector.properties.bootstrap.servers' = '',
'update-mode' = 'append',
'format.type' = 'json',
'format.derive-schema' = 'true'
);

CREATE TABLE sink_table (
vul_name STRING,
) WITH (
'connector.type' = 'kafka',
'connector.version' = 'universal',
'connector.startup-mode' = 'latest-offset',
'connector.topic' = 'vul_agg',
'connector.properties.zookeeper.connect' = '',
'connector.properties.bootstrap.servers' = '',
'update-mode' = 'append',
'format.type' = 'json',
'format.derive-schema' = 'true'
);

INSERT INTO sink_table SELECT
bulletin
FROM source_table;

`
执行上述job时候，web_ui 页面无法看到metric数据(接收了多少数据，发送了多数据)，不理解这是为什么。
还请孙老师能够讲解下，什么情况下面可以看到web_ui metric数据，什么情况下看不到。

远程env

本地可以像java那样生成远程执行环境吗？
pyflink好像只能调用java的gateway确定当前环境，不能指定远程环境.
ExecutionEnvironment env = ExecutionEnvironment.createRemoteEnvironment("flink-jobmanager", 8081, "/home/user/udfs.jar")

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.