Code Monkey home page Code Monkey logo

docker-centos's Introduction

docker-centos

Centos image stacks for develop and deploy components

中文版 Readme

background

blog-Use Dockerfile to build develop image

Target: save my day Use dockerfile and compose file to build docker image and run develop environment.

extend blog: Use Docker Compose to start hdfs cluster locally

Use Docker Compose to start zookeeper cluster locally

project structure

  • deployments

    • compose: deployment on docker compose

  • Dockerfiles

    • system: centos basic image
    • dev: development env such as golang, java, python, nodejs
    • frontend: frontend service such as: vue admin, ant design
    • backend: backend service such as: zk, mysql, redis
    • emr: bigdata component, such as jupyter, hue
    • net: for network debug

  • Makefile

    • Makefile.vars.repo: software repo
    • Makefile.vars: service version
    • Makefile: image build, push and service deploy single cmd

All Image And How To Use

basic image

type component/image name build command feature
basic base # full centos base image
make build_base		 centos8, yum repo(tsinghua), s6, oh-my-zsh, crontab
minimal # minimal centos base image
make build_minimal
# build centos7 version
make build_minimal_7		 centos, yum repo(tsinghua), s6, crontab
develop golang make build_dev_golang golang 1.19
java # full, include java, maven, gradle
make build_dev_java
# minimal, only include java
make build_dev_java_minimal		 java develop env
openjdk
JAVA_HOME: jdk8
JDK_HOME: jdk11 for vscode java extension
maven: 3.8.4
gradle: 7.0.2
python make build_dev_python python develop env
miniforge: 4.13.0
python: 3.8
nodejs make build_dev_nodejs nodejs develop env
nodejs: v16.15.0
full make build_dev_full include all develop env above
frontend vue admin make build_vue_admin vue-admin 4.4.0
ant design make build_ant_design ant-design 5.2.0
middleware mysql make build_mysql mysql 8.0.27
postgresql make build_pgsql postgresql 14
redis make build_redis redis 7.0-rc2
mongodb make build_mongo mongo 6.0.0
prometheus make build_prometheus prometheus 2.33.4
alertmanager 0.23.0
pushgateway 1.4.3
node exporter 1.3.1
grafana make build_grafana grafana 8.4.2
zookeeper make build_zookeeper zookeeper 3.9.1
kafka make build_kafka kafka 3.2
nginx make build_nginx nginx 1.23.2
emr airflow make build_airflow airflow 2.7.3
hdfs make build_hdfs hdfs 3.3.2
hive make build_hive hive 3.1.2
knox make build_knox knox 1.6.1
hdfs_full make build_hdfs hdfs 3.3.2
knox 1.6.1
hive 3.1.2
spark 3.2
flink 1.15
hue make build_hue hue 4.3.0 fix branch: dev_bugfix
jupyter make build_jupyter jupyterlab 3.3.3
notebook 6.4.10
zeppelin make build_zeppelin zeppelin 0.10.1
presto make build_presto presto 0.275
trino make build_trino trino 403
flink make build_flink flink 1.15
spark make build_spark spark 3.4.1
superset make build_superset superset 2.0.0
azkaban make build_azkaban azkaban master branch
prefect make build_prefect prefect 2.7.7
dolphinscheduler make build_dolphinscheduler dolphinscheduler 3.1.4
datalink make build_datalink datalink dev branch
elasticsearch make build_es elasticsearch 8.4.1
kibana make build_kibana kibana 8.4.1
atlas make build_atlas atlas 2.2.0
clickhouse make build_clickhouse clickhouse 21.7.8
doris make build_doris doris 2.0.2
starrocks make build_starrocks starrocks 2.5.3
minio make build_minio minio release
net xrdp make build_xrdp centos with xrdp
easyconnect make build_ec easyconnect 7.6.7.3
clash 1.10.6
firefox
entertainment navidrome make build_navidrome navidrome

note: build image & run container defail command refer: Makefile

dockerhub

you can also pull these image from dockerhub, no need to build locally, just execute command in Makefile with REPO var. e.g.

# dev_full
REPO=mzsmieli make run_dev_full
# will pull mzsmieli/centos:base8_dev_full_1.0

# jupyter
REPO=mzsmieli make run_jupyter
# will pull mzsmieli/centos:base8_jupyter3.3

# nacos
REPO=mzsmieli make run_nacos_mysql
# will pull mzsmieli/centos:minimal8_mysql8 and mzsmieli/centos:base8_nacos2.1.2

services

type service run command feature
dev tools code server REPO=mzsmieli make run_code_server code server: http://localhost:8080
oauth server REPO=mzsmieli SERVER_HOST=server_host make run_oauth_server oauth client: http://${server_host}:19094
middleware zookeeper REPO=mzsmieli make run_zookeeper_cluster zookeeper cluster(3 node)
connect on host: zkCli.sh -server localhost:12181
refer
kafka REPO=mzsmieli make run_kafka_cluster kafka cluster(3 node)
nacos REPO=mzsmieli make run_nacos_mysql nacos+mysql
address: http://localhost:8848
prometheus+grafana REPO=mzsmieli make run_prometheus_grafana grafana: http://localhost:3000
prometheus: http://localhost:3001
kafka+efak REPO=mzsmieli make run_kafka_efak efak: http://localhost:38042
admin/123456
emr hadoop REPO=mzsmieli make run_hdfs_cluster hadoop cluster
hdfs: http://localhost:8443/gateway/sandbox/hdfs
yarn: http://localhost:8443/gateway/sandbox/yarn
hive: localhost:10000
mysql: localhost:33306
hue REPO=mzsmieli make run_hue hue: http://localhost:8281
hue+hdfs+hive REPO=mzsmieli make run_hue_hive hue: http://localhost:8281
hue+hdfs+hive+presto REPO=mzsmieli make run_hue_presto hue: http://localhost:8281
azkaban REPO=mzsmieli make run_azkaban azkaban web: http://localhost:8020
datalink REPO=mzsmieli make run_datalink_singleton datalink: http://localhost:18080
admin/admin
clickhouse REPO=mzsmieli make run_clickhouse_cluster 3 clickhouse nodes: localhost:18123,localhost:28123,localhost:38123
starrocks REPO=mzsmieli make run_starrocks_cluster 3 starrocks nodes, master fe: localhost:19030
doris REPO=mzsmieli make run_doris_cluster 3 doris nodes，master fe: localhost:19030
es+kibana REPO=mzsmieli make run_es_kibana es: http://localhost:9200
kibana: http://localhost:5601
flink REPO=mzsmieli make run_flink flink ui: http://localhost:8081

docker-centos's People

Contributors

smiecj avatar

Stargazers

avatar avatar avatar leo avatar Alan Liang avatar

Watchers

avatar avatar avatar

Forkers

cnopens sancijun

docker-centos's Issues

docker-compose 创建 zk 集群

使用docker-compose创建集群，java通过Curator无法连接，报错如下

2022-09-09 14:38:34.746 [main-EventThread] INFO  org.apache.curator.framework.imps.EnsembleTracker Line:201 - New config event received: {server.1=0.0.0.0:2888:3888:participant;0.0.0.0:2181, version=0, server.3=zk3:2888:3888:participant;0.0.0.0:2181, server.2=zk2:2888:3888:participant;0.0.0.0:2181}
2022-09-09 14:38:34.776 [main] INFO  com.zaxxer.hikari.HikariDataSource Line:123 - HikariPool-1 - Start completed.
2022-09-09 14:38:34.794 [main-EventThread] INFO  org.apache.curator.utils.Compatibility Line:47  - Using org.apache.zookeeper.server.quorum.MultipleAddresses
2022-09-09 14:38:34.798 [main-EventThread] ERROR org.apache.curator.framework.imps.CuratorFrameworkImpl Line:733 - Background exception was not retry-able or retry gave up
java.lang.NullPointerException: null
        at org.apache.curator.utils.Compatibility.getHostAddress(Compatibility.java:116)
        at org.apache.curator.framework.imps.EnsembleTracker.configToConnectionString(EnsembleTracker.java:185)
        at org.apache.curator.framework.imps.EnsembleTracker.processConfigData(EnsembleTracker.java:206)
        at org.apache.curator.framework.imps.EnsembleTracker.access$300(EnsembleTracker.java:50)
        at org.apache.curator.framework.imps.EnsembleTracker$2.processResult(EnsembleTracker.java:150)
        at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:926)
        at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:683)
        at org.apache.curator.framework.imps.WatcherRemovalFacade.processBackgroundOperation(WatcherRemovalFacade.java:152)
        at org.apache.curator.framework.imps.GetConfigBuilderImpl$2.processResult(GetConfigBuilderImpl.java:222)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:644)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:563)
2022-09-09 14:38:34.802 [main-EventThread] INFO  org.apache.curator.framework.imps.EnsembleTracker Line:201 - New config event received: {server.1=0.0.0.0:2888:3888:participant;0.0.0.0:2181, version=0, server.3=zk3:2888:3888:participant;0.0.0.0:2181, server.2=zk2:2888:3888:participant;0.0.0.0:2181}
2022-09-09 14:38:34.803 [main-EventThread] ERROR org.apache.curator.framework.imps.CuratorFrameworkImpl Line:733 - Background exception was not retry-able or retry gave up
java.lang.NullPointerException: null
        at org.apache.curator.utils.Compatibility.getHostAddress(Compatibility.java:116)
        at org.apache.curator.framework.imps.EnsembleTracker.configToConnectionString(EnsembleTracker.java:185)
        at org.apache.curator.framework.imps.EnsembleTracker.processConfigData(EnsembleTracker.java:206)
        at org.apache.curator.framework.imps.EnsembleTracker.access$300(EnsembleTracker.java:50)
        at org.apache.curator.framework.imps.EnsembleTracker$2.processResult(EnsembleTracker.java:150)
        at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:926)
        at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:683)
        at org.apache.curator.framework.imps.WatcherRemovalFacade.processBackgroundOperation(WatcherRemovalFacade.java:152)
        at org.apache.curator.framework.imps.GetConfigBuilderImpl$2.processResult(GetConfigBuilderImpl.java:222)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:644)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:563)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.