Code Monkey home page Code Monkey logo

palo-deploy's Introduction

palo-deploy

使用shell脚本部署Apache Doris (incubating)(原百度palo)

快速部署

  1. 安装java环境
  • 查询jdk下载地址:
curl -s https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html|grep linux-x64.rpm
  • 复制filepath地址,浏览器中进行访问,完成登录验证后进行下载,下载完毕后进行安装:
rpm -ivh jdk-8u211-linux-x64.rpm 
ls -ld /usr/java/jdk1.8.0_211-amd64/
ln -s /usr/java/jdk1.8.0_211-amd64 /usr/java/jdk
echo "export JAVA_HOME=/usr/java/jdk" >> /etc/profile
echo "export JRE_HOME=/usr/java/jdk/jre" >> /etc/profile
  1. 安装supervisor环境
yum install supervisor -y
echo "files = supervisord.d/*.conf" >> /etc/supervisord.conf
systemctl enable supervisord
systemctl start supervisord.service
systemctl status supervisord.service

注:supervisor主配置文件路径为:/etc/supervisord.conf,子配置目录为:/etc/supervisord.d/,若配置非标准,需要手工修改部署脚本。

  1. 拷贝palo编译完成后output文件至指定目录,如:/usr/local/src/output
cp -rf /your/local/path/incubator-doris-DORIS-x.x.x-release/output /usr/local/src/output

注:在部署集群时,不可能将整个源码+编译文件都拷贝到集群服务器,此时只需打包output目录,拷贝至集群各服务器,解压使用:

# 编译服务器上打包output目录,并上传至ftp
cd /your/local/path/incubator-doris-DORIS-x.x.x-release/
tar zcf output.tar.gz output
curl -s -k -T output.tar.gz output -u <u>:<p> ftp://x.x.x.x//

# 从ftp拷贝打包文件至集群各服务器,并解压
wget -P /usr/local/src/ ftp://<u>:<p>@x.x.x.x:21/output.tar.gz
tar zxf /usr/local/src/output.tar.gz -C /usr/local/src/
  1. 部署Frontendroot环境下执行以下命令:
sh deploy-fe.sh palo /usr/local/src/output /data/palo_fe_log /home/soft/java

注:以下参数必须指定后执行,缺一不可:

  • palo:以该用户执行palo;
  • /usr/local/src/output:palo编译完成后output文件路径;
  • /data/palo_fe_log:指定存放FE日志文件路径;
  • /home/soft/java:指定JAVA_HOME目录;
  1. 部署Backendroot环境下执行以下命令:
sh deploy-be.sh 1 /data1/palo palo /usr/local/src/output /data1/palo_be_log

注:以下参数必须指定后执行,缺一不可:

  • 1:由于同一台服务器有部署多个be的场景,故指定BE ID,注意该id与BE端口相关联,必须在0~9之内;
  • /data1/palo:指定存放BE数据文件路径;
  • palo:以该用户执行palo;
  • /usr/local/src/output:palo编译完成后output文件路径;
  • /data1/palo_be_log:指定存放BE日志文件路径;

验证服务

  • 验证FE集群
mysql -h <fe_ip> -P 9030 -uroot
ALTER SYSTEM ADD FOLLOWER "<fe_ip>:9010";
show proc '/frontends';
  • 验证BE集群
mysql -h <fe_ip> -P 9030 -uroot
ALTER SYSTEM ADD BACKEND "<be_ip>:<be_heartbeat_service_port>";
SHOW PROC '/backends';

升级palo

升级脚本通常用在大版本升级(已验证v0.8.2至v0.9.0),采用了类似重新部署的方式,文件将全部替换;如果是小版本升级,BE只需升级palo_be;而FE只需升级palo-fe.jar

  1. 拷贝新版palo编译完成后output文件至指定目录,如:/usr/local/src/output

  2. 升级Backendroot环境下执行以下命令(各执行参数同部署脚本,不再重复):

sh upgrade-be.sh 3 /data3/palo palo /usr/local/src/output /data3/palo_be_log
  1. 升级Frontendroot环境下执行以下命令(各执行参数同部署脚本,不再重复):
sh upgrade-fe.sh palo /usr/local/src/output /data2/palo_fe_log /home/soft/java
su - palo
cd /home/palo/fe/palo-meta/image/
cp VERSION VERSION.bak
vi VERSION
token=xxx
clusterId=xxx

注:必须指定token及clusterId,否则FE加入集群会失败:ErrMsg: invalid cluster id: xxx

  1. BE及FE全部升级完毕后,重启supervisor,使之生效:
supervisorctl reload
supervisorctl status

palo-deploy's People

Contributors

zhoufwind avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.