Code Monkey home page Code Monkey logo

zhangyongtian / incubator-seatunnel-web Goto Github PK

View Code? Open in Web Editor NEW

This project forked from apache/seatunnel-web

0.0 0.0 0.0 14.01 MB

SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).

Home Page: https://seatunnel.apache.org/

License: Apache License 2.0

Shell 1.78% JavaScript 1.07% Java 66.59% TypeScript 29.95% CSS 0.18% HTML 0.21% Dockerfile 0.22%

incubator-seatunnel-web's Introduction

Apache SeaTunnel (Incubating)

seatunnel logo

Backend Workflow Slack Twitter Follow


EN doc

SeaTunnel was formerly named Waterdrop , and renamed SeaTunnel since October 12, 2021.


So, What we are?

An open-source web console to manage your seatunnel-script, and would push them to any scheduling-system easily. Click it if your want to know more about our design. ๐Ÿ‘‰๐ŸปDesign

How to start

1 Preparing the Apache DolphinScheduler environment

1.1 Install Apache DolphinScheduler

If you already have Apache DolphinScheduler environment, you can skip this step and go to [Create Tenant and User for SeaTunnel Web](#1.2 Create Tenant and User for SeaTunnel Web)

Because running SeaTunnel Web must rely on the DolphinScheduler, if you do not have a DS environment, you need to first install and deploy a DolphinScheduler (hereinafter referred to as DS). Taking DS version 3.1.5 as an example.

Reference https://dolphinscheduler.apache.org/zh-cn/docs/3.1.5/guide/installation/standalone to install a standalone DS.

1.2 Create Tenant and User for SeaTunnel Web

If you already have a DS environment and decide to use existing users and tenants for SeaTunnel Web, you can skip this step and go to [Create Project for SeaTunnel Web](#1.3 Create Project for SeaTunnel Web).

Because SeaTunnel Web needs to call the interface of DS to create workflows and tasks, it is necessary to submit the projects, users, and tenants created in DS for SeaTunnel to use.

  1. Create Tenant

"Security" -> "Tenant Manage" -> "Create Tenant"

image

  1. For simplicity, use the default user admin of DS directly here

1.3 Create Project for SeaTunnel Web

image

1.4 Create Token for SeaTunnel Web

image

2 Run SeaTunnel Web in IDEA

If you want to deploy and run SeaTunnel Web, Please turn to [3 Run SeaTunnel Web In Server](#3 Run SeaTunnel Web In Server)

2.1 Init database

  1. Edit whaletunnel-server/whaletunnel-app/src/main/resources/script/seatunnel_server_env.sh file, Complete the installed database address, port, username, and password. Here is an example:

    export HOSTNAME="localhost"
    export PORT="3306"
    export USERNAME="root"
    export PASSWORD="123456"
    
  2. Run init shell sh seatunnel-server/seatunnel-app/src/main/resources/script/init_sql.sh If there are no errors during operation, it indicates successful initialization.

2.2 Config application and Run SeaTunnel Web Backend Server

  1. Edit seatunnel-server/seatunnel-app/src/main/resources/application.yml Fill in the database connection information and DS interface related information in the file.

image

  1. Run seatunnel-server/seatunnel-app/src/main/java/org/apache/seatunnel/app/SeatunnelApplication.java If there are no errors reported, the seatunnel web backend service is successfully started.

2.3 Run SeaTunnel Web Front End

cd seatunnel-ui
npm install
npm run dev

If there are no issues with the operation, the following information will be displayed:

  โžœ  Local:   http://127.0.0.1:5173/
  โžœ  Network: use --host to expose
  โžœ  press h to show help

Accessing in a browser http://127.0.0.1:5173/login Okay, the default username and password are admin/admin.

3 Run SeaTunnel Web In Server

3.1 Build Install Package From Code

cd incubator-seatunnel-web
sh build.sh code

Then you can find the installer package in dir incubator-seatunnel-web/seatunnel-server/seatunnel-app/target/apache-seatunnel-web-incubating-${project.version}.tar.gz.

3.2 Install

Copy the apache-seatunnel-web-incubating-${project.version}.tar.gz to your server node and unzip it.

tar -zxvf apache-seatunnel-web-incubating-${project.version}.tar.gz

3.3 Init database

  1. Edit apache-seatunnel-web-incubating-${project.version}/script/seatunnel_server_env.sh file, Complete the installed database address, port, username, and password. Here is an example:

    export HOSTNAME="localhost"
    export PORT="3306"
    export USERNAME="root"
    export PASSWORD="123456"
    
  2. Run init shell sh apache-seatunnel-web-incubating-${project.version}/script/init_sql.sh If there are no errors during operation, it indicates successful initialization.

3.4 Config application and Run SeaTunnel Web Backend Server

Edit apache-seatunnel-web-incubating-${project.version}/config/application.yml Fill in the database connection information and DS interface related information in the file.

image

3.5 Start SeaTunnel Web

cd apache-seatunnel-web-incubating-${project.version}
sh bin/seatunnel-backend-daemon.sh start

Accessing in a browser http://127.0.0.1:8801/ui/ Okay, the default username and password are admin/admin.

How to use it

After all the pre-work is done, we can open the following URL: 127.0.0.1:7890(please replace it according to your configuration) to use it.

Now ,let me show you how to use it.

User manage

img.png

Datapipeline manage

img.png

Job manage

img.png

Task manage

img.png

incubator-seatunnel-web's People

Contributors

asdf2014 avatar ashulin avatar calvinkirs avatar cnmac avatar dijiekstra avatar ericjoy2048 avatar garyelephant avatar hailin0 avatar hisoka-x avatar ic4y avatar kid-xiong avatar laglangyue avatar legendtkl avatar lhyundeadsoul avatar mans2singh avatar nielifeng avatar rickyhuo avatar ruanwenjun avatar simon824 avatar songjianet avatar tmljob avatar tobezhou33 avatar tyrantlucifer avatar wuchunfu avatar xleoken avatar yx91490 avatar zhaomin1423 avatar zhongjiajie avatar zhuangchong avatar zixi0825 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.