Code Monkey home page Code Monkey logo

featinsight's Introduction

FeatInsight - a Feature Platform Based on OpenMLDB

Introduction

FeatInsight is a feature computation and store platform, leveraging OpenMLDB for efficient feature computation, management and orchestration. FeatInsight provides a user-friendly user interface, allowing users to perform the entire process of feature engineering for machine learning, including data import, viewing and update, feature generation, store and online deployment. For offline scenarios, user can choose features for training sample generation for ML training; for online scenarios, user can deploy online feature services for real-time feature computations.

The main objective for FeatInsight is to tackle engineering problems in the process of machine learning, provide a convenient tool for users to perform feature extraction

The main objective of FeatInsight is to address common challenges in machine learning development, including facilitating easy and quick feature extraction, transformation, combination, and selection, managing feature lineage, enabling feature reuse and sharing, version control for feature services, and ensuring consistency and reliability of feature data used in both training and inference processes. Application scenarios include deploying online feature services, building MLOps workflows, establishing FeatureStore platforms, reusing open-source feature solutions, and using it as a business component in applications such as recommendation systems, natural language processing, finance, healthcare, and other areas of machine learning implementation.

FeatInsight offers key functionalities including data management, feature management, online feature services deployment, offline samples generation, SQL Playground, pre-computed features, and more.

Installation and Deployment

FeatInsight provides multiple ways for deployment. For more details, please refer to the documentation.

Docker

Prepare configuration file and name it as application.yml.

server:
  port: 8888
 
openmldb:
  zk_cluster: 127.0.0.1:2181
  zk_path: /openmldb
  apiserver: 127.0.0.1:9080

Start Docker container.

docker run -d -p 8888:8888 -v `pwd`/application.yml:/app/application.yml registry.cn-shenzhen.aliyuncs.com/tobe43/featinsight

All-in-One Image

This image additionally includes a portable OpenMLDB deployment and configuration files, good for testing purposes.

docker run -d -p 8888:8888 registry.cn-shenzhen.aliyuncs.com/tobe43/portable-openmldb

Installation Package

Prepare configuration file application.yml.

wget https://openmldb.ai/download/featinsight/featinsight-0.1.0-SNAPSHOT.jar

java -jar ./featinsight-0.1.0-SNAPSHOT.jar

Usage

FeatInsight service can be accessed using any web browser at http://127.0.0.1:8888/.

The general usage process is as follows:

  1. Importing Data: Create databases, create data tables, import online data, and import offline data using SQL commands or frontend forms.
  2. Creating Features: Define feature views using SQL statements. FeatInsight will use a SQL compiler to analyze the features and create corresponding entities.
  3. Offline Scenario: Select the desired features to import. You can choose features from different feature views simultaneously and use distributed computing to import sample files into local or distributed storage.
  4. Online Scenario: Select the desired features to go live. Publish them as an online feature extraction service with one click, and then use an HTTP client to request and return online feature extraction results.

We provide a comprehensive guide using dummy data to show a whole usage process, please refer to QuickStart.

Documentation

For more details, please refer to FeatInsight Documentation.

featinsight's People

Contributors

aceforeverd avatar elliezza avatar lumianph avatar qichenx avatar shanbinbin avatar tobegit3hub avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

featinsight's Issues

Resolve the security issue of log4j

Here is the security report.

漏洞名称    
Apache Log4j安全漏洞(CVE-2021-45046)
漏洞标签    
必修漏洞
远程利用
存在POC
漏洞类型    
应用漏洞
威胁等级    严重
CVE编号    
CVE-2021-45046
披露时间    
2021-12-15
漏洞描述    
ApacheLog4j是美国阿帕奇(Apache)基金会的一款基于Java的开源日志记录工具。Log4j2.16.0之前的版本(不包括2.3.1, 2.12.2)存在安全漏洞,该漏洞源于当日志配置使用非默认模式布局和上下文查找或线程上下文映射模式使用JNDI查找模式制作恶意输入数据,从而导致拒绝服务攻击。
修复方案
修复方案    
目前厂商已发布升级补丁以修复漏洞,建议受影响的用户,及时升级至2.16.0及以上版本。
补丁获取链接:https://logging.apache.org/log4j/2.x/security.html。
扫描到服务器存在漏洞风险,建议立即对相关主机进行快照备份,避免遭受损失。

参考链接    
http://www.openwall.com/lists/oss-security/2021/12/14/4

https://logging.apache.org/log4j/2.x/security.html

https://www.cve.org/CVERecord?id=CVE-2021-44228

影响主机范围(15)
全部
     主机名称/实例ID    IP地址    
防护版本
服务器状态
扫描时间    说明    状态    操作
     
ins-j7e7564v
暂无标签
公152.136.144.33
内172.21.80.57
运行中    
首次:2024-02-22 01:44:57
最近:2024-04-07 04:10:31
/home/common-user/featinsight_80/./featinsight-0.1.1-SNAPSHOT.jar/BOOT-INF/lib/log4j-core-2.14.1.jar:2.14.1 /home/common-user/featinsight/featinsight-0.1.1-SNAPSHOT.jar/BOOT-INF/lib/log4j-core-2.14.1.jar:2.14.1 /home/common-user/featinsight/./featinsight-0.1.1-SNAPSHOT.jar/BOOT-INF/lib/log4j-core-2.14.1.jar:2.14.1 /home/common-user/featinsight_80/featinsight-0.1.1-SNAPSHOT.jar/BOOT-INF/lib/log4j-core-2.14.1.jar:2.14.1    
待修复

Update active menu item when watching the changes

Here is the example code.

<template>
  <div>
    <a-menu :default-selected-keys="[activeMenu]" mode="vertical" theme="dark">
      <a-menu-item key="component-a">
        <router-link to="/component-a">Component A</router-link>
      </a-menu-item>
      <a-menu-item key="component-b">
        <router-link to="/component-b">Component B</router-link>
      </a-menu-item>
      <a-menu-item key="component-c">
        <router-link to="/component-c">Component C</router-link>
      </a-menu-item>
      <a-menu-item key="component-d">
        <router-link to="/component-d">Component D</router-link>
      </a-menu-item>
    </a-menu>
    <router-view />
  </div>
</template>

<script>
export default {
  data() {
    return {
      activeMenu: 'component-a',
    };
  },
  watch: {
    $route(to, from) {
      this.activeMenu = to.name || 'component-a';
    },
  },
};
</script>

Support multiple lines SQL for feature services

Now we can not construct the correct deployment SQL for multiple lines SQL.

image

{"name":"gg","db":"DOG","sql":"SELECT ID, COUNT(NAME) over w as agg from DOG.GOLDEN \nWINDOW w as ( PARTITION BY NAME order by BIRTH ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)","description":"","featureNames":["ID","agg"],"featureDescriptionMap":{}}
executeSQL fail: [1001] create logic plan tree failed--Syntax error: Unclosed string literal [at 1:116]
...description, feature_names) values ('gg', 'DOG', 'SELECT ID, COUNT(NAME) o...

nginx reverse proxy

Make sure reverse proxy can work, especially check the case below:

http {
    server {
        listen 80;
        location /foo/ {

The location is not just /.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.