Code Monkey home page Code Monkey logo

monaco-sql-languages's Introduction

Monaco SQL Languages

NPM version NPM downloads

English | 简体中文

This project is based on the SQL language project of Monaco Editor, which was forked from the monaco-languages.The difference is that Monaco SQL Languages supports various SQL languages and the corresponding advanced features for the Big Data field.


Feature highlights

  • Code Highlighting
  • Syntax Validation
  • Code Completion

Powered By dt-sql-parser


Online Preview

https://dtstack.github.io/monaco-sql-languages/

Powered By molecule.


Supported SQL Languages

  • MySQL
  • Flink
  • Spark
  • Hive
  • Trino (Presto)
  • PostgreSQL
  • Impala

Installing

npm install monaco-sql-languages

Tips: Monaco SQL Languages is only guaranteed to work stably on [email protected] for now.


Integrating


Usage

  1. Import language contributions

    Tips: If integrated via MonacoEditorWebpackPlugin, it will help us to import contribution files automatically. Otherwise, you need to import the contribution files manually.

    import 'monaco-sql-languages/esm/languages/mysql/mysql.contribution';
    import 'monaco-sql-languages/esm/languages/flink/flink.contribution';
    import 'monaco-sql-languages/esm/languages/spark/spark.contribution';
    import 'monaco-sql-languages/esm/languages/hive/hive.contribution';
    import 'monaco-sql-languages/esm/languages/trino/trino.contribution';
    import 'monaco-sql-languages/esm/languages/pgsql/pgsql.contribution';
    import 'monaco-sql-languages/esm/languages/impala/impala.contribution';
    
    // Or you can import all language contributions at once.
    // import 'monaco-sql-languages/esm/all.contributions';
  2. Setup language features

    You can setup language features via setupLanguageFeatures. For example, setup code completion feature of flinkSQL language.

    import { LanguageIdEnum, setupLanguageFeatures } from 'monaco-sql-languages';
    
    setupLanguageFeatures(LanguageIdEnum.FLINK, {
        completionItems: {
            enable: true,
            triggerCharacters: [' ', '.'],
            completionService: //... ,
        }
    });

    By default, Monaco SQL Languages only provides keyword autocompletion, and you can customize your completionItem list via completionService.

    import { languages } from 'monaco-editor/esm/vs/editor/editor.api';
    import {
        setupLanguageFeatures,
        LanguageIdEnum,
        CompletionService,
        ICompletionItem,
        SyntaxContextType
     } from 'monaco-sql-languages';
    
    const completionService: CompletionService = function (
        model,
        position,
        completionContext,
        suggestions, // syntax context info at caretPosition
        entities // tables, columns in the syntax context of the editor text
    ) {
        return new Promise((resolve, reject) => {
            if (!suggestions) {
                return Promise.resolve([]);
            }
            const { keywords, syntax } = suggestions;
            const keywordsCompletionItems: ICompletionItem[] = keywords.map((kw) => ({
                label: kw,
                kind: languages.CompletionItemKind.Keyword,
                detail: 'keyword',
                sortText: '2' + kw
            }));
    
            let syntaxCompletionItems: ICompletionItem[] = [];
    
            syntax.forEach((item) => {
                if (item.syntaxContextType === SyntaxContextType.DATABASE) {
                    const databaseCompletions: ICompletionItem[] = []; // some completions about databaseName
                    syntaxCompletionItems = [...syntaxCompletionItems, ...databaseCompletions];
                }
                if (item.syntaxContextType === SyntaxContextType.TABLE) {
                    const tableCompletions: ICompletionItem[] = []; // some completions about tableName
                    syntaxCompletionItems = [...syntaxCompletionItems, ...tableCompletions];
                }
            });
    
            resolve([...syntaxCompletionItems, ...keywordsCompletionItems]);
        });
    };
    
    setupLanguageFeatures(LanguageIdEnum.FLINK, {
        completionItems: {
            enable: true,
            completionService: //... ,
        }
    });
  3. Create the Monaco Editor instance and specify the language you need

    import { LanguageIdEnum } from 'monaco-sql-languages';
    
    monaco.editor.create(document.getElementById('container'), {
        value: 'select * from tb_test',
        language: LanguageIdEnum.FLINK // languageId
    });

Monaco Theme

Monaco SQL Languages plan to support more themes in the future.

Monaco SQL Languages provides built-in Monaco Theme that is named vsPlusTheme. vsPlusTheme inspired by vscode default plus colorTheme and it contains three styles of themes inside:

  • darkTheme: Inherited from monaco built-in theme vs-dark;
  • lightTheme: Inherited from monaco built-in theme vs;
  • hcBlackTheme: Inherited from monaco built-in theme hc-black;

Use Monaco SQL Languages built-in vsPlusTheme

import { vsPlusTheme } from 'monaco-sql-languages';
import { editor } from 'monaco-editor';

// import themeData and defineTheme, you can customize the theme name, e.g. sql-dark
editor.defineTheme('sql-dark', vsPlusTheme.darkThemeData);
editor.defineTheme('sql-light', vsPlusTheme.lightThemeData);
editor.defineTheme('sql-hc', vsPlusTheme.hcBlackThemeData);

// specify the theme you have defined
editor.create(null as any, {
    theme: 'sql-dark',
    language: 'flinksql'
});

Customize your own Monaco theme

import { TokenClassConsts, postfixTokenClass } from 'monaco-sql-languages';

// Customize the various tokens style
const myThemeData: editor.IStandaloneThemeData = {
    base: 'vs-dark',
    inherit: true,
    rules: [
        { token: postfixTokenClass(TokenClassConsts.COMMENT), foreground: '6a9955' },
        { token: postfixTokenClass(TokenClassConsts.IDENTIFIER), foreground: '9cdcfe' },
        { token: postfixTokenClass(TokenClassConsts.KEYWORD), foreground: '569cd6' },
        { token: postfixTokenClass(TokenClassConsts.NUMBER), foreground: 'b5cea8' },
        { token: postfixTokenClass(TokenClassConsts.STRING), foreground: 'ce9178' },
        { token: postfixTokenClass(TokenClassConsts.TYPE), foreground: '4ec9b0' }
    ],
    colors: {}
};

// Define the monaco theme
editor.defineTheme('my-theme', myThemeData);

postfixTokenClass is not required in most cases, but since Monaco SQL Languages has tokenPostfix: 'sql' internally set for all SQL languages, in some cases your custom style may not work if you don't use postfixTokenClassClass to handle TokenClassConsts.*.


Dev: cheat sheet

  • initial setup

    pnpm install
  • open the dev web

    pnpm watch-esm
    cd website
    pnpm install
    pnpm dev
  • build

    pnpm build
  • run test

    pnpm test
    

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.


License

MIT

monaco-sql-languages's People

Contributors

abacigil avatar aeschli avatar alanvf avatar alefragnani avatar alexdima avatar avelino avatar finalfantasia avatar fonsecas72 avatar goober99 avatar haydenorz avatar hirse avatar langpavel avatar larshp avatar lessmost avatar liuxy0551 avatar luckyfbb avatar marlenecota avatar masad-frost avatar mbtools avatar mradkov avatar nrayburn-cyber avatar olane avatar ozgunozen avatar rcjsuen avatar rebornix avatar sabolch avatar salvoravida avatar serhioromano avatar spahnke avatar wewoor avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

monaco-sql-languages's Issues

How to make work with code splitting/async loading?

It seems the setup happens in an odd way, which makes it hard to work when we need to rely on state to load the completionService (load table, catalog metadata).

We are not seeing the language be registered correctly (when we navigate within SPA and monaco is already loaded)

Is there a way to register the language/completionService from e.g. a useEffect or programmatically, that doesn't rely on global loading?

Roadmap 2023

Roadmap 2023

Q1

Q2

  • FlinkSQL 语法完善, 主要是 Query、Create、Insert 几种语法

Q3

  • HiveSQL 语法纠错与完善,补充单元测试,并支持自动补全
  • SparkSQL 语法纠错与完善,补充单元测试,并支持自动补全
  • TrinoSQL 语法纠错与完善,补充单元测试,并支持自动补全

Q4

  • GenericSQL 语法纠错与完善,补充单元测试,并支持自动补全
  • PostgreSQL 语法纠错与完善,补充单元测试,并支持自动补全
  • ImpalaSQL 语法纠错与完善,补充单元测试,并支持自动补全
  • 集成新版本 dt-sql-parser 到 BatchWorks

References

PGSQL support?

Hi, on the homepage, there is a mention of PGSQL support.

Where can I find the relative worker?

cannot read properties of undefined

I'm getting this error trying to use with trinosql:

TypeError: Cannot read properties of undefined (reading 'get')
    at _CodeCompletionCore.collectCandidates (http://localhost:3000/trinosql.worker.js:68773:33)
    at TrinoSQL.getSuggestionAtCaretPosition (http://localhost:3000/trinosql.worker.js:50222:29)
    at TrinoSQLWorker.<anonymous> (http://localhost:3000/trinosql.worker.js:63071:41)
    at Generator.next (<anonymous>)
    at http://localhost:3000/trinosql.worker.js:63030:67
    at new Promise (<anonymous>)
    at __awaiter (http://localhost:3000/trinosql.worker.js:63012:10)
    at TrinoSQLWorker.doComplete (http://localhost:3000/trinosql.worker.js:63068:12)
    at EditorSimpleWorker.fmr (http://localhost:3000/trinosql.worker.js:61720:58)
    at SimpleWorkerServer._handleMessage (http://localhost:3000/trinosql.worker.js:58454:59)

Update docs

主要内容

  1. 目前已支持的功能

  2. 如何在项目中使用

    • 在webpack 项目中如何使用
    • 在 vite 项目中如何使用
    • 浏览器原生环境中使用
  3. 本地运行 demo 脚本

  • yarn run compile
  • cd website && yarn run dev

Integrating new pgsql parser

TODO List

  • 支持 pgsql parser 自动补全功能
  • 高亮功能优化,包括关键字,内置函数,操作符,数据类型,各种字面量规则等等,对应的语言文件所有的规则和设置都需要重新检查
  • 更新 website

使用正则表达式实现语法高亮

我想要使用正则表达式进行语法高亮,类似于@23asdfsaf, @dsgw434536, @iohiyo, 请问哪里有类似的案例,具体在第几行到第几行之间

Is the demo breaking?

main.js 404

After clicking parse there's an error.

Nothing valuable in the problems & output tab.

image

support monaco.languages.CompletionList.incomplete

I need to trigger the 'incomplete' logic in the return value of Monaco Editor's provideCompletionItems. Currently, it seems that only 'suggestion' is exported. If it doesn't interfere with your normal plans, I can provide a pull request. My fork: https://github.com/Kijin-Seija/monaco-sql-languages


我需要触发Monaco Editor provideCompletionItems 返回值中的 incomplete 逻辑,我看现在只导出了suggestion。
如果不影响你们正常计划的话,我可以提供pr,我的fork: https://github.com/Kijin-Seija/monaco-sql-languages

image

image

【BUG】Dependency path error problem

First of all, I appreciate your efforts to fix the wrong path issue with dt-sql-parser.

I found a problem: installing monaco-sql-languages still encounters the dt-sql-parser wrong path (node_modules\dt-sql-parser\src\xxx') problem. So I hope monaco-sql-languages can also fix this error. As well as being able to update the documentation

Thanks again.

替换 Monaco 内置的图标

Why?

monaco-sql-languages 支持的 language 都是各种 SQL,与之对应的就会有许多符号/icon是SQL语言特有的,比如 database 、table 等,而monaco 内置并不支持这些符号/icon,目前自动补全项左侧的 icon 问题尤为明显:

image

方案概述

  1. 制作自动补全项对应的 iconfont
  2. 新增 css 文件,覆盖monaco 内置的字体图标

字体图标风格

与monaco/vscode 的字体图标风格保持一致 https://code.visualstudio.com/api/references/icons-in-labels#icon-listing
image

字体图标表

类型 icon name
keyword 关键字 symbol-keyword
Snippet 代码片段 symbol-snippet
catalog symbol-catalog
database/schema 数据库 symbol-database
table 表 symbol-table
view 视图 symbol-view
column 字段 symbol-column
function 函数 symbol-function
procedure 存储过程 symbol-procedure

SQL 补全项类型与 monaco 内置的 CompletionItemKind 映射表

自动补全项左侧会根据不同的自动补全项的 CompletionItemKind 属性显示不同的 icon,但是内置的类型并不能覆盖所有的 SQL 补全项类型,比如 table、database 等,所以需要一个类型映射

类型 CompletionItemKind
keyword 关键字 CompletionItemKind.Keyword
Snippet 代码片段 CompletionItemKind.Snippet
catalog CompletionItemKind.Folder
database/schema 数据库 CompletionItemKind.File
table 表 CompletionItemKind.Class
view 视图 CompletionItemKind.Interface
column 字段 CompletionItemKind.Field
function 函数 CompletionItemKind.Function
procedure 存储过程 CompletionItemKind.Constructor

同时,monaco-sql-languages 需要新增公开的辅助方法,用于将 dt-sql-parser 中的 实体类型(EntityContextType/ SyntaxContextType )转换为 CompletionItemKind

import { SyntaxContextType } from 'dt-sql-parser';
import { languages } from 'monaco-editor/esm/vs/editor/editor.api';

export function toCompletionItemKind(type: SyntaxContextType): languages.CompletionItemKind {
        switch(type) {
		case SyntaxContextType.CATALOG:
			return languages.CompletionItemKind.Folder;
		case SyntaxContextType.DATABASE:
			return languages.CompletionItemKind.File;
		// ...
		default:
			 return languages.CompletionItemKind.Text;
	}
}

覆盖 icon 的 css 文件参考

每一种类型对应的icon 类名格式为: .codicon-${built-in-icon-name}

内置的 icon 名称看这里: https://code.visualstudio.com/api/references/icons-in-labels#icon-listing

自动补全项的icon名称都是以 symbol- 为前缀的

  .codicon-symbol-text:before {
      content: '\eb01';
  }
  .codicon-symbol-function:before {
      content: '\eb02';
  }
  .codicon-symbol-folder:before {
      content: '\eb03';
  }
  .codicon-symbol-field:before {
      content: '\eb04';
  }

Can not load language due to _EditorSimpleWorker.loadForeignModule error

Using Vite and standard configuration like yours https://github.com/DTStack/monaco-sql-languages/blob/main/website/vite.config.ts

languageWorker.ts

self.MonacoEnvironment = {
  async  getWorker(_, label) {
    let worker;
    
    switch (label) {
      case 'trinosql':
        worker = await import('monaco-sql-languages/out/esm/trinosql/trinosql.worker?worker');
        break;
      default:
        worker = await import('monaco-editor/esm/vs/editor/editor.worker?worker');
    }

    return new worker.default();
  },
};

monacoInit.ts

import { loader } from '@monaco-editor/react';
import * as monaco from 'monaco-editor';

import 'monaco-sql-languages/out/esm/trinosql/trinosql.contribution';
import {
  setupLanguageFeatures,
  LanguageIdEnum,
} from 'monaco-sql-languages';

import './language-worker.ts';
import { theme } from './theme';

setupLanguageFeatures({
  languageId: LanguageIdEnum.TRINO,
  completionItems: false
});

loader.config({ monaco });

loader.init().then((monaco) => {
  monaco.editor.defineTheme('theme', theme);
});

Editor.tsx

import MonacoEditor from '@monaco-editor/react';
import { LanguageIdEnum } from 'monaco-sql-languages';

export const Editor = (editorOptions) => {
 return (
    <MonacoEditor
      height="100%"
      width="100%"
      defaultLanguage={LanguageIdEnum.TRINO}
      options={editorOptions}
    />
  );
}

image

构建相关

  • monaco-sql-languages 源的话 有 Npm Yarn Pnpm. 废弃采用Pnpm
  • website 工程 scripts
    • dev vs preview 两个脚本区别?是否要废弃1个

Uncaught ReferenceError: Cannot access 'languageDefinitions' before initialization

Version
"monaco-editor": "0.31.0",
"monaco-sql-languages": "0.12.0-beta.10",

The Type of SQL
SparkSQL | FlinkSQL | HiveSQL | Impala

Your Code
chainWebpack: (config) => {
config.plugin('definePlugin').use(new webpack.DefinePlugin({
'process.env.NODE_DEBUG': process.env.NODE_DEBUG,
}))
config.plugin('monaco').use(new MonacoEditorWebpckPlugin({
features: [], // 包含你所需要的 Monaco Editor 功能
languages: [], // 包含你所需要的 Monaco Editor 内置语言功能
customLanguages: [
// 包含 Monaco SQL Languages 所提供的语言功能
{
label: 'mysql',
entry: 'monaco-sql-languages/out/esm/mysql/mysql.contribution',
worker: {
id: 'monaco-sql-languages/out/esm/mysql/mySQLWorker',
entry: 'monaco-sql-languages/out/esm/mysql/mysql.worker'
}
},
{
label: 'flinksql',
entry: 'monaco-sql-languages/out/esm/flinksql/flinksql.contribution',
worker: {
id: 'monaco-sql-languages/out/esm/flinksql/flinkSQLWorker',
entry: 'monaco-sql-languages/out/esm/flinksql/flinksql.worker'
}
},
{
label: 'sparksql',
entry: 'monaco-sql-languages/out/esm/sparksql/sparksql.contribution',
worker: {
id: 'monaco-sql-languages/out/esm/sparksql/sparkSQLWorker',
entry: 'monaco-sql-languages/out/esm/sparksql/sparksql.worker'
}
},
{
label: 'hivesql',
entry: 'monaco-sql-languages/out/esm/hivesql/hivesql.contribution',
worker: {
id: 'monaco-sql-languages/out/esm/hivesql/hiveSQLWorker',
entry: 'monaco-sql-languages/out/esm/hivesql/hivesql.worker'
}
}
]
}));

// create
setupLanguageFeatures({
languageId: LanguageIdEnum.FLINK,
// completionService,
});
monaco.editor.create(editorRef.current as HTMLDivElement, {
value: 'SELECT * FROM tb_test',
theme: 'sql-dark',
language: LanguageIdEnum.FLINK,
});

Problem
Uncaught ReferenceError: Cannot access 'languageDefinitions' before initialization

Worker not found and Unexpected usage at EditorSimpleWorker.loadForeignModule

我的Wepack是在vue.config.js里配置的:
My Webpack is configured in vue.config.js:

module.exports = {
  configureWebpack: {
    entry: {
      'sparksql.worker':
        'monaco-sql-languages/out/esm/sparksql/sparksql.worker.js',
      'sql.worker': 'monaco-sql-languages/out/esm/sql/sql.worker.js',
    }
  }
}

在封装的monaco-editor组件中:
Inside my monaco-editor component:

import * as monaco from 'monaco-editor';
import 'monaco-sql-languages/out/esm/sparksql/sparksql.contribution';
import 'monaco-sql-languages/out/esm/sql/sql.contribution';

MonacoEnvironment = {
  getWorkerUrl: function (moduleId, label) {
    console.log(moduleId, label);
    switch (label) {
      case 'sparksql': {
        return './sparksql.worker.js';
      }
      default: {
        return './editor.worker.js';
      }
    }
  },
};

可是项目跑起来后报错:
But I get the following error in the console:

GET http://localhost:3000/xxx/sparksql.worker.js 404 (Not Found)
GET http://localhost:3000/xxx/editor.worker.js 404 (Not Found)
Error: Unexpected usage
    at EditorSimpleWorker.loadForeignModule (editorSimpleWorker.js:460:1)
    at eval (webWorker.js:38:1)

以下是部分package.json
Here is part of my package.json

"dt-sql-parser": "^4.0.0-beta.3.2",
"monaco-editor": "0.33",
"monaco-editor-webpack-plugin": "^7.0.1",
"monaco-sql-languages": "^0.11.0",
"webpack": "5.61.0",

这个应该怎么解决呢?
How can I resolve this?

在monaco-editor中无法使用hive-sql插件

使用monaco-sql- language 0.9.4,无法实现关键词提示、错误提示功能。
我这边用的monaco-editor版本是0.25.0

使用其他自带的如:html.worker,关键词提示都是正常的

如何添加一个自定义语言实现语法高亮

72d1c68c6db15be7cf6e62659d3f89f
如果按照这个添加语言的方法去添加4个文件在里面,但是这些文件都是本地文件

依据taier的案例来看,taier导入是依据导入下载包的方法,这个方法是通过yarn指令下载monaco-sql-languages到node_modules中,这个与本地文件没什么关系把
222 (2)

Module has no exports.

I'm trying to incorporate monaco-sql-languages into existing project. I have followed the docs. When I run my app I get this error:

export 'default' (imported as 'FlinkSQLWorker') was not found in 'monaco-sql-languages/out/esm/flinksql/flinksql.worker?worker' (module has no exports)

Support more monaco theme

可以参考 vscode 相关插件

Todo List

  • 整理并统一各个类型 sql 的Monarch 配置内部的 token
  • 开发对应的 theme
    • lightTheme
    • darkTheme
    • hcTheme
    • hcDarkTheme

Presto/Trino SQL Syntax?

Hi! Thanks for this.

Any chance you could implement Trino SQL (formerly known as Presto SQL) syntax?

Sql Formatter support

Hi there, thanks for this lib!

I have added support for formatting, lmk if you'd like a PR

it's pretty simple, something like this:

import { languages } from 'monaco-editor/esm/vs/editor/editor.api';
import { format } from 'sql-formatter';

languages.registerDocumentFormattingEditProvider('sql', {
   provideDocumentFormattingEdits(model, options) {
      const formatted = format(model.getValue(), {
         indentStyle: 'standard',
      });
      return [
         {
            range: model.getFullModelRange(),
            text: formatted,
         },
      ];
   },
});

languages.registerDocumentRangeFormattingEditProvider('sql', {
   provideDocumentRangeFormattingEdits(model, range, options) {
      const formatted = format(model.getValueInRange(range), {
         indentStyle: 'standard',
      });
      return [
         {
            range: range,
            text: formatted,
         },
      ];
   },
});

Monaco CompletionItem documentation not showing?

Has anyone been able to get Monaco completion documentation to show? I am returning the documentation property in my CompletionItem from my completionService, but I don't see it render. The label and detail show correctly. I am not sure why, am trying to rule out if it is caused by this package or some other setting.

Documentation should look like the following btw (the right window)

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.