A versatile script engine abstraction layer
ScriptX is a script engine abstraction layer. A variety of script engines are encapsulated on the bottom and a unified API is exposed on the top, so that the upper-layer caller can completely isolate the underlying engine implementation (back-end).
ScriptX not only isolates several JavaScript engines, but can even isolate different scripting languages, so that the upper layer can seamlessly switch between scripting engine and scripting language without changing the code.
In ScriptX terminology, "front-end" refers to the external C++ API, and "back-end" refers to different underlying engines. The currently implemented back-ends include: V8, node.js, JavaScriptCore, WebAssembly, Lua.
backend | language | version | states |
---|---|---|---|
V8 | JavaScript | 7.4+ (tested versions) |
done |
JavaScriptCore | JavaScript | 7604.1.38.0.7+ (iOS 10+/macOS10.12+) |
done |
Node.js | JavaScript | 14.x+ | done |
QuickJs | JavaScript | 2024-01-13 | done |
WebAssembly | JavaScript | Emscripten-2.0.5+ | done |
Lua | Lua | 5.4+ | done |
CPython | Python | todo | |
YARV | Ruby | todo | |
Mono | C# | todo |
The interface of ScriptX uses modern C++ features. And to be 100% in line with the C++ standard, completely cross-platform.
All APIs are exposed in the ScriptX.h
aggregate header file.
Design goals: Multi-language | Multi-engine implementation | High performance | API easy to use | Cross-platform
We use a relatively complete code to leave an overall impression of ScriptX.
EngineScope enter(engine);
try {
engine->eval("function fibo(x) {if (x<=2) return 1; else return fibo(x-1) + fibo(x-2) }");
Local<Function> fibo = engine->get("fibo").asFunction();
Local<Value> ret = fibo.call({}, 10);
ret.asNumber().toInt32() == 55;
auto log = Function::newFunction(
[](const std::string& msg) {
std::cerr << "[log]: "<< msg << std::endl;
});
// or use: Function::newFunction(std::puts);
engine->set("log", log);
engine->eval("log('hello world');");
auto json = engine->eval(R"( JSON.parse('{"length":1,"info":{"version": "1.18","time":132}}'); )")
.asObject();
json.get("length").asNumber().toInt32() == 1;
auto info = json.get("info").asObject();
info.get("version").asString().toString() == "1.18";
info.get("time").asNumber().toInt32() == 132;
Local<Object> bind = engine->eval("...").asObject();
MyBind* ptr = engine->getNativeInstance<MyBind>(bind);
ptr->callCppFunction();
} catch (Exception& e) {
FAIL() << e.message() << e.stacktrace();
// or FAIL() << e;
}
- Use
EngineScope
to enter the engine environment - Most APIs can accept C++ native types as parameters and automatically convert types internally
- Script functions can be created directly from C/C++ functions (native binding)
- Support script exception handling
- API strong typing
At the beginning of the design of ScriptX, the goal was to support multiple scripting languages, and the engine package of V8 and JavaScriptCore was implemented on JavaScript. In order to verify the multi-language design of ScriptX, a complete Lua binding was implemented. Currently support for WebAssembly has also been completed.
API design conforms to modern C++ style, such as:
- Three reference types Local/Global/Weak, using copy and move semantics to achieve automatic memory management (automatic reference counting)
- Use variadic template to support the very convenient Function::call syntax
- Use Template Meta-Programing to directly bind C++ functions
Modern language features, refer to null pointer safety (nullability safety please refer to the concept of kotlin).
Note: ScriptX requires C++17 (or 1z) or higher compiler support, and needs to turn on the exception feature (you can turn off the RTTI feature).
High performance is an important indicator in the design of ScriptX. The C++ idea of Zero-Overhead is also fully embodied in the implementation process. And pass relevant performance tests when adding functional features.
Test indicator: single time consuming from JS to C++ function call, microsecond
Test environment: iMac i9-9900k 32G RAM @macOS 10.15
Test Index: single function call from js to C++ in micro seconds. Test Environment: iMac i9-9900k 32G RAM @macOS 10.15
The performance test shows that in Release mode, ScriptX can achieve almost the same performance as the native binding. (Because ScriptX uses a large number of templates, please do not perform performance testing in the Debug version)
ScriptX has realized the ability of script exceptions and C++ exceptions to communicate with each other through a series of technical means. There is no need to judge the return value when calling the engine API, and unified exception handling can be used to avoid crashes.
For example, users can catch exceptions thrown by js code in the C++ layer, and get the message and stack; they can also throw a C++ exception (script::Exception
) in the native function and pass it through to the js code.
For details, please refer to ExceptionTest and Related Documents
Easy-to-use API => Happy Engineer => Efficient => High Quality
ScriptX was designed with full consideration of the ease of use of the API, including friendly and simple operation, not easy to make mistakes, obvious error messages, and easy to locate problems. Under this guiding ideology, ScriptX has done a lot of things that native engines can't do.
For example: V8 does not perform GC when destroying, resulting in many bound native classes that cannot be released. ScriptX does additional logic to handle this situation.
V8 and JSCore require that other APIs of ScriptX cannot be called in the finalize callback, otherwise it will crash, which also makes the code logic difficult to implement. ScriptX uses MessageQueue to avoid this problem perfectly.
The global references of V8 and JSCore must be released before the engine is destroyed, otherwise it will cause problems such as crash and failure to destroy. ScriptX guarantees to actively reset all Global / Weak references during Engine destruction.
When an app is used as a host to use a scripting engine, it is usually necessary to inject a large number of native-bound functions/classes to provide capabilities for the scripting logic. The ClassDeifine
related binding API designed by ScriptX is simple and easy to use, and can support direct binding of C++ functions, which greatly improves work efficiency.
While ScriptX provides engine encapsulation, it also provides a set of tools and methods to achieve mutual conversion between native types and ScriptX types.
For details, please refer to InteroperateTest and Related Documents
High code quality requirements
- Hundreds of test cases, UnitTests coverage rate reaches 90+%
- The cyclomatic complexity is only 1.18.
- Use clang-format to ensure uniform code format.
- Use clang-tidy to find potential problems.
- Both clang and MSVC compilers have opened "warning as error" level error messages.
root
├── README.md
├── src
│ ├── Engine.h
│ └── ...
├── backend
│ ├── JavaScriptCore
│ ├── Lua
│ ├── Python
│ ├── QuickJs
│ ├── Ruby
│ ├── SpiderMonkey
│ ├── Template
│ ├── V8
│ ├── WKWebView
│ └── WebAssembly
├── docs
│ ├── Basics.md
│ └── ...
└── test
├── CMakeLists.txt
└── src
├── Demo.cc
└── ...
src
: External API, mainly header filesbackend
: Implementation of various engine backendsdocs
: Rich documentationtest
: Various unit tests
Some important classes in ScriptX:
ScriptEngine
EngineScope
Exception
Value
,Null
,Object
,String
,Number
,Boolean
,Function
,Array
,ByteBuffer
,Unsupported
Local<Value>
,Local<Null>
,Local<Object>
,Local<String>
,Local<Number>
,Local<Boolean>
,Local<Function>
,Local<Array>
,Local<ByteBuffer>
,Local<Unsupported>
Global<T>
,Weak<T>
Before officially using ScriptX, please spend half an hour read carefully the project documents, and be familiar with several concepts in ScriptX.
scriptx's People
Forkers
ryecrow xiaoying18 agindage wurq skymysky qicosmos kerwinzxc hanhailong alenstarx vn-os jimmy54 bobhan1 zzh-527 dingyong4223 mensong dut3062796s linecode yqs112358 brinkqiang hadryan cageq warfarecode andycall qingyangmoke externalrepositories bryainzhang metricix gerhobbelt liuwjchinal wenyu826 heshan0x1 wittech wldp andy-spitfire watch-later redchew-fork 1506978348 hwrom archie3d xaioxw vbirds jrpat zhangshuangjun woeoio getsync wonderinteractive isabella232 liteldev lzhice ltears cl1933404370 stevezhou6 elizaher dl-scripts kennisl frazurbluu shism2 lyoko-jeremie thinkingpanda caut sosovia dangyajun meowboy326 ezhangle beijiaan andyjtb telluriumdevscriptx's Issues
Add new APIs to "eval a file" in Engine.h
如题,至今引擎加载源码都只能依靠读取文件然后eval,在适配其他引擎过程中此种加载方式会带来一些困扰
比如
- V8 engine 和 embedded NodeJs 需要使用.mjs后缀判定此文件为ES Module文件,以启动ES Module支持
- Python解释器用import可以更方便地实现实例隔离,而非使用其标准中尚不太完善的subinterpreter
- 等等
因此有必要为Engine.h提供支持加载指定路径文件的eval,并在不同引擎做特殊处理
请问如果使用v8 on windows 是要单独编译下v8吗
请问如果使用v8 on windows 是要单独编译下v8吗
cmake 的项目 看文档中还要 libv8.a
关于引入到项目中的文档 可以再详细点吗 门槛感觉有点高
Memory leak in QuickJs backend
Hi,
When compiled with QuickJs backend I experience memory leaks reported by MSVC compiler.
I use std::shared_ptr<script::ScriptEngine>
with provided script::ScriptEngine::Deleter()
.
One problem is inside QjsEngine::destroy()
which is missing the delete this;
statement.
And another one is in MessageQueue.cc
because of the global static runningQueue_
.
Could be a false-positive, but it can be fixed by moving the runningQueue_
inside the getRunningQueue()
function:
using RunnineQueueMap = std::unordered_map<MessageQueue*, int>;
static inline RunnineQueueMap& getRunningQueue() {
SCRIPTX_THREAD_LOCAL(RunnineQueueMap, runningQueue_);
return internal::getThreadLocal(runningQueue_);
}
Thanks!
Great project!
在同一次构建中更换编译开关重复构建ScriptX时报错
本来打算在同一次构建中对每种语言分别生成一个Target的输出
每次include( ScriptX/CMakeLists.txt )前先修改SCRIPTX_BACKEND
结果cmake报错
CMake Error at src/ScriptX/CMakeLists.txt:50 (add_library):
[cmake] add_library cannot create target "ScriptX" because another target with the
[cmake] same name already exists. The existing target is a static library created
[cmake] in source directory
也就是说在同一次构建中不能有两个同名对象
那该如何实现同时构建多种语言的多个target呢
或者有无办法实现构建完一个target后执行clean
demo代码运行异常.failed to run game can't cast value as Object [failed to obtain stacktrace]
开发环境是VS2022 C++20
void exportHostAbility(const std::shared_ptrscript::ScriptEngine& engine) {
using host_ability::HostImage;
// wrapper inherits from HostImage and ScriptClass
class HostImageWrapper : public HostImage, public script::ScriptClass {
public:
using script::ScriptClass::ScriptClass;
};
static script::ClassDefine<HostImageWrapper> hostImageDef =
script::defineClass<HostImageWrapper>("Image")
.constructor()
.instanceProperty("src", &HostImage::getSrc, &HostImage::setSrc)
.instanceProperty("width", &HostImage::getWidth, nullptr)
.instanceProperty("height", &HostImage::getHeight, nullptr)
.instanceFunction("drop", &HostImage::drop)
.build();
engine->registerNativeClass(hostImageDef);
auto drawImageFunc =
script::Function::newFunction([](HostImageWrapper* img) { host_ability::drawImage(img); });
engine->set("_drawImage", drawImageFunc);
auto sendMessageFunc = script::Function::newFunction(host_ability::sendMessage);
engine->set("_sendMessage", sendMessageFunc);
}
auto engine = createEngine();
// 1. export function
{
script::EngineScope enter(engine.get());
try {
exportHostAbility(engine);//这里异常了
}
catch (const script::Exception& e) {
std::cerr << "failed to run game " << e;
}
}
// 2. inject host base lib
{
script::EngineScope enter(engine.get());
try {
engine->eval(getScriptBaseLibrary());
}
catch (const script::Exception& e) {
std::cerr << "failed to run game " << e;
}
}
// 3. run downloaded game script
{
script::EngineScope enter(engine.get());
try {
engine->eval(downloadGameScript());
}
catch (const script::Exception& e) {
std::cerr << "failed to run game " << e;
}
}
// exit and shutdown.
是否可以给Local<Number>加上区分整数 / 浮点数的能力
尽管说Js引擎内部统一使用64位浮点数来表示数字,但是Local无法区分类型这个情况还是经常导致ScriptX数据类型在转换到C++类型时出现一定的问题,特别是做ScriptX和Json类型互转的过程当中不知该如何处理整数 / 浮点数相关的东西
Python单元测试
请问如何配置python单元测试?
######## ScriptX config ##########
# 1. import ScriptX
# set which backend engine to use
set(SCRIPTX_BACKEND Python CACHE STRING "" FORCE)
我将test文件夹的cmake改成这样
结果编译时仍有v8的引用
配置cmake时我发现有仓库的克隆
但是文件里面我并没有发现git链接
就感觉很奇怪
能提供JavaScriptCore的在安卓上面编译的文档么?
跪求 😭
V8引擎作为DLL嵌入工作时出错
编译正确无警告,使用的是你预编译的V8,在单独的exe项目工作正常,,到作为DLL加载时就出问题了
不知有无遇到过这样的问题
经测试lua版dll运行正常。
由于目前没找到编译玩的x64的jscore或者quickjs所以只能搞v8。。。或者不知可否发一下您precompile完测试用的jsc或qjs(泪目)
用V8作为后端启动时遇到问题
问题描述:把小游戏demo单独提取出来,选择V8为backend,把test自动从github拉取的那个作者编译好的v8提取出来,加入项目并链接。接下来启动demo的时候报错
#
# Fatal error in , line 0
# Failed to deserialize the V8 snapshot blob. This can mean that the snapshot blob file is corrupted or missing.
#
#
#
#FailureMessage Object: 0000005F167AF088
看起来像是V8快照加载失败
不知该如何解决
另外问一下v8该如何编译才能符合作为scriptx后端的要求
PHP support
Add PHP support please
希望可以增加对 Js ESM 的支持
lua原生require可以正常使用。但在V8 Js测试 ESM 时提示
Uncaught SyntaxError: Cannot use import statement outside a module
原生V8支持ESM需要通过.mjs后缀,但是由于ScriptX是基于eval的加载,没法让引擎知道当前文件需要支持ESM
貌似QuickJs就原生支持了ESM,ScriptX可以在V8这边做一些处理吗
循环引用内存泄漏
const image = new Image(60, 45); // Using optional size for image
image.onload = function() {
canvas.draw(image)
};
image.src = 'https://mdn.mozillademos.org/files/5397/rhino.jpg';
如果Image是c++实现的,就会出现内存泄漏吧
Local<Array>'s behavior in lua may be wrong
SCRIPTX_BACKEND gets erased by ScriptX/CMakeLists.txt
Environment
- cmake: version 3.22.1
- ScriptX: latest master branch
Problem
When I try to import ScriptX into my project, I follow the ScriptX cmake guide:
set(SCRIPTX_BACKEND QuickJs)
include(ScriptX/CMakeLists.txt)
However, I get an error the first time I run cmake:
CMake Error at ScriptX/CMakeLists.txt:81 (message):
Platform not set. Please set SCRIPTX_BACKEND. Candidates:
V8;JavaScriptCore;SpiderMonkey;QuickJs;WebAssembly;WKWebView;Lua;Python;Ruby;Empty
If I run cmake again, it works now. 🧐
Reason
This seems to be because line 42 of CMakeLists.txt resets the variable to a blank string the first time it is run.
You can see this by adding some logging:
message("backend: [${SCRIPTX_BACKEND}]") # backend: [QuickJs]
set(SCRIPTX_BACKEND "" CACHE STRING "choose which ScriptX Backend(Implementation) to use, V8 or JavaScriptCore or etc...")
message("backend: [${SCRIPTX_BACKEND}]") # backend: []
According to the documentation, this is expected if policy CMP0126 is set to OLD
.
That policy was introduced in version 3.21. However, since ScriptX does cmake_minimum_required(VERSION 3.0)
, this policy is set to the OLD
behavior (see here).
Workaround
If you are using cmake 3.21 or newer, you can set the policy explicitly:
# add this line first:
set(CMAKE_POLICY_DEFAULT_CMP0126 NEW)
# ...now we can do these lines:
set(SCRIPTX_BACKEND QuickJs)
include(ScriptX/CMakeLists.txt)
If you are running an older version, you can copy line 42 into your file and change it, like this:
set(SCRIPTX_BACKEND QuickJs CACHE STRING "choose which ScriptX Backend(Implementation) to use, V8 or JavaScriptCore or etc...")
include(ScriptX/CMakeLists.txt)
Solution
It seems there are a few options.
- Change line 42 to make it capture the user variable:
set(SCRIPTX_BACKEND "${SCRIPTX_BACKEND}" CACHE STRING "choose which ScriptX Backend(Implementation) to use, V8 or JavaScriptCore or etc...")
- Change the cmake guide to recommend the workarounds above.
- Change the minimum required version to 3.21. I'm not sure if this is desirable for the project, since some users may be using older versions.
Option 1 works in my tests and seems like the cleanest fix, but I'm not sure what is best.
Thanks
ScriptX is a really impressive library! It's well-designed and well-engineered, and promises to a be a huge benefit to my project. Thank you for making it 🎉
scriptDynamicCast返回NULL导致空指针崩溃
- Engine.hpp:39 是把引擎类型从ScriptEngine向下转型为V8Engine, scriptDynamicCast返回了NULL;
- 工程开启了rtti, scriptDynamicCast的实现使用了dynamic_cast;
scriptDynamicCast参数类型:
R: V8Engine
T: ScriptEngine
scriptDynamicCast里测试代码来看:
- 使用static_cast 转换时t_static_cast非空;
- 使用dynamic_cast 转换时t_forward_dynamic_cast 为空;
请问是否scriptDynamicCast的实现在向下类型转换时有问题?
JavaScriptCore的JscWeakRef和 OC里面的JSManagedValue 语义好像不一样吧
JSManagedValue 是用来解决循环应用的,并且保证不会被GC
JscWeakRef 是不是类似c++ weak_ptr, 还是会被GC掉?
Exception when using promises with QuickJs backend
Hi,
When running the following code with QuickJs backend an exception is thrown
const promise = new Promise((resolve, reject) => {
setTimeout(() => { resolve('Ok') }, 300);
});
promise.then(x => { console.log(x); }); // Exception gets thrown here
The exception is about the missing engine scope, thrown from QjsEngine::newPrototype
newRawFunction
lambda. This is because when executing the promise_reaction_job
(that is the .then()
part) and eventually the lambda function created in QjsEngine::newPrototype
, the engine scope is not set (the promise resolution is called on the message loop) so the args.thiz()
in newRawFunction
throws an exception.
It seems this can be solved by creating a scope in there:
// QjsEngine.hpp
// QjsEngine::newPrototype
auto fun = newRawFunction(
this, const_cast<FuncDef*>(&f), definePtr,
[](const Arguments& args, void* data1, void* data2, bool) {
EngineScope scope(args.engine()); // <---------------------------- *HERE*
auto ptr = static_cast<InstanceClassOpaque*>(
JS_GetOpaque(qjs_interop::peekLocal(args.thiz()), kInstanceClassId));
if (ptr == nullptr || ptr->classDefine != data2) {
throw Exception(u8"call function on wrong receiver");
}
auto f = static_cast<FuncDef*>(data1);
Tracer tracer(args.engine(), f->traceName);
return (f->callback)(static_cast<T*>(ptr->scriptClassPolymorphicPointer), args);
});
是否支持直接在iOS平台跑起来?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.