blindmindstudios / angelscript-jit-compiler Goto Github PK
View Code? Open in Web Editor NEWA Just-In-Time compiler for the AngelScript language on x86 processors.
Home Page: www.blind-mind.com
A Just-In-Time compiler for the AngelScript language on x86 processors.
Home Page: www.blind-mind.com
int64 count=0;
for(int64 i=0;i<1000000000;++i)
{
count+=i;
}
The code above gives a result of 500000003794967296 in 32 bit angelscript when jit was enabled, but the result is 499999999500000000 in c++ code and in angelscript when jit was disabled.x64 is all right.
Compiler is MSVC of VS2013.
I routinely get an access violation when executing the JIT-compiled code for asBC_POW* instructions. It looks like the pointer arguments to the *pow_wrapper functions are being truncated, preserving only the lower DWORD. I think this is because the JIT compiler is writing to 32-bit registers. The following modification seems to correct the issue.
Original
case asBC_POWd:
{
eax.copy_address(*edi-offset1);
ecx.copy_address(*edi-offset2);
edx.copy_address(*esp+local::overflowRet);
ebx.copy_address(*edi-offset0);
cpu.call_cdecl((void*)dpow_wrapper, "rrrr", &eax, &ecx, &edx, &ebx);
ecx = as<char>(*esp + local::overflowRet);
as<char>(ecx) &= as<char>(ecx);
ReturnCondition(NotZero);
} break;
Revised
case asBC_POWd:
{
#ifdef JIT_64
Register arg0 = as<void*>(cpu.intArg64(0, 0));
Register arg1 = as<void*>(cpu.intArg64(1, 1));
Register arg2 = as<void*>(cpu.intArg64(2, 2));
Register arg3 = as<void*>(cpu.intArg64(3, 3));
#else
Register arg0 = ecx;
Register arg1 = eax;
Register arg2 = edx;
Register arg3 = ebx;
#endif
arg0.copy_address(*edi-offset1);
arg1.copy_address(*edi-offset2);
arg2.copy_address(*esp+local::overflowRet);
arg3.copy_address(*edi-offset0);
cpu.call_cdecl((void*)dpow_wrapper, "rrrr", &arg0, &arg1, &arg2, &arg3);
ecx = as<char>(*esp + local::overflowRet);
as<char>(ecx) &= as<char>(ecx);
ReturnCondition(NotZero);
} break;
Hi!
It's possible to use JIT on iOS ?
I noticed the readme states that the JIT was last made compatible with 2.27 but as_jit.cpp references op codes that were only added in 2.28.0. Since I'm trying to track down some stack corruption in the JIT that I'm seeing on Windows using gcc4.8.1 I want to make sure I'm working with a known compatible version.. thanks..
I don't know whether or not this is a bug in AngelScript or this JIT compiler, but I am getting improper behavior when suspending a script context from within a function called from a JIT block.
I posted the bug on the AngelScript forum already, but just in case this is a bug with the JIT compiler, I wanted to raise it as an issue here. I am using Revision 52 of this JIT compiler and AngelScript version 2.31.0
http://www.gamedev.net/topic/677758-jit-compiler-suspend-blindmindstudios/
Apparently, a null pointer is stored in the object register after the constructor of a script class is executed. I'm using the method outlined in the manual to instantiate a script class from C++. The code worked fine in 2.26.3, but it does not in 2.27.0 with JIT enabled.
void Object::CallConstructor()
{
if (constructorCalled)
return;
asIScriptContext *context = scriptManager->GetContext();
if (context == NULL)
return;
if (context->Prepare( funcConstructor) < 0)
{
printf( "There was an error preparing the context.\n");
return;
}
*((Object**)context->GetAddressOfArg(0)) = this;
//AddRef();
int r = context->Execute();
if (r != asEXECUTION_FINISHED)
{
if (r == asEXECUTION_EXCEPTION)
{
const char *exception = context->GetExceptionString();
printf( "An Exception, '%s', Occurred In The Constructor.\n", exception);
return;
}
if (r == asERROR)
{
printf( "An Unexpected Error Occurred In The Constructor.\n");
return;
}
}
else
{
pScrObj = *((asIScriptObject**)context->GetAddressOfReturnValue());// <--- NULL HERE
pScrObj->AddRef(); //Must Be Called or else Pure Virtual Error!
context->Unprepare();
constructorCalled = true;
}
}
asCContext members now starts with m_ prefix.
I building on Ubuntu 14.04 LTS 64 bit with gcc 4.8.2 using the latest JIT compiler code and AngelScript 2.29, but I have tried 2.28 and 2.27 and have gotten the same results. I am still narrowing down the problem. But running valgrind with --tool=memcheck detects an invalid write with a script containing an empty main function, here is part of the output:
==10471== Invalid write of size 1
==10471== at 0x5BA821: assembler::Processor& assembler::Processor::operator<< (unsigned char) (virtual_asm.h:216)
==10471== by 0x5B2E66: assembler::Processor& assembler::Processor::operator<< assembler::RegPrefix(assembler::RegPrefix) (virtual_asm_x64.cpp:471)
==10471== by 0x5B2EE4: assembler::Processor::push(assembler::Register&) (virtual_asm_x64.cpp:476)
==10471== by 0x583185: asCJITCompiler::CompileFunction(asIScriptFunction_, void (__)(asSVMRegisters_, unsigned long)) (as_jit.cpp:421)
==10471== by 0x6841BD: asCScriptFunction::JITCompile() (as_scriptfunction.cpp:1422)
==10471== by 0x6F16C0: asCModule::JITCompile() (as_module.cpp:241)
==10471== by 0x6F183B: asCModule::Build() (as_module.cpp:288)
I wanted to post this here while I look into it further just in case anyone on the team sees anything in the output. Just for completeness the line referenced above is:
(T)op = b; op += sizeof(T);
So it at first glance it seems like op is going out of bounds, and I'm not sure why or if I have done anything to cause it to happen. I'm still debugging it a bit.
I'll update here when I have something more concrete to show, or close it if I find I did something in my code to cause it.. thanks..
AS 2.27.1 has moved byteCode and jitFunction data members of asCScriptFunction inside the inner ScriptFunctionData struct which can be reference through the scriptData pointer.. so lines like the following:
ptr = (void*)func->jitFunction;
become:
ptr = (void*)func->scriptData->jitFunction;
There are only a handful of indirections that need to be added in the file and I was able to compile fine against AS 2.27.1 after I made the changes..
class Test
{
double Value(){return 0.0;}
};
Test t;
"result" is a completely random value. It is probably a regression due to the recent changes in the THIS_CALL implementation, but I have not checked yet if previous versions had the problem. The same code with floats work fine.
When using angelscript 2.28.2 (rev 1887) and a method registered with asCALL_THISCALL_ASGLOBAL, the code compiled with the JIT calls the registered method with the wrong pointer. It runs fine if the JIT is deactivated, or when used on other platforms (Mac OS X 64-bit or Windows 32/64).
Example:
1- Register a global function that call a method on an object:
r = engine->RegisterGlobalFunction("void print(const string &in)", asMETHOD(CASEngine, Print), asCALL_THISCALL_ASGLOBAL, this);
2- Compile a script that uses this method:
void main void()
{
print("Hello World");
}
The Print method on the object is called with an invalid "this" pointer.
What the title says.
We are using AngelScript version 2.31.1 and the latest asJIT compiler. Everything works fine, but we are facing a rare-ish bug where our game crashes at the same (JIT'd) function, on this line: (Function was fetched by dumping all function signatures and JIT addresses and matching it with the instruction pointer in an mdmp, line is an educated guess based on the byte offset and disassembly around the instructions)
if (di.Melee)
Here, di
is of type DamageInfo
and is passed to the function by value, and Melee
is a bool. The class is defined in scripts with 2 constructors, a regular constructor (DamageInfo()
) and one with a bunch of params for its properties. (So we're using the default copy constructor)
This is the disassembly corresponding to the script line above, where the crash occurs on 35745A9F
:
35745A9C | 83 C3 1E | add ebx,1E |
35745A9F | 8B 03 | mov eax,dword ptr ds:[ebx] |
35745AA1 | 25 FF 00 00 00 | and eax,FF |
35745AA6 | 89 47 EC | mov dword ptr ds:[edi-14],eax |
35745AA9 | 8B 5F EC | mov ebx,dword ptr ds:[edi-14] |
35745AAC | 20 DB | and bl,bl |
35745AAE | 0F 84 39 00 00 00 | je 35745AED |
So far we've only seen this crash happen on Windows 32 bit (we don't have 64 bit builds though), but we haven't tested on our Linux builds to confirm whether it happens there, too.
This is a minor issue, but I use the Microsoft VC++ compiler, and I have have the warning level set to 4 (which includes 4xxx warnings). This line (file: as_jit.cpp line: 3800) issues a C4267 warning: "conversion from 'size_t' to 'unsigned int', possible loss of data.
#ifdef _MSC_VER
unsigned offset = ((size_t)func->func) >> 2;
offset *= sizeof(void*);
as<void*>(pax) += offset;
as<void*>(pax) = as<void*>(*pax);
#endif
To me it looks like the conversion is safe in this context, but it would be nice to address the warning. I am currently using this patch:
#ifdef _MSC_VER
unsigned offset = (unsigned)(((size_t)func->func) >> 2);
offset *= sizeof(void*);
as<void*>(pax) += offset;
as<void*>(pax) = as<void*>(*pax);
#endif
CriticalSecion object allocates memory for a CRITICAL_SECTION but never frees it.
CriticalSection::CriticalSection() {
auto* section = new CRITICAL_SECTION;
InitializeCriticalSection(section);
pLock = section;
}
CriticalSection::~CriticalSection() {
DeleteCriticalSection((CRITICAL_SECTION*)pLock);
delete pLock; // <- Add this line to free the CRITICAL_SECTION structure.
}
AngelScript 2.30 released yesterday changed the CallSystemFunction signature. Previously it was
int CallSystemFunction(int id, asCContext *context, void *objectPointer);
Now it is:
int CallSystemFunction(int id, asCContext *context);
It is used only twice in JIT, but it uses the object pointer argument (in one case, it's nullptr in the other), so I'm not sure what needs replacing.
Hi,
Sorry to open a new issue... While testing the JIT with or without the new JIT_FAST_REFCOUNT feature, I found that an exception is thrown by the JIT on Mac OSX (XCode 4) when not using JIT_FAST_REFCOUNT (JIT_NOSUSPEND active), when the script is being compiled. Example script to reproduce:
class Object
{
void doStuff()
{
}
};
The exception is thrown in void Processor::end_short_jump(void* p):
if(offset < CHAR_MIN || offset > CHAR_MAX)
throw "Short jump too long.";
offset value here is 135
The full stack is the following:
#1 0x00000001001a1dc7 in assembler::Processor::end_short_jump(void*) at virtual_asm_x64.cpp:548
#2 0x00000001001844f6 in asCJITCompiler::CompileFunction(asIScriptFunction_, void (__)(asSVMRegisters_, unsigned long)) at as_jit.cpp:1794
#3 0x000000010016eac0 in asCScriptFunction::JITCompile() at as_scriptfunction.cpp:1409
#4 0x0000000100103253 in asCModule::JITCompile() at as_module.cpp:241
#5 0x00000001001033c9 in asCModule::Build() at as_module.cpp:288
the jump is called at the end of a asBC_FREE instruction, of "release" type:
as<void*>(*edi-offset0) = nullptr;
cpu.end_short_jump(p);
The name of the function called right before the jump is "beh_6" and ends up in call_viaAS()
Trying to skip the exception just crashes at execution time, so I guess the offset is indeed too large.
All of below is tested on AngelScript 2.31.0 and 2.31.1.
It seems there's a problem in AngleScipt+JIT.
I registered in AS the POD type.
The problem is in its member function taking 4 int32 args (not including the POD itself).
After this function call it seems the stack became corrupted (maybe not stack but some memory).
I think so because after this call the AS engine throws an exception in absolutely different part of code (in "dictionary" addon: function CScriptDictValue_opCast).
The POD type:
struct Vector4Stub {
Vector4 v;
};
Its registration:
res = engine->RegisterObjectType("Vector4", sizeof(Vector4),
asOBJ_VALUE | asOBJ_POD);
assert(res >= 0);
Function registration:
res = engine->RegisterObjectMethod("Vector4", "void reload(
int32 nX, int32 nY, int32 nZ, int32 nW)",
asFUNCTION(Vector4Reload), asCALL_CDECL_OBJLAST);
assert(res >= 0);
The C++ function as very simple:
void Vector4Reload(int nX, int nY, int nZ, int nW, Vector4Stub& self) {
self.v[0] = nX;
self.v[1] = nY;
self.v[2] = nZ;
self.v[3] = nW;
}
And then in AS after the call like
v.reload(0, 100, 80, 30);
an exception appears in 100% of cases (several lines later and in the code not connected to the Vector4).
I also found that if I register function with just 3 arguments, all seems to be working properly:
void Vector4Reload(int nX, int nY, int nZ, Vector4Stub& self);
I tried to analyze an assembly code around the call of Vector4Reload and found strange difference between 4-args and 3-args versions:
4-args version (suspicious line is marked with ***):
0000000140A0BB1C and r10,r10
0000000140A0BB1F jne 0000000140A0BB30
0000000140A0BB21 mov r10,5A3CBFCh
0000000140A0BB2B jmp 0000000140A09D36
*** 0000000140A0BB30 push r10 ***
0000000140A0BB32 mov r9d,dword ptr [r13+14h]
0000000140A0BB36 mov r8d,dword ptr [r13+10h]
0000000140A0BB3A mov edx,dword ptr [r13+0Ch]
0000000140A0BB3E mov ecx,dword ptr [r13+8]
0000000140A0BB42 sub rsp,20h
0000000140A0BB46 call Vector4Reload (01401F1C6Ah)
0000000140A0BB4B add rsp,20h
0000000140A0BB4F add r13,18h
0000000140A0BB53 mov rax,qword ptr [rsp+18h]
0000000140A0BB58 mov qword ptr [rax],0
0000000140A0BB5F mov cl,byte ptr [rbp+30h]
0000000140A0BB62 and cl,cl
0000000140A0BB64 je 0000000140A0BBA1
0000000140A0BB66 mov rax,qword ptr [rbp+38h]
0000000140A0BB6A mov edx,dword ptr [rax+18h]
0000000140A0BB6D cmp edx,6
0000000140A0BB73 je 0000000140A0BB84
3-args version (no push like above at all):
000000013FBBBB56 mov qword ptr [rsp+18h],rax
000000013FBBBB5B mov r9,qword ptr [r13]
000000013FBBBB5F and r9,r9
000000013FBBBB62 jne 000000013FBBBB73
000000013FBBBB64 mov r10,52A48BCh
000000013FBBBB6E jmp 000000013FBB9D36
000000013FBBBB73 mov r8d,dword ptr [r13+10h]
000000013FBBBB77 mov edx,dword ptr [r13+0Ch]
000000013FBBBB7B mov ecx,dword ptr [r13+8]
000000013FBBBB7F sub rsp,20h
000000013FBBBB83 call Vector4Reload (013F3A1C6Fh)
000000013FBBBB88 add rsp,20h
000000013FBBBB8C add r13,14h
000000013FBBBB90 mov rax,qword ptr [rsp+18h]
000000013FBBBB95 mov qword ptr [rax],0
000000013FBBBB9C mov cl,byte ptr [rbp+30h]
000000013FBBBB9F and cl,cl
000000013FBBBBA1 je 000000013FBBBBDE
000000013FBBBBA3 mov rax,qword ptr [rbp+38h]
000000013FBBBBA7 mov edx,dword ptr [rax+18h]
000000013FBBBBAA cmp edx,6
000000013FBBBBB0 je 000000013FBBBBC1
Maybe I'm wrong but after the call of "push r10" nobody pops the value back from the stack.
An even If this problem's reason is in different place it's definitely exist.
Hi,
I know that according to the readme AS v2.27 is supported, but ThyReaper states in this comment:
We're using the SVN build which has variably been 2.26, 2.27, and 2.28 over time.
So which AS version is supported by the current jit-compiler implementation? And if v2.28 is not supported, when will it be updated?
thx in advance.
in as_jit.cpp
void stdcall i64_srl(unsigned long long* a, asDWORD* b, unsigned long long* r) {
*r = *a << *b;
}
void stdcall i64_sra(long long* a, asDWORD* b, long long* r) {
*r = *a << *b;
}
Is it right?
When I use AngelScript-JIT my application crashes (with corrupted stack so no backtrace) when I use formatInt in scripts. When I disable JIT it works fine. Strings themselves also works fine, but only formatFloat and formatInt crashes. parseFloat and parseInt doesn't crash.
Maybe others can replicate this?
AS version is AngelScript 2.29.2.
Hi,
I found an issue when using AngelScript 2.35.1 (note that this is probably not version specific) and MSVC
In the function SystemCall::call_64conv
in as_jit.cpp
there is this code:
if(sFunc->DoesReturnOnStack()) {
Register arg0 = as<void*>(cpu.intArg64(firstPos, firstPos, pax));
if(pos == OP_None || objPointer)
arg0 = as<void*>(*esi);
else
arg0 = as<void*>(*esi + sizeof(asPWORD));
if(acceptReturn)
as<void*>(*esp + local::retPointer) = arg0;
retPointer = true;
argOffset += sizeof(void*);
if(func->hostReturnInMemory) {
if(!cpu.isIntArg64Register(firstPos, firstPos)) {
stackBytes += cpu.pushSize();
retOnStack = true;
}
++intCount;
++a;
firstPos += 1;
}
}
I found that for pos == OP_Last
, pos == OP_First
and pos == OP_This
the return value was not being placed into the correct VM stack location and was infact clobbering an argument instead of using the reserved return space.
This works fine with the following AngelScript function (presumably) because the arguments are local copies, releasing the return value string generally has no ill effect:
string foo( string, string )
however if the function is changed to:
string foo( const string& in, const string& in )
then the return value is released and this corrupts the overwritten string reference and causes a crash.
I have modified our version of call_64conv
to the following:
if(sFunc->DoesReturnOnStack()) {
Register arg0 = as<void*>(cpu.intArg64(firstPos, firstPos, pax));
switch (pos)
{
case OP_First:
case OP_Last:
{
//always second item in the array
arg0 = as<void*>(*esi + sizeof(asPWORD));
}
case OP_This:
case OP_None:
{
arg0 = as<void*>(*esi);
}
break;
default:
throw std::exception("unknown operation");
}
if(acceptReturn)
as<void*>(*esp + local::retPointer) = arg0;
retPointer = true;
argOffset += sizeof(void*);
if(func->hostReturnInMemory) {
if(!cpu.isIntArg64Register(firstPos, firstPos)) {
stackBytes += cpu.pushSize();
retOnStack = true;
}
++intCount;
++a;
firstPos += 1;
}
}
This is now working for me and the arguments are written into the correct VM stack location. Maybe you can come up with a better fix for this?
Thanks,
Tom
Hi,
using the latest version from trunk, the following script will crash when JIT_FAST_REFCOUNT is not defined (currently tested on windows 32 and 64-bit):
Begin script >>>>>
class Object
{
void method()
{
}
};
void main()
{
{
// functor call
Object f;
Object@ ptr=f;
ptr.method();
}
}
<End Script <<<<<<
I thought you might want to check it out. I haven't had the time to try to debug it yet.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.