genjvm's People
Forkers
sundanceresearchgenjvm's Issues
std/java
The standard library is a bit of a mess. There's some __java__
which have to be replaced, as well as @:functionCode
. I'm not sure how to go about this without making a huge #if-fest.
TObjectDecl
We need a good way to generate anonymous objects.
Stack overflow on recursive signatures
E.g.
typedef T = haxe.ds.Vector<T -> Void>;
class Main {
static public function main() {
}
static function x(t:T) { }
}
This probably affects all compound signature types.
Enum value printing
Enum values are currently not printed properly. This is related to #6 and we should make sure to generate the necessary information in a way that doesn't waste memory unnecessarily.
Method name clashes
Since all classes implicitly inherit from java.lang.Object at run-time, we have to deal with name clashes. The following methods are final
and have to be escaped:
Class<?> getClass()
void notify()
void notifyAll()
void wait(long timeout)
void wait(long timeout, int nanos)
void wait()
Note that the signature is relevant and we only have to escape if a method matches in both name and signature.
The following methods are not final and might be overridden:
int hashCode()
boolean equals(Object obj)
Object clone()
String toString()
void finalize()
The question here is what we want to do with those. We can treat toString
as a separate case because it's defined in Haxe with the same semantics, so we always want to allow overriding it in classes. For the others, there are two options:
- Escape by default, allow to not-escape (and thus override) with metadata.
- Don't escape (and thus override) by default, allow to to escape with metadata.
Not sure which option is better. From a pragmatic point of view I'm leaning towards 2.
As for how to handle the escaping, I propose that we prefix with _hx_
and make sure reflection respects that. This should be a matter of stripping a leading _hx_
in some places, and adding it in some others. That's pretty annoying, but I don't think we can avoid it.
Exhaustive switches on compile-time-finite values
This is currently not generated correctly:
enum E {
A;
B;
}
class Main {
static public function main() {
switch (a) {
case A:
return 1;
case B:
return 2;
}
}
static var a = A;
}
I think that in cases like this we should treat the last case as default
to make JVM happy.
Reflect.hx
- hasField (only supported on anons as specified)
- field
- setField
- getProperty (can't remember if this has can just go by convention or actually has to check something)
- setProperty (same)
- callMethod (there's an implementation but probably 20000 corner cases)
- fields (only supported on anons as specified)
- isFunction (should come down to an instanceof check against MethodHandle)
- compare (partial implementation, have to make it robust)
- compareMethods (have to check the field closure case)
- isObject (this one is annoying... see below)
- isEnumValue (I have this already in Type.hx, just have to move/call it)
- deleteField (should only have to call
_hx_deleteField
on the object) - copy (uhm... cast to java.lang.Object and clone?)
- makeVarArgs (headache)
Regarding isObject
: The problem with this is that we always box primitive values when assigning to Dynamic, as is the case here. This means that isObject(12)
and isObject((12 : Null<Int>))
are indistinguishable within the function body.
The genjava implementation solves this by doing a whole bunch of Std.is
checks against java.lang.Number
, java.lang.Boolean
etc. The question here is if we're talking about "object" in the run-time sense or the Haxe-sense.
Something is wrong with closures
The unit tests throw this at me:
Error: Expr function(c2:Int) {
return a26 + b17 + c2;
}
Stack error
ops :
121 ldc_w 329 [(Lunit/TestArrowFunctions$2;I)I]
121 new 318 [, (Lunit/TestArrowFunctions$2;I)I]
121 dup [, , (Lunit/TestArrowFunctions$2;I)I]
121 aload_0 [Ljava/lang/Integer;, , , (Lunit/TestArrowFunctions$2;I)I]
121 iload_1 [I, Ljava/lang/Integer;, , , (Lunit/TestArrowFunctions$2;I)I]
line: 121
operation: invokespecial unit/TestArrowFunctions$2.<init>:(II)V
expected : [I, I, Lunit/TestArrowFunctions$2;]
actual : [I, Ljava/lang/Integer;, , , (Lunit/TestArrowFunctions$2;I)I]
Method unit.TestArrowFunctions.testSyntax
This probably means that there's some disconnect between types somewhere. Relying on v_type
is bad because the run-time type could be something else if captured.
haxe.CallStack
yeah...
Bytecode-level tests
So the reason I worked on format.jvm.Reader was because I wanted to add support for bytecode-level tests in order to catch regressions there. The basic idea is to get the jvm-tests to read their own .jar file and then compare it to some specific input.
This requires writing a printer for https://github.com/Simn/format/blob/jvm/format/jvm/Data.hx#L47 which fetches items from the constant pool. Once we have that, we can come up with some nice way to define the expected bytecode in-line. I imagine a macro like this would work nicely:
bytecodeEquals(<>
iconst_0
istore_1
iinc 1, 1
</>, {
var i = 0;
i++;
});
Last `case` does not need a goto
Reminder issue for me to fix that at some point. Control flow can just fall through from the last case.
enums again
Something I didn't realize is that genjava allows using java-native enums as Haxe enums. The only way to support that would be to make our enums extend java.lang.Enum
so there's a common API.
This isn't entirely free because the java.lang.Enum
constructor wants a string of the enum name. It would also mean dropping (parts of) the annotation-based reflection approach.
switch on string
Let's do something smart here.
interface fields
The JVM doesn't allow non-method non-statics on interfaces. Is there something clever we can do here, or do we need fully dynamic lookup?
Boolop generation isn't ideal
I don't know what I'm doing but I know it's not the correct way to handle this.
StringBuf.add vs. Int
This broke in my overload changes:
class Main {
static public function main() {
var buf = new StringBuf();
buf.add(12);
trace(buf.toString());
}
}
I really want to blame the StringBuf.add implementation for that though:
public function add<T>( x : T ) : Void {
if (Std.is(x, Int))
{
var x:Int = cast x;
var xd:Dynamic = x;
b.append(xd);
} else {
b.append(x);
}
}
Type.hx
- getClass
- getEnum
- getSuperClass
- getClassName (problem with String vs. java.lang.String)
- getEnumName
- resolveClass
- resolveEnum
- createInstance (probably 20000 corner cases still)
- createEmptyInstance (this one sucks)
- createEnum
- createEnumIndex
- getInstanceFields (should just be busywork)
- getClassFields (same)
- getEnumConstructs
- typeof
- enumEq (I always forget how this is specified and implement something else instead...)
- enumConstructor
- enumParameters
- enumIndex
- allEnums (lol...)
Improve StackMapTable
Currently, all stack maps are generated as StackFull
because the documentation went a bit over my head. We should
- read the documentation until we understand it and
- figure out the diff between the last and current stack frame and use one of the other options.
Assertion on nested native arrays
class Main {
static public function main() {
var a = new java.NativeArray<java.NativeArray<Int>>(0);
}
}
This hits the assertion in NativeArray.create
. I have to check what we're supposed to do with this.
constructors
We have to do the super-before-everything-else dance. Also have to make sure that the verifier is happy.
Comparison operator flipping
The generator currently flips some comparison operators, which is not safe to do because of NaN stuff.
The reason it does that is because I initially got mega-confused by the if instructions. They branch on comparison with 0, so if_eq
with 0/false on the stack branches.
[quest] Haxe unit test compilation
-
src/unit/TestOverloads.hx:194: characters 6-11 : Could not find overload
eq(child.someField(ChildJava2), 51);
No idea why this is so special.
-
src/unit/issues/Issue3024.hx:20: characters 2-32 : Recursive signature?
That's #30
-
src/unit/issues/Issue7335.hx:8: characters 17-25 : Unuspported unop on Ljava/lang/Object;
That's #29 (comment)
Getting close!
Std.hx
- is (I think this can just call Jvm.instanceof, but the genjava version does some other stuff too)
- string
- int (this just casts which emits a
d2i
instruction and returns the result) - parseInt (the genjava version is huge... surely there's something in the Java API for this)
- parseFloat (same)
- instance (uuhm... how did this one work again...?)
- random (Math.random + cast I guess)
Signature vs. native type parameters
This currently errors:
import java.jvm.annotation.EnumReflectionInformation;
class Main {
static public function main() {
var exprDefClass = Type.resolveClass("haxe.macro.ExprDef");
var exprDefNative = exprDefClass.native(); // same value but typed as java.lang.Class
var annotation:java.lang.annotation.Annotation = exprDefNative.getAnnotation(cast EnumReflectionInformation);
}
}
java.lang.NoSuchMethodError: java.lang.Class.getAnnotation(Ljava/lang/Class;)Ljava/lang/Object;
I think it's because of the return type being `Ljava/lang/Object;``.
The Haxe dump looks fine:
[Var annotation(1388):java.lang.annotation.Annotation]
[Call:java.lang.annotation.Annotation]
[Field:(param1 : java.lang.Class<java.lang.annotation.Annotation>) -> java.lang.annotation.Annotation]
[Local exprDefNative(1387):java.lang.Class<Dynamic>:java.lang.Class<Dynamic>]
[FInstance:(param1 : java.lang.Class<java.lang.annotation.Annotation>) -> java.lang.annotation.Annotation]
java.lang.Class<Dynamic>
getAnnotation:(param1 : java.lang.Class<getAnnotation.A>) -> getAnnotation.A
[Cast:java.lang.Class<java.lang.annotation.Annotation>] [TypeExpr haxe.jvm.annotation.EnumReflectionInformation:Class<java.jvm.annotation.EnumReflectionInformation>]
I have to check that the generator uses the correct type for this.
Emojis
class Main {
static public function main() {
Sys.println("๐");
}
}
java.lang.ClassFormatError: Illegal UTF8 string in constant pool in class file Main
I thought I didn't have to worry about this kind of stuff but... looks like I do.
Debugging
We should check what we need to make our code debuggable properly. There's good chance that this already works quite well given how nice the decompiled code looks.
I don't know how to use Java debuggers though. :P
`@:strict`
I just found out from the unit tests that genjava supports Java annotations through @:strict
:
@:strict(MyClass_MyAnnotation({ author:"author", currentRevision: 2 }))
public function testAnnotations()
It's probably not worth inventing something new here, so we should just adhere to this. Requires writing a function that transforms expressions to https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.7.16.1.
P.S.: This is the last error in the unit tests before we can move on to failures.
Exceptions
The straightforward approach is to always catch Throwable and do a series of Std.is checks. I wonder if it could be worth to create exception wrappers classes for thrown classes that are not aliased (no parent class or interface). This way we could have StringException
and others and directly catch them through the JVM instruction.
Top-down-casting
In #1 (comment) I've found a case where casting is not ideal. Instead of casting to the expression type Haxe gives us, it would be better to cast to the type we actually expect.
This requires piping the expected type through the texpr-to-instruction converted. We already have RValue
to track if we're in a value place or not, so I think we can simply attach a jsignature option
argument to that and use it where appropriate.
haxe-checkstyle failures
FAIL: massive.munit.AssertionException: Value [null] was not equal to expected value [eol] at checkstyle.detect.DetectCodingStyleTest#testDetectOperatorWrap (486)
FAIL: massive.munit.AssertionException: Value [null] was not equal to expected value [^[A-Z][A-Z0-9]*(_[A-Z0-9_]+)*$] at checkstyle.detect.DetectCodingStyleTest#testDetectConstantName (322)
FAIL: massive.munit.AssertionException: Value [null] was not equal to expected value [eol] at checkstyle.detect.DetectCodingStyleTest#testDetectSeparatorWrap (506)
@AlexHaxe's hint:
might it be an issue, that all three of these checks are child classes and the failing property comes from base class?
at least that's what all three have in common and (I think) all others don't
Reflection
It has to be done eventually. This requires generating some auxiliary type information in a way that it doesn't bother the run-time too much. I don't really want to think about this though.
Interfaces vs. type parameters
interface I<T> {
public function get(t:T):Void;
}
class Main implements I<String> {
static public function main() {
var i:I<String> = new Main();
i.get("foo");
}
function new() { }
public function get(s:String) { }
}
Exception in thread "main" java.lang.AbstractMethodError: Method Main.get(Ljava/lang/Object;)V is abstract
at Main.get(src/Main.hx)
at Main.main(src/Main.hx:8)
The problem is this invokeinterface
instruction:
14: invokeinterface #17, 2 // InterfaceMethod I.get:(Ljava/lang/Object;)V
I wonder if this is supposed to use the concrete type...
super.calls()
This currently doesn't compile:
class Base {
function test() {
trace("Base.test");
}
}
class Main extends Base {
static public function main() { }
function new() { }
override function test() {
trace("Main.test");
super.test();
}
}
I have to check which instruction to use for this. I think it's invokespecial
.
operands in the wrong order when comparing with zero
class Main {
static function main() {
var a = 1;
var x = 0 < a;
}
}
public static void main(java.lang.String[]);
Code:
0: iconst_1
1: istore_1
2: iload_1
3: ifge 10
6: iconst_1
7: goto 11
10: iconst_0
11: istore_2
12: return
public static void main(String[] args) {
int a = 1;
boolean x = a < 0;
}
interesting, even tho instruction is ifge
, intellij decompiler says <
...
Closures
Both field and local closures don't work at the moment.
Haxe extern vs. native signatures
Math.floor is a good example:
class Main {
@:analyzer(ignore)
static public function main() {
var d:Dynamic = Math;
Sys.println(d.floor(f)); // 12.0
Sys.println(Math.floor(f)); // Exception in thread "main" java.lang.NoSuchMethodError: java.lang.Math.floor(D)I
}
static var f = 12.5;
}
For the first one, the result has to be cast to Int
. We cannot detect this at compile-time in the general case, so the dynamic resolution has to deal with this. This may be a good use-case for annotations.
In the second one we have to patch the signature at compile-time, but still make sure the result is cast back to Int
. This is hardcoded in genjava, but I would like look into having a more flexible solution (metadata?).
VerifyError: Method expects a return value
class Main {
static function main() {
f();
}
static function f():Int {
while (true) {
// throw "oops";
return 1;
}
}
}
public static int f();
Code:
0: iconst_1
1: ifeq 6
4: iconst_1
5: ireturn
6: return
looks like we always add a final return
in cases like this, but we really should generate a proper instruction based on the return type
Branching in constructor arguments doesn't work
class Main {
static public function main() {
new Main(f ? 1 : 2);
}
function new(_) {}
static var f = false;
}
java.lang.VerifyError: Inconsistent stackmap frames at branch target 13
Exception Details:
Location:
Main.main([Ljava/lang/String;)V @13: iconst_2
Reason:
Type uninitialized 0 (current frame, stack[0]) is not assignable to 'Main' (stack map, stack[0])
The problem is that in the JVM a new
instruction causes the verification type of the stack item to be Uninitialized
until the invokespecial
instruction happens. My own stack handling doesn't respect that and sets the type to the actual verification type immediately.
I'll have to see how to handle this because my entire stack handling is based on signatures, and there's no signature for "uninitialized". I might have to encode this in the signature structure. There's TObjectInner
which is unused and I don't think could appear on the stack, so that would be an option.
Box-variance
HaxeFoundation/haxe#4342 is still a big problem. This code causes a NullPointerException:
class Main {
static function main() {
var n:Null<Int> = null;
var closure:Null<Int>->Void = printA;
closure(n);
}
static function printA(a:Int) {
trace(a);
}
}
This has to be solved when we detect the assignment of printA
to closure
, but I'm not sure what exactly we should do here. The notion of "adapter methods" has come up and I think MethodHandle
supports that somehow, but my brain can't handle any more Java documentation for the moment.
Note that this fails on genjava too.
Invalid boxed primitve comparison
class Main {
static function main() {
var b:Null<Bool> = null;
trace(b == false);
var i:Null<Int> = null;
trace(i == 0);
}
}
traces true
x2, however it must be false
.
This is because we call Jvm.toBoolean/toInt
which converts null
to default values, but we should not do that for comparisons.
API patches (like String methods)
Ideally, we want to support both "foo".substr
and ("foo" : Dynamic).substr
. There are several options for the first one, but the second one is a bit harder.
An idea would be to collect all these methods at compile-time and create a map from the accessed field to the implementation field. Something along the lines of:
["String" => [ "substr" => ["StringExt", "substr"], "substring" => ["StringExt", "substr"]]]
The dynamic lookup can then (either eagerly or on failure) index into that map with the type and field name, and rewrite the field accordingly.
This could be done explicitly through some @:jvm.reroute("StringExt", "substr")
metadata on the field, or we do it implicitly for all extern
fields that have expressions.
Generated constructors
typeloadFields.ml has this:
(* add_constructor does not deal with overloads correctly *)
if not ctx.com.config.pf_overload then TypeloadFunction.add_constructor ctx c cctx.force_constructor p;
That means that constructors are not auto-generated, which leads to problems. This is handles for genjava through overloadingConstructor.ml, but that thing is pretty gencommon.
I'll probably just enable add_constructor
for the time being because constructors with wrong overloads are slightly better than no constructors.
[quest] Haxe unit test verification endgame
This commit shows what currently doesn't verify:
- TestJava.hx
This sometimes fails with this error:
Exception in thread "main" java.lang.NoClassDefFoundError: haxe/test/Base$InnerClass
at unit.TestMain.main(src/unit/TestMain.hx:81)
Caused by: java.lang.ClassNotFoundException: haxe.test.Base$InnerClass
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
And sometimes with this:
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
unit/TestJava.testIssue2964()V @313: checkcast
Reason:
Type integer (current frame, stack[2]) is not assignable to 'java/lang/Object'
Current Frame:
bci: @313
flags: { }
locals: { 'unit/TestJava', 'haxe/test/MyClass', 'java/lang/Boolean', null }
stack: { 'unit/TestJava', 'haxe/test/MyClass', integer }
The latter is a cast(1, Integer)
case which I don't think should be a thing... but we should be able to support it regardless.
The first one I have no idea.
- Issue2281.hx
Something still going wrong with crement ops. This is likely missing a cast to some expected type.
- Issue3118.hx
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
unit/issues/Issue3118.test()V @27: invokevirtual
Reason:
Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'Array'
Current Frame:
bci: @27
flags: { }
locals: { 'unit/issues/Issue3118', 'unit/issues/Final' }
stack: { 'java/lang/Object', 'java/lang/Integer' }
Another missing cast, probably after field access.
- Issue4002.hx
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
unit/issues/Issue4002.hx_closure$0(Ljava/lang/Object;)I @1: getfield
Reason:
Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'unit/issues/_Issue4002/A'
Current Frame:
bci: @1
flags: { }
locals: { 'java/lang/Object' }
stack: { 'java/lang/Object' }
That could be bad, or it could be another missing cast.
- Issue5124.hx
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
unit/issues/Issue5124.<init>()V @4: invokevirtual
Reason:
Type uninitializedThis (current frame, stack[2]) is not assignable to 'java/lang/Object'
Current Frame:
bci: @4
flags: { flagThisUninit }
locals: { uninitializedThis }
stack: { uninitializedThis, 'java/lang/invoke/MethodHandle', uninitializedThis }
Something wrong with constructor logic vs. field inits. Oh no...
- the rest
Increment issues
class Main {
static public function main() {
x++;
}
static var x:Null<Int> = 0;
}
Fails in the generator with:
Stack error
ops :
3 getstatic Main.x:Ljava/lang/Integer; [Ljava/lang/Integer;]
3 invokestatic haxe/jvm/Jvm.toInt:(Ljava/lang/Object;)I [I]
3 iconst_1 [I, I]
3 iadd [I]
line: 3
operation: putstatic Main.x:Ljava/lang/Integer;
expected : [Ljava/lang/Integer;]
actual : [I]
Method Main.main
I think it's because the TUnop itself is typed as Int:
[Unop:Int]
++
Postfix
[Field:Null<Int>]
[TypeExpr Main:Class<Main>]
[FStatic:Null<Int>]
Main
x:Null<Int>
javac synchronized equivalent
There are two instructions for this:
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5.monitorenter
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5.monitorexit
Sounds surprisingly straightforward actually.
OpAdd
All the variants of +
have to be supported (with conversion to string).
[boss battle] invokedynamic
We really have to understand invokedynamic
. The idea would be to use that whenever we call something but aren't sure what we're calling.
My concern is that this is not enough to cover our semantics. For every unknown.call()
there's also (unknown.call)()
. However, maybe these are just cascading problems: The dynamic field access would return "something", and then the invokedynamic
would figure out how to call it.
Relevant documentation:
What I understood so far is that this is a two-step process:
- When the JVM finds
invokedynamic
while jitting, it calls a bootstrap method which returns something (I thinkCallSite
?) that is associated with the instruction. - At run-time, that returned something is called, somehow.
Missing super call on field-init-generated constructor
class Base {
public function new() { }
}
class Main extends Base {
static public function main() {
new Main();
}
var s:String = "foo";
}
java.lang.VerifyError: Constructor must call super() or this() before return
Exception Details:
Location:
Main.<init>()V @6: return
Reason:
Error exists in the bytecode
I probably shouldn't use add_field_inits
at all anyway...
NativeArray wrapping
We currently always create java.NativeArray (should move that btw...) as a reference type and wrap the values, mostly because "that's easy". We should use the array specializations for basic types here and generate the instructions accordingly.
The problem with this is that Haxe's type system is retarded and allow this kind of stuff:
class Main {
static public function main() {
var ia = new java.NativeArray<Int>(1);
var fa = new java.NativeArray<Float>(1);
var da = new java.NativeArray<Dynamic>(1);
da = ia;
da = fa;
f(ia);
f(fa);
}
static function f<T>(_:java.NativeArray<T>) { }
}
I think we have three options:
- Introduce some
@:absolutelyNoVariance
metadata in Haxe core which does not allow this. That's somewhat straightforward for the assignment toDynamic
, but could be a huge mess for type parameters. I think we would basically have to disable type inference for this, like for the old@:generic
implementation. - Generate reference arrays by default (like now), but do some escape analysis to detect if the array is never assigned and aliased. In these cases we can safely use the unwrapped types. This is quite similar to the inline constructor analysis, but less local because it should work for fields as well.
- Don't care and simply let the JVM verifier fail.
We'll start with 3 either way, but the question is how much nicer we can/should make this and with what priority...
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.