Comments (22)
@jtulach Any thoughts?
Most of the language implementations I've seen to date work around these issues with bootstrapping tricks. It would be nice to have a better solution.
from graal.
I haven't responded as I didn't know what to say. The text in this issue is so vague that it cannot be treated as a real bug report. Unless I see the real problems, I can't discuss solutions.
from graal.
@jtulach I've tried my best to give an initial problem statement, but I'll try again:
When language-specific code (e.g. a language-specific REPL implementation) wants to interface with its language, it's natural that it wants to work directly with the value types, AST nodes, and exceptions used by the language implementation. In such a case, PolyglotEngine
's abstractions such as Value
and wrapping every exception with IOException
get in the way, and the fact that only a Source
(but not a Node
) can be executed is limiting.
Here is how SOMns and JRuby/Truffle work around this:
https://github.com/smarr/SOMns/blob/master/src/som/interpreter/SomLanguage.java#L150
I believe these constructions are only necessary because there is no supported way to execute anything other than a Source
in a PolyglotEngine
managed context.
Has @chrisseaton talked to you about this? We had a discussion in the Gitter channel, and he wanted to propose the "lower-level execution mode" idea to you.
from graal.
My idea was
vm.executeInLanguageContext('application/x-my-language', () -> {
whatever you want here
})
It would just be syntactic sugar for the current workarounds in the other languages.
That was my proposed solution to what you wanted, but really I'm happy with the current system - I only have the workaround in a couple of places so I don't see it as a huge burden.
from graal.
@chrisseaton Keep in mind that your understanding is way above that of the average Truffle user, and that other language implementations may have additional constraints that make such workarounds more complicated.
from graal.
When language-specific code (e.g. a language-specific REPL implementation) wants to interface
with its language, it's natural that ...
...they want to take shortcuts. Yes, in such system it may seem that the overhead of PolyglotEngine
is too big and unnecessary. However it isn't true, as soon as you find yourself talking with other language or using some language agnostic instrument - you'll benefit from using the PolyglotEngine
concepts.
it wants to work directly with the value types, AST nodes, and exceptions used by the language
implementation. In such a case, PolyglotEngine's abstractions such as Value and wrapping every > exception with IOException get in the way, and the fact that only a Source (but not a Node) can be > executed is limiting.
- I'll start working on removing the
IOException
soon. Value
has methodas
which you can use to extract your internal type value- implementing your own parsing function and invoking it via
Value.execute
is also possible - with any parameters
Using these tricks is dangerous - unless you are careful, you may find out things like debugger don't work - to guarantee that I would need to tighten things up, not relax them.
from graal.
@pniederw I think we should be able to implement the method I proposed, using the existing Truffle API stuff. We could that in your code, so you don't need a modification to the API. Let me know if you want to try that.
from graal.
...they want to take shortcuts. Yes, in such system it may seem that the overhead of PolyglotEngine is too big and unnecessary. However it isn't true, as soon as you find yourself talking with other language or using some language agnostic instrument - you'll benefit from using the PolyglotEngine concepts.
The point I'm trying to make is that something seems to be missing from PolyglotEngine
's concepts. It shouldn't be necessary for every other language implementation, even those by highly skilled Truffle experts, to funnel some bootstrapping code through PolyglotEngine
. Or if that's really the way to go, it's critical to have this pattern documented.
To give one example from my own language, my tooling sometimes needs to programmatically construct an AST and execute that. It's not clear to me how to achieve this with PolyglotEngine
, at least not without significant workarounds.
By now I've implemented a complete Truffle interpreter for my language (which is roughly as complex as SOMns) and some APIs/tooling around it. How to fit in PolyglotEngine
has been the hardest part of this (!), the only part I haven't been able to figure out yet. That's the feedback I have for you. Oh, and thanks for this great technology!
from graal.
@chrisseaton I'd love to try that. The only constraint I have is that (unfortunately) I can't share any of my code at this time.
from graal.
I thought I could do this cleanly, because I thought that one of the public PolyglotEngine
eval methods accepted arguments, but it looks like that isn't the case. If you don't care about races you can do it like below, or maybe you could allocate some kind of unique name to recognise a particular callback. You'd have to GC those somehow.
diff --git a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java
index 1f52d88..f6e86ca 100644
--- a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java
+++ b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java
@@ -52,9 +52,11 @@ import com.oracle.truffle.api.Truffle;
import com.oracle.truffle.api.TruffleLanguage;
import com.oracle.truffle.api.debug.DebuggerTags;
import com.oracle.truffle.api.frame.MaterializedFrame;
+import com.oracle.truffle.api.frame.VirtualFrame;
import com.oracle.truffle.api.instrumentation.ProvidedTags;
import com.oracle.truffle.api.instrumentation.StandardTags;
import com.oracle.truffle.api.nodes.Node;
+import com.oracle.truffle.api.nodes.RootNode;
import com.oracle.truffle.api.source.Source;
import com.oracle.truffle.sl.nodes.SLEvalRootNode;
import com.oracle.truffle.sl.nodes.SLRootNode;
@@ -89,6 +91,20 @@ public final class SLLanguage extends TruffleLanguage<SLContext> {
@Override
protected CallTarget parse(Source source, Node node, String... argumentNames) throws IOException {
+ if (source.getCode().equals("@callback")) {
+ final Runnable runnableWhenCreated = SLMain.runnable;
+
+ return Truffle.getRuntime().createCallTarget(new RootNode(SLLanguage.class, null, null) {
+
+ @Override
+ public Object execute(VirtualFrame frame) {
+ runnableWhenCreated.run();
+ return null;
+ }
+
+ });
+ }
+
Map<String, SLRootNode> functions;
try {
/*
diff --git a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java
index ab994d5..9cff745 100644
--- a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java
+++ b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java
@@ -208,6 +208,14 @@ public final class SLMain {
PolyglotEngine engine = PolyglotEngine.newBuilder().setIn(in).setOut(out).build();
assert engine.getLanguages().containsKey(SLLanguage.MIME_TYPE);
+ runInSLContext(engine, new Runnable() {
+
+ public void run() {
+ System.err.println("hello");
+ }
+
+ });
+
try {
Value result = engine.eval(source);
@@ -290,4 +298,20 @@ public final class SLMain {
}
return result.toString();
}
+
+ public static Runnable runnable;
+
+ private static void runInSLContext(PolyglotEngine engine, Runnable runnable) {
+ SLMain.runnable = runnable;
+
+ final Source source = Source.newBuilder("@callback").name("(callback)").mimeType(SLLanguage.MIME_TYPE).build();
+
+ try {
+ engine.eval(source);
+ } catch (IOException e) {
+ // TODO Auto-generated catch block
+ e.printStackTrace();
+ }
+ }
+
}
from graal.
Hello Chris,
can't we do it like this (speaking in JavaScript):
Runnable r = // your own code
enter = engine.eval("(function(f) { f(); })");
TruffleObject callback = JavaInterop.asTruffleFunction(r);
and then any time later:
enter.execute(null, callback); // null is JavaScript specifiec thing
I believe I am using this style in my own work at
http://source.apidesign.org/hg/html~html4j/rev/47cd2110ed8d
and it seems to work somehow.
-jt
Chris Seaton : 10. 7. 2016 @ 05:52
I thought I could do this cleanly, because I thought that one of the public
PolyglotEngine
eval methods accepted arguments, but it looks like that
isn't the case. If you don't care about races you can do it like below, or
maybe you could allocate some kind of unique name to recognise a particular
callback. You'd have to GC those somehow.diff --git a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java index 1f52d88..f6e86ca 100644 --- a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java +++ b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLLanguage.java @@ -52,9 +52,11 @@ import com.oracle.truffle.api.Truffle; import com.oracle.truffle.api.TruffleLanguage; import com.oracle.truffle.api.debug.DebuggerTags; import com.oracle.truffle.api.frame.MaterializedFrame; +import com.oracle.truffle.api.frame.VirtualFrame; import com.oracle.truffle.api.instrumentation.ProvidedTags; import com.oracle.truffle.api.instrumentation.StandardTags; import com.oracle.truffle.api.nodes.Node; +import com.oracle.truffle.api.nodes.RootNode; import com.oracle.truffle.api.source.Source; import com.oracle.truffle.sl.nodes.SLEvalRootNode; import com.oracle.truffle.sl.nodes.SLRootNode; @@ -89,6 +91,20 @@ public final class SLLanguage extends TruffleLanguage<SLContext> { @Override protected CallTarget parse(Source source, Node node, String... argumentNames) throws IOException { + if (source.getCode().equals("@callback")) { + final Runnable runnableWhenCreated = SLMain.runnable; + + return Truffle.getRuntime().createCallTarget(new RootNode(SLLanguage.class, null, null) { + + @Override + public Object execute(VirtualFrame frame) { + runnableWhenCreated.run(); + return null; + } + + }); + } + Map<String, SLRootNode> functions; try { /* diff --git a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java index ab994d5..9cff745 100644 --- a/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java +++ b/truffle/com.oracle.truffle.sl/src/com/oracle/truffle/sl/SLMain.java @@ -208,6 +208,14 @@ public final class SLMain { PolyglotEngine engine = PolyglotEngine.newBuilder().setIn(in).setOut(out).build(); assert engine.getLanguages().containsKey(SLLanguage.MIME_TYPE); + runInSLContext(engine, new Runnable() { + + public void run() { + System.err.println("hello"); + } + + }); + try { Value result = engine.eval(source); @@ -290,4 +298,20 @@ public final class SLMain { } return result.toString(); } + + public static Runnable runnable; + + private static void runInSLContext(PolyglotEngine engine, Runnable runnable) { + SLMain.runnable = runnable; + + final Source source = Source.newBuilder("@callback").name("(callback)").mimeType(SLLanguage.MIME_ TYPE).build(); + + try { + engine.eval(source); + } catch (IOException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + } + }
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#183 (comment)
from graal.
Yes that works. And I guess if you had a language where you couldn't natively write that, you could introduce a special form for it.
from graal.
We are going to continue to support to use the Truffle core APIs without PolyglotEngine. However the instrumentation and multi-language interop will only work embedded in PolyglotEngine.
@pniederw : @jtulach has explained how PolyglotEngine can be used to achieve your goals. Can you work with that? Or do you want to make other concrete suggestions on how to improve? Please close this issue if all your questions are answered.
from graal.
ok, so, I should report NullPointerExceptions for stuff that breaks when not initializing the PolyglotEngine
?
from graal.
We are going to continue to support to use the Truffle core APIs without PolyglotEngine.
Without PolyglotEngine
, lookup of the execution context (a concept exposed by TruffleLanguage
) doesn't work. Hence I'd argue that use of PolyglotEngine
is already mandatory.
@jtulach has explained how PolyglotEngine can be used to achieve your goals. Can you work with that?
My language doesn't currently support language interop, and supporting it is not a priority. The "special form" solution suggested by @chrisseaton should work, but feels like an obscure workaround for a common need. However, it does demonstrate that running non-program code within a context is already possible, which means that adding a public API for this wouldn't lead to any loss of control (which was used as an argument for not adding such an API).
Or do you want to make other concrete suggestions on how to improve?
@chrisseaton's suggestion would solve my problem immediately: #183 (comment)
from graal.
Is there any consensus on whether something like @chrisseaton's vm.executeInLanguageContext()
idea should be added?
from graal.
It's up to @jtulach whether the new method is added, and from his previous comments it looks like he's not keen on it.
All I can suggest is helping you to implement the tiny amount of interop needed to make the eval pattern work for you.
from graal.
I believe that common things should be easy, complex possible. Here is a working code that gets anyone into the language context:
public final class Main {
public void callMeInContext() {
Thread.dumpStack();
}
public static void main(String[] args) throws IOException {
PolyglotEngine engine = PolyglotEngine.newBuilder().build();
final Source source = Source.
newBuilder("function() { return this.callMeInContext(); }").
name("callback.js").
mimeType("text/javascript").
build();
final PolyglotEngine.Value invoke = engine.eval(source);
final TruffleObject callback = JavaInterop.asTruffleObject(new Main());
invoke.execute(callback);
invoke.execute(callback);
}
}
I don't think the solution is that complex and the request that common to justify addition of new method into the API.
from graal.
@pniederw PolyglotEngine is not mandatory if you are not using TruffleLanguage. Only TruffleRuntime, CallTarget, RootNode, Node can be used as well. TruffleLanguage is the SPI side of PolyglotEngine.
To execute your language you would parse on your own and then call into a CallTarget. Thats what PolyglotEngine is currently doing for you, but if you, as you say, don't care about instrumentation/interop feel free skip PolyglotEngine and TruffleLanguage.
from graal.
@jtulach There's no OSS Truffle JS implementation at this point in time, and shipping a JS implementation with my language just to solve this problem doesn't make sense. From my perspective, language interop is a complex solution to this common problem.
@chrisseaton I'm keen on giving this a go. Can you give me some pointers on how to implement the necessary amount of language interop? Unfortunately I can't share my code at this point.
@chumer Good point. Ditching all ofPolyglotEngine
, TruffleLanguage
, and ExecutionContext
seems like an option.
from graal.
You need to subclass ObjectType
and override getForeignAccessFactory
, returning a ForeignAccess
. You can create one of those using a DSL. The Ruby one is only a couple of hundred lines, and it's only complicated because it needs to be fast.
If you are just using this for Jaroslav's pattern, you could put simple call logic here:
Catch me on Gitter and I'll walk you through it.
from graal.
Seems like the issue is resolved. Low-level execution without any polyglot support is possible for a custom language. I would however not recommend it as it eliminates many benefits, like interop/instruments/debugging.
from graal.
Related Issues (20)
- [GR-52238] Oracle GraalVM crashes while compiling Aeron MediaDriver HOT 4
- SandboxPolicy.UNTRUSTED can't be used in multi-module SpringBoot based application with Oracle GraalVM HOT 1
- Micro-optimization opportunity: Optimizing out default branch from RangeTableSwitchOp HOT 2
- native-image hangs after inlining methods (Windows x64) HOT 6
- Native image running error with Springboot 3.1 application HOT 1
- [GR-52267] Can't compile with Onnxruntime library, conflict with svm-enterprise.jar HOT 5
- Native-image compilation fails in windows: with parsing error when the user home dir contains spaces. HOT 3
- [GR-52339] ExitOnOutOfMemoryError not waiting for HeapDumpOnOutOfMemoryError HOT 1
- Add JFR events and track peak usage for native memory tracking (NMT)
- GraalVM error with sbt nativeImage plugin : Registering type as reachable after analysis when using google-cloud-compute library HOT 3
- Static binary compiled with musl reports question mark for (System/getProperty "user.home") HOT 2
- [GR-52400] Native: Introduce flag to favor build time improvements at the cost of using more resources and vice versa HOT 2
- Serialization of float[] or other arrays is not supported in Native Image HOT 8
- docs/getting-started/windows/ has misleading pictures HOT 1
- Caused by: java.lang.IllegalArgumentException: Invalid logger interface org.hibernate.validator.internal.util.logging.Log (implementation not found in jdk.internal.loader.ClassLoaders$AppClassLoader@2f7c7260) HOT 7
- macOS Native Image libawt fails to load due to JNU_NewStringPlatform failure HOT 2
- Native-image of CLI application is failing with less than 32 GB of virtual memory HOT 3
- reflect-config-schema-v1.0.0.json doesn't mention certain attributes that can actually be supplied
- Problem java.lang.NoClassDefFoundError: sun/misc/Unsafe
- [GR-52454] Native Image shutdown hooks don't run upon SIGINT HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from graal.