babashka / sci Goto Github PK

This is because the reader does a first pass and we are trying to macroexpand it fully. What we should do instead of let the reader expand this to a normal fn syntax which we then macroexpand later.

Transcript:

borkdude@MBA2015 ~/Dropbox/dev/clojure/babashka (master*) $ echo 'nil' | lein bb "(1 2 3)"
Cannot call 1 as a function.
borkdude@MBA2015 ~/Dropbox/dev/clojure/babashka (master*) $ echo 'nil' | lein bb '("foo" 2 3)'
Cannot call "foo" as a function.
borkdude@MBA2015 ~/Dropbox/dev/clojure/babashka (master*) $ echo 'nil' | lein bb "({:a 1} 2 3)"
3
borkdude@MBA2015 ~/Dropbox/dev/clojure/babashka (master*) $ echo 'nil' | lein bb "((hash-map :a 1) 2 3)"
3
borkdude@MBA2015 ~/Dropbox/dev/clojure/babashka (master*) $ echo 'nil' | lein bb "((hash-map :a 1) :a 2)"
1

This works:

$ echo '[1 2 3]' | ./bb '(map-indexed #(vector %1 %2) *in*)'
([0 1] [1 2] [2 3])

but this doesn't:

$ echo '[1 2 3]' | ./bb '(map-indexed #(do [%1 %2]) *in*)'

There also seems a bug with var-args:

(apply (fn [& xs] xs) 1 2 [3 4])

version
babashka v0.0.18

platform
Ubuntu 16.04

problem
when trying to use a function with an argument called "comment" the actual value of the argument is ignored and it is used as a 'comment' object

repro
file repro.bb:

(defn comment-bug [comment]
  (println (type comment))
  (println comment))

(comment-bug *in*)

then execute:

$ bb --stream -io -f ~/repro.bb << EOF
please print this
EOF

expected output
java.lang.String
please print this

actual output
clojure.lang.Symbol
comment

more (maybe meaningless) information
this: bb --stream -io -e '(def comment *in*) (println comment)'
and this bb --stream -io -e '(def comment "in") (println comment)'

produce a similar bug

(def c (atom 0))
(defn hello []
  (swap! c inc)
  (if (< @c 100) (hello) @c))
(println (hello))
java.lang.Exception: unexpected: clojure.lang.AFunction$1@1102b2b10 [at line 5, column 1]

Why don't we see this error when using 1 argument?

./bb '(defn hello [x]
  (if (< x 2400) (hello (inc x)) x))
(hello 0)'
2400

Note about dealing with stack overflows, use trampoline:

(defn hello [x]
  (if (< x 10000) #(hello (inc x)) x))
(println (trampoline hello 0))

Maybe with a trick we might be able to use trampoline for recursive functions under the hood.

See: https://clojure.org/reference/reader#_dispatch

Ignore next form (#_)

The form following #_ is completely skipped by the reader. (This is a more complete removal than the comment macro which yields nil).

user=> (require '[sci.core :as sci])
nil
user=> (def f (sci/eval-string "#(< % 10)"))
#'user/f
user=> (f 11)
false
user=> (def f (sci/eval-string "#(x % 10)"))
#'user/f
user=> (f 11)
Exception Could not resolve symbol: x  sci.impl.interpreter/resolve-symbol (interpreter.cljc:159)

That's likely also beneficial for performance.

sci.impl.main=> (sci/eval-string "(println :hello)" {:namespaces {'clojure.core {'println println}}})
ExceptionInfo Could not resolve symbol: println [at line 1, column 2]  clojure.core/ex-info (core.clj:4739)

Workaround for now: add these vars to :bindings.

We might go about it as follows:

(defn do-recur!
  [f & args]
  (let [ret (apply f args)]
    (if-let [m (meta ret)]
      (if (:sci.impl/recur m)
        (recur f ret)
        ret)
      ret)))

(defn recur! [& args]
  (with-meta args
    {:sci.impl/recur true}))

(defn hello [x]
  (if (< x 10000) (recur! (inc x)) x))

(println (do-recur! hello 0)) ;;=> 100000

and use do-recur! instead of apply when calling functions.

(defmacro let-bindings []
  (zipmap (map (fn [sym]
                 (list 'quote sym)) (keys &env)) (keys &env)))

(let [x 1 y 2] (println (let-bindings)))

(defn foo [x y z]
  (println (let-bindings)))

(foo 1 2 3) ;; <- problematic

The last call is problematic. We probably should mark some symbols as implementation details.

(-> '(1 2 3))

=> cannot call 1 as a function

Implement :allow option to only allow certain functions/macros.

Would be great if sci also supported also unqualified source code as input, maybe sci.core/eval? Currently, a hop via pr-str is needed. Also, fn* is not supported:

Current status

sci:

((sci/eval-string (pr-str `#(< 10 % 18))) 17)
;Syntax error compiling at (src/malli/core.cljc:575:1).
;Could not resolve symbol: fn*

eval:

((eval `#(< 10 % 18)) 17)
; => true

((eval (read-string (pr-str `#(< 10 % 18)))) 17)
; => true

Proposed:

((sci/eval `#(< 10 % 18)) 17)
; => true

((sci/eval-string (read-string (pr-str `#(< 10 % 18)))) 17)
; => true

E.g. it would be nice if this worked:

(defmacro lets []
  (zipmap (map (fn [sym]
                 (list 'quote sym)) (keys &env)) (keys &env)))

(println
 (let [x 1 y 2]
   (lets))) ;;=> {x 1, y 2}

Not super high priority probably.

&form is mostly used for either error reporting or capturing form metadata

I really love this project and I hope to be able to contribute.
in the README:

You want to evaluate code from user input, but eval isn't safe or simply doesn't work.

Could you elaborate about what are the situations in which eval doesn't work?
I guess that when we compile a clojure program with GraalVM, eval doesn't work. Right?

Is it something essential to GraalVM? Could eval be available some day on GraalVM? Could we image a super fast Clojure REPL?

Please share as much details as you can ...

We might get an idea how to implement the templating from looking at:

• https://clojure.github.io/clojure/clojure.template-api.html
• https://github.com/brandonbloom/backtick/blob/master/src/backtick.clj.

Try:

--initialize-at-run-time=java.lang.Math\$RandomNumberGeneratorHolder

instead of the patch for the rand* functions.

After compiling to JS with advanced this works:

$ node
Welcome to Node.js v12.8.0.
Type ".help" for more information.
> x = require('./out/main.js');
{ sci: { core: { eval_string: [Function] } } }
> eval_string = x.sci.core.eval_string
[Function: bx] { a: [Function], b: [Function], h: 2 }
> eval_string("(defn foo [] 1) (foo)")
1

Rationale

Is this actually useful for anyone?

API

What should a proper JS API look like and how do we get this from CLJS?

Function return values

Functions returned from SCI are CLJS functions. These are not directly callable from JS but you can use them like this: f.call(null, 1,2,3). What should we do to fix this?

References

• https://github.com/swannodette/mori
• https://shadow-cljs.github.io/docs/UsersGuide.html#target-node-library
• https://github.com/tonsky/datascript/tree/master/release-js
• https://github.com/xapix-io/axel-f/blob/master/src/axel_f/api.cljs
  • JS docs: https://github.com/xapix-io/axel-f/tree/master/release-js
  • output-wrapper: https://github.com/xapix-io/axel-f/blob/master/axel_f.min.js.edn#L15-L65
  • https://clojurescript.org/reference/compiler-options#output-wrapper
  • .circleci config: https://github.com/xapix-io/axel-f/blob/9bb72e13699b9559142392c4fa19041b1d072244/.circleci/config.yml#L97-L134

On master this comes through:

(sci/eval-string "clojure.core/inc" {:deny '[clojure.core/inc]})
#object[Function]

• bb '[ (-> 3 inc inc inc) ] '
• bb '{:a (-> 3 inc inc inc)} '

Example:

(def f (sci/eval-string "(fn [coll n] (if (zero? n) (first coll) (recur (rest coll) (dec n))))" {:deny '[nth]}))

(f '(1 2 3) 1)
((2 3) 0)

Workaround: use a name and use the name instead of recur.

Possible solution: maybe it's possible to do the recur check inside the function body?

The edamame library has been extracted from sci, but every time a fix gets applied to edamame we have to apply it here too. It would be better to just use edamame as a library.

(map #(let [[ns v] %]
        (symbol (str ns) (str v)))
     '[[foo f1] [foo f2]])

results in:

Could not resolve symbol: foo

This flag will add recur, loop add to the :deny list (#79)
This flag will also set a sane default for :realize-max unless it is already set.

See oracle/graal#1610 (comment).

This seems to returnexecutable so we can use that to detect if we're in a binary:

System.out.println(System.getProperty("org.graalvm.nativeimage.kind"));

See https://clojuredocs.org/clojure.core/comment

This should work:

`(let [x# 1] ~`(let [x# 2] x#))