cognitect / transit-cljs Goto Github PK

Transit for ClojureScript

License: Apache License 2.0

Clojure 89.51% Shell 7.83% HTML 2.12% JavaScript 0.54%

transit-cljs's Introduction

transit-cljs

Transit is a data format and a set of libraries for conveying values between applications written in different languages. This library provides support for marshalling Transit data to/from ClojureScript. Unlike the Java and Clojure implementations it relies on the non-streaming JSON parsing mechanism of the host JavaScript environment.

This implementation's major.minor version number corresponds to the version of the Transit specification it supports.

NOTE: Transit is intended primarily as a wire protocol for transferring data between applications. If storing Transit data durably, readers and writers are expected to use the same version of Transit and you are responsible for migrating/transforming/re-storing that data when and if the transit format changes.

Releases and Dependency Information

Latest release: 0.8.280
All Released Versions

Leiningen

Add the following to your project.clj :dependencies:

[com.cognitect/transit-cljs "0.8.280"]

Maven

Maven dependency information:

<dependency>
  <groupId>com.cognitect</groupId>
  <artifactId>transit-cljs</artifactId>
  <version>0.8.280</version>
</dependency>

Usage

(ns example
  (:require [cognitect.transit :as t]))

(defn roundtrip [x]
  (let [w (t/writer :json)
        r (t/reader :json)]
    (t/read r (t/write w x))))

(defn test-roundtrip []
  (let [list1 [:red :green :blue]
        list2 [:apple :pear :grape]
        data  {(t/integer 1) list1
               (t/integer 2) list2}
        data' (roundtrip data)]
    (assert (= data data'))))

Default Type Mapping

Abbreviations:

cc = cljs.core
gm = goog.math
cct = com.cognitect.transit

Transit type	Write accepts	Read returns
null	null	null
string	String	String
boolean	Boolean	Boolean
integer	Number, gm.Long	Number, gm.Long
decimal	Number	Number
keyword	cc.Keyword	cc.Keyword
symbol	cc.Symbol	cc.Symbol
big integer	cct.TaggedValue	cct.TaggedValue
big decimal	cct.TaggedValue	cct.TaggedValue
time	Date	Date
uri	cct.URI	cct.URI
uuid	cct.UUID	cct.UUID
char	String	String
array	cc.IVector	cc.IVector
list	cc.IList	cc.IList
set	cc.ISet	cc.ISet
map	cc.IMap	cc.IMap
link	cct.Link	cct.Link
cmap	cct.IMap	cct.IMap

Contributing

This library is open source, developed internally by Cognitect. We welcome discussions of potential problems and enhancement suggestions on the transit-format mailing list. Issues can be filed using GitHub issues for this project. Because transit is incorporated into products and client projects, we prefer to do development internally and are not accepting pull requests or patches.

Development

Dependencies

Install dependencies with

lein deps

Running the tests & benchmarks

Running the tests:

lein cljsbuild once test

In order to run the transit-format bin/verify you must first clone transit-format into the same parent directory as your transit-cljs checkout and build the roundtrip file:

lein cljsbuild once roundtrip

To run the benchmarks you must first clone transit-format into the same parent directory as your transit-cljs checkout.

Running the benchmarks:

lein cljsbuild once bench
node target/transit.bench.js

Build

Build JAR for ClojureScript

Assuming you have a JDK and Maven installed, the following will install a JAR suitable for use from ClojureScript into your local Maven repository.

build/package_local

Copyright and License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

transit-cljs's People

Contributors

Stargazers

Watchers

Forkers

pwightman weaver-viii tangramhq day8 lahorichargha ajchemist salewski spinningtopsofdoom msuess matthavener bhb ykankaya mk souenzzo lambdaisland clojure-land lnostdal nhunzaker azurecloudmonk

transit-cljs's Issues

With :json, caching strips some values due to serialization counting errors

This took a while to track down. I reduced it down to this minimal input:

(def pathological
  [{:any-value {["this vector makes this a cmap"] "any value"
                "any string" :victim}}
   {:victim :any-other-value}])

transit-cljs fails to serialize that properly. I've verified this by roundtripping it, and testing in Clojure. You can see below that it strips the second :victim.

ClojureScript: Fails

(defn clojurescript-test [v]
  (let [_ (println "ORIGINAL VALUE:")
        _ (println v)
        serialized (transit/write (transit/writer :json) v)
        _ (println "SERIALIZED:")
        _ (println serialized)
        roundtrip (transit/read (transit/reader :json) serialized)]
    (if (= v roundtrip)
      (println "Successful roundtrip")
      (do
        (println "FAILURE")
        (println "ROUNDTRIP VALUE:")
        (println roundtrip)))))

(clojurescript-test pathological)

OUTPUT:

ORIGINAL VALUE:
[{:any-value {[this vector makes this a cmap] any value, any string :victim}} {:victim :any-other-value}]
SERIALIZED:
[["^ ","~:any-value",["~#cmap",[["this vector makes this a cmap"],"any value","any string","~:victim"]]],["^ ","^3","~:any-other-value"]]
FAILURE
ROUNDTRIP VALUE:
[{:any-value {[this vector makes this a cmap] any value, any string :victim}} {nil :any-other-value}]

Clojure: Succeeds

(defn clojure-test [v]
  (let [_ (println "ORIGINAL VALUE:")
        _ (println v)
        serialized (let [out (ByteArrayOutputStream.)]
                     (transit/write (transit/writer out :json) v)
                     (.toString out))
        _ (println "SERIALIZED:")
        _ (println serialized)
        roundtrip (let [in (-> serialized
                               (.getBytes "UTF-8")
                               ByteArrayInputStream.)]
                    (transit/read (transit/reader in :json)))]
    (if (= v roundtrip)
      (println "Successful roundtrip")
      (do
        (println "FAILURE")
        (println "ROUNDTRIP VALUE:")
        (println roundtrip)))))

(clojure-test pathological)

OUTPUT:

ORIGINAL VALUE:
[{:any-value {[this vector makes this a cmap] any value, any string :victim}} {:victim :any-other-value}]
SERIALIZED:
[["^ ","~:any-value",["~#cmap",[["this vector makes this a cmap"],"any value","any string","~:victim"]]],["^ ","^2","~:any-other-value"]]
Successful roundtrip

With latest ClojureScript (1.9.76), get warning "uuid? already refers to: cljs.core/uuid? being replaced by: cognitect.transit/uuid?"

project.clj version and tag mismatch

Why is there a tag for v0.8.207 and yet project.clj has 0.8.202 as current version?

update Getting Started page

Small thing: in the getting started page,

(def w
  (t/writer :json-verbose
    {:handlers 
      {Point (PointHandler.)
       Size  (SizeHandler.)
       Rect  (RectHandler.)}})

a closing parenthesis is missing! Was copying and pasting into the repl and was wondering why the code wasn't returning...

Bump transit-js dep in pom.xml to fix metadata encoding

Trying to encode metadata in our project didn't work as expected, setting the :transform option just didn't have an effect. Tests are passing here because project.clj is using another version of transit than pom.xml.

transit-cljs/project.clj

Line 7 in ada722e

[com.cognitect/transit-js "0.8.861"]]

transit-cljs/pom.xml

Lines 134 to 136 in ada722e

    
           <groupId>com.cognitect</groupId> 
        
           <artifactId>transit-js</artifactId> 
        
           <version>0.8.846</version>

As a workaround, adding a explicit require in deps.edn on the latest transit-js fixed the issue for me:

com.cognitect/transit-js {:mvn/version "0.8.861"}

I'm happy to provide a PR for this one-line change in pom.xml it it helps.

Consider better support for DCE

Currently, because of the way the writer is set up, Google Closure will not DCE any CLJS core data structures that are otherwise unused by the developer.

Though, transit-js already support dynamically looking up an alias on the object itself. Thus by adding the alias to the prototype of the types, it'd enable proper DCE on lesser used types.

So by using, for instance:

(goog.object/set (.-prototype PersistentQueueSeq) "transitTag" List)
(goog.object/set (.-prototype PersistentQueue) "transitTag" List)
(goog.object/set (.-prototype PersistentTreeSet) "transitTag" PersistentHashSet)

And removing the entries from the map in transito.core/writer, we enable DCE without losing any functionality.

This also works fine with GCC's advanced minification.

Edit: Should probably not reuse the types since user's may provide customs handler for types in which case the alias could fail. We could introduce some simple dummy deftypes that can for sure handle the conversion and alias it to those instead.

So:

(deftype GenericListHandler [])
(deftype GenericSetHandler [])
(goog.object/set (.-prototype PersistentQueueSeq) "transitTag" GenericListHandler)
(goog.object/set (.-prototype PersistentQueue) "transitTag" GenericListHandler)
(goog.object/set (.-prototype PersistentTreeSet) "transitTag" GenericSetHandler)

;; Then in the writer:
(merge
           {GenericListHandler list-handler
            GenericSetHandler set-handler
          .....})

cognitect.transit/uuid conflicts with cljs.core/uuid

When using CLJS version 3291 or later

WARNING: uuid already refers to: cljs.core/uuid being replaced by: cognitect.transit/uuid at line 337 file:/Users/prokopov/.m2/repository/com/cognitect/transit-cljs/0.8.215/transit-cljs-0.8.215.jar!/cognitect/transit.cljs
WARNING: uuid already refers to: cljs.core/uuid being replaced by: cognitect.transit/uuid at line 337 /Users/prokopov/Dropbox/ws/datascript-todo/target/cljsbuild-compiler-0/cognitect/transit.cljs

May I suggest that simple (:refer-clojure :exclude [uuid]) should solve the problem

Stabilized specification yet? Aching to start using transit for storage

I have a React Native app that does quite a lot of serialization when I'm saving my state to localStorage, pr-str sometimes takes up to 100 ms which is not ideal. Thus I'm just wondering if the specification is stabilized yet so I could start using it for storage or if there are specification changes on the horizon?

Add support for sorted map entries

transit-clj supports writing map entries (as 2-element vectors). Here is an example:

$ clj -Sdeps '{:deps {com.cognitect/transit-clj {:mvn/version "0.8.313"}}}'
Clojure 1.10.1
user=> (require '[cognitect.transit :as transit])
nil
user=> (import [java.io ByteArrayInputStream ByteArrayOutputStream])
java.io.ByteArrayOutputStream
user=> (def out (ByteArrayOutputStream. 4096))
#'user/out
user=> (def writer (transit/writer out :json))
#'user/writer
user=> (transit/write writer (first (sorted-map :a 1)))
nil
user=> (.toString out)
"[\"~:a\",1]"

But, transit-cljs doesn't support this. Here is an example, illustrating attempting to write a cljs.core/BlackNode:

$ clj -Sdeps '{:deps {com.cognitect/transit-cljs {:mvn/version "0.8.256"} org.clojure/clojurescript {:mvn/version "1.10.520"}}}' -m cljs.main
ClojureScript 1.10.520
cljs.user=> (require '[cognitect.transit :as t])
nil
(defn roundtrip [x]
  (let [w (t/writer :json)
        r (t/reader :json)]
    (t/read r (t/write w x))))
#'cljs.user/roundtrip
cljs.user=> (roundtrip (first (sorted-map :a 1)))
Execution error (Error) at (<cljs repl>:1).
Cannot write

Additionally, note that support for seqs on sorted maps ostensibly exists, but any such seq would convey cljs.core/BlackNode and/or cljs.core/RedNode entries. Adding handlers for these two types fixes the direct round-tripping of them as described above, but also enables round-tripping of seqs on sorted maps (which works in Clojure).

Documentation missing requirement for transit

The readme should clarify that the user should also clone repository transit-format on the parent folder of transit-cljs before running

node target/transit.bench.js

ClojureScript 0.0-2342 compatibility?

Just tried to upgrade to 0.0-2342 and got this:

WARNING: Use of undeclared Var cljs.core/PersistentArrayMap.fromArray at line 75 file:/Users/sean/.m2/repository/com/cognitect/transit-cljs/0.8.188/transit-cljs-0.8.188.jar!/cognitect/transit.cljs

WARNING: Use of undeclared Var cljs.core/PersistentVector.fromArray at line 82 file:/Users/sean/.m2/repository/com/cognitect/transit-cljs/0.8.188/transit-cljs-0.8.188.jar!/cognitect/transit.cljs

List is missing in Default Type Mapping

The Default Type Mapping Table in the README does not have an entry for list. It should be

Transit type	Write accepts	Read returns
list	cc.IList	cc.IList

Error compiling

When I run "lein cljsbuild once" I get the following errors:

WARNING: No such namespace: com.cognitect.transit.eq at line 15 file:/Users/cwilliams/.m2/repository/com/cognitect/transit-cljs/0.8.199/transit-cljs-0.8.199.jar!/cognitect/transit.cljs
WARNING: No such namespace: com.cognitect.transit.types at line 15 file:/Users/cwilliams/.m2/repository/com/cognitect/transit-cljs/0.8.199/transit-cljs-0.8.199.jar!/cognitect/transit.cljs
WARNING: No such namespace: com.cognitect.transit at line 15 file:/Users/cwilliams/.m2/repository/com/cognitect/transit-cljs/0.8.199/transit-cljs-0.8.199.jar!/cognitect/transit.cljs

Then when I try to load the compiled file, I get this error:

[Error] goog.require could not find: com.cognitect.transit.eq
require (sheet.js, line 108)
global code (sheet.js, line 28738)
[Error] Error: goog.require could not find: com.cognitect.transit.eq
require (sheet.js, line 110)
global code (sheet.js, line 28738)

A couple of details that might be relevant:

I'm using version 0.8.199
It's being imported into my namespace using cljs-ajax

Decimal numbers are turned to ints when used as map keys

When using decimal numbers such as 123.456 as keys in maps, they are incorrectly turned into integers (~i123.456) when writing them with transit:

cljs.user=> (transit/write (transit/writer :json) {123.456 :foo})
"[\"^ \",\"~i123.456\",\"~:foo\"]"

Standalone decimals are fine:

cljs.user=> (transit/write (transit/writer :json) 123.456)
"[\"~#'\",123.456]"

Transit’s UUID should be IComparable to cljs.core/UUID and vice versa

Documentation/fix for default handlers

Currently we are missing docs on how to use default read/write handlers. Also the current implementation only accepts "default" as string, not :default.

In case someone gets here by looking for how to make default handlers, here is some sample code:

; default write

(deftype DefaultHandler []
  Object
  (tag [this v] "unknown")
  (rep [this v] (pr-str v)))

(def write-handlers
  {"default" (DefaultHandler.)})

bigint and bigint? have typos in their docstrings

The docstrings for bigint and bigint? both say "big decimal" instead of "big integer".

transit uuid conflicts with cljs uuid

WARNING: uuid already refers to: cljs.core/uuid being replaced by: cognitect.transit/uuid at line 337 file:/Users/Yogthos/.m2/repository/com/cognitect/transit-cljs/0.8.215/transit-cljs-0.8.215.jar!/cognitect/transit.cljs

(write) failes when sequence/transducer is passed.

When using sequence with transducers, transit/write failes with an exception.

Consider:

(def transit-writer
  (transit/writer :json))

(defn transform-map
  [m]
  (assoc m :x 1))

(def data
  [{:a 1} {:a 1}])

(prn "p1. Regular map")
(prn (transit/write transit-writer (map transform-map data)))

(prn "p2. Transducer")
(prn (transit/write transit-writer (sequence (map transform-map) data)))

(prn "p3. Done")

Regular map (p1) will print just fine, however, p2 fails with an exception:

Uncaught Error: Cannot write 
    writer.maybeQuoted @ writer.js:464
    writer.marshalTop @ writer.js:471
    writer.Writer.write @ writer.js:501
    cognitect$transit$write @ transit.cljs?rel=1478250205512:256
    (anonymous function) @ core.cljs?rel=1478251095137:53

Because of that, obviously p3 never gets printed.

Latest transit: 0.8.239. I'm able to reproduce it on ClojureScript 1.9.293 (latest) as well as 1.9.76 (haven't tried other versions).

Switch docs toYUIDoc or autodoc

YUIDoc is not tied to specific versions of ClojureScript and can work quite well. If Clojure autodoc is enhanced to use ClojureScript compiler public apis only that could also work.

Round-trip fails when write handler returns extension type

It appears that when a write handler returns an object which is not a Transit ground type, the corresponding read handler gets called with a decoder.Tag value instead of the fully-decoded value.

This only applies in :json mode, not :json-verbose.

Sample ClojureScript code:

(ns example
  (:require [cognitect.transit :as transit]))

(defrecord VectorBox [value])

(defrecord ListBox [value])

;; This Transit write handler returns a Vector, which is a Transit
;; ground type.
(deftype VectorBoxWriteHandler []
  Object
  (tag [_ box] "vectorbox")
  (rep [_ box] (vec (:value box)))
  (stringRep [_ box] nil))

;; This Transit write handler returns a List, which is an extension
;; type in Transit, not a ground type.
(deftype ListBoxWriteHandler []
  Object
  (tag [_ box] "listbox")
  (rep [_ box] (list* (:value box)))
  (stringRep [_ box] nil))

(def my-write-handlers
  {VectorBox (VectorBoxWriteHandler.)
   ListBox (ListBoxWriteHandler.)})

(def my-read-handlers
  {"vectorbox" (fn [rep] (->VectorBox (vec rep)))
   "listbox" (fn [rep] (->ListBox (vec rep)))})

(defn round-trip
  "Returns the result of writing value with Transit and then reading
  it back. transit-type is either :json or :json-verbose"
  [value transit-type]
  {:pre [(contains? #{:json :json-verbose} transit-type)]}
  (transit/read
   (transit/reader transit-type {:handlers my-read-handlers})
   (transit/write
    (transit/writer transit-type {:handlers my-write-handlers})
    value)))

(def test-value {:listbox (->ListBox [1 2 3])
                 :vectorbox (->VectorBox [1 2 3])})

(defn test-round-trip-verbose
  "Asserts that we can successfully round-trip a value through transit
  using :json-verbose encoding."
  []
  (assert (= test-value (round-trip test-value :json-verbose))))

(defn test-round-trip-json
  "Asserts that we can successfully round-trip a value through transit
  using :json encoding. This test fails."
  []
  (assert (= test-value (round-trip test-value :json))))

Tested with these dependencies:

 [com.cognitect/transit-cljs "0.8.232"]
   [com.cognitect/transit-js "0.8.755"]
 [org.clojure/clojure "1.7.0"]
 [org.clojure/clojurescript "1.7.170"]

Read handlers clobber rather than merge

Using [com.cognitect/transit-cljs "0.8.178"]

When constructing a transit writer, the :handlers key in options get merged in with the default handlers.

This doesn't seem to be true with the read handlers:

(require '[cognitect.transit :as t])

(.read (t/reader :json) "[\"~:foo\", 1]")
;=>  [:foo 1]

(.read (t/reader :json {:handlers {"custom" (fn [x] x)}}) "[\"~:foo\", 1]")
;=> [#<:foo> 1]

Consider allowing vals for tag-fn, similar to transit-clj

In transit-clj, the write-handler function allows for the tag-fn (and other fns) to be values. From the doc:

If a non-fn is passed as an argument, implemented
handler method returns the value unaltered.
https://github.com/cognitect/transit-clj/blob/master/src/cognitect/transit.clj#L73-L76

In transit-cljs you must always pass a function. Making write-handler behave the same in clj and cljs
makes sharing code for write-handler fns in cljc files a bit easier.

API Docs public website contains no content

Not sure if this is just a temporary blip or a symptom of some deeper problem, but the API Docs website for this repo, linked directly from the top of the home page, is currently showing no useful content for me. I get headers and top-level navigation but nothing else. Screen shot attached so you can see what I mean, in case you're seeing something different.

Add a CHANGELOG

:transform is not implemented on writer

I was trying to use the :transform feature from the writer but seems like it's not implemented in the CLJS version, this is an example code using Clojure transit (this works):

(let [x (with-meta [:a] {:foo "bar"})
        baos (ByteArrayOutputStream.)
        w    (transit/writer baos :json {:transform transit/write-meta})
        _    (transit/write w x)
        ret  (.toString baos)]
    (.reset baos)
    ret)
=> "[\"~#with-meta\",[[\"~:a\"],[\"^ \",\"~:foo\",\"bar\"]]]"

But doing a similar run in CLJS yields:

(-> (transit/writer :json {:transform transit/write-meta}) 
      (transit/write (with-meta [:a] {:foo "data"})))
=> "[\"~:a\"]"

The docs on CLJS suggest this should work. So I guess it was copied the docs but never implemented?

Add documentation to point out that UUID is a cct type, not a cc type

In the past people have been confused using UUID's in Transit. Transit-cljs uses a custom cct UUID type, not a native CLJS UUID. It would be good to explicitly point this out in the documentation and show how people can override this behaviour if they want to get CLJS UUID's.

This could be something like:

Transit-cljs returns custom com.cognitect.transit types by default for some values, including UUID's. To override this and return native ClojureScript UUID's, add a custom read handler for u. This will override the default read handler.
(t/reader :json {:handlers {"u" cljs.core/uuid} })

Relates to #11

Publishing a changelog

It would be really helpful if this library published a changelog so I can see what has been changed, added, deprecated, e.t.c. when upgrading. For example 0.8.225 depends on a version of CLJS 1.7.107 that hasn't been marked stable yet, and may be using features introduced in that version.

Exclude uri?

Much like #26, we could use an exclude for uri? because with clojure/clojurescript@35ad08e we now get a :redef warning:

WARNING: uri? already refers to: cljs.core/uri? being replaced by: cognitect.transit/uri? at line 332 out/cognitect/transit.cljs

Add support for cljs.core/Eduction if it exists

Repro:

{:deps {org.clojure/clojurescript {:mvn/version "1.9.946"}
        com.cognitect/transit-cljs {:mvn/version "0.8.243"}}}

$ clj -m cljs.repl.node
ClojureScript Node.js REPL server listening on 55348
To quit, type: :cljs/quit
cljs.user=> (exists? cljs.core/Eduction)
true
cljs.user=> (require '[cognitect.transit :as t])
nil
cljs.user=> (t/write (t/writer :json) (seq (eduction (take 3) (range))))
"[\"~#list\",[0,1,2]]"
cljs.user=> (t/write (t/writer :json) (eduction (take 3) (range)))
repl:13
throw e__8152__auto__;
^

Error: Cannot write
    at Object.writer.maybeQuoted (/private/tmp/transit-cljs/.cljs_node_repl/com/cognitect/transit/impl/writer.js:464:23)
    at Object.writer.marshalTop (/private/tmp/transit-cljs/.cljs_node_repl/com/cognitect/transit/impl/writer.js:471:57)
    at Transit$Writer.writer.Writer.write (/private/tmp/transit-cljs/.cljs_node_repl/com/cognitect/transit/impl/writer.js:501:26)
    at cognitect$transit$write (/private/tmp/transit-cljs/.cljs_node_repl/cognitect/transit.js:886:10)
    at repl:1:103
    at repl:9:3
    at repl:14:4
    at Script.runInThisContext (vm.js:65:33)
    at Object.runInThisContext (vm.js:199:38)
    at Domain.<anonymous> ([stdin]:50:34)

Ability to ignore tags when reading JSON?

I'm using transit-cljs because it's way faster than JSON.parse + js->clj. That's awesome.

I'm decoding user provided JSON files that happen to include ["~some stuff", ....] and these get decoded as [TaggedValue....] which breaks stuff. Users include my .js, point it to a JSON file so I can't transit-encode them upfront.

Is there a possibility to tell the transit/reader to stop treating tags in a special way?

Data is being attached to the wrong keys on read

I am reading the same file in through clojure and clojurescript and in reading the clojurescript data is being tied the wrong keys. I have included an example and the transit file in question to allow anyone to reproduce the issue.

ClojureScript REPL

development:cljs.user=> (require '[cljs-node-io.core :as io :refer [slurp spit]])
                   #_=> (require '[cognitect.transit :as ct])
                   #_=> (def tmp
                   #_=>      (let [ins (slurp "/Users/ckirkendall/Development/gr/one-loan-client/resources/trans-ast.transit")
                   #_=>                rdr (ct/reader :json)]
                   #_=>        (ct/read rdr ins)))
                   #_=> (get-in tmp [:loanProductData :children :prepaymentPenalties])
nil
nil
#'cljs.user/tmp
{:type :leaf, :field [:1946], :field-type "string"}

Clojure REPL

user=> (require '[clojure.java.io :as io])

user=> (require '[cognitect.transit :as ct])
nil
user=> (def tmp
  #_=>      (let [ins (io/input-stream (io/file "/Users/ckirkendall/Development/gr/one-loan-client/resources/trans-ast.transit"))
  #_=>                rdr (ct/reader ins :json)]
  #_=>        (ct/read rdr)))
#'user/tmp
user=> (get-in tmp [:loanProductData :children :prepaymentPenalties])
{:type :array, :children [{:idx 1, :children {:fullPrepaymentPenaltyOptionType {:type :leaf, :field [:1971], :field-type "string"}, :termMonthsCount {:type :leaf, :field [:1972], :field-type "int"}, :prepaymentPenaltyPercent {:type :leaf, :field [:1973], :field-type "decimal"}}} {:idx 2, :children {:fullPrepaymentPenaltyOptionType {:type :leaf, :field [:1974], :field-type "string"}, :termMonthsCount {:type :leaf, :field [:1975], :field-type "int"}, :prepaymentPenaltyPercent {:type :leaf, :field [:1976], :field-type "decimal"}}} {:idx 3, :children {:fullPrepaymentPenaltyOptionType {:type :leaf, :field [:1977], :field-type "string"}, :termMonthsCount {:type :leaf, :field [:1978], :field-type "int"}, :prepaymentPenaltyPercent {:type :leaf, :field [:1979], :field-type "decimal"}}} {:idx 4, :children {:fullPrepaymentPenaltyOptionType {:type :leaf, :field [:1980], :field-type "string"}, :termMonthsCount {:type :leaf, :field [:1981], :field-type "int"}, :prepaymentPenaltyPercent {:type :leaf, :field [:1982], :field-type "decimal"}}}]}

Here is the file in question so this issue can be reproduced:
trans-ast.transit.zip

Transit `TaggedValue` should implement `IComparable`

Does it make sense for Transit BigIntegers and BigDecimals to be comparable?

The specific issue that I ran into is that Transit values cannot be indexed by DataScript:

Uncaught Error: Cannot compare [TaggedValue: n, 37502364230516917834597274526242608522740435501611847496456113203943353843903] to [TaggedValue: n, 37746944873274413711073286601178981016490453790917975016109108374206467768511]
    at cljs$core$compare (core.cljs:2241)
    at datascript$db$cmp_val (db.cljc:288)
    at datascript$db$cmp_datoms_avet (db.cljc:311)
    at datascript$btset$binary_search_l (btset.cljc:72)
    at datascript$btset$_seek (btset.cljc:951)
    at datascript$btset$_slice (btset.cljc:979)
    at Function.datascript.btset.slice.cljs$core$IFn$_invoke$arity$3 (btset.cljc:990)
    at datascript$btset$slice (btset.cljc:985)
    at Function.datascript.btset.slice.cljs$core$IFn$_invoke$arity$2 (btset.cljc:988)
    at datascript$btset$slice (btset.cljc:985)

Add support for cljs.core/Repeat if it exists

With ClojureScript 1.9.946

{:deps {org.clojure/clojurescript {:mvn/version "1.9.946"}
        com.cognitect/transit-cljs {:mvn/version "0.8.243"}}}

$ clj -m cljs.repl.node
ClojureScript Node.js REPL server listening on 55429
To quit, type: :cljs/quit
cljs.user=> (exists? cljs.core/Repeat)
false
cljs.user=> (require '[cognitect.transit :as t])
nil
cljs.user=> (t/write (t/writer :json) (repeat 3 :hi))
"[\"~#list\",[\"~:hi\",\"^1\",\"^1\"]]"

But with ClojureScript 1.10.x

{:deps {org.clojure/clojurescript {:mvn/version "1.10.191"}
        com.cognitect/transit-cljs {:mvn/version "0.8.243"}}}

$ clj -m cljs.repl.node
ClojureScript 1.10.191
cljs.user=> (exists? cljs.core/Repeat)
true
cljs.user=> (require '[cognitect.transit :as t])
nil
cljs.user=> (t/write (t/writer :json) (repeat 3 :hi))
repl:13
throw e__6371__auto__;
^

Error: Cannot write
    at Object.writer.maybeQuoted (/private/var/folders/gx/nymj3l7x4zq3gxb97v2zwzb40000gn/T/out2256590624295300548180118500448100/com/cognitect/transit/impl/writer.js:464:23)
    at Object.writer.marshalTop (/private/var/folders/gx/nymj3l7x4zq3gxb97v2zwzb40000gn/T/out2256590624295300548180118500448100/com/cognitect/transit/impl/writer.js:471:57)
    at Transit$Writer.writer.Writer.write (/private/var/folders/gx/nymj3l7x4zq3gxb97v2zwzb40000gn/T/out2256590624295300548180118500448100/com/cognitect/transit/impl/writer.js:501:26)
    at cognitect$transit$write (/private/var/folders/gx/nymj3l7x4zq3gxb97v2zwzb40000gn/T/out2256590624295300548180118500448100/cognitect/transit.js:923:10)
    at repl:1:103
    at repl:9:3
    at repl:14:4
    at Script.runInThisContext (vm.js:65:33)
    at Object.runInThisContext (vm.js:199:38)
    at Domain.<anonymous> ([stdin]:56:38)

back ~r Uri type with goog.Uri

URIs as tagged values aren't very useful generally speaking, so right now I'm doing:

(deftype URIHandler []
    Object
  (tag [this v] "r")
  (rep [this v] (.toString v))
  (stringRep [this v] nil))

(def transit-reader
  (transit/reader :json
                  {:handlers
                   {"r" (fn [uri] (Uri. uri))}}))

(def transit-writer
  (transit/writer :json
                  {:handlers
                   {goog.Uri (URIHandler.)}}))

Would be nice to actually have this built into transit-cljs, though I'm not sure what the implications for running anywhere that goog.closure has a hard time reaching (self-hosting issue?).

transit.cljs:285 (ty/integer s) undefined

Seeing this in my console trying to use (cognitect.transit/integer "30")
Looking in my js output directory, it seems like it's missing from com/cognitect/transit/types.js

Using transit-cljs version "0.8.207"

Bug: read-handler with multiple arities breaks

Description

If a read-handler function has more than one arity, transit-cljs/read attempts to call the incorrect arity when deserializing.

If the incorrect arity happens to be defined, this can lead to bizarre errors originating far from the transit code or top-level constructor; if not defined, it results in "Error: Invalid arity: X".

I discovered this by using partial to define a read handler, which of course constructs a function with many arities. The extra argument being passed to the function appears to be a Transit$TaggedValue, defined in transit-js.

Steps to Reproduce

Use a read-handler function with more than one arity. A complete minimal example, based on the official tutorial, can be found here:

https://gist.github.com/galdre/cc875e28590d253b2c40cdfd3f5ecd07

Add support for map entries

transit-clj supports writing map entries (as 2-element vectors). Here is an example:

$ clj -Sdeps '{:deps {com.cognitect/transit-clj {:mvn/version "0.8.300"}}}'
Clojure 1.9.0
user=> (require '[cognitect.transit :as transit])
nil
user=> (import [java.io ByteArrayInputStream ByteArrayOutputStream])
java.io.ByteArrayOutputStream
(def out (ByteArrayOutputStream. 4096))
#'user/out
(def writer (transit/writer out :json))
#'user/writer
user=> (transit/write writer "foo")
nil
user=> (transit/write writer (into [] (sorted-map :a 1)))
nil
user=> (transit/write writer (into [] {:b 2}))
nil
user=> (.toString out)
"[\"~#'\",\"foo\"] [[\"~:a\",1]] [[\"~:b\",2]]"
user=> (def in (ByteArrayInputStream. (.toByteArray out)))
#'user/in
user=> (def reader (transit/reader in :json))
#'user/reader
user=> (prn (transit/read reader))
"foo"
nil
user=> (transit/read reader)
[[:a 1]]
user=> (type (first *1))
clojure.lang.PersistentVector
user=> (transit/read reader)
[[:b 2]]
user=> (type (first *1))
clojure.lang.PersistentVector

But, transit-cljs doesn't support this. Here is an example, illustrating attempting to a cljs.core/BlackNode and a cljs.core/MapEntry:

$ clojure -Sdeps '{:deps {com.cognitect/transit-cljs {:mvn/version "0.8.243"} org.clojure/clojurescript {:git/url "https://github.com/clojure/clojurescript" :sha "c6961d518beb69a8ecd84421b13778fab33bdaab"}}}' -m cljs.main
To quit, type: :cljs/quit
cljs.user=> (require '[cognitect.transit :as t])

cljs.user=> (defn roundtrip [x]
  (let [w (t/writer :json)
        r (t/reader :json)]
    (t/read r (t/write w x))))
#'cljs.user/roundtrip
cljs.user=> (roundtrip (into [] (sorted-map :a 1)))
Error: Cannot write
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:446:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:172:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:183:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:436:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:471:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:501:0)
	 cognitect$transit$write (.cljs_nashorn_repl/cognitect/transit.cljs:256:3)
	 (NO_SOURCE_FILE <eval>:5:0)
	 (NO_SOURCE_FILE <eval>:1:0)
	 (NO_SOURCE_FILE <eval>:1:0)
	 (NO_SOURCE_FILE <eval>:1:0)
cljs.user=> (roundtrip (into [] {:a 1}))
Error: Cannot write
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:446:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:172:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:183:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:436:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:471:0)
	 (.cljs_nashorn_repl/com/cognitect/transit/impl/writer.js:501:0)
	 cognitect$transit$write (.cljs_nashorn_repl/cognitect/transit.cljs:256:3)
	 (NO_SOURCE_FILE <eval>:5:0)
	 (NO_SOURCE_FILE <eval>:1:0)
	 (NO_SOURCE_FILE <eval>:1:0)
	 (NO_SOURCE_FILE <eval>:1:0)

As an aside: The motivating scenario where this arose is that the ClojureScript compiler serializes out, under the key :cljs.spec/registry-ref a sequence of map entries, into its analysis cache. This works fine for for the Clojure-implementation of ClojureScript when it uses transit-clj, but the same use fails if you attempt to do the same with self-hosted ClojureScript, using transit-cljs.

Write fails when data contains a js/Function instance

I am trying to use transit as a means to send cljs data structures to a chrome extension without having to use clj->js on one end, send, and then js->clj on the other end. If the data structure contains a js/Function instance an error is thrown, cannot write. It seems to work if there is a js/Object instance as data.

I cannot guarantee that the data will not contain js/Function instances as data. Is there any way around this?

Comparing UUIDs is unnecessarily slow

With the current implementation:

(extend-protocol IComparable
  UUID
  (-compare [this other]
    (if (or (instance? UUID other)
            (instance? ty/UUID other))
      (compare (.toString this) (.toString other))
      (throw (js/Error. (str "Cannot compare " this " to " other)))))

The compare call will again check the strings for IComparable and even call (slow) native-satisfies this is very undesirable since I'm relying on fast comparisons in datascript datoms.

It'd be better if a call to garray/defaultCompare were made (as is done in cljs.core).

Byte arrays -> Uint8Array and comparability

When encoding a byte array with transit-clj, and then decoding it with transit-cljs, a Uint8Array type is used. This breaks several things, i.e. such data structures cannot be compared:

using = (can get around that with extend-type, use array-seq in there)
using set/difference (can't get around that, as it uses identical?).

(One possible solution would be to provide custom handler for "b", but it is currently prohibited in transit.impl.decoder in transit-js library - the "Cannot override handler for ground type "b")" exception is being thrown.)

Wrong number of args on java 1.9

By instance:

user=>(require '[cognitect.transit :as t])
nil
user=> (def r (t/reader :json))

CompilerException clojure.lang.ArityException: Wrong number of args (1) passed to: transit/reader, compiling:(form-init2906799654242627519.clj:1:8)

Support for MapEntry writing

New CLJS type MapEntry should be handled by the writer.

Encoding for float ground type

In Javascript platform I would like to make {:person/weight 200.0} transmit as a float, not an int. By default Javascript will turn that to Number: 200 and encode an int, JVM will decode int and JVM doesn't have the context to know that it should be coerced to float. It is the client's job to transmit the data properly and deal with platform quirks at the point that the value escapes the platform.

Forgoing the above, I would like to argue that this is a foundational flaw in transit as argued here on ask.clojure.org

Add support for uuid?

There is code in transit-cljs that supports transparently treating core ClojureScript UUID types and the Transit-specific UUID type as being equivalent.

Recently, cljs.core/uuid? was added, but it returns false for Transit-specific UUIDs.

Perhaps this could be fixed by doing something like (extend-type ty/UUID IUUID) in the Transit cljs core namespace.

write-read roundtrip results in different value, when key of a map is a decimal

My test generator found following, wrong behavior.

(let [w (transit/writer :json) 
      r (transit/reader :json)]
      (transit/read r (transit/write w {0.5 0})))
==> {0 0}

(let [w (transit/writer :json)]
     (transit/write w {0.5 0.5}))
==> "[\"^ \",\"~i0.5\",0.5]"

Version: [com.cognitect/transit-cljs "0.8.239"]

Uncaught Error: Undefined nameToPath for com.cognitect.transit.eq

I get the following warnings on cljsbuild. Am I possibly doing something wrong, or is this release broken at the moment? (note I tried with 0.8.188 as well)

WARNING: No such namespace: com.cognitect.transit.eq at line 15 file:/Users/julian/.m2/repository/com/cognitect/transit-cljs/0.8.186/transit-cljs-0.8.186.jar!/cognitect/transit.cljs
WARNING: No such namespace: com.cognitect.transit.types at line 15 file:/Users/julian/.m2/repository/com/cognitect/transit-cljs/0.8.186/transit-cljs-0.8.186.jar!/cognitect/transit.cljs
WARNING: No such namespace: com.cognitect.transit at line 15 file:/Users/julian/.m2/repository/com/cognitect/transit-cljs/0.8.186/transit-cljs-0.8.186.jar!/cognitect/transit.cljs

I couldn't see how the dependencies were included in the cljs file (given my relative lack of CLJS/CLJ skill, apologies in advance if I'm barking up the wrong tree)

Not all tests being run via lein

If you do lein cljsbuild once test you will see:

Testing transit.test.core

Ran 10 tests containing 46 assertions.

The run-tests macro is sitting in -main that is too high up in the file, prematurely expanding to only the tests that have been defined at that point in the source:

transit-cljs/test/transit/test/core.cljs

Line 161 in 6d59713

(run-tests))

If you move it down to just before (set! *main-cli-fn* -main) you will see all of the tests being run:

Testing transit.test.core

Ran 19 tests containing 58 assertions.

Use of aget / aset when options present triggers checked-arrays

If you specify options to t/writer with :checked-arrays enabled, this code will trigger.

Here is a working example without the compiler flag:

$ clj -Srepro -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.520"} com.cognitect/transit-cljs {:mvn/version "0.8.256"} com.cognitect/transit-js {:mvn/version "0.8.861"}}}' -m cljs.main -re node -r
ClojureScript 1.10.520
cljs.user=> (require '[cognitect.transit :as t])
nil
cljs.user=> (t/write (t/writer :json {:transform t/write-meta}) ^:foo [1 2])
"[\"~#with-meta\",[[1,2],[\"^ \",\"~:foo\",true]]]"

And here is an example with the flag enabled at the :warn level:

$ clj -Srepro -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.520"} com.cognitect/transit-cljs {:mvn/version "0.8.256"} com.cognitect/transit-js {:mvn/version "0.8.861"}}}' -m cljs.main -co '{:checked-arrays :warn}' -re node -r
ClojureScript 1.10.520
cljs.user=> (require '[cognitect.transit :as t])
nil
cljs.user=> (t/write (t/writer :json {:transform t/write-meta}) ^:foo [1 2])
Error: Assert failed: (or (array? array) (goog/isArrayLike array))
    at Function.cljs.core.checked_aget.cljs$core$IFn$_invoke$arity$2 (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cljs/core.js:524:8)
    at cljs$core$checked_aget (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cljs/core.js:490:31)
    at cognitect$transit$opts_merge (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cognitect/transit.js:132:37)
    at Function.cognitect.transit.writer.cljs$core$IFn$_invoke$arity$2 (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cognitect/transit.js:906:97)
    at cognitect$transit$writer (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cognitect/transit.js:883:33)
    at repl:1:148
    at repl:9:3
    at repl:14:4
    at ContextifyScript.Script.runInThisContext (vm.js:50:33)
    at Object.runInThisContext (vm.js:139:38)
Error: Assert failed: (or (array? array) (goog/isArrayLike array))
    at Function.cljs.core.checked_aset.cljs$core$IFn$_invoke$arity$3 (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cljs/core.js:601:8)
    at cljs$core$checked_aset (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cljs/core.js:567:31)
    at cognitect$transit$opts_merge (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cognitect/transit.js:133:24)
    at Function.cognitect.transit.writer.cljs$core$IFn$_invoke$arity$2 (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cognitect/transit.js:906:97)
    at cognitect$transit$writer (/private/var/folders/yw/77skdn6s2099kb4_43vvrg1c0000gn/T/out5495452781688741529186887019006156/cognitect/transit.js:883:33)
    at repl:1:148
    at repl:9:3
    at repl:14:4
    at ContextifyScript.Script.runInThisContext (vm.js:50:33)
    at Object.runInThisContext (vm.js:139:38)
"[\"~#with-meta\",[[1,2],[\"^ \",\"~:foo\",true]]]"

Unify clj/cljs APIs where possible

Though some aspects of the API are necessarily different (IO stream vs. String decoding/encoding), many aspects of the APIs could probably benefit from unification so that users do not shim the differences themselves.

	<groupId>com.cognitect</groupId>
	<artifactId>transit-js</artifactId>
	<version>0.8.846</version>