Code Monkey home page Code Monkey logo

guile-json's Introduction

guile-json

GNU Guile 2.2 GNU Guile 3.0

guile-json is a JSON module for Guile. It supports parsing and building JSON documents according to the http://json.org specification.

  • Complies with http://json.org specification.

  • Supports JSON Text Sequences (RFC 7464).

  • Supports parsing concatenated JSON documents.

  • Builds JSON documents programmatically using scheme data types.

  • Allows JSON pretty printing.

Installation

Download the latest tarball and untar it:

If you are cloning the repository make sure you run this first:

$ autoreconf -vif

Then, run the typical sequence:

$ ./configure --prefix=<guile-prefix>
$ make
$ sudo make install

Where <guile-prefix> should preferably be the same as your system Guile installation directory (e.g. /usr).

If everything installed successfully you should be up and running:

$ guile
scheme@(guile-user)> (use-modules (json))
scheme@(guile-user)> (scm->json #(1 2 3))
[1,2,3]

It might be that you installed guile-json somewhere differently than your system's Guile. If so, you need to indicate Guile where to find guile-json, for example:

$ GUILE_LOAD_PATH=/usr/local/share/guile/site guile

Usage

guile-json provides a few procedures to parse and build a JSON document. A JSON document is transformed into or from native Guile values according to the following table:

JSON Guile
string string
number number
object alist
array vector
true #t
false #f
null 'null

Why are JSON arrays converted to vectors and JSON objects to alists? See this discussion for details.

By default the value of JSON "null" is mapped to the symbol 'null. However, all guile-json functions allow changing the default null value by specifying the #:null keyword argument with another value. This other value needs to be recognized by eq?.

To start using guile-json procedures and macros you first need to load the module:

scheme@(guile-user)> (use-modules (json))

Reading JSON documents

  • (json->scm #:optional port #:key null ordered concatenated) : Reads a JSON document from the given port, or from the current input port if none is given.

    Optional arguments:

    • port : is optional, it defaults to the current input port.

    Keyword arguments:

    • null : value for JSON's null, it defaults to the 'null symbol.

    • ordered : indicate whether JSON objects order should be preserved or not (the default).

    • concatenated : if true it tells the parser that more JSON documents might be present after a properly parsed document, otherwise the parser will fail if additional data is present after the first document (this is the default).

  • (json-string->scm str #:key null ordered) : Reads a JSON document from the given string.

    Keyword arguments:

    • null : value for JSON's null, it defaults to the 'null symbol.

    • ordered : indicate whether JSON objects order should be preserved or not (the default).

Building JSON documents

  • (scm->json native #:optional port #:key solidus unicode null validate pretty) : Creates a JSON document from the given native Guile value. The JSON document is written into the given port, or to the current output port if non is given.

    Optional arguments:

    • port : it defaults to the current output port.

    Keyword arguments:

    • solidus : if true, the slash (/ solidus) character will be escaped (defaults to false).

    • unicode : if true, additional to control characters, non-ASCII characters will be escaped as well (defaults to false).

    • null : value for JSON's null (defaults to the 'null symbol).

    • validate : if true, the native value will be validated before starting to print the JSON document (defaults to true).

    • pretty : if true, the JSON document will be pretty printed (defaults to false).

  • (scm->json-string native #:key solidus unicode null validate pretty) : Creates a JSON document from the given native Guile value into a string.

    See keyword arguments for scm->json.

    Note that when using alists to build JSON objects, symbols or numbers might be used as keys and they both will be converted to strings.

Reading JSON Text Sequences

  • (json-seq->scm #:optional port #:key null ordered handle-truncate truncated-object) : Reads a stream of JSON documents from the given port, or from the current input port if none is given.

    Optional arguments:

    • port : is optional, it defaults to the current input port.

    Keyword arguments:

    • null : value for JSON's null, it defaults to the 'null symbol.

    • ordered : indicate whether JSON objects order should be preserved or not (the default).

    • handle-truncate : defines how to handle data loss. Possible values:

      • 'throw: throw an exception.
      • 'stop: stop parsing and end the stream.
      • 'skip: skip corrupted fragment and return next object (default).
      • 'replace: skip corrupted fragment and return object specific by truncated-object.
    • truncated-object : use this object if an object could not be parsed (to be used when setting handle-truncate to 'replace value).

  • (json-seq-string->scm str #:key null ordered handle-truncate truncated-object) : Reads a stream of JSON documents from the given string.

    See keyword arguments for json-seq->scm.

Building JSON Text Sequences

  • (scm->json-seq objects #:optional port #:key solidus null validate) : Creates a JSON document sequence from the given list of native Guile objects. The JSON document sequence is written into the given port, or to the current output port if non is given.

    Optional arguments:

    • port : it defaults to the current output port.

    Keyword arguments:

    • solidus : if true, the slash (/ solidus) character will be escaped (defaults to false).

    • null : value for JSON's null (defaults to the 'null symbol).

    • validate : if true, the native value will be validated before starting to print the JSON document (defaults to true).

  • (scm->json-seq-string objects #:key solidus null validate) : Creates a JSON document sequence from the given list of native Guile objects into a string.

    See keyword arguments for scm->json-seq.

Exceptions

A json-invalid exception is thrown if an error is found during the JSON parsing with a single port argument. The line or column where the error occured can be easily obtained from the port by calling port-line or port-column.

When building a JSON document from a native type a json-invalid exception might be thrown with the offending value as an argument (see table above for supported types).

JSON Objects and Records

guile-json 4.5.0 introduces JSON types, a new feature that allows converting JSON objects into record types and vice versa in a very straight forward way. This was built on top of define-json-mapping which was introduced in version 4.2.0.

Let's take a look at an example. Imagine we have the following user account information:

{
  "id": 1234,
  "username": "jane"
}

We can easily create a record representing that data with define-json-type by simply doing:

> (define-json-type <account>
    (id)
    (username))

This will define the record constructor, the predicate and conversion procedures like json->account or account->json (see define-json-type for more details).

We can now create a new account and check its contents as with regular records:

> (define account (make-account "1234" "jane"))
> (account-id account)
"1234"
> (account-username account)
"jane"

Or we can use the auto-generated scm->account to create the account:

> (define account (scm->account '(("id" . "1234") ("username" . "jane"))))

It is also possible to convert the record to a JSON string:

> (account->json account)
"{\"id\":\"1234\",\"username\":\"jane\"}"

Or from a JSON string to a new record:

> (define json-account "{\"id\":\"1234\",\"username\":\"jane\"}")
> (json->account json-account)
#<<account> id: "1234" username: "jane">

We could also create a list of accounts:

> (define-json-type <accounts-list>
    (accounts "accounts" #(<account)))

In which case we would do:

> (scm->accounts-list '(("accounts" . #((("id" . "1234") ("username" . "jane"))
                                        (("id" . "4321") ("username" . "joe"))))))
#<<accounts-list> accounts: (#<<account> id: "1234" username: "jane"> #<<account> id: "4321" username: "joe">)>

Note how the accounts field is stored as a list inside the record field, this is to simplify creating records and working with field getters. For example, to create the same but directly using records we would use:

> (make-accounts-list (list (make-account "1234" "jane") (make-account "4321" "joe")))

Macros

  • (define-json-type rtd (field key type) ...) : Define a new mapping between a JSON object and a record type. This will automatically define the record constructor, the predicate, all field getters and record to/from JSON conversions. For more control use define-json-mapping.

    • rtd : the name of the record type.

    • ((field key type) ...) : a series of field specifications.

      • field : the name of a JSON object field.

      • key : a different name for the field of this JSON object. If given, this name will be used instead of the field name when serializing or deserializing.

      • type : indicates that this field contains a record type. It is also possible to indicate that the field contains an array of objects of the same record type by using the vector syntax #(type). However, note that in order to simplify things a list will be stored in the record field (see examples).

  • (define-json-mapping rtd ctor pred json->record [<=> record->json [<=> scm->record]] (field getter key ...) ...) : Define a new mapping between a JSON object and a record type, à la SRFI-9.

    • rtd : the name of the record type.

    • ctor : the name for the record constructor procedure.

    • pred : a predicate procedure to check if a given argument holds a record of this type.

    • json->record : the name of the procedure to convert a JSON object, read from a port, string, or alist, into a record of this type.

    • <=> record->json : optional name of the procedure to convert a record of this type to a JSON string.

    • <=> scm->record : optional name of the procedure to convert a JSON object, represented as an alist, into a record of this type. This is equivalent to json->record when an alist is passed.

    • <=> record->scm : optional name of the procedure to convert a record of this type to a JSON objected represented as an alist.

    • ((field getter ...) ...) : a series of field specifications.

      • field : the name of a JSON object field.

      • getter : the name of the procedure to get the value of this field given a record of this type.

      • key : a different name for the field of this JSON object. If given, this name will be used instead of the field name when serializing or deserializing.

      • scm->value : an optional procedure that will be used to convert native values supported by guile-json to the value contained in the record. Used when reading JSON.

      • value->scm : an optional procedure that will be used to convert the value contained in the record to a native value supported by guile-json Used when writing JSON.

Records and null fields

When serializing a record to a JSON object it is possible to set a field to the *unspecified* value in order to omit it from serialization. Also, when deserializing a JSON object to a record, missing record fields in the JSON object will be set to *unspecified* in the record.

Examples

  • Build the string "hello world":
> (scm->json "hello world")
"hello world"
  • Build the [1, 2, 3] array:
> (scm->json #(1 2 3))
[1,2,3]
  • Build the object { "project" : "foo", "author" : "bar" } using an alist:
> (scm->json '(("project" . "foo") ("author" . "bar")))
{"project":"foo","author":"bar"}
  • Build the same object but this time using symbols:
> (scm->json '((project . foo) ("author" . "bar")))
{"project":"foo","author":"bar"}
  • Build the object { "values" : [ 234, 98.56 ] }:
> (scm->json '(("values" . #(234 98.56))))
{"values":[234,98.56]}
  • Build the object { "values" : [ 234, 98.56 ] } again, this time using a variable:
> (define values #(234 98.56))
> (scm->json `(("values" . ,values)))
{"values":[234,98.56]}
  • Default null value is the 'null symbol:
> (scm->json 'null)
null
  • The default null value can be changed to something else:
> (scm->json #nil #:null #nil)
null

License

Copyright (C) 2013-2022 Aleix Conchillo Flaque [email protected]

guile-json is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

guile-json is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with guile-json. If not, see https://www.gnu.org/licenses/.

guile-json's People

Contributors

aconchillo avatar christopherlam avatar civodul avatar davexunit avatar ijp avatar janneke avatar jason-earl avatar kutsurak avatar lee-thomp avatar mazechazer avatar mothacehe avatar rain-1 avatar sirikid avatar vagrantc avatar zelphirkaltstahl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

guile-json's Issues

The builder doesn't validate the native object first.

Doing this requires an extra tree walk, but for example if you call (json-scm (vector 1 2 3 #u8(1 2 3))) it will output [1,2,3 before throwing an exception. This is not the Right Thing: a procedure that does output should work correctly or fail cleanly without outputting anything.

Json record type public procedures

Hi!

I have a few decently sized json types that I am defining with define-json-type.

It would be nice to be able to have something like define-json-type-public so that I don't have to export all of the generated procedures.

Is this possible or is there any kind of workaround other than listing all of the symbols in a module's #:export?

Thanks

Symbols are output as JSON strings

This will be problematic if anyone depends on it and you decide to switch to the symbol null for JSON null, as I suggested in my previous email. I think symbols should be invalid as JSON values.

Mapping nested JSON objects

I access an api which sends its response back in the following shape:

{
  "results": {
    "sunrise": "9:52:03 AM",
    "sunset": "12:20:22 AM",
    "first_light": "8:01:33 AM",
    "last_light": "2:10:52 AM",
    "dawn": "9:21:29 AM",
    "dusk": "12:50:56 AM",
    "solar_noon": "5:06:12 PM",
    "golden_hour": "11:41:50 PM",
    "day_length": "14:28:19",
    "timezone": "UTC"
  },
  "status": "OK"
}

I would like this data (or, in the future, any nested json) to be accessible using guile's record type. When I feed this into my record constructor, I wind up with the 'results' field containing all the data and fields like 'sunrise' and 'dawn' are all unspecified. Is there a way to go from the JSON above directly to the record type without first extracting all the fields under the "results" section and using that as the data provided to the record constructor?

Thanks for a great module!

nullable fields in define-json-type macro

Thanks for the awesome work.
I was wondering if it's possible to somehow allow nullable fields in the define-json-type macro.
I think that this is indeed the expected behavior.

Example

(define-json-type <student>
  (id)
  (name)
  (major "major" <major>))

(define-json-type <major>
  (id)
  (name)
  (parent "parent" <major>))

(define json-str "{ \"id\": 1, \"name\": null, \"major\": null}")
(json->student json-str)

error

In procedure assoc: Wrong type argument in position 2 (expecting association list): null

changing the json-str to the below

builds a correct (not unspecified) scheme object

(define json-str "{ \"id\": 1, \"name\": null, \"major\": {}}")
(json->student json-str)

output

=> #<<student> id: 1 name: null major: #<<major> id: #<unspecified> name: #<unspecified> parent: #<unspecified>>>

also emitting the field major will produce correct results.

(define json-str "{ \"id\": 1, \"name\": null }")
(json->student json-str)

output

=> #<<student> id: 1 name: null major: #<unspecified>>

Anything wrong with my approach?
Thanks again for your great work.

JSON numbers must be exact integers or inexact floats

JSON numbers have to be, in Scheme terms, either exact integers or inexact reals excluding +inf.0, -inf.0, and +nan.0. Guile-json will blithely output any of these three as well as 1/2, 3.4+5.6i, etc., which will not work with other JSON parsers.

Error while running make on Ubuntu

I've tried to call

./configure
make

but got error from make:

../env  compile -Wunbound-variable -Warity-mismatch -Wformat -o "builder.go" "builder.scm"
../env: 17: exec: compile: not found

Licensing

Hello,
Not a complaint about quality of code. This module is superb.
I am exploring plugging it into Gnucash reporting engine. But Gnucash uses GPLv2 or GPLv3.
I don't know if it is acceptable, or rude, to ask for this module to be GPLv2. I apologize in advance if that's the case. The relevant work is at Gnucash/gnucash#316 to upgrade the old jqplot code to chartjs.

array datatypes - how to create blank slots?

I'm not sure if this is a genuine issue, or valid json. But I hope I can make sense.

In scheme I can create a blank vector with (make-vector 3) which creates #(#unspecified #unspecified #unspecified). This vector is currently impossible to convert to json. Could this be convertible to [,,]?

In javascript I can create an array with empty slots: a = [0,1,2,,,5] whereby a[3] is undefined. Converting this to scm leads to #(0 1 2 5) which is imho not desirable. A good scm output would be #(0 1 2 #unspecified #unspecified 5)?

Unicode characters in string input and output

I have the following code:

(use-modules (json))

(define* (print-json the-json)
  (display
   (simple-format #f "~a\n" (scm->json-string the-json #:escape #f #:pretty #t))))

(define* (get-json-from-file file-path)
  ;; json->scm takes an optional port, so we can call it this way.
  (call-with-input-file file-path json->scm))

(define* (put-json-to-file file-path the-json #:key (escape-slashes #f) (pretty #f))
  (call-with-output-file file-path
    (λ (port)
      (set-port-encoding! port "UTF-8")
      (scm->json the-json port #:escape escape-slashes #:pretty pretty))))

In my JSON file, I have Unicode characters in strings, for example like this:

{
    "000": {
        "metadata": {
            "learned": false,
            "description": "",
            "relevance": 0,
            "difficulty": 0,
            "similar_words": [],
            "mnemonics": [],
            "explanations": [],
            "tags": [],
            "usage_examples": []
        },
        "translation_data": {
            "english": "eight",
            "pinyin_numbered": "ba1",
            "pinyin": "",
            "simplified": "",
            "traditional": ""
        }
    },
    ...
}

When I read it in and display it, it all looks fine, but when I write the file again using the procedure put-json-to-file, the following comes out, no matter whether or not I set the encoding for the port:

{
      "translation_data" : 
      {
        "traditional" : "\u516b",
        "simplified" : "\u516b",
        "pinyin" : "b\u0101",
        "pinyin_numbered" : "ba1",
        "english" : "eight"
      },
      "metadata" : 
      {
        "usage_examples" : [],
        "tags" : [],
        "explanations" : [],
        "mnemonics" : [],
        "similar_words" : [],
        "difficulty" : 0,
        "relevance" : 0,
        "description" : "",
        "learned" : false
      }
    }

The problem is, that now the file is no longer human readable. I cannot know from looking at the file, what words there are, because they are written in this way.

I searched the repository and could only find the following relating to UTF-8 encoding:

;; A single byte UTF-8

But I don't see a way to stop the library from doing that.

Optional nested records

Consider the following:

(define-json-type <account>
  (id)
  (username)
  (omitted "omitted" <omitted>)
  (boolean))

(define-json-type <omitted>
  (name))


(define test-json-account
  "{\"id\":\"11111\",\"username\":\"jane\",\"boolean\":false}")

(define test-account (json->account test-json-account))

Executing this returns:

In unknown file:
           0 (assoc "name" #<unspecified>)

ERROR: In procedure assoc:
In procedure assoc: Wrong type argument in position 2 (expecting association list): #<unspecified>

Could we get support for records in which a nested record may be missing from incoming JSON?

P.S. This library is really well designed and is a joy to use. Thank you for your work!

Please consider dual licensing under the MIT license

I would like to write a SRFI based on the interface of guile-json and to provide guile-json as the implementation that the SRFI process requires. In order to do so, however, it must be possible to publish the implementation under the MIT license so that it can be incorporated into any Scheme implementation or program.

If you cannot see your way to doing that, I will probably reimplement it clean-room (I have not looked at the code at all so far), but I would like to avoid doing so.

Will you go ahead and dual-license it?

installation fails

Hello

I configured guile-json 1.3.0 lie this

$ ./configure --prefix=$HOME/opt GUILD=/usr/bin/guild GUILE_CONFIG=/usr/bin/guile-config GUILE_TOOLS=/usr/bin/guile-tools

and this is make install:

$ make install
Making install in json
make[1]: Entering directory '/home/catonano/projects/guile-json/json'
make[2]: Entering directory '/home/catonano/projects/guile-json/json'
make[2]: Nothing to be done for 'install-exec-am'.
 /bin/mkdir -p '/json'
/bin/mkdir: cannot create directory ‘/json’: Permission denied
Makefile:317: recipe for target 'install-nobase_modDATA' failed
make[2]: *** [install-nobase_modDATA] Error 1
make[2]: Leaving directory '/home/catonano/projects/guile-json/json'
Makefile:460: recipe for target 'install-am' failed
make[1]: *** [install-am] Error 2
make[1]: Leaving directory '/home/catonano/projects/guile-json/json'
Makefile:446: recipe for target 'install-recursive' failed
make: *** [install-recursive] Error 1

it attempts to create a /json folder, that is a folder in the file system root

Shouldn't it try to create a folder in the target folder ?

Thanks

Fields in JSON mappings can no longer be #f

Hi!

Starting from version 4.4.1 or thereabouts (the bug was definitely not in 4.3.2), it is no longer possible for a JSON mapping to set a field to #f as this is interpreted as "unspecified field", as in this example:

(define-json-mapping <foo> make-foo foo?
  json->foo
  (x foo-x "x" (lambda (x)
                 (not (string=? x "fals")))))

(json->foo "{ \"x\": \"fals\" }")
=> #<<foo> x: #<unspecified>>

This looks like a bug to me as it should be possible to use any value for fields; WDYT?

(This was originally reported at https://issues.guix.gnu.org/45615.)

Nested json mappings generate incorrect json

Hi!

So, in json mappings you can create procedures to transform json scheme syntax to a record, and a record to a json string. For nested mappings (in my case, a <jrd-record> has a field that contains a list of <link-record>s), this is not ideal. To generate records, I can simply specify that the reader is json->link-record, but if I use link-record->json as the writer, the value is a string, instead of a json object.

Wouldn't it be better to have the reader and writer both convert from the same type of objects (namely record <-> scm)? That way nesting would work better, I think. Here again is my webfinger example, this time with the full mappings:

(define-json-mapping <link-record> make-link-record link-record?
  json->link-record <=> link-record->json
  (rel        link-record-rel) ; string
  (type       link-record-type) ; string
  (href       link-record-href) ; string
  (titles     link-record-titles) ; alist whose keys are languages or "und" and values ar strings
  (properties link-record-properties)) ; alist whose keys and values are strings

(define-json-mapping <jrd-record> make-jrd-record jrd-record?
  json->jrd-record <=> jrd-record->json
  (subject    jrd-record-subject) ; string
  (aliases    jrd-record-aliases "aliases" ; string list
              (lambda (val)
                (if val (array->list val) '()))
              (lambda (val)
                (if (null? val) *unspecified* (list->array 1 val))))
  (properties jrd-record-properties) ; alist whose keys and values are strings
  (links      jrd-record-links "links" ; list of links
              (lambda (val)
                (if val (map json->link-record (array->list val)) '()))
              (lambda (val) (list->array 1 (map link-record->json val)))))

So if I do something like

(jrd-record->json
  (make-jrd-record *unspecified* '() *unspecified*
    (list (make-link-record "http://openid.net/specs/connect/1.0/issuer" *unspecified* "https://openid.example.com" *unspecified* *unspecified*))))

I get this:

"{\"links\":[\"{\\\"rel\\\":\\\"http://openid.net/specs/connect/1.0/issuer\\\",\\\"href\\\":\\\"https://openid.example.com\\\"}\"]}"

Note how links is a list of strings that contain the json representation of my object, instead of a list of objects. Well, I can always use json-string->scm on that, but it feels unnatural. Wdyt?

optional entries in json mapping

Hi!

I'm using define-json-mapping to parse and serialize jrd records (for the webfinger protocol: https://tools.ietf.org/html/rfc7033#section-3.1). Here's my current code for the link record as an example:

(define-json-mapping <link-record> make-link-record link-record?
  json->link-record <=> link-record->json
  (rel        link-record-rel) ; string
  (type       link-record-type) ; string
  (href       link-record-href) ; string
  (titles     link-record-titles) ; alist whose keys are languages or "und" and values are strings
  (properties link-record-properties)) ; alist whose keys and values are strings

nice and easy, json->link-record works as intended: unspecified fields get #f. Now, when I try to generate that back, as in:

(link-record->json "http://openid.net/specs/connect/1.0/issuer" #f "https://openid.example.com" #f #f)

the result is not what I expect (I tried with #nil too):

{ "rel": "http://openid.net/specs/connect/1.0/issuer",
  "type": false,
  "href": "https://openid.example.com",
  "titles": false,
  "propeties": false
}

But I expected "type", "titles" and "properties" to not be defined at all. So maybe guile-json could not cons the value when it's #f or #nil (but it's hard to distinguish from the case of an empty list or the literal false), or provide a way to specify a predicate under which to add the field or not.

Wdyt?

Prefer Arrays over Objects

guile-json currently prefers objects over arrays in certain circumstances. For example, this code:

(scm->json '((1 2 3) (4 5 6)))

returns:

{"1":[2,3],"4":[5,6]}

Sometimes I would prefer it to return something like this:

[[1,2,3],[4,5,6]]

I have a branch prefer_array that I used to test what I needed. I don't pretend to be any good at this, but it might be helpful to see what I mean.

Failing test-suite

Hi, I was testing guile-json with some of the tests described here.

[,1]

This invalid json is accepted by guile-json (parsed as: '(1) ) while it should fail.

Also I get 2 segmentation fault on guile 3.0.4 on some files in the testsuite

Files causing a segfault:

[0.4e00669999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999969999999006]
[123e-10000000]

Include `configure` script.

The package doesn't include the configure script. One could run autoreconf, but may not have it installed.

json macro doesn't support unquote-splicing

The json macro supports unquote, but not unquote-splicing.

This:

(define x '(2 3))
(json (array 1 ,@x 4))

Should yield the output (1 2 3 4)

It would be very useful for json to behave like quasiquote. Thanks!

When reading JSON, Error in procedure read-control-char: In procedure integer->char: Argument 1 out of range: 55357

When trying to read the JSON at "https://www.reddit.com/r/programming/comments/.json?count=25&after=t1_fyuzqiw", this error is returned:

Backtrace:
In ice-9/boot-9.scm:
  1736:10 18 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
          17 (apply-smob/0 #<thunk 7fb403483e60>)
In ice-9/boot-9.scm:
    718:2 16 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
In ice-9/eval.scm:
    619:8 15 (_ #(#(#<directory (guile-user) 7fb40307bf00>)))
In ice-9/boot-9.scm:
   2806:4 14 (save-module-excursion _)
  4351:12 13 (_)
In /home/itsme/guile-read-json.scm:
    34:12 12 (get-all-authors #:subreddit _ #:after _)
In json/parser.scm:
   327:15 11 (json->scm _ #:null _)
   177:20 10 (json-read-object #<input: string 7fb40162f5b0> null)
   159:18  9 (read-pair #<input: string 7fb40162f5b0> null)
   177:20  8 (json-read-object #<input: string 7fb40162f5b0> null)
   159:18  7 (read-pair #<input: string 7fb40162f5b0> null)
   214:21  6 (json-read-array #<input: string 7fb40162f5b0> null)
   177:20  5 (json-read-object #<input: string 7fb40162f5b0> null)
   159:18  4 (read-pair #<input: string 7fb40162f5b0> null)
   177:20  3 (json-read-object #<input: string 7fb40162f5b0> null)
   159:18  2 (read-pair #<input: string 7fb40162f5b0> null)
   279:18  1 (json-read-string #<input: string 7fb40162f5b0>)
    249:2  0 (read-control-char _)

json/parser.scm:249:2: In procedure read-control-char:
In procedure integer->char: Argument 1 out of range: 55357

To reproduce this, run this in Guile:

(use-modules (web client) (web response) (json) (rnrs bytevectors))

(define* (get-reddit-response)
  (let ((uri "https://www.reddit.com/r/programming/comments/.json?count=25&after=t1_fyuzqiw"))
  
  (json-string->scm
   (utf8->string
    (u8-list->bytevector
     (u8vector->list
      (read-response-body (http-request uri #:streaming? #t))))))))

(get-reddit-response)

I asked about this in the #guile channel when I tried with another page of Reddit JSON comments, there was some discussion about the cause of the bug and how to fix it, but I don't know Guile well enough to fix it myself:

<pkill9> how can I deal with unsupported characters in text that gets passed to guile-json
<pkill9> i want to just replace all non-supported characters in a string basically
<pkill9> the json parser passes some text to integer->char, and then an error: json/parser.scm:249:2: In procedure read-control-char:
<pkill9> In procedure integer->char: Argument 1 out of range:
<pkill9> 55358
<pkill9> 55358 i believe is a hugging emoji, lol
<RhodiumToad> no its not
<RhodiumToad> 55358 = U+D83E, which is a high surrogate, and therefore not a valid character
<RhodiumToad> I believe json encodes surrogate pairs separately? so you might have to fetch both parts of the pair and convert them together
<RhodiumToad> U+D83E could be the first half of U+1F917, which would encode as U+D83E U+DD17
<RhodiumToad> i.e. "\uD83E\uDD17"  in json
<RhodiumToad> (U+1F917 == hugging face emoji)
<dsmith-work> So that's a bug in guile-json?
<RhodiumToad> if it's trying to decode \uXXXX independently when the value is a surrogate, yes
<dsmith-work> Ugh.
<RhodiumToad> (it's kind of ick that json uses that encoding in the first place...)
<RhodiumToad> haha. "strictly complies to http://json.org specification".
<RhodiumToad> anyway, yes, definite bug in guile-json.
<pkill9> guix's guile-json is 3 stable versions out of date, so it may have been fixed, i'll see
<RhodiumToad> I was looking at what was on savannah
<RhodiumToad> http://git.savannah.nongnu.org/cgit/guile-json.git/tree/json/parser.scm#n275
<holomorph> what happened to ice-9 json
<RhodiumToad> it is not correct that "Characters in Guile match the JSON representation" so that breaks
<RhodiumToad> I guess the fix should be to read 4 hex digits, and if the value is in DC00-DFFF throw error, if it's D800-DBFF then read in an immediately following \uXXXX and error if that isn't DC00-DFFF

`define-json-mapping` allows nonsensical `key` subforms

I recently wrote something like the following buggy code:

(define-json-mapping <project-info> make-project-info project-info?
  json->project-info
  (license project-info-license
           (lambda (x)
             (if (string? x)
                 (spdx-string->license x)
                 #f))))

and was confused to find that project-info-license always produced *unspecified*.

Eventually, I worked out that I needed instead to write:

(define-json-mapping <project-info> make-project-info project-info?
  json->project-info
  (license project-info-license
           "license" (lambda (x)
                       (if (string? x)
                           (spdx-string->license x)
                           #f))))

i.e. that writing "license" was required even though it matched license.

Originally, I had read the grammar at:

guile-json/README.md

Lines 331 to 348 in 81bc5da

- *((field getter ...) ...)* : a series of field specifications.
- *field* : the name of a JSON object field.
- *getter* : the name of the procedure to get the value of this field
given a record of this type.
- *key* : a different name for the field of this JSON object. If given, this
name will be used instead of the field name when serializing or
deserializing.
- *scm->value* : an optional procedure that will be used to convert native
values supported by guile-json to the value contained in the record. Used
when reading JSON.
- *value->scm* : an optional procedure that will be used to convert the
value contained in the record to a native value supported by guile-json
Used when writing JSON.

to mean the field and getter subforms were required, but the key, scm->value, and value->scm subforms were optional, so I thought I would only need to specify key if it were different than field. However, it seems like what I intended as a scm->value expression was being treated as a key, and of course there would never be a key equal? to that procedure, so I ended up with *unspecified* and no scm->value procedure to convert it.

I can see a few possible improvements, depending on what behavior is preferred:

  1. If you want key to be optional in the way I originally imagined:

    1. The expansion of define-json-mapping could check at run-time if the first of the optional sub-forms evaluates to a string and treat it as key if so; scm->value otherwise.

    2. If key is restricted to a string literal, rather than an expression that evaluates to a string, define-json-mapping could perform such a check at compile-time.

  2. If you want key to be required in order to provide scm->value, I think define-json-mapping should still check, either at run-time or compile-time, that key actually is/evaluates to a string, since supplying a procedure for key would never be a reasonable thing to do.

Add scm->value and value->scm specifications to define-json-type

Hi Aleix. Thank you for this solid library I use daily.

It's kind of painful when a JSON type with a nested record type needs to implement some value conversion.

Example

(use-modules (json))

(define simplified-real-world-json
"{\"initializationDate\":\"2018-06-19T08:13:04.29+02:00\",\"product\":\"ICE\",\"trainNumber\":\"369\",\"serviceId\":\"1529131014\",\"wagons\":[{\"group\":0,\"type\":\"TRIEBKOPF\",\"id\":\"938054015830\",\"wagonNumber\":null,\"status\":\"OFFEN\"}]}")

(define-json-type <manifest>
  (initialization-date "initializationDate")
  (product)
  (train-number "trainNumber")
  (service-id "serviceId")
  (wagons "wagons" #(<wagon>)))

(define-json-type <wagon>
  (group) (type) (id) (number "wagonNumber") (status))

Now let's say we'd like to operate on the train number (simpler than the date for illustration purposes), delivered to us by the grace of German engineering in string format. Reformatting according to define-json-mapping is a little tedious but overall worth the convenience/granularity tradeoff in my opinion. The painful part is that we also have to manually map any nested records, which isn't related to the problem we're working on.

(use-modules (ice-9 match))

(define wagon-converter
  (match-lambda (#((= scm->wagon wagons) ...) wagons)
                (((= wagon->scm wagons) ...) (list->vector wagons))))

(define-json-mapping <manifest> manifest manifest?
  json->manifest <=> manifest->json
  (initialization-date manifest-initialization-date "initializationDate")
  (product manifest-product)
  (train-number manifest-train-number "trainNumber" string->number number->string)
  (service-id manifest-service-id "serviceId")
  (wagons manifest-wagons "wagons" wagon-converter wagon-converter))

(call-with-input-string simplified-real-world-json json->manifest)

Am I just missing something simple? Is there any reason why define-json-type can't support the scm->value and value->scm field specifications? Have you considered using keywords over positional parameters, or would you be open to that change in the future? Thanks again for your support.

Change representation of JSON null to symbol null

In the current version of guile-json, JSON null is represented internally as #nil. Because of the magic behavior of #nil, this causes certain problems when trying to discriminate between various internal JSON representations. For example, the programmer will expect (list? j) to detect a JSON object and (null? j) to detect an empty JSON object. But these type discriminators are not correct, because they will also return #t on #nil. So one must write unidiomatic things like (or (pair? j) ((eq? j '())) instead of (list? j).

I suggest (even though it is yet another breaking change) switching to the Scheme symbol null. This is portable, printable and rereadable, and not subject to the above problems. It is also what Racket and the Chicken medea egg (not the json egg) use.

Wrong error on bad object key

If you pass anything but a string, a symbol, or a number as an object key, you get a low-level error from string->list instead of a proper json-invalid error.

Cannot read stream of json objects

I tried using this library to read the output of a program which prints a stream of json objects. However, the reader won't simply consume the first object of a stream.

The following snippet should, but doesn't work

(use-modules (json))
(with-input-from-string "{} {}" json->scm)

Make doesn't fail when tests fail

Hello, Aleix, and thanks for the library!

I think your test runner is wrong, test-end returns the whole runner rather than value returned from test-runner-on-final function and tests always succeed.

(exit (if (test-end "test-builder") 0 1))

(exit (if (test-end "test-parser") 0 1))

(exit (if (test-end "test-record") 0 1))

Guile

guile (GNU Guile) 3.0.5
Copyright (C) 2021 Free Software Foundation, Inc.

License LGPLv3+: GNU LGPL 3 or later <http://gnu.org/licenses/lgpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

How to reproduce.

  1. Add an always failing test somewhere
Diff
diff --git a/tests/test-parser.scm b/tests/test-parser.scm
index 1ff254c..d2ae962 100644
--- a/tests/test-parser.scm
+++ b/tests/test-parser.scm
@@ -111,6 +111,8 @@
 (test-error #t (json-string->scm "[1,2,3] extra"))
 (test-error #t (json-string->scm "{} extra"))
 
+(test-eq #t #f)
+
 (exit (if (test-end "test-parser") 0 1))
 
 ;;; (tests test-parser) ends here
  1. Run make check
Output
-*- mode: compilation; default-directory: "~/workspace/guile-json/" -*-
Compilation started at Fri May 21 18:23:32

make check
Making check in json
make[1]: Entering directory '/home/ivan/workspace/guile-json/json'
make[1]: Nothing to be done for 'check'.
make[1]: Leaving directory '/home/ivan/workspace/guile-json/json'
Making check in tests
make[1]: Entering directory '/home/ivan/workspace/guile-json/tests'
make  check-TESTS
make[2]: Entering directory '/home/ivan/workspace/guile-json/tests'
make[3]: Entering directory '/home/ivan/workspace/guile-json/tests'
PASS: test-builder.scm
PASS: test-parser.scm
PASS: test-record.scm
============================================================================
Testsuite summary for guile-json 4.5.2
============================================================================
# TOTAL: 3
# PASS:  3
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[3]: Leaving directory '/home/ivan/workspace/guile-json/tests'
make[2]: Leaving directory '/home/ivan/workspace/guile-json/tests'
make[1]: Leaving directory '/home/ivan/workspace/guile-json/tests'
make[1]: Entering directory '/home/ivan/workspace/guile-json'
make[1]: Nothing to be done for 'check-am'.
make[1]: Leaving directory '/home/ivan/workspace/guile-json'

Compilation finished at Fri May 21 18:23:32
test-parser.log
;;; note: source file ../json/builder.scm
;;;       newer than compiled /home/ivan/.cache/guile/ccache/3.0-LE-8-4.4/home/ivan/workspace/guile-json/json/builder.scm.go
;;; note: source file ../json/parser.scm
;;;       newer than compiled /home/ivan/.cache/guile/ccache/3.0-LE-8-4.4/home/ivan/workspace/guile-json/json/parser.scm.go
[pass] line:36, test: 
[pass] line:37, test: 
[pass] line:38, test: 
[pass] line:39, test: 
[pass] line:40, test: 
[pass] line:41, test: 
[pass] line:42, test: 
[pass] line:43, test: 
[pass] line:44, test: 
[pass] line:45, test: 
[pass] line:46, test: 
[pass] line:47, test: 
[pass] line:48, test: 
[pass] line:49, test: 
[pass] line:50, test: 
[pass] line:53, test: 
[pass] line:54, test: 
[pass] line:55, test: 
[pass] line:56, test: 
[pass] line:57, test: 
[pass] line:59, test: 
[pass] line:60, test: 
[pass] line:62, test: 
[pass] line:63, test: 
[pass] line:65, test: 
[pass] line:66, test: 
[pass] line:67, test: 
[pass] line:68, test: 
[pass] line:71, test: 
[pass] line:72, test: 
[pass] line:75, test: 
[pass] line:76, test: 
[pass] line:79, test: 
[pass] line:80, test: 
[pass] line:81, test: 
[pass] line:82, test: 
[pass] line:83, test: 
[pass] line:84, test: 
[pass] line:85, test: 
[pass] line:86, test: 
[pass] line:87, test: 
[pass] line:90, test: 
[pass] line:91, test: 
[pass] line:92, test: 
[pass] line:93, test: 
[pass] line:94, test: 
[pass] line:95, test: 
[pass] line:96, test: 
[pass] line:97, test: 
[pass] line:98, test: 
[pass] line:103, test: 
[pass] line:104, test: 
[pass] line:105, test: 
[pass] line:108, test: 
[pass] line:109, test: 
[pass] line:110, test: 
[pass] line:111, test: 
[pass] line:112, test: 
[fail] line:114, test: 
test-parser
 -> expected: #t
 -> obtained: #f
Source:tests/test-parser.scm
pass = 58, fail = 1
PASS test-parser.scm (exit status: 0)
  1. Expected result
Output
-*- mode: compilation; default-directory: "~/workspace/guile-json/" -*-
Compilation started at Fri May 21 18:42:45

make clean check
Making clean in json
make[1]: Entering directory '/home/ivan/workspace/guile-json/json'
test -z "builder.go parser.go record.go" || rm -f builder.go parser.go record.go
make[1]: Leaving directory '/home/ivan/workspace/guile-json/json'
Making clean in tests
make[1]: Entering directory '/home/ivan/workspace/guile-json/tests'
test -z "test-builder.log test-parser.log test-record.log" || rm -f test-builder.log test-parser.log test-record.log
test -z "test-builder.log test-parser.log test-record.log" || rm -f test-builder.log test-parser.log test-record.log
test -z "test-builder.trs test-parser.trs test-record.trs" || rm -f test-builder.trs test-parser.trs test-record.trs
test -z "test-suite.log" || rm -f test-suite.log
make[1]: Leaving directory '/home/ivan/workspace/guile-json/tests'
make[1]: Entering directory '/home/ivan/workspace/guile-json'
test -z "json.go" || rm -f json.go
make[1]: Leaving directory '/home/ivan/workspace/guile-json'
Making check in json
make[1]: Entering directory '/home/ivan/workspace/guile-json/json'
../env /usr/sbin/guild3 compile  -Wunbound-variable -Warity-mismatch -Wformat -o "builder.go" "builder.scm"
wrote `builder.go'
../env /usr/sbin/guild3 compile  -Wunbound-variable -Warity-mismatch -Wformat -o "parser.go" "parser.scm"
wrote `parser.go'
../env /usr/sbin/guild3 compile  -Wunbound-variable -Warity-mismatch -Wformat -o "record.go" "record.scm"
wrote `record.go'
make[1]: Leaving directory '/home/ivan/workspace/guile-json/json'
Making check in tests
make[1]: Entering directory '/home/ivan/workspace/guile-json/tests'
make  check-TESTS
make[2]: Entering directory '/home/ivan/workspace/guile-json/tests'
make[3]: Entering directory '/home/ivan/workspace/guile-json/tests'
PASS: test-builder.scm
FAIL: test-parser.scm
PASS: test-record.scm
============================================================================
Testsuite summary for guile-json 4.5.2
============================================================================
# TOTAL: 3
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to [email protected]
============================================================================
make[3]: *** [Makefile:492: test-suite.log] Error 1
make[3]: Leaving directory '/home/ivan/workspace/guile-json/tests'
make[2]: *** [Makefile:600: check-TESTS] Error 2
make[2]: Leaving directory '/home/ivan/workspace/guile-json/tests'
make[1]: *** [Makefile:659: check-am] Error 2
make[1]: Leaving directory '/home/ivan/workspace/guile-json/tests'
make: *** [Makefile:449: check-recursive] Error 1

Compilation exited abnormally with code 2 at Fri May 21 18:42:47

How can I mutate the value?

For example, here's a s-expr parsed from a json:

(("aaa" . 1) ("bbb" . 2))

Now I want to modify the value of "aaa" and send to a certain API. The problem is that the quoted list in Guile is always immutable, so I can't change the value of "aaa" with assoc-set!.

Of course, I can construct a new alist for this simple case. But my actual case is a complex and nested json structure, so reconstructing it with new values becomes hard.

Is there any idea about such a situation?

Generation of JSON lists could be better

Right now with (scm->json ... #:pretty #t), a list of strings is generated into one line, which leads to very long lines if there are many items in a list. What would you think about adding a line break after each item?

Instead of

[ "foo", "bar", "baz"]

it would look like

[
  "foo",
  "bar",
  "baz"
]

If you like the idea, I can work on a patch to make this happen.

Cannot write the empty object

Hello!

In version 3.1.0, (json-string->scm "{}") returns the empty list, but (scm->json-string '()) raises an error. So it seems we cannot emit the empty object currently, right?

Thanks,
Ludo'.

Duplicate keys in objects

Hello,

According to the references in this answer from stack overflow when there are duplicate keys in an object, later values overwrite previous values. That is
the object

{"a": 1, "b": 2, "a": 3}

is equivalent to the object

{"a": 3, "b":2}

after parsing.

I tested the following parsers

  • Firefox
  • Chromium (and node.js)
  • Python (json from standard library and ujson)

and all of them behave as described above.

Other parsers (like this for example) simply refuse to parse an object
with duplicate keys.

This is not the behavior of guile-json. The biggest issue with the current implementation, in my opinion, is that it can return inconsistent results:

(assoc "a" (json-string->scm "{\"a\": 1, \"b\": 2, \"a\": 3}"))

$6 = ("a" . 3)

but

(assoc "a" (json-string->scm (scm->json-string (json-string->scm "{\"a\": 1, \"b\": 2, \"a\": 3}"))))

$5 = ("a" . 1)

It would be nice to have the following properties:

(equal? (scm->json-string (json-string->scm x)) x)

(equal? (json-string->scm (scm->json-string y)) y)

I have a patch that behaves like most parsers I mentioned above but I wanted to discuss with you first, if you think this change is something that would fit this project.

Thank you for maintaining such a useful package.

Where is ./configure?

I noticed there is no ./configure. So I ran the following from within the project directory, which resulted in errors (Fedora 25):

$ autoconf
configure.ac:27: error: possibly undefined macro: AM_INIT_AUTOMAKE
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:28: error: possibly undefined macro: AM_SILENT_RULES

$ autoreconf
configure.ac:27: error: required file 'build-aux/install-sh' not found
configure.ac:27: 'automake --add-missing' can install 'install-sh'
configure.ac:27: error: required file 'build-aux/missing' not found
configure.ac:27: 'automake --add-missing' can install 'missing'
autoreconf: automake failed with exit status: 1

These resulted in generating the ./configure script, but that ran into further errors:

$ ./configure
configure: error: cannot find install-sh, install.sh, or shtool in build-aux "."/build-aux

Please provide clear directions on how to generate a clean 'configure' script, and any system/package level dependencies if those need to be pre-installed on Fedora/Centos hosts.

Thank you.

reader syntax for hash tables

(define-module (guile-hashtable-reader)
    #:use-module (ice-9 textual-ports)
    #:export ())

(define (read-hashtable _ p)
  (unless (eqv? #\space (read-char p))
    (error 'read-hashtable "must start with #,{,space"))
  (let loop ((h '()))
    (let ((key (read p)))
      (if (equal? (string->symbol "}") key)
	  (let ((h (list 'quasiquote h)))
	    `((@ (ice-9 hash-table) alist->hash-table) ,h))
	  (let ((val (read p)))
	    (when (eof-object? val)
	      (error 'read-hashtable "need an even number of key value pairs for a hashtable"))
	    (loop (cons (cons key (list 'unquote val)) h)))))))

(read-hash-extend #\{ read-hashtable)

;; scheme@(guile-user)> (define x 7)
;; scheme@(guile-user)> (hash-map->list cons #{ 1 (+ 3 5) x x } )
;; $1 = ((x . 7) (1 . 8))

Something like this could work maybe, It would be a separate library from guile-json but guile-json could point to it. Any thoughts?

scm->json validation error triggered by the use of a port?

Hi!

I'm trying to convert the S-exp defined in the attached file to JSON:
jami-dummy-account.scm.txt; it works when not using a port:

 scheme@(guile-user)> (scm->json %jami-account-content-sexp)
{"RINGCAKEY":"LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUpRd0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQ1Mwd2dna3BBZ0VBQW9JQ0FRQzBxWUozSkYvTzhQRGEKRnUwRnpRcHBCaDgybGJMdURrNTlVU0I0MUJSaS9kdDZGV1BRN
[...]
,"Account.allowCertFromTrusted":"true","Account.allowCertFromHistory":"true","Account.allowCertFromContact":"true","Account.allModeratorEnabled":"true","Account.alias":"dummy","Account.activeCallLimit":"-1","Account.accountPublish":"false","Account.accountDiscovery":"false"}

But somehow fails when attempting to use a port:

scheme@(guile-user)> (let ((port (open-output-file "/tmp/account.gz")))
           (scm->json port %jami-account-content-sexp))
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `json-invalid' with args `(#<output: /tmp/account.gz 16>)'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In json/builder.scm:
   227:18  2 (scm->json #<output: /tmp/account.gz 16> _ #:solidus _ #:unicode _ #:null _ #:validate _ # …)
    189:9  1 (json-valid? #<output: /tmp/account.gz 16> _)
In ice-9/boot-9.scm:
  1669:16  0 (raise-exception _ #:continuable? _)

Even when setting the validate argument to #58

scheme@(guile-user)> (let ((port (open-output-file "/tmp/account.gz")))
           (scm->json port %jami-account-content-sexp #:validate #f))
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `json-invalid' with args `(#<output: /tmp/account.gz 17>)'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In json/builder.scm:
    201:9  1 (json-build #<output: /tmp/account.gz 17> _ _ _ null #f 0)
In ice-9/boot-9.scm:
  1669:16  0 (raise-exception _ #:continuable? _)

Ideas?

Thank you :-)

the closing curly brace won't come

I'm afraid I'm running into a new bug

Or I am misunderstanding something

I built guile-json from the master branch

I copied the ping server provided as an example from the Fiber library

Then I changed the server code a bit and this is a relevant excerpt

(define (client-loop port addr store)
  (setvbuf port 'block 1024)
  ;; Disable Nagle's algorithm.  We buffer ourselves.
  (setsockopt port IPPROTO_TCP TCP_NODELAY 1)
  (let loop ()
    ;; TODO: Restrict read-line to 512 chars.
    (let ((line (read-line port)))
      (cond
       ((eof-object? line)
        (close-port port))
       (else
        ;; TODO this is where the server is gonna cook up its replies

	(put-string port "Content-Length: 121\r\n")
	(put-string port "\r\n")

	(scm->json
	  (list
	   (cons "jsonrpc" "2.0")
	   (cons "id" 1)
	   (cons "method" "textDocument/didOpen"))  port #:pretty #t )  
	
        (force-output port)
	
        (loop))))))

So i have a Guile process running this server

I open another Guile and at the REPL I dive down in a client module and then I try to connect to such server and display its reply

I type:

scheme@(lsp-client)> (getaddrinfo "localhost" "11211")
$1 = (#(0 2 1 6 #(2 2130706433 11211) #f) #(0 2 2 17 #(2 2130706433 11211) #f) #(0 2 3 0 #(2 2130706433 11211) #f))
scheme@(lsp-client)> (car $1)
$2 = #(0 2 1 6 #(2 2130706433 11211) #f)
scheme@(lsp-client)> (connect-to-server $2)
$3 = #<input-output: socket 13>
scheme@(lsp-client)> 
scheme@(lsp-client)> (put-string $3 "Hi !")
scheme@(lsp-client)> (put-char $3 #\newline)
scheme@(lsp-client)> (force-output $3)

so I sent a request to the server, now let's see what it replies back

scheme@(lsp-client)> scheme@(lsp-client)> (read-line $3)
$4 = "Content-Length: 121\r"

so far, so good

scheme@(lsp-client)> (read-line $3)
$5 = "\r"
scheme@(lsp-client)> (read-line $3)
$6 = "{"
scheme@(lsp-client)> (read-line $3)
$7 = "  \"jsonrpc\" : \"2.0\","
scheme@(lsp-client)> (read-line $3)
$8 = "  \"id\" : 1,"
scheme@(lsp-client)> (read-line $3)
$9 = "  \"method\" : \"textDocument/didOpen\""
scheme@(lsp-client)> (read-line $3)

at this point I was expecting a closing brace, like this }

but instead it hangs, it doesn't even return the prompt back

Note that if I try the same operation at the REPL

scheme@(lsp-client)>    (scm->json
          (list
           (cons "jsonrpc" "2.0")
           (cons "id" 1)
           (cons "method" "textDocument/didOpen"))  #:pretty #t )  

{
  "jsonrpc" : "2.0",
  "id" : 1,
  "method" : "textDocument/didOpen"
}scheme@(lsp-client)>

it does print the closing brace and the it returns the prompt

Maybe I'm misunderstanding the Fiber use ?

configure fails for 3.0.0

I just tried to ./configure version 3.0.0 and it fails. Here is the output I see:

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
configure: checking for guile 3.0
configure: checking for guile 2.2
configure: found guile 2.2
checking for guile-2.2... no
checking for guile2.2... no
checking for guile-2... no
checking for guile2... no
checking for guile... /usr/local/bin/guile
checking for Guile version >= 2.2... 2.2.4
checking for guild... /usr/local/bin/guild
checking for guile-config... /usr/local/bin/guile-config
checking for Guile site directory... /usr/local/share/guile/site/2.2
checking for Guile site-ccache directory using pkgconfig... /usr/local/lib/guile/2.2/site-ccache
checking for Guile extensions directory... /usr/local/lib/guile/2.2/extensions
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating json/Makefile
config.status: error: cannot find input file: `tests/Makefile.in'

I checked and there is really no tests/Makefile.in, but a tests/Makefile.am.

What --prefix should I use on Ubuntu?

I've installed guile-json by using:

$ ./configure
$ make
$ sudo make install

but when I call (use-modules (json)) I've got error ERROR: no code for module (json)

json->scm reverses order of keys in object

json->scm reverses the order of keys in an object. For example, json-scm converts {"spam": 1, "ham": 2, "eggs": 3} to (("eggs" . 3) ("ham" . 2) ("spam" . 1)). Now, I get that the order of keys is supposed to be insignificant. Nevertheless, it would be nice if the order were preserved.

My specific use case is that I am working on a GraphQL implementation for Guile, and having json->scm reverse the keys feels weird.

Thank you and happy new year!

What is the response of parsing JSON object

I've try to run this code:

(let ((alist (json-string->scm "{\"foo\": \"bar\", \"bar\": 20}")))
    (let ((pair (assoc "foo" alist)))
        (if (not (eq? pair #f))
            (display (cdr pair)))))

but it don't show anything, I've also tried to get 'foo with no success.

Converting a json file with multiple values to scm

If our json file looks like the following, only the first entry will be converted to Hashtable with (json->scm).
{ "b": 2, "a": 1, "c": [ 1.23, 1.61 ], "d": "nod" }, { "a": "gesture", "b": 0, "c": 3 }

Merge README and README.org

It's difficult to keep these in sync (at this very moment they are out of sync). Pick one, preferably the one that GitHub will display.

control characters are not escaped correctly

scheme@(guile−user)> (scm−>json (list−>string (map integer−>char (iota 60))))(newline)
$3 = "\"\x00\x01\x02\x03\x04\x05\x06\a\\b\\t\\n\v\\f\\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\\\"#$%&'()⋆+,−.\\/0123456789:;\""

As you can see, we get guile escapes in this string, which means that in the output from display we will get control characters. JSON does not allow raw control characters and requires them to be escaped.

precompiled version of the lib have wrong extension

When I first include the lib using: (use-modules (json)) I've got this messages:

;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /usr/share/guile/site/json.scm
;;; compiling /usr/share/guile/site/json/builder.scm
;;; compiled /home/kuba/.cache/guile/ccache/2.0-LE-4-2.0/usr/share/guile/site/json/builder.scm.go
;;; compiling /usr/share/guile/site/json/parser.scm
;;; compiled /home/kuba/.cache/guile/ccache/2.0-LE-4-2.0/usr/share/guile/site/json/parser.scm.go
;;; compiling /usr/share/guile/site/json/syntax.scm
;;; compiled /home/kuba/.cache/guile/ccache/2.0-LE-4-2.0/usr/share/guile/site/json/syntax.scm.go
;;; compiled /home/kuba/.cache/guile/ccache/2.0-LE-4-2.0/usr/share/guile/site/json.scm.go

Note the extension is scm.go not just go.

json-string->scm not reversible

This is with guile-json 3.1.0.

I have got the following JSON string (created using the copy function at the docker API docs: https://docs.docker.com/engine/api/v1.38/#operation/ContainerCreate):

{
  "Hostname": "",
  "Domainname": "",
  "User": "",
  "AttachStdin": false,
  "AttachStdout": true,
  "AttachStderr": true,
  "Tty": false,
  "OpenStdin": false,
  "StdinOnce": false,
  "Env": [
    "FOO=bar",
    "BAZ=quux"
  ],
  "Cmd": [
    "date"
  ],
  "Entrypoint": "",
  "Image": "ubuntu",
  "Labels": {
    "com.example.vendor": "Acme",
    "com.example.license": "GPL",
    "com.example.version": "1.0"
  },
  "Volumes": {
    "/volumes/data": {}
  },
  "WorkingDir": "",
  "NetworkDisabled": false,
  "MacAddress": "12:34:56:78:9a:bc",
  "ExposedPorts": {
    "22/tcp": {}
  },
  "StopSignal": "SIGTERM",
  "StopTimeout": 10,
  "HostConfig": {
    "Binds": [
      "/tmp:/tmp"
    ],
    "Links": [
      "redis3:redis"
    ],
    "Memory": 0,
    "MemorySwap": 0,
    "MemoryReservation": 0,
    "KernelMemory": 0,
    "NanoCPUs": 500000,
    "CpuPercent": 80,
    "CpuShares": 512,
    "CpuPeriod": 100000,
    "CpuRealtimePeriod": 1000000,
    "CpuRealtimeRuntime": 10000,
    "CpuQuota": 50000,
    "CpusetCpus": "0,1",
    "CpusetMems": "0,1",
    "MaximumIOps": 0,
    "MaximumIOBps": 0,
    "BlkioWeight": 300,
    "BlkioWeightDevice": [
      {}
    ],
    "BlkioDeviceReadBps": [
      {}
    ],
    "BlkioDeviceReadIOps": [
      {}
    ],
    "BlkioDeviceWriteBps": [
      {}
    ],
    "BlkioDeviceWriteIOps": [
      {}
    ],
    "MemorySwappiness": 60,
    "OomKillDisable": false,
    "OomScoreAdj": 500,
    "PidMode": "",
    "PidsLimit": -1,
    "PortBindings": {
      "22/tcp": [
        {
          "HostPort": "11022"
        }
      ]
    },
    "PublishAllPorts": false,
    "Privileged": false,
    "ReadonlyRootfs": false,
    "Dns": [
      "8.8.8.8"
    ],
    "DnsOptions": [
      ""
    ],
    "DnsSearch": [
      ""
    ],
    "VolumesFrom": [
      "parent",
      "other:ro"
    ],
    "CapAdd": [
      "NET_ADMIN"
    ],
    "CapDrop": [
      "MKNOD"
    ],
    "GroupAdd": [
      "newgroup"
    ],
    "RestartPolicy": {
      "Name": "",
      "MaximumRetryCount": 0
    },
    "AutoRemove": true,
    "NetworkMode": "bridge",
    "Devices": [],
    "Ulimits": [
      {}
    ],
    "LogConfig": {
      "Type": "json-file",
      "Config": {}
    },
    "SecurityOpt": [],
    "StorageOpt": {},
    "CgroupParent": "",
    "VolumeDriver": "",
    "ShmSize": 67108864
  },
  "NetworkingConfig": {
    "EndpointsConfig": {
      "isolated_nw": {
        "IPAMConfig": {
          "IPv4Address": "172.20.30.33",
          "IPv6Address": "2001:db8:abcd::3033",
          "LinkLocalIPs": [
            "169.254.34.68",
            "fe80::3468"
          ]
        },
        "Links": [
          "container_1",
          "container_2"
        ],
        "Aliases": [
          "server_x",
          "server_y"
        ]
      }
    }
  }
}

Which I transform to a scm value:

(define data (json-string->scm "{
  \"Hostname\": \"\",
  \"Domainname\": \"\",
  \"User\": \"\",
  \"AttachStdin\": false,
  \"AttachStdout\": true,
  \"AttachStderr\": true,
  \"Tty\": false,
  \"OpenStdin\": false,
  \"StdinOnce\": false,
  \"Env\": [
    \"FOO=bar\",
    \"BAZ=quux\"
  ],
  \"Cmd\": [
    \"date\"
  ],
  \"Entrypoint\": \"\",
  \"Image\": \"ubuntu\",
  \"Labels\": {
    \"com.example.vendor\": \"Acme\",
    \"com.example.license\": \"GPL\",
    \"com.example.version\": \"1.0\"
  },
  \"Volumes\": {
    \"/volumes/data\": {}
  },
  \"WorkingDir\": \"\",
  \"NetworkDisabled\": false,
  \"MacAddress\": \"12:34:56:78:9a:bc\",
  \"ExposedPorts\": {
    \"22/tcp\": {}
  },
  \"StopSignal\": \"SIGTERM\",
  \"StopTimeout\": 10,
  \"HostConfig\": {
    \"Binds\": [
      \"/tmp:/tmp\"
    ],
    \"Links\": [
      \"redis3:redis\"
    ],
    \"Memory\": 0,
    \"MemorySwap\": 0,
    \"MemoryReservation\": 0,
    \"KernelMemory\": 0,
    \"NanoCPUs\": 500000,
    \"CpuPercent\": 80,
    \"CpuShares\": 512,
    \"CpuPeriod\": 100000,
    \"CpuRealtimePeriod\": 1000000,
    \"CpuRealtimeRuntime\": 10000,
    \"CpuQuota\": 50000,
    \"CpusetCpus\": \"0,1\",
    \"CpusetMems\": \"0,1\",
    \"MaximumIOps\": 0,
    \"MaximumIOBps\": 0,
    \"BlkioWeight\": 300,
    \"BlkioWeightDevice\": [
      {}
    ],
    \"BlkioDeviceReadBps\": [
      {}
    ],
    \"BlkioDeviceReadIOps\": [
      {}
    ],
    \"BlkioDeviceWriteBps\": [
      {}
    ],
    \"BlkioDeviceWriteIOps\": [
      {}
    ],
    \"MemorySwappiness\": 60,
    \"OomKillDisable\": false,
    \"OomScoreAdj\": 500,
    \"PidMode\": \"\",
    \"PidsLimit\": -1,
    \"PortBindings\": {
      \"22/tcp\": [
        {
          \"HostPort\": \"11022\"
        }
      ]
    },
    \"PublishAllPorts\": false,
    \"Privileged\": false,
    \"ReadonlyRootfs\": false,
    \"Dns\": [
      \"8.8.8.8\"
    ],
    \"DnsOptions\": [
      \"\"
    ],
    \"DnsSearch\": [
      \"\"
    ],
    \"VolumesFrom\": [
      \"parent\",
      \"other:ro\"
    ],
    \"CapAdd\": [
      \"NET_ADMIN\"
    ],
    \"CapDrop\": [
      \"MKNOD\"
    ],
    \"GroupAdd\": [
      \"newgroup\"
    ],
    \"RestartPolicy\": {
      \"Name\": \"\",
      \"MaximumRetryCount\": 0
    },
    \"AutoRemove\": true,
    \"NetworkMode\": \"bridge\",
    \"Devices\": [],
    \"Ulimits\": [
      {}
    ],
    \"LogConfig\": {
      \"Type\": \"json-file\",
      \"Config\": {}
    },
    \"SecurityOpt\": [],
    \"StorageOpt\": {},
    \"CgroupParent\": \"\",
    \"VolumeDriver\": \"\",
    \"ShmSize\": 67108864
  },
  \"NetworkingConfig\": {
    \"EndpointsConfig\": {
      \"isolated_nw\": {
        \"IPAMConfig\": {
          \"IPv4Address\": \"172.20.30.33\",
          \"IPv6Address\": \"2001:db8:abcd::3033\",
          \"LinkLocalIPs\": [
            \"169.254.34.68\",
            \"fe80::3468\"
          ]
        },
        \"Links\": [
          \"container_1\",
          \"container_2\"
        ],
        \"Aliases\": [
          \"server_x\",
          \"server_y\"
        ]
      }
    }
  }
}"))

This value will be output as follows in the REPL:

$24 = (("NetworkingConfig" ("EndpointsConfig" ("isolated_nw" ("Aliases" . #("server_x" "server_y")) ("Links" . #("container_1" "container_2")) ("IPAMConfig" ("LinkLocalIPs" . #("169.254.34.68" "fe80::3468")) ("IPv6Address" . "2001:db8:abcd::3033") ("IPv4Address" . "172.20.30.33"))))) ("HostConfig" ("ShmSize" . 67108864) ("VolumeDriver" . "") ("CgroupParent" . "") ("StorageOpt") ("SecurityOpt" . #()) ("LogConfig" ("Config") ("Type" . "json-file")) ("Ulimits" . #(())) ("Devices" . #()) ("NetworkMode" . "bridge") ("AutoRemove" . #t) ("RestartPolicy" ("MaximumRetryCount" . 0) ("Name" . "")) ("GroupAdd" . #("newgroup")) ("CapDrop" . #("MKNOD")) ("CapAdd" . #("NET_ADMIN")) ("VolumesFrom" . #("parent" "other:ro")) ("DnsSearch" . #("")) ("DnsOptions" . #("")) ("Dns" . #("8.8.8.8")) ("ReadonlyRootfs" . #f) ("Privileged" . #f) ("PublishAllPorts" . #f) ("PortBindings" ("22/tcp" . #((("HostPort" . "11022"))))) ("PidsLimit" . -1) ("PidMode" . "") ("OomScoreAdj" . 500) ("OomKillDisable" . #f) ("MemorySwappiness" . 60) ("BlkioDeviceWriteIOps" . #(())) ("BlkioDeviceWriteBps" . #(())) ("BlkioDeviceReadIOps" . #(())) ("BlkioDeviceReadBps" . #(())) ("BlkioWeightDevice" . #(())) ("BlkioWeight" . 300) ("MaximumIOBps" . 0) ("MaximumIOps" . 0) ("CpusetMems" . "0,1") ("CpusetCpus" . "0,1") ("CpuQuota" . 50000) ("CpuRealtimeRuntime" . 10000) ("CpuRealtimePeriod" . 1000000) ("CpuPeriod" . 100000) ("CpuShares" . 512) ("CpuPercent" . 80) ("NanoCPUs" . 500000) ("KernelMemory" . 0) ("MemoryReservation" . 0) ("MemorySwap" . 0) ("Memory" . 0) ("Links" . #("redis3:redis")) ("Binds" . #("/tmp:/tmp"))) ("StopTimeout" . 10) ("StopSignal" . "SIGTERM") ("ExposedPorts" ("22/tcp")) ("MacAddress" . "12:34:56:78:9a:bc") ("NetworkDisabled" . #f) ("WorkingDir" . "") ("Volumes" ("/volumes/data")) ("Labels" ("com.example.version" . "1.0") ("com.example.license" . "GPL") ("com.example.vendor" . "Acme")) ("Image" . "ubuntu") ("Entrypoint" . "") ("Cmd" . #("date")) ("Env" . #("FOO=bar" "BAZ=quux")) ("StdinOnce" . #f) ("OpenStdin" . #f) ("Tty" . #f) ("AttachStderr" . #t) ("AttachStdout" . #t) ("AttachStdin" . #f) ("User" . "") ("Domainname" . "") ("Hostname" . ""))

However, when I try to reverse this, I get an error:

(scm->json data)

Error:

{"NetworkingConfig":{"EndpointsConfig":{"isolated_nw":{"Aliases":["server_x","server_y"],"Links":["container_1","container_2"],"IPAMConfig":{"LinkLocalIPs":["169.254.34.68","fe80::3468"],"IPv6Address":"2001:db8:abcd::3033","IPv4Address":"172.20.30.33"}}}},"HostConfig":{"ShmSize":67108864,"VolumeDriver":"","CgroupParent":"","StorageOpt":srfi/srfi-1.scm:640:9: In procedure for-each:
Throw to key `json-invalid' with args `()'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.

I think it has to do with how guile-json handles empty objects.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.