crystal-community / toml.cr Goto Github PK

View Code? Open in Web Editor NEW

58.0 58.0 16.0 65 KB

TOML parser for Crystal

Crystal 100.00%

toml.cr's People

Contributors

Stargazers

Watchers

Forkers

vjdhama timjs hackervera jamestaylr lukc foliea chipper1 maiha andraantariksa xmonader bcardiff manveru summer-alice rocx nobodywasishere willhbr

toml.cr's Issues

Issue new release?

Hi! The latest release is from 2020, and I just found myself in a rabbit hole that made me realize I needed 7852db9.

I recommend issuing a new release so that others can easily get these fixes.

Spec not passed on long integer

When I run crystal spec, it will give me a 2 test error, which caused by StackOverFlow. Actually, the JSON implementation also have a StackOverFlow when it try to parse -9223372036854775808. So, I think the test it's pretty trivial and I change it into -9223372036854775807 which works fine

require "json"

puts Int64.from_json("-9223372036854775808")

https://play.crystal-lang.org/#/r/80z2

Can I make a pull request to change the test?

Be able to generate TOML as well as parse it.

What would be required to write TOML to a file, as well as just reading it?

Can't parse a bit complex toml file

I've a problem parsing a bit complex toml file (parsing it in python works fine).
The following file contains the config.toml example and the crystal test file.
I tried https://github.com/crystal-community/toml.cr and https://github.com/Stygmates/crystal-toml

Unhandled exception: expected hardware_types to be an Array, not {"amd" => {"name" => "amd"}} at 20:17 (TOML::ParseException)
  from lib/toml/src/toml/parser.cr:284:7 in 'raise'
  from lib/toml/src/toml/parser.cr:153:11 in 'parse_array_table_header'
  from lib/toml/src/toml/parser.cr:95:14 in 'parse_table_header'
  from lib/toml/src/toml/parser.cr:26:9 in 'parse'
  from lib/toml/src/toml/parser.cr:4:5 in 'parse'
  from lib/toml/src/toml.cr:11:5 in 'parse'
  from test.cr:3:1 in '__crystal_main'
  from ../../../../../../usr/share/crystal/src/crystal/main.cr:105:5 in 'main_user_code'
  from ../../../../../../usr/share/crystal/src/crystal/main.cr:91:7 in 'main'
  from ../../../../../../usr/share/crystal/src/crystal/main.cr:114:3 in 'main'
  from __libc_start_main
  from _start
  from ???

example.zip

[inputSimulatorConfig]
hws = ["amd_starter","amd_big"]
growth_values = [50000, 200000, 1000000]
token_prices = [0.15, 0.3, 3.0]
cloudunits_price_ranges = [3]
export_path = ""

[interpolation]
cpr_improve = "0:0,60:40"
cpr_sales_price_decline = "0:0,60:40"
utilization = "20:80,40:90"
token_price = ""
cost_rack_unit = ""
cost_power_kwh = ""

[hardware_types]
[hardware_types.amd]
name = "amd"

[[hardware_types.amd.components]]
name = "margin1"
description = "Margin per node for threefold and its partners"
cost = 400.0
power = 0
rackspace_u = 0.0
cru = 0.0
sru = 0.0
hru = 0.0
mru = 0.0
su_perc = 0.0
cu_perc = 0.0
link = ""
passmark = 0
[[hardware_types.amd.components]]
name = "margin2"
description = "margin per node for threefold and its partners"
cost = 900.0
power = 0
rackspace_u = 0.0
cru = 0.0
sru = 0.0
hru = 0.0
mru = 0.0
su_perc = 0.0
cu_perc = 0.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "chassis"
description = "Chassis for Ryzen CPU (no cpu inside), 4 HD slots, 1 SSD slots, 10 gbit nic"
cost = 300.0
power = 30
rackspace_u = 2.0
cru = 0.0
sru = 0.0
hru = 0.0
mru = 0.0
su_perc = 0.0
cu_perc = 0.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "hd8"
description = "Seagate Baracuda 8TB (6-8 watt)"
cost = 200.0
power = 10
rackspace_u = 0.0
cru = 0.0
sru = 0.0
hru = 8000.0
mru = 0.0
su_perc = 100.0
cu_perc = 0.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "hd12"
description = "Seagate Baracuda 12TB (6-8 watt)"
cost = 330.0
power = 10
rackspace_u = 0.0
cru = 0.0
sru = 0.0
hru = 12000.0
mru = 0.0
su_perc = 100.0
cu_perc = 0.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "amd1"
description = "AMD Ryzen 3 3600X (12 logical cores)"
cost = 240.0
power = 80
rackspace_u = 0.0
cru = 12.0
sru = 0.0
hru = 0.0
mru = 0.0
su_perc = 0.0
cu_perc = 100.0
link = ""
passmark = 18000

[[hardware_types.amd.components]]
name = "amd2"
description = "AMD Ryzen 7 3800X (16 logical cores)"
cost = 377.0
power = 105
rackspace_u = 0.0
cru = 16.0
sru = 0.0
hru = 0.0
mru = 0.0
su_perc = 0.0
cu_perc = 100.0
link = ""
passmark = 24500

[[hardware_types.amd.components]]
name = "ssd05"
description = "0.5 TB SSD"
cost = 120.0
power = 5
rackspace_u = 0.0
cru = 0.0
sru = 500.0
hru = 0.0
mru = 0.0
su_perc = 100.0
cu_perc = 0.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "ssd1"
description = "1 TB SSD"
cost = 240.0
power = 5
rackspace_u = 0.0
cru = 0.0
sru = 1000.0
hru = 0.0
mru = 0.0
su_perc = 100.0
cu_perc = 0.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "mem16_ddr4"
description = "mem 16GB"
cost = 140.0
power = 8
rackspace_u = 0.0
cru = 0.0
sru = 0.0
hru = 0.0
mru = 16.0
su_perc = 0.0
cu_perc = 100.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "mem32_ddr4"
description = "mem 32GB"
cost = 220.0
power = 10
rackspace_u = 0.0
cru = 0.0
sru = 0.0
hru = 0.0
mru = 32.0
su_perc = 0.0
cu_perc = 100.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "mem64_ddr4"
description = "mem 64GB"
cost = 400.0
power = 16
rackspace_u = 0.0
cru = 0.0
sru = 0.0
hru = 0.0
mru = 64.0
su_perc = 0.0
cu_perc = 100.0
link = ""
passmark = 0

[[hardware_types.amd.components]]
name = "ng2"
description = "48 ports 10 gbit + 4 ports 10 gbit sfp: fs.com + cables"
cost = 4500.0
power = 100
rackspace_u = 1.0
cru = 0.0
sru = 0.0
hru = 0.0
mru = 0.0
su_perc = 0.0
cu_perc = 0.0
link = ""
passmark = 0

[[hardware_types.amd.devices]]
name = "amd_starter"
[[hardware_types.amd.devices.components]]
name = "chassis"
nr = 1

[[hardware_types.amd.devices.components]]
name = "amd1"
nr = 1

[[hardware_types.amd.devices.components]]
name = "hd8"
nr = 1

[[hardware_types.amd.devices.components]]
name = "mem32_ddr4"
nr = 1

[[hardware_types.amd.devices.components]]
name = "ssd05"
nr = 1

[[hardware_types.amd.devices.components]]
name = "margin1"
nr = 1


[[hardware_types.amd.devices]]
name = "amd_big"
[[hardware_types.amd.devices.components]]
name = "chassis"
nr = 1

[[hardware_types.amd.devices.components]]
name = "amd2"
nr = 1

[[hardware_types.amd.devices.components]]
name = "hd12"
nr = 4

[[hardware_types.amd.devices.components]]
name = "mem64_ddr4"
nr = 2

[[hardware_types.amd.devices.components]]
name = "ssd1"
nr = 1

[[hardware_types.amd.devices.components]]
name = "margin2"
nr = 1


[[hardware_types.amd.devices]]
name = "switch"
[[hardware_types.amd.devices.components]]
name = "ng2"
nr = 1



[hardware]

[hardware.amd_big]
name = "amd_big"
hardware_type = "amd"
[[hardware.amd_big.nodes]]
name = "compute"
template = "amd_big"
nr = 20
[[hardware.amd_big.overhead]]
name = "switch"
template = "swtich"
nr = 2


[hardware.amd_starter]
name = "amd_starter"
hardware_type = "amd"
[[hardware.amd_starter.nodes]]
name = "compute"
template = "amd_starter"
nr = 1

crystal code

require "toml"

h = TOML.parse(File.read("config.toml")).as(Hash)
puts h.inspect

Question regarding mistake in the grammar

There are some flaw on the library that are not following the TOML grammar rule. I have add some test from the TOML example and it's failed

# Bare keys may only contain ASCII letters, ASCII digits, underscores, and dashes (A-Za-z0-9_-). Note that bare keys are allowed to be composed of only ASCII digits, e.g. 1234, but are always interpreted as strings.
it_parses "123abc_- = 123", { "123abc_-" => 123 }
it_parses "-_123abc = 123", { "-_123abc" => 123 }

# Since v0.5.0
# Dotted keys are a sequence of bare or quoted keys joined with a dot. This allows for grouping similar properties together:
it_parses "physical.color = \"orange\"", { "physical" => { "color" => "orange" } }
it_parses "site.\"google.com\" = true", { "site" => { "google.com" => true } }

it_parses %(
    [dog."tater.man"]
    type.name = "pug"
    ),
    {"dog" => {"tater" => {"man" => {"type" => {"name" => "pug"}}}} }

It appears that the lexer read 123abc_- as 2 token, 123 as a number and abc_
as a key. And for the -_123abc, because the key started with character -, the lexer then expect the token as a negative number. I'd like to make a pull request to fix this issue.

Can you give me some pointers to fix this? I'm thinking to recreate some parts of the lexer and parser.

each do give weird behavior

This is my TOML file

[sites]


[sites.prod]

  server_ext_port = '8888'
  server_ext_ip = '10.0.0.1'
  server_int_port = '8888'
  server_int_ip = '192.168.143.128'

[sites.dev]

  server_ext_port = '8888'
  server_ext_ip = '192.168.1.103'
  server_int_port = '8888'
  server_int_ip = '192.168.143.128'

I'm doing something like this:

#conf is the file loaded via toml.load_file
conf[sites].each do |site|
  puts site
end

In ruby, using Ruby TOML, this code will return a [String, Hash] per loop, in Crystal-TOML, I get back a string with the sites name String

I believe this is a bug.

TOML v1.0.0

TOML is out of beta

Any chance to support v1.0.0?

Documentation error

Documentation @ http://docrystal.org/github.com/manastech/crystal-toml/master/index.html fails to generate with the following error:

Incompatible with Crystal 1.0.0

shards.yml miss the crystal: >=1.0.0

Also it compiles with deprecation warnings on Crystal 0.36.1 that become errors in Crystal 1.0.0.

On Crystal 0.36.1

In spec/parser_spec.cr:7:12

 7 | actual.should eq(expected), file, line
            ^-----
Warning: Deprecated Hash(String, TOML::Type)#should. Use named arguments `.should(expectation, file: file, line: line)`

On Crystal 1.0.0

In spec/parser_spec.cr:7:12

 7 | actual.should eq(expected), file, line
            ^-----
Error: wrong number of arguments for 'Hash(String, TOML::Type)#should' (given 3, expected 1..2)

Overloads are:
 - Spec::ObjectExtensions#should(expectation : BeAExpectation(T), failure_message : String | ::Nil = nil, *, file = __FILE__, line = __LINE__)
 - Spec::ObjectExtensions#should(expectation, failure_message : String | ::Nil = nil, *, file = __FILE__, line = __LINE__)

Recursive configuration don't work

for example: (https://github.com/toml-lang/toml)

[servers]

Indentation (tabs and/or spaces) is allowed but not required

[servers.alpha]
ip = "10.0.0.1"
dc = "eqdc10"

[servers.beta]
ip = "10.0.0.2"
dc = "eqdc10"

should create:

 {"servers" => {alpha = > {ip => "10.0.0.1", dc => "eqdc10"}, beta => {ip = "10.0.0.2", dc = "eqdc10"}}}

but for me this just creates an empty hash:

{}

Using TOML is hard becuase of how the compiler is working

For example

config = TOML.parse_file("./daemon.conf")
threads = [] of Thread
if config.is_a?(Hash)
  pid_path = config["main_process"]["pid_path"]
  daemon_port = config["main_process"]["port"].to_i
  log_path = config["main_process"]["log_path"]
else
  exit 1
end

Now, apparently for the compiler it's not enough to check if config is a Hash, but it also suspects
config["main_process"] of being a Int64, or in other word, not a Hash.

crystal build src/rsaccess-daemon.cr 
Error in ./src/rsaccess-daemon.cr:47: instantiating 'Rsaccess::Daemon::Execute:Class#run()'

Rsaccess::Daemon::Execute.run
                          ^~~

in ./src/rsaccess-daemon.cr:16: undefined method '[]' for Int64

        pid_path = config["main_process"]["pid_path"]
                                         ^

Support for TOML-to-class mappings

Thanks for the work with the TOML shard thus far; it's very much appreciated.

It would be nice to have support for mapping TOML to Crystal classes like YAML#mapping and JSON#mapping offer, as this would help alleviate the need for type casting around the code where the TOML parameters are accessed.

Issue compiling using Crystal 0.8.0

Getting this error:

in ./src/rsaccess-daemon.cr:142: instantiating 'TOML:Module#parse_file(String)'

        config = TOML.parse_file("./daemon.conf")
                      ^~~~~~~~~~

in ./libs/toml/toml.cr:15: instantiating 'parse(String)'

    parse File.read(filename)
    ^~~~~

in ./libs/toml/toml.cr:10: instantiating 'TOML::Parser:Class#parse(String)'

    Parser.parse(string)
           ^~~~~

in ./libs/toml/toml/parser.cr:4: instantiating 'new(String)'

    new(string).parse
    ^~~

instantiating 'TOML::Parser#initialize(String)'

in ./libs/toml/toml/parser.cr:8: instantiating 'TOML::Lexer:Class#new(String)'

    @lexer = Lexer.new(string)
                   ^~~

instantiating 'TOML::Lexer#initialize(String)'

in ./libs/toml/toml/lexer.cr:6: undefined constant CharReader

    @reader = CharReader.new(string)
              ^~~~~~~~~~

Release needed

Last released version doesn't work with Crystal 0.35.1, but the master branch do. So to avoid having people (like me) using a commit hash in their shard.yml files, a release would be appreciated 😃