Code Monkey home page Code Monkey logo

vlang-yaml's Introduction

Native YAML support for V-lang

I think this YAML parser covers many important aspects of the YAML spec, but certainly not everything. I've been using the YAML spec and this online tool whenever I was in doubt.

This is an initial version. Please apply some caution and report issues back to me.

Since this is an initial version, it is not yet performance tested. I'm sure there is room for improvements.

Key Features

  • Read and parse yaml files (or data blocks); utf-8 only
  • Tokenize the YAML file and derive YAML events (tokens)
  • Convert YAML events to a JSON string (and leverage V's built-in JSON decoder to load json)
  • Load the YAML file into a tree-like structure of dynamic YamlValue's (V is currently lacking the feature to map (YAML) values to struct variables. The respective JSON logic is build into the compiler. It is not possible for users to define their own [attributes] right now)
  • Value types, such as string, int, float and bool are auto-detected and carried forward
  • Tag definitions and references are mostly supported. By default references get replaced with their definition.
  • Multiple YAML documents within a file is supported (mostly)
  • un-escape and interpolate strings, hex numbers, etc.

Architecture

TextScanner: split text into string tokens (generic)
    -> Scanner: map into basic yaml-like tokens
        -> Tokenizer: derive proper yaml tokens (events)
            -> JSON output: convert into JSON string
                -> print
                -> json.decode() V built-in: load into V struct's
            -> Reader: load into dynamic YamlValue structure
                -> Accessor: path-like access to tree-like YamlValue structure

TextScanner: A generic module, not YAML specific, with re-useable functions to detect newline, handle line count, skip whitespace, move to end-of-line, set a marker at a certain position and retrieve the text between the marker and the current postion, read quoted strings, etc.

Scanner: Leverage TextScanner to parse the YAML file into basic YAML tokens, properly determine indentation (very important in YAML), the different multi-line text variants, identify special chars such as -:{}[],!---..., etc. The token generated consist of a type, the indentiation level and the associated string.

Tokenizer: What we really want are YAML events such as start-list, close, start-object, key, value, end-of-document, tag-definition, tag-reference, etc.. YAML files very human readable, but not very computer friendly. The Tokenizer creates a nice stream of YAML events that can easily be leveraged for different purposes, such as generate JSON, or create dynamic YamlValue's.

JSON output: Convert the stream of YAML events into a JSON string. This json string can be used by V's build-in decoder to load the YAML data into V struct's.

YAML Reader: User defined [attributes] are not yet supported. Hence the YAML data can only be loaded into V struct's via JSON's built-in JSON decoder. Reader creates a completely dynamic tree-like structure, based on YamlValue's, reflecting lists, objects, key and values, including value types (string, int, float, bool). This is a little bit like in dynamic languages, such as Python.

Accessor: Traversing the tree of YamlValue's that make up a yaml file, is not especially pleasant. Accessor provides getter functions, so that by means of a 'path' the value can be accessed. Additionally type converters are provided, to return i8, i16, int, i64, f32, f64, bool etc. values.

API

	import yaml
	
	content := os.read_file("myfile.yaml")?
	scanner := yaml.yaml_scanner(content, debug)?
	for i, tok in scanner.tokens { .. }

	tokenizer := yaml.yaml_tokenizer(content, replace_tags: true, debug: debug)?
	for i, tok in tokenizer.tokens { .. }

	json_data := yaml.yaml_to_json(content, replace_tags: true, debug: debug)?
	println(json_data)

	docs := yaml.yaml_reader(content)?
	x := docs.get(0)	// Most files have only 1 (implict) document
	assert x.get("american", 0)?.string()? == "Boston Red Sox"

	// with additional options
	docs := yaml.yaml_reader(content, replace_tags: yaml.ReplaceTagsEnum.in_reader, debug: debug)?

	// Path-like getter for YAML values
	assert x1.get(0, "center", "x")?.int()? == 73

Examples

There is a reasonable amount of test cases for each major component. Probably a good starting point for anybody who want to dig a bit deeper.

Yaml-to-json command line tool

In the ./examples folder is a little command line utility that reads a yaml file and prints json to the console

vlang-yaml's People

Contributors

abom avatar jdonnerstag avatar juergendo avatar ken0x0a avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

vlang-yaml's Issues

YamlValues are not currently indexable

We currently cannot iterate through block collections of the .YAML file. This functionality is extremely useful.

....
for key in doc {
    foo(doc)
}
error: for in: cannot index 'yaml.YamlValue'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.