Hey, I just wanted to say thanks for the library. I needed a simple YAML parser that I could strip down for a custom application, and poyo fits the bill. So far I've only stripped out even more features, but if I end up expanding the test suite or anything like that I'll be sure to submit a pull request. Thanks again!

Cookiecutter is now using pyyaml instead of poyo. See cookiecutter/cookiecutter#1489

Unlike pyyaml, poyo mistakes blank: as {'blank': {}} instead of {'blank': None}.

Hello @hackebrot,

I'm trying out poyo as a replacement for pyyaml in one of my projects. The project's config file looks more or less like this:

bind: 0.0.0.0:8081      # address and port to bind on
allow-transfers: true   # allow log file downloads
relative-root: /        # web app root path (default: '')
commands: [tail, grep]  # allowed commands
wrap-lines: true        # initial line-wrapping state

files:
  - 'logs/apache/www.tailon.org/'

This currently breaks poyo in two places:

1. Poyo tries to parse 0.0.0.0:8081 as a float. Here's the debug output I added to read_from_tag():

pattern: '(?P<quotes>[\'\\"]?).*(?P=quotes)'
group:  'bind'
string: 'bind'

pattern: '([-+]?(\\.[0-9]+|[0-9]+(\\.[0-9]*)?)([eE][-+]?[0-9]+)?)'
group:  '0.0'
string: '0.0.0.0:8081'

Perhaps the regex can be fixed or read_from_tag() can continue trying patterns until the one that matches most 'fully' is found?

2. The relative-root: / # web app root path (default: '') line breaks the parser, because of the : in the comment:

pattern: '(?P<quotes>[\'\\"]?).*(?P=quotes)'
group:  'relative-root: "/"    # web app root path (default'
string: 'relative-root: "/"    # web app root path (default'

pattern: '(?P<quotes>[\'\\"]?).*(?P=quotes)'
group:  "''"
string: "''"

Kind regards,
Georgi

Since flake8 is in the tests, it would be helpful to include the required settings in the repo. I have different default settings than the defaults in my home directory, so when putting together the changes for #5, flake8 passed for me locally but failed on Travis.

https://pypi.org/project/black/

Consider the following YAML, which I'll call escape.yaml:

escaped_string: "line\twith\ttabs\nline without tabs"

This is allowed.
https://yaml.org/spec/history/2001-08-01.html#:~:text=In%20YAML%2C%20text%20scalars%20can,%5C%5C%20to%20represent%20the%20backslash.

In YAML, text scalars can be surrounded by quotes enabling escape sequences such as \n to represent a new line, \t to represent a tab, and \ to represent the backslash.

https://yaml.org/spec/1.1/current.html#escaping%20in%20double-quoted%20style/

(see also e.g. https://yaml-multiline.info/ for some cases when escape sequences are ignored)

poyo appears to ignore escape sequences in quoted strings. Instead, it reads each backslash as a literal backslash.

>>> poyo.parse_string(open('escape.yaml').read())
{'escaped_string': 'line\\twith\\ttabs\\nline without tabs'}

I believe yaml.safe_load shows the correct behavior:

>>> yaml.safe_load(open('escape.yaml'))
{'escaped_string': 'line\twith\ttabs\nline without tabs'}
>>> print(yaml.safe_load(open('escape.yaml'))['escaped_string'])
line	with	tabs
line without tabs

See https://github.com/hackebrot/labels/blob/main/.github/workflows/run-checks.yml

Eg:

Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ruamel import yaml
>>> from io import StringIO
>>> fh = StringIO()
>>> yaml.safe_dump({'foobar': 'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer lobortis ligula sapien, eget luctus odio porta ut.'}, fh, indent=4, default_flow_style=False, allow_unicode=True)
>>> fh.getvalue()
'foobar: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer lobortis\n    ligula sapien, eget luctus odio porta ut.\n'
>>> import poyo

>>> poyo.parse_string(fh.getvalue())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/python3.5/site-packages/poyo/parser.py", line 241, in parse_string
    return parser()
  File "/lib/python3.5/site-packages/poyo/parser.py", line 234, in __call__
    match = self.find_match()
  File "/lib/python3.5/site-packages/poyo/parser.py", line 224, in find_match
    ''.format(self.source[self.pos:])
poyo.exceptions.NoMatchException: None of the known patterns match for     ligula sapien, eget luctus odio porta ut.

Version: 0.4.2
OS: OSX (but irrelevant)

I came across this issue when attempting to render a cookiecutter template from a user config
where there was a space between a key and the colon.

While this is very trivial, it is hard to root cause because the error message is something like
Error message: 'dict object' has no attribute 'python_3_6' and only careful examination of the dumped structure shows the problem

"interpreter_versions": {
            "python_2_7": "no",
            "python_3_5": "no",
            "python_3_6 ": "yes",
            "python_3_7": "yes"
        },

How to reproduce

If you take this (valid) YAML example and parse it through poyo

python_3_6 : yes

you get (note the space after the python_3_6 key

>>> from poyo import parse_string
>>> config = parse_string("python_3_6 : yes")
>>> config
{'python_3_6 ': 'yes'}

whereas it should have been {'python_3_6': 'yes'}

Using this simple YAML file:

---
default_context:
  project_name: whazz
  version: 0.0.1
  year: 2016

I get this error

 cookiecutter.exceptions.InvalidConfiguration: Unable to parse YAML file /home/jenkins/workspace/gate-puppet-openstack-cookiecutter-puppet-3-cookiecutter-check/default-config.yaml. Error: None of the known patterns match for ---

According to the specification, a YAML file should start with "---" on
a line of its own: http://www.yaml.org/spec/1.2/spec.html#id2760395

If you change the test file like this :

diff --git a/tests/foobar.yml b/tests/foobar.yml
index a0d62ba..3c2cee2 100644
--- a/tests/foobar.yml
+++ b/tests/foobar.yml
@@ -1,3 +1,4 @@
+---
 default_context: # foobar
     greeting: ?????
     email: "[email protected]"

You can easily reproduce it.

It seems currently poyo doesn't support list of dicts, for example:

list_val:
  - key: key1
  - key: key2

Will be parsed intto:

{'list_val': ['key: key1', 'key: key2']}

And:

list_val:
  - key: key1
    value: value1
  - key: key2
    value: value2

Will just throw ValueError while parsing:

ValueError: Parent of ChildMixin instance needs to be a Container.

Is there any plan to support them?

I'm using poyo v0.4.0

Submitted a PR to demonstrate the bug: #7

Thanks!

Or is it safe by default?

A brief note on this, perhaps in the README, would be helpful.

I created poyo to address issues with the YAML parser that we used in cookiecutter. Now that cookiecutter moved to a new GitHub org it makes sense to also transfer this repository.

https://github.com/cookiecutter 🍪

What do you think @pydanny?

• CONTRIBUTING.rst
• CONTRIBUTORS.rst
• README.rst

HI,

Here's an example of the problem:

poyo.parse_string("a: \n  a: 2")

Causes the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/python3.8/site-packages/poyo/parser.py", line 280, in parse_string
    return parser()
  File "/python3.8/site-packages/poyo/parser.py", line 273, in __call__
    match = self.find_match()
  File "/python3.8/site-packages/poyo/parser.py", line 254, in find_match
    node = callback(match)
  File "/python3.8/site-packages/poyo/parser.py", line 67, in _wrapper
    result = wrapped_function(parser, match, **kwargs)
  File "/python3.8/site-packages/poyo/parser.py", line 166, in parse_simple
    value = self.read_from_tag(groups["value"])
  File "/python3.8/site-packages/poyo/parser.py", line 128, in read_from_tag
    raise NoTypeException(error_message.format(string))
poyo.exceptions.NoTypeException: Unable to determine type for "
 a: 2"

I'm not 100% sure if trailing spaces are actually allowed in yaml, but I was under the impression it's allowed because other parsers accept it.

If this parser doesn't accept trailing spaces, it should show a helpful message.

LIST pattern silently eats away and anything between nonsibling LIST_ITEM patterns. The only exception is if a comment exists between lists.

For the following input

a:
  - 1
  - 2 # comment between a and b
b:
  - 3
  - 4
c:
  - 5
  - 6
# comment between c and d
d:
  - 7
  - 8

parse_string returns

{
    u'a': [
        1,
        2,
    ],
    u'b': [
        3,
        4,
        5,
        6,
    ],
    u'd': [
        7,
        8,
    ],
}

while the expected output is

{
    u'a': [
        1,
        2,
    ],
    u'b': [
        3,
        4,
    ],
    u'c': [
        5,
        6,
    ],
    u'd': [
        7,
        8,
    ],
}