joshtemple / lkml Goto Github PK
View Code? Open in Web Editor NEWA speedy LookML parser & serializer implemented in pure Python.
License: MIT License
A speedy LookML parser & serializer implemented in pure Python.
License: MIT License
The parser complains
f"Unable to find a matching expression for '{token.id}' "
SyntaxError: Unable to find a matching expression for ']' on line
if it is receives a list with a final comma such as
timeframes: [
date,
week,
month,
]
However, the Looker validator passes this as valid
For a dimension definition:
{
"name": "field_name",
"sql": "${TABLE}.field_name ;;"
}
When the lookml is generated, another ;;
is appended. I'm seeing two options (there's probably more):
;;
during lookml generationIf I process a lkml view like this:
view: orders {
sql_table_name: order_1 ;;
dimension: easy {
description: "This should be easy"
sql: ${TABLE}.id ;;
}
dimension: hard {
description: "This is
probably harder isn't it?"
sql: ${TABLE}.id ;;
}
}
Then when I parse the lkml I get the following description for the hard
dimension:
This is\n probably harder isn't it?
In looker the new line and indentation is ignored. Is this something you'd want to handle in lkml? I'm just trying a workaround in our own code but would be great to fix upstream if it's in scope
For query fields, the dimensions and measures keys are special - they are kept in plural form, with a list as the value.
When the simple parser parses this field, it sees that it's a pluralized key, and unnests them. Example:
In [98]: lkml.dump(
...: {
...: "explores": [
...: {
...: "name": "queried_explore",
...: "queries": [
...: {
...: "dimensions": ["first_dim"],
...: "measures": ["first_meas"],
...: "pivots": ["first_pivot"],
...: "name": "aquery"
...: }
...: ]
...: }
...: ]
...: }
...: )
Out[98]: 'explore: queried_explore {\n query: aquery {\n dimension: first_dim\n measure: first_meas\n pivots: [first_pivot]\n }\n}'
Many of the docstrings in lkml are outdated, especially in simple.DictParser
.
Doesn't look like the tests cover the action
param and the parser choked on ones in my schema
https://docs.looker.com/reference/field-params/action
I can grab the error it was throwing if that's helpful
For example:
import lkml
lkml.dump({"dimensions": [{"name": "active_this_week", "type": "yesno", "sql": "${TABLE}.atw"}], "measures": {"name": "active_this_week", "filters": {"active_this_week": "yes"}}})
This results in:
'dimension: active_this_week {\n type: yesno\n sql: ${TABLE}.atw ;;\n}\n\nmeasures: active_this_week {\n filters: {\n active_this_week: "yes"\n }\n}'
I would expect:
'dimension: active_this_week {\n type: yesno\n sql: ${TABLE}.atw ;;\n}\n\nmeasures: active_this_week {\n filters: [\n active_this_week: "yes"\n ]\n}'
The current version fails LookML validation.
filters: [field: value, field: value]
lkml
would benefit from adding docstrings explaining the various functionality of methods throughout the codebase.
This project does not currently support parsing the project manifest file, but it should be possible because the manifest file follows a syntax that is essentially LookML.
To support this file, we would need to review the keys defined by Looker, update lkml/keys.py
, plus ample test cases (ideally with public Looker projects found on GitHub) to confirm it works in all cases.
Also see: #75
Hi! First of all, thanks so much for the work here.
I've noticed that #63 was not included in the latest release -- and I was wondering if a new release with this could be done.
Thanks!
I found this that suggests that refinements are parsed (and the functional tests + sample refinement).
I have been trying with the provided example and the parsing does not fail; however, I don't think the behaviour is what I would expect: I would expect the end object to have only the base view, with each of the fields inside updated as per the layers of refinements. Instead, the object contains multiple views (the base view and each of the refinements).
I didn't find any reference in the docs to refinements, so maybe that is the intended behaviour.
e.g. input:
view_txt = '''
view: tickets {
label: "foo"
}
view: +tickets {
label: "baz"
}
view: +tickets {
label: "bar"
}
'''
parsed = lkml.load(view_txt)
end_object = lkml.dump(parsed)
My expectation would be that 'end_object' is:
view: tickets {
label: "bar"
}
Or, if that is not the end_object, that I could retrieve it from the parsed object (as the refinements are position dependant).
One thing we want to do is ingest an entire directory of LookML and resolve references between files.
For example, a Looker view is bound to a specific model when the the view is included in the model definition file. Until you bind a view to a specific model, you actually do not know which connection the view is referencing.
Another example, to understand a view definition completely, you need to resolve view extends. Since Looker views can extend other views in other files, again you have to ingest the entire directory of LookML and resolve references.
This might be completely out of scope, but it would be nice to have an interface to ingest a directory on the filesystem and build a graph of Looker objects with dependencies resolved between them.
Dimensions prefaced with html
or sql
throw parsing errors, likely because they are valid expected keywords.
My guess is that similar issues would happen with other keywords. I'd expect this not to happen.
If I get a chance to dig around in the code to see how to fix it, I'll open a PR.
$ pip list | grep lkml
lkml 0.1.1
$ cat lkml_test.view.lkml
view: blah {
sql_table_name: test ;;
# these prefixes break parsing
dimension: html_blah {}
dimension: sql_stuff {}
}
$ lkml lkml_test.view.lkml
Traceback (most recent call last):
File "/usr/local/bin/lkml", line 11, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/lkml/__init__.py", line 59, in cli
lookml = load(args.file)
File "/usr/local/lib/python3.7/site-packages/lkml/__init__.py", line 13, in load
tokens = lexer.scan()
File "/usr/local/lib/python3.7/site-packages/lkml/lexer.py", line 81, in scan
self.tokens.append(self.scan_expression_block())
File "/usr/local/lib/python3.7/site-packages/lkml/lexer.py", line 92, in scan_expression_block
if self.peek() == "\n":
File "/usr/local/lib/python3.7/site-packages/lkml/lexer.py", line 13, in peek
return self.text[self.index]
IndexError: string index out of range
In the LookML editor, SQL statements have a leading space (you can see the highlighted blue background). However, this is probably confusing for users, as they wouldn't think to insert a leading space when they write or modify the SQL statements, leading to confusion when the LookML is generated without one.
We could manage this automatically in the DictParser
class by checking for the presence of leading whitespace and inserting a space if none is found.
It could be useful to have a line width option that, when supplied to the serializer, adds line breaks where needed to keep the maximum line width within a certain number of characters. I would suggest a default of 80 characters.
Thanks for the amazing project @joshtemple :)
I've run into another issue, similar to #59.
The following LookML works fine.
explore: dataset_owners {
query: my_owner_query {
sorts: [ownership_count: desc, urn: asc ]
}
}
But adding a space after desc
causes issues. This validates just fine in the Looker online IDE.
explore: dataset_owners {
query: my_owner_query {
sorts: [ownership_count: desc , urn: asc ]
}
}
When parsing this with lkml==1.2.0, it fails with this error
$ lkml -v model_failure_1.lkml
...
SyntaxError: Unable to find a matching expression for '<whitespace>' on line 3
I've observed this for both sorts
and filters
within a query. Additionally, it also happens when the comma is on the following line.
It doesn't seem like this package supports parsing of LookML dashboard files?
Currently getting this error:
Traceback (most recent call last):
File /opt/anaconda3/envs/flask/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3433 in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
Cell In [10], line 17
result = lkml.load(f)
File /opt/anaconda3/envs/flask/lib/python3.9/site-packages/lkml/__init__.py:50 in load
tree: DocumentNode = parse(text)
File /opt/anaconda3/envs/flask/lib/python3.9/site-packages/lkml/__init__.py:29 in parse
tree: DocumentNode = parser.parse()
File /opt/anaconda3/envs/flask/lib/python3.9/site-packages/lkml/parser.py:214 in parse
container = self.parse_container()
File /opt/anaconda3/envs/flask/lib/python3.9/site-packages/lkml/parser.py:39 in wrapper
result = fn(self, *args, **kwargs)
File /opt/anaconda3/envs/flask/lib/python3.9/site-packages/lkml/parser.py:254 in parse_container
raise SyntaxError(
File <string>
SyntaxError: Unable to find a matching expression for '<literal>' on line 1
If there is a space after "CLOSED"
, lkml throws an error: SyntaxError: Unable to find a matching expression for '<whitespace>' on line 15
connection: "my_connection"
explore: second_model {
label: "Second model!"
description: "Lorem ipsum"
measure: bookings_measure {
label: "Number of new bookings"
group_label: "New bookings"
description: "A distinct count of all new bookings"
sql: ${booking_id} ;;
type: count_distinct
filters: [ state: "CLOSED" ,
name: "New Bookings"]
}
}
Removing the space between the "CLOSED" and the comma works.
Full debug logs below:
❯ lkml -v second.model.lkml
lkml.parser DEBUG: Check StreamStartToken() == StreamStartToken
lkml.parser DEBUG: Check LiteralToken(connection) == CommentToken or WhitespaceToken
lkml.parser DEBUG: Try to parse [expression] = (block / pair / list)*
lkml.parser DEBUG: . Check LiteralToken(connection) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . Check LiteralToken(connection) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . Check LiteralToken(connection) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(connection) == LiteralToken
lkml.parser DEBUG: . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check QuotedLiteralToken(my_connection) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Successfully parsed key.
lkml.parser DEBUG: . . Check QuotedLiteralToken(my_connection) == LiteralToken
lkml.parser DEBUG: . . Check QuotedLiteralToken(my_connection) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check QuotedLiteralToken(my_connection) == BlockStartToken
lkml.parser DEBUG: . Try to parse [pair] = key value
lkml.parser DEBUG: . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . Check LiteralToken(connection) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(connection) == LiteralToken
lkml.parser DEBUG: . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check QuotedLiteralToken(my_connection) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Successfully parsed key.
lkml.parser DEBUG: . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . Check QuotedLiteralToken(my_connection) == LiteralToken
lkml.parser DEBUG: . . . Check QuotedLiteralToken(my_connection) == QuotedLiteralToken
lkml.parser DEBUG: . . Successfully parsed value.
lkml.parser DEBUG: . Successfully parsed pair.
lkml.parser DEBUG: . Check WhitespaceToken('\n\n') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . Check LiteralToken(include) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . Check LiteralToken(include) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . Check WhitespaceToken('\n\n') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(include) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(include) == LiteralToken
lkml.parser DEBUG: . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check QuotedLiteralToken(/view_declarations.view) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Successfully parsed key.
lkml.parser DEBUG: . . Check QuotedLiteralToken(/view_declarations.view) == LiteralToken
lkml.parser DEBUG: . . Check QuotedLiteralToken(/view_declarations.view) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check QuotedLiteralToken(/view_declarations.view) == BlockStartToken
lkml.parser DEBUG: . Try to parse [pair] = key value
lkml.parser DEBUG: . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . Check WhitespaceToken('\n\n') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(include) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(include) == LiteralToken
lkml.parser DEBUG: . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check QuotedLiteralToken(/view_declarations.view) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Successfully parsed key.
lkml.parser DEBUG: . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . Check QuotedLiteralToken(/view_declarations.view) == LiteralToken
lkml.parser DEBUG: . . . Check QuotedLiteralToken(/view_declarations.view) == QuotedLiteralToken
lkml.parser DEBUG: . . Successfully parsed value.
lkml.parser DEBUG: . Successfully parsed pair.
lkml.parser DEBUG: . Check WhitespaceToken('\n\n') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . Check LiteralToken(explore) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . Check LiteralToken(explore) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . Check WhitespaceToken('\n\n') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(explore) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(explore) == LiteralToken
lkml.parser DEBUG: . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(second_model) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Successfully parsed key.
lkml.parser DEBUG: . . Check LiteralToken(second_model) == LiteralToken
lkml.parser DEBUG: . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check BlockStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check BlockStartToken() == BlockStartToken
lkml.parser DEBUG: . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Try to parse [expression] = (block / pair / list)*
lkml.parser DEBUG: . . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(label) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(label) == LiteralToken
lkml.parser DEBUG: . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Second model!) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . Check QuotedLiteralToken(Second model!) == LiteralToken
lkml.parser DEBUG: . . . . Check QuotedLiteralToken(Second model!) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check QuotedLiteralToken(Second model!) == BlockStartToken
lkml.parser DEBUG: . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(label) == LiteralToken
lkml.parser DEBUG: . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Second model!) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Second model!) == LiteralToken
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Second model!) == QuotedLiteralToken
lkml.parser DEBUG: . . . . Successfully parsed value.
lkml.parser DEBUG: . . . Successfully parsed pair.
lkml.parser DEBUG: . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(description) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(description) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(description) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(description) == LiteralToken
lkml.parser DEBUG: . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Lorem ipsum) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . Check QuotedLiteralToken(Lorem ipsum) == LiteralToken
lkml.parser DEBUG: . . . . Check QuotedLiteralToken(Lorem ipsum) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check QuotedLiteralToken(Lorem ipsum) == BlockStartToken
lkml.parser DEBUG: . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(description) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(description) == LiteralToken
lkml.parser DEBUG: . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Lorem ipsum) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Lorem ipsum) == LiteralToken
lkml.parser DEBUG: . . . . . Check QuotedLiteralToken(Lorem ipsum) == QuotedLiteralToken
lkml.parser DEBUG: . . . . Successfully parsed value.
lkml.parser DEBUG: . . . Successfully parsed pair.
lkml.parser DEBUG: . . . Check WhitespaceToken('\n\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(measure) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(measure) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(measure) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(measure) == LiteralToken
lkml.parser DEBUG: . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(bookings_measure) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . Check LiteralToken(bookings_measure) == LiteralToken
lkml.parser DEBUG: . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check BlockStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check BlockStartToken() == BlockStartToken
lkml.parser DEBUG: . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Try to parse [expression] = (block / pair / list)*
lkml.parser DEBUG: . . . . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(label) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(label) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(Number of new bookings) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(Number of new bookings) == LiteralToken
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(Number of new bookings) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(Number of new bookings) == BlockStartToken
lkml.parser DEBUG: . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check LiteralToken(label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(label) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(Number of new bookings) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(Number of new bookings) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(Number of new bookings) == QuotedLiteralToken
lkml.parser DEBUG: . . . . . . Successfully parsed value.
lkml.parser DEBUG: . . . . . Successfully parsed pair.
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(group_label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(group_label) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(group_label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(group_label) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(New bookings) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(New bookings) == LiteralToken
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(New bookings) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(New bookings) == BlockStartToken
lkml.parser DEBUG: . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(group_label) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(group_label) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(New bookings) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(New bookings) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(New bookings) == QuotedLiteralToken
lkml.parser DEBUG: . . . . . . Successfully parsed value.
lkml.parser DEBUG: . . . . . Successfully parsed pair.
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(description) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(description) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(description) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(description) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(A distinct count of all n ... ) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(A distinct count of all n ... ) == LiteralToken
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(A distinct count of all n ... ) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check QuotedLiteralToken(A distinct count of all n ... ) == BlockStartToken
lkml.parser DEBUG: . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(description) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(description) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(A distinct count of all n ... ) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(A distinct count of all n ... ) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check QuotedLiteralToken(A distinct count of all n ... ) == QuotedLiteralToken
lkml.parser DEBUG: . . . . . . Successfully parsed value.
lkml.parser DEBUG: . . . . . Successfully parsed pair.
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(sql) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(sql) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(sql) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(sql) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check ExpressionBlockToken(${booking_id}) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check ExpressionBlockToken(${booking_id}) == LiteralToken
lkml.parser DEBUG: . . . . . . Check ExpressionBlockToken(${booking_id}) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check ExpressionBlockToken(${booking_id}) == BlockStartToken
lkml.parser DEBUG: . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(sql) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(sql) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check ExpressionBlockToken(${booking_id}) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . Check ExpressionBlockToken(${booking_id}) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ExpressionBlockToken(${booking_id}) == QuotedLiteralToken
lkml.parser DEBUG: . . . . . . . Check ExpressionBlockToken(${booking_id}) == ExpressionBlockToken
lkml.parser DEBUG: . . . . . . . Check ExpressionBlockEndToken() == ExpressionBlockEndToken
lkml.parser DEBUG: . . . . . . Successfully parsed value.
lkml.parser DEBUG: . . . . . Successfully parsed pair.
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(type) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(type) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(type) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(type) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(count_distinct) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check LiteralToken(count_distinct) == LiteralToken
lkml.parser DEBUG: . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check LiteralToken(filters) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check LiteralToken(filters) == BlockStartToken
lkml.parser DEBUG: . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(type) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(type) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(count_distinct) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . Check LiteralToken(count_distinct) == LiteralToken
lkml.parser DEBUG: . . . . . . Successfully parsed value.
lkml.parser DEBUG: . . . . . Successfully parsed pair.
lkml.parser DEBUG: . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(filters) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(filters) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(filters) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(filters) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check ListStartToken() == LiteralToken
lkml.parser DEBUG: . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check ListStartToken() == BlockStartToken
lkml.parser DEBUG: . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(filters) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(filters) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == QuotedLiteralToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == ExpressionBlockToken
lkml.parser DEBUG: . . . . . Try to parse [list] = key '[' csv? ']'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(filters) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(filters) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check ListStartToken() == ListStartToken
lkml.parser DEBUG: . . . . . . Try to parse [csv] = (literal / quoted_literal) ("," (literal / quoted_literal))* ","?
lkml.parser DEBUG: . . . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . . . Check LiteralToken(state) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . . . Check LiteralToken(state) == LiteralToken
lkml.parser DEBUG: . . . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . . . Check QuotedLiteralToken(CLOSED) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . . . Check QuotedLiteralToken(CLOSED) == LiteralToken
lkml.parser DEBUG: . . . . . . . . . Check QuotedLiteralToken(CLOSED) == QuotedLiteralToken
lkml.parser DEBUG: . . . . . . . . Successfully parsed value.
lkml.parser DEBUG: . . . . . . . Successfully parsed pair.
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check CommaToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check CommaToken() == ListEndToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommaToken
lkml.parser DEBUG: . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check LiteralToken(state) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check LiteralToken(state) == ListEndToken
Traceback (most recent call last):
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/bin/lkml", line 8, in <module>
sys.exit(cli())
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/__init__.py", line 119, in cli
result: dict = load(args.file)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/__init__.py", line 50, in load
tree: DocumentNode = parse(text)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/__init__.py", line 29, in parse
tree: DocumentNode = parser.parse()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 213, in parse
container = self.parse_container()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 237, in parse_container
block = self.parse_block()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 296, in parse_block
container = self.parse_container()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 237, in parse_container
block = self.parse_block()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 296, in parse_block
container = self.parse_container()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 253, in parse_container
raise SyntaxError(
SyntaxError: Unable to find a matching expression for '<whitespace>' on line 15
Hello, how to generate SQL use lkml ?
This is a change (and I assume a regression), sometime between 0.2.2 and 1.0.0 (installed today).
sql_table_name: entrepot.annotated_donations ;;
view['sql_table_name'] returns: entrepot.an...ed_donations
with leading and trailing whitespace, rather than entrepot.annotated_donations
This is not how Looker thinks about it (Looker strips whitespace). And, this is the recommended style to use by Looker.
We have this situation in one of our project's manifest file:
project_name: "abc"
constant: app_name {
value: "ABC"
}
constant: database_name {
value: "ABC"
}
local_dependency: {
project: "pqr"
override_constant: app_name {
value: "@{app_name}"
}
override_constant: database_name {
value: "@{database_name}"
}
}
remote_dependency: xyz {
url: "[email protected]:xyz/xyz.git"
ref: "main"
override_constant: app_name {
value: "@{app_name}"
}
override_constant: database_name {
value: "@{database_name}"
}
}
This looks to be a valid syntax in LookML, but when we try to parse it, we get this error:
KeyError: 'Key "override_constant" already exists in tree and would overwrite the existing value.'
The parser supports multiple constant
keys at the top level, can it be updated to support multiple override_constant
keys in the local_dependency and remote_dependency blocks?
User story: As a developer, I want to autoformat LookML (via a round trip through lkml.load
and lkml.dump
, for example) in conjunction w/ a style linter.
Blocker: lkml
currently does not store commented out code. The lams
linter uses comments to ignore warnings. Therefore, LAMS would fail on the following code if it was formatted w/ lkml
.
view: order {
sql_table_name: sales.order ;;
dimension: customer_name {
label: "Customer Name"
type: string
sql: ${customer.name} ;;
# LAMS
# rule_exemptions: {
# F1: "This is really important to keep for some reason"
# }
}
}
Looker's validator allows multiple duplicate top-levels keys. For instance, it consider this as valid LookML
connection: "prod-data-playground-std-bq"
connection: "prod-data-playground-std-bq"
label: "CIE - Memberhip Facts"
label: "CIE - Memberhip Facts"
week_start_day: sunday
week_start_day: sunday
include: "*.view.lkml" # include all views in this project
include: "*.view.lkml" # include all views in this project
include: "attendance_base.explore.lkml"
include: "attendance_base.explore.lkml"
and then just takes last value of each key.
However, lkml
parser errors out with errors such as
f'Key "{key}" already exists in tree '
KeyError: 'Key "week_start_day" already exists in tree and would overwrite the existing value.'
It doesn't look like lkml
likes the filters: [name:...]
feature
carl.anderson@nycCANDERSONmbp ~/code/lookml pip freeze | grep lkml
lkml==0.2.2
This example was taken from the looker docs:
carl.anderson@nycCANDERSONmbp ~/code/lookml more filter.lkml
view: view_name {
measure: field_name {
filters: [dimension_name: "filter expression", dimension_name: "filter expression"]
}
}
carl.anderson@nycCANDERSONmbp ~/code/lookml lkml filter.lkml
Traceback (most recent call last):
File "/opt/anaconda3/bin/lkml", line 8, in
sys.exit(cli())
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/init.py", line 99, in cli
lookml = load(args.file)
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/init.py", line 33, in load
result = parser.parse()
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 153, in parse
return self.parse_expression()
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 250, in parse_expression
block = self.parse_block()
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 322, in parse_block
expression = self.parse_expression()
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 250, in parse_expression
block = self.parse_block()
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 322, in parse_block
expression = self.parse_expression()
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/lkml/parser.py", line 267, in parse_expression
f"Unable to find a matching expression for '{token.id}' "
SyntaxError: Unable to find a matching expression for ':' on line 3
I was looking at using this for CI and my use case would be to automatically add a description field if its not populated in the lookml for any dimension. The issue is when I read in the lookml file and add that description field to a dimension we lose all the comments that are important for us to maintain the code base since the DictVisitor().visitor(tree)
doesn't return the comments even though they're in the tree structure. Is it possible to keep the comments so we can consider using this in our CI pipeline?
When reading a lkml file that has something like the following:
measure: min_example {
label: "example_label"
description: "Example Description"
type: min
sql: ${TABLE}.n_example;;
filters: [n_example: ">0"]
}
We get this error:
Traceback (most recent call last):
File "main.py", line 19, in searching_all_files
data = lkml.load(stream)
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/__init__.py", line 33, in load
result = parser.parse()
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 153, in parse
return self.parse_expression()
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 250, in parse_expression
block = self.parse_block()
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 322, in parse_block
expression = self.parse_expression()
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 250, in parse_expression
block = self.parse_block()
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 322, in parse_block
expression = self.parse_expression()
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 89, in wrapper
result = method(self, *args, **kwargs)
File "/Users/user/.local/share/virtualenvs/example/lib/python3.8/site-packages/lkml/parser.py", line 266, in parse_expression
raise SyntaxError(
SyntaxError: Unable to find a matching expression for ':' on line 105
The library is throwing an error when it tries to parse: filters: [date_day_date: "yesterday"]
The traceback error is in parser.py
. Would fix it myself but I don't have enough context on this library.
We have this situation in one of our derived view in LookML:
view: user_activity_by_active_days{
derived_table: {
explore_source: active_users_count {
column: date_grouping {}
column: user_id {}
column: space_name {}
column: app_role {}
bind_filters: {
to_field: active_users_count.timestamp_date
from_field: user_activity_by_active_days.filter_date
}
bind_filters: {
to_field: active_users_count.event_name
from_field: user_activity_by_active_days.event_name
}
}
}
filter: filter_date {
type: date
}
filter: event_name {
type: string
}
}
This looks to be a valid syntax in LookML, but when I try to parse it, I get this error:
KeyError: 'Key "bind_filters" already exists in tree and would overwrite the existing value.'
The error makes sense, as the key is really duplicated, but tool should most likely create list of bind filters there.
Currently, you can only load and parse in Python using lkml.load
, which requires a file object as an input. This means that if you want to load a string from Python, you have to put it in a StringIO
instance or similar.
lkml
should allow strings as well in its input.
In general, the JSON format is well structured and very easy to work with. View files have a key views
with a list of views. Explores have an explores
key with a list of explores. However, there is nothing to specify that the JSON is from a model file.
Take these examples:
carl.anderson@nycCANDERSONmbp ~/code/ww-tech-lookml-tools-pythonparser pythonparser ✚ ● ? more test/grapher_lkml/some_explore.exlore.lkml
include: "*.view.lkml"
explore: some_explore {
from: some_view
join: some_other_view {
from: some_other_view
}
}
carl.anderson@nycCANDERSONmbp ~/code/ww-tech-lookml-tools-pythonparser pythonparser ✚ ● ? more test/grapher_lkml/some_model.model.lkml
connection: "some_db_connection"
include: "*.view"
explore: some_explore {
from: some_view
join: enrollment_date {
from: some_other_view
type: inner
relationship: many_to_one
sql_on: ${membership.enrollment_date_id} =${enrollment_date.date_id} ;;
}
}
carl.anderson@nycCANDERSONmbp ~/code/ww-tech-lookml-tools-pythonparser pythonparser ✚ ● ? lkml test/grapher_lkml/some_explore.exlore.lkml
{
"includes": [
"*.view.lkml"
],
"explores": [
{
"from": "some_view",
"joins": [
{
"from": "some_other_view",
"name": "some_other_view"
}
],
"name": "some_explore"
}
]
}
carl.anderson@nycCANDERSONmbp ~/code/ww-tech-lookml-tools-pythonparser pythonparser ✚ ● ? lkml test/grapher_lkml/some_model.model.lkml
{
"connection": "some_db_connection",
"includes": [
"*.view"
],
"explores": [
{
"from": "some_view",
"joins": [
{
"from": "some_other_view",
"type": "inner",
"relationship": "many_to_one",
"sql_on": "${membership.enrollment_date_id} =${enrollment_date.date_id}",
"name": "enrollment_date"
}
],
"name": "some_explore"
}
]
}
carl.anderson@nycCANDERSONmbp ~/code/ww-tech-lookml-tools-pythonparser pythonparser ✚ ● ?
I can't tell from this last parsed JSON that it is from a model. It is very similar to the explore file above. Have you considered pushing the JSON down a level under a models
key. This would provide consistency
With such consistency, you can write cleaner code against it, such as
if 'views' in json_data:
for v in json_data['views']:
self.node_map[v['name']] = NodeType.VIEW
elif 'models' in json_data:
for m in json_data['models']:
self.node_map[m['name']] = NodeType.MODEL
[self.process_explores(m, e) for e in m['explores']]
elif 'explores' in json_data:
for e in json_data['explores']:
self.process_explores(None, e)
For ease of use in Python, lkml
should convert values that are yes
or no
strings into True
and False
boolean values.
Hi there 👋
This is not an issue at all just a question but I couldn't find a way to reach out to you so I'm posting this here.
I'm trying to figure out a way to add back comments to a parsed lookML file. I have gone through the advanced parsing doc here and I understand the general data structures involved in parsing a file but I'm still not able to synthesize an approach to manipulating the tree to get the comments out in lkml.dump
. Any hints/directions would be much appreciated!
Hi!
First off thanks for all the work that went into this library, it has exactly what I was looking for!
I'm creating a class which adds descriptions to dimensions when they do not exist. I'm using the advanced parsing method as I want to make sure all comments and formatting are kept. Here is the current iteration of my class (and the initial creation of the DocumentNode):
import lkml
from dataclasses import replace
from lkml.visitors import BasicTransformer
from lkml.tree import BlockNode, PairNode, SyntaxToken
with open(file_path, 'r') as file:
tree = lkml.parse(file.read())
class DescriptionGeneratorTransformer(BasicTransformer):
def visit_block(self, block: BlockNode) -> BlockNode:
"""Visit each block and adds a description node if one does not exist."""
block = super().visit_block(block)
if block.type in ['dimension', 'measure', 'dimension_group']:
nodes = [node.type.value for node in block.container.items]
if 'description' not in nodes:
old_block = block.container.items
old_prefix = block.container.items[0].value.prefix
print(len(old_prefix))
old_line_number = block.container.items[0].value.line_number
print(old_line_number)
new_node = PairNode(type=SyntaxToken('description', prefix=' '), value=SyntaxToken(f'"This is a description for the {block.name} dimension."', suffix='\n'))
new_block = old_block + (new_node,)
replacement_block = replace(block, container=replace(block.container, items=new_block))
return replacement_block
else:
return block
else:
return block
new_tree = tree.accept(DescriptionGeneratorTransformer())
For the new description
PairNode
I'm trying to dynamically assign the type
's SyntaxToken
's prefix
value so that it aligns with the other objects. My idea was that it could just copy the prefix
value from the first PairNode
in the same BlockNode
. When I try to grab that value it returns an empty string (see the print(len(old_prefix))
. My expectation is that it should return a string with 2 whitespaces
. I could hardcode it (like I have it in the above class definition) but I would rather use that old_prefix
value in case other LookML files are formatted differently.
Additionally, I would love to have the description
become the first parameter in the dimension. Currently the above class places it last. I was thinking this is where I could use SyntaxToken
's line_number
value; however, I haven't been able to get it to work. My fear is that this would actually be a pretty complex change because I would potentially have to change the line_number
for every SyntaxToken
in the BlockNode
(e.g. move them down one). Any thoughts or ideas on how I could make the description the first element?
Current:
dimension: ascent_status {
type: number
sql: ${TABLE}."ASCENT_STATUS" ;;
description: "This is a description for the ascent_status dimension."
}
Goal:
dimension: ascent_status {
description: "This is a description for the ascent_status dimension."
type: number
sql: ${TABLE}."ASCENT_STATUS" ;;
}
The test cases included in this repo show the explore filters as follows:
lkml/tests/resources/model_with_all_fields.model.lkml
Lines 104 to 109 in 46b7917
However the official looker docs show explore filters with a different format (brackets instead of curly braces):
https://docs.looker.com/reference/explore-params/always_filter
This causes parsing of looker model files to fail if they contain the proper looker filter syntax. I tried the curly braces format in Looker's UI and it failed their LookML validation, so I think the proper syntax is indeed the brackets.
Hey, thanks for this library!
I'm parsing Looker datagroups and it seems the interval_trigger
subparameter isn't supported. Here's an example:
>>> datagroup_dict = {"datagroups": [{
'name': 'name',
'label': 'label',
'sql_trigger': 'SELECT 1',
'description': 'description',
'max_cache_age': '24 hours',
'interval_trigger': '6 hours'
}]}
>>> datagroup = lkml.dump(datagroup_dict)
>>> print(datagroup)
datagroup: name {
label: "label"
sql_trigger: SELECT 1 ;;
description: "description"
max_cache_age: "24 hours"
interval_trigger: 6 hours
}
Note the interval_trigger is unquoted. I took a look at the source and I don't see it mentioned, for example in the QUOTED_LITERAL_KEYS
. We've solved this downstream with some regex replacement but just wanted to note it upstream here as well. I'm happy to take a closer look too.
This is a fun one - turns out Looker is accepting of lkml code that has multiple colon (:
) characters in a row, possibly with spaces between.
This works in both measure/dimension definitions and in value_format, and probably in other places too.
view: my_view {
# other stuff
measure: :: ::::: my_measure {
type: number
value_format:: : "0.00%"
}
}
The current lkml parser (v1.3.1) throws an exception on this input.
With the following lkml file:
view: my_view {
set: my_set {
fields: [
, field_a
, field_b
, field_c
]
}
}
The lkml
library crashes:
❯ lkml -v comma-issue.view.lkml
lkml.parser DEBUG: Check StreamStartToken() == StreamStartToken
lkml.parser DEBUG: Check LiteralToken(view) == CommentToken or WhitespaceToken
lkml.parser DEBUG: Try to parse [expression] = (block / pair / list)*
lkml.parser DEBUG: . Check LiteralToken(view) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . Check LiteralToken(view) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . Check LiteralToken(view) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(view) == LiteralToken
lkml.parser DEBUG: . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(my_view) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Successfully parsed key.
lkml.parser DEBUG: . . Check LiteralToken(my_view) == LiteralToken
lkml.parser DEBUG: . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check BlockStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check BlockStartToken() == BlockStartToken
lkml.parser DEBUG: . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Check LiteralToken(set) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . Try to parse [expression] = (block / pair / list)*
lkml.parser DEBUG: . . . Check LiteralToken(set) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . Check LiteralToken(set) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . Check LiteralToken(set) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(set) == LiteralToken
lkml.parser DEBUG: . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(my_set) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . Check LiteralToken(my_set) == LiteralToken
lkml.parser DEBUG: . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check BlockStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check BlockStartToken() == BlockStartToken
lkml.parser DEBUG: . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Check LiteralToken(fields) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . Try to parse [expression] = (block / pair / list)*
lkml.parser DEBUG: . . . . . Check LiteralToken(fields) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . Check LiteralToken(fields) == StreamEndToken or BlockEndToken
lkml.parser DEBUG: . . . . . Try to parse [block] = key literal? '{' expression '}'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check LiteralToken(fields) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(fields) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check ListStartToken() == LiteralToken
lkml.parser DEBUG: . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check ListStartToken() == BlockStartToken
lkml.parser DEBUG: . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check LiteralToken(fields) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(fields) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Try to parse [value] = literal / quoted_literal / expression_block
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == QuotedLiteralToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == ExpressionBlockToken
lkml.parser DEBUG: . . . . . Try to parse [list] = key '[' csv? ']'
lkml.parser DEBUG: . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . Check LiteralToken(fields) == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check LiteralToken(fields) == LiteralToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ValueToken() == ValueToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken(' ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Successfully parsed key.
lkml.parser DEBUG: . . . . . . Check ListStartToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check ListStartToken() == ListStartToken
lkml.parser DEBUG: . . . . . . Try to parse [csv] = (literal / quoted_literal) ("," (literal / quoted_literal))* ","?
lkml.parser DEBUG: . . . . . . . Try to parse [pair] = key value
lkml.parser DEBUG: . . . . . . . . Try to parse [key] = literal ':'
lkml.parser DEBUG: . . . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . . . Check CommaToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . . . Check CommaToken() == LiteralToken
lkml.parser DEBUG: . . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check CommaToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . . Check CommaToken() == LiteralToken or QuotedLiteralToken
lkml.parser DEBUG: . . . . . . Check WhitespaceToken('\n ') == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check CommaToken() == CommentToken or WhitespaceToken
lkml.parser DEBUG: . . . . . . Check CommaToken() == ListEndToken
Traceback (most recent call last):
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/bin/lkml", line 8, in <module>
sys.exit(cli())
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/__init__.py", line 119, in cli
result: dict = load(args.file)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/__init__.py", line 50, in load
tree: DocumentNode = parse(text)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/__init__.py", line 29, in parse
tree: DocumentNode = parser.parse()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 213, in parse
container = self.parse_container()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 237, in parse_container
block = self.parse_block()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 296, in parse_block
container = self.parse_container()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 237, in parse_container
block = self.parse_block()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 296, in parse_block
container = self.parse_container()
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 39, in wrapper
result = fn(self, *args, **kwargs)
File "/Users/hsheth/projects/datahub/metadata-ingestion/venv/lib/python3.9/site-packages/lkml/parser.py", line 253, in parse_container
raise SyntaxError(
SyntaxError: Unable to find a matching expression for ',' on line 4
If I type this directly into Looker, it is valid syntax as it passes validation.
Hi
I was trying the tool (good job BTW!!!) and I have found out, that it cannot parse the LookML dashboard files.
It will fail on this error:
SyntaxError: Unable to find a matching expression for '<literal>' on line 1
are you planing to add this functionality?
Thanks for this awesome project!
Would you consider pushing a new minor release to PyPI?
v1.1.0, the most recent release on PyPI, went up on January 26, 2021. There have been a handful of bugfixes since then, and I'd like to be able to use a version of lkml
with those fixes form a package manager.
I'm currently relying on lkml
through acryl-datahub
to do metadata ingestion for LookML files, and am pinning to a build from GitHub here to get the fixes from #64.
Thanks for your time and consideration.
As a user, I would like to be able to easily modify the parse tree to change LookML programmatically. This is really difficult to do right now because the parse tree is immutable, so any change requires the tree being rebuilt.
Adding a Transformer class and/or helper functions would make this a lot more manageable and allow people to deeply and programmatically modify the parsed LookML.
It would be useful to have a dump
function similar to other Python packages like json
and yaml
. This function would accept a Python object and would serialize it into a string of LookML, the exact opposite of the load
function.
In my opinion, lkml
should not be responsible for generating LookML that is always functional in Looker (e.g. valid property names). The only requirement is that the generated LookML can also be parsed successfully with lkml
.
When deserialising a quote (") in a string using lkml.dump(…)
quotes are added around the whole string, but the quotes inside the string aren't escaped leading to invalid lkml.
Present workaround is to do str.replace(""", "\"") before setting the value of the field.
In Looker 6.20, users can specify data tests in LookML. This is new syntax that will need to be supported and parsed by lkml
.
Plural keys will need to be added for test
and assert
.
Data tests can also be specified in independent data test files with extension .lkml
.
The following LookML is accepted by the Looker validator, but not by lkml, due to the space after the -
. This is because lkml does not consider a space to be a valid part of a literal.
explore: explore_name {
fields: [- ALL_FIELDS*]
}
To my knowledge, you cannot define more than one sql_where key as part of a JOIN.
https://docs.looker.com/reference/explore-params/sql_where
This line should be deleted:
https://github.com/joshtemple/lkml/blob/master/lkml/keys.py#L35
Currently when a lookml file contains a sql_preamble
parameter lkml hangs and there is no obvious response from the program until you comment out the offending code.
File "/opt/python/lkml/parser.py", line 266, in parse_expression
Error: Unable to find a matching expression for ''
Even the double quotes are escaped.
dimension: CA_34__MONITOR {
sql: ${TABLE}."CA_34__MONITOR" ;;
label: "34\" Monitor"
type: yesno
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.