an2deg / pyraml-parser Goto Github PK
View Code? Open in Web Editor NEWpyraml-parser - Python parser to RAML, the RESTful API Modeling Language.
pyraml-parser - Python parser to RAML, the RESTful API Modeling Language.
Following RAML:
#%RAML 0.8
---
title: Schema with double title entry in YAML
title: Schema with double title entry in YAML
version: v1
baseUri: https://sample.com/api
/media:
get: !!null
has title
defined twice.
raml-parser
command line tool properly detects this as an error with a message:
while validating while validating root properties
root property already used: 'title'
in "/home/javl/sandbox/pyraml-parser/tests/samples/invalid/invalid-double-title.yaml", line 4, column 1
pyraml-parser does not detect this as an error.
According to the RAML documentation and samples the api version does not have to be restricted to strings.
Hello.
Please add mediaType support.
It is allowed to specify mediaType only once at top level, and then omit it in responses body.
Currently this valid raml will fail with error:
pyraml.model.ValidationError: {'body': "test' expected to be <class 'pyraml.entities.RamlBody'> or dict"}
My valid raml:
#%RAML 0.8
title: Test API
baseUri: http://localhost:8001
mediaType: application/json
schemas:
- test: !include schemas/test.json
/resources:
get:
description: Get all resources
responses:
200:
body:
schema: test
from flask import Flask
app = Flask(name)
import pyraml.parser
@app.route("/")
def hello():
p= pyraml.parser.load('schema.raml')
return p
if name == "main":
app.run(host='0.0.0.0',port=8087,debug=True,threaded=True)
it throws an exception, did I miss something?
TypeError
TypeError: 'RamlRoot' object is not callable
Traceback (most recent call last)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1836, in call
return self.wsgi_app(environ, start_response)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Library/Python/2.7/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1478, in full_dispatch_request
response = self.make_response(rv)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1577, in make_response
rv = self.response_class.force_type(rv, request.environ)
File "/Library/Python/2.7/site-packages/werkzeug/wrappers.py", line 841, in force_type
response = BaseResponse(*_run_wsgi_app(response, environ))
File "/Library/Python/2.7/site-packages/werkzeug/test.py", line 867, in run_wsgi_app
app_rv = app(environ, start_response)
TypeError: 'RamlRoot' object is not callable
To see this happen run the parser with the github api: http://api-portal.anypoint.mulesoft.com/github/api/github-api-v3/github-api-v3.raml
This has a global mediaType defined as application/json, so it is not explicitly defined in each body...
/members:
type: collection
get:
description: |
List team members.
In order to list members in a team, the authenticated user must be a member
of the team.
responses:
200:
body:
schema: |
{
"$schema": "http://json-schema.org/draft-03/schema",
"type": "array",
"list": [
{
"properties": {
"login": {
"type": "string"
},
"id": {
"type": "integer"
},
"avatar_url": {
"type": "string"
},
"gravatar_id": {
"type": "string"
},
"url": {
"type": "string"
}
},
"type": "object"
}
]
}
example: |
[
{
"login": "octocat",
"id": 1,
"avatar_url": "https://github.com/images/error/octocat_happy.gif",
"gravatar_id": "somehexcode",
"url": "https://api.github.com/users/octocat"
}
]
This currently yields the error:
Traceback (most recent call last):
File "parser.py", line 10, in <module>
p = pyraml.parser.load('github.raml')
File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 107, in load
return parse(c, relative_path)
File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 139, in parse
resources[property_name] = parse_resource(context, property_name, root)
File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 194, in parse_resource
resources[property_name] = parse_resource(new_context, property_name, resource)
File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 179, in parse_resource
_method = parse_method(ParseContext(_method, c.relative_path), resource)
File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 263, in parse_method
method.responses = c.get_property_with_schema("responses", RamlMethod.responses)
File "/Users/tsmith5/dev/pyraml-parser/pyraml/parser.py", line 70, in get_property_with_schema
return property_schema.to_python(property_value)
File "/Users/tsmith5/dev/pyraml-parser/pyraml/fields.py", line 379, in to_python
for key, val in value.items()])
File "/Users/tsmith5/dev/pyraml-parser/pyraml/fields.py", line 462, in to_python
value = self.ref_class.from_json(value)
File "/Users/tsmith5/dev/pyraml-parser/pyraml/model.py", line 122, in from_json
raise ValidationError(errors)
pyraml.model.ValidationError
This can be fixed by specifying appplication/json in each body section, but ideally the parser should know that a global media type was specified
Please release this on the Python Package Index.
It seems that method.queryParameters is not implemented (although the queryParameters do appear in the traits). If this is not intentional, I suggest the following patch:
diff --git a/pyraml/entities.py b/pyraml/entities.py
index 930c301..84614f5 100644
--- a/pyraml/entities.py
+++ b/pyraml/entities.py
@@ -79,6 +79,7 @@ class RamlMethod(Model):
description = String()
body = Reference(RamlBody)
responses = Map(Int(), Reference(RamlBody))
+ queryParameters = Map(String(), Reference(RamlQueryParameter))
class RamlResource(Model):
diff --git a/pyraml/parser.py b/pyraml/parser.py
index e4d37d5..7b0cae3 100644
--- a/pyraml/parser.py
+++ b/pyraml/parser.py
@@ -10,7 +10,7 @@ from collections import OrderedDict
from raml_elements import ParserRamlInclude
from fields import String, Reference
-from entities import RamlRoot, RamlResource, RamlMethod, RamlBody, RamlResourceType, RamlTrait
+from entities import RamlRoot, RamlResource, RamlMethod, RamlBody, RamlResourceType, RamlTrait, RamlQueryParameter
from constants import RAML_SUPPORTED_FORMAT_VERSION
import bootstrap
@@ -263,6 +263,7 @@ def parse_method(c, parent_object):
method.responses = c.get_property_with_schema("responses", RamlMethod.responses)
method.description = c.get_string_property("description")
method.body = parse_body(ParseContext(c.get("body"), c.relative_path), method)
+ method.queryParameters = c.get_property_with_schema("queryParameters", RamlMethod.queryParameters)
return method
I tried to use this today to flatten out some RAML files (so I could import them into postman) and discovered that the pyraml models don't re-encode cleanly. Using a simple script like this one:
import pyraml.parser
import yaml
p = pyraml.parser.load('main.raml')
print yaml.dump(p)
... elements encode using a variety of non-generic types like:
!!python/object:pyraml.entities.RamlTrait
!!python/object:pyraml.entities.RamlResource
!!python/object:pyraml.entities.RamlMethod
!!python/unicode
(for embedded JSON schemas)And so forth.
I haven't yet tested this but it looks like fixing some of these might be as easy as extending BaseModel
from yaml.YAMLObject
instead of plain object
.
Is this something you'd be open to having fixed (even if it means creating a pyraml.parser.dump(p)
type method)? If so, I can keep poking at it.
Hi, I'm having trouble including an example file for a response field. I've tried a lot of different things. Am I doing something wrong?
For example:
/me:
get:
description: Get detailed profile information about the current user (including the current user’s username)
responses:
200:
body:
application/json:
example: !include me-example.json
It's breaking with this stack trace:
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
<ipython-input-31-2eb03608abae> in <module>()
----> 1 p = pyraml.parser.load('my-spec.raml')
/Library/Python/2.7/site-packages/pyraml/parser.pyc in load(uri)
105 c, _ = _load_local_file(uri)
106
--> 107 return parse(c, relative_path)
108
109
/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse(c, relative_path)
137 for property_name in context.__iter__():
138 if property_name.startswith("/"):
--> 139 resources[property_name] = parse_resource(context, property_name, root)
140
141 if resources > 0:
/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse_resource(c, property_name, parent_object)
192 for property_name in new_context.__iter__():
193 if property_name.startswith("/"):
--> 194 resources[property_name] = parse_resource(new_context, property_name, resource)
195
196 if resources > 0:
/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse_resource(c, property_name, parent_object)
177 _method = new_context.get(_http_method)
178 if _method:
--> 179 _method = parse_method(ParseContext(_method, c.relative_path), resource)
180 methods[_http_method] = _method
181 elif _http_method in new_context.data:
/Library/Python/2.7/site-packages/pyraml/parser.pyc in parse_method(c, parent_object)
261
262 method = RamlMethod()
--> 263 method.responses = c.get_property_with_schema("responses", RamlMethod.responses)
264 method.description = c.get_string_property("description")
265 method.body = parse_body(ParseContext(c.get("body"), c.relative_path), method)
/Library/Python/2.7/site-packages/pyraml/parser.pyc in get_property_with_schema(self, property_name, property_schema)
68 def get_property_with_schema(self, property_name, property_schema):
69 property_value = self.get(property_name)
---> 70 return property_schema.to_python(property_value)
71
72 def _load_include(self, file_name):
/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
316 value = OrderedDict([
317 (self._key_type.to_python(key), self._value_type.to_python(val))
--> 318 for key, val in value.items()])
319
320 self.validate(value)
/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
399 elif isinstance(value, dict):
400 # Value is JSON object, convert it to `ref_class`
--> 401 value = self.ref_class.from_json(value)
402 elif value is None:
403 # Value empty, just instantiate empty `ref_class`
/Library/Python/2.7/site-packages/pyraml/model.pyc in from_json(cls, json_object)
97 # Validate and process a field of JSON object
98 try:
---> 99 value = field_type.to_python(json_object.get(model_field_name, None))
100 setattr(rv, model_field_name, value)
101 except ValueError as e:
/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
316 value = OrderedDict([
317 (self._key_type.to_python(key), self._value_type.to_python(val))
--> 318 for key, val in value.items()])
319
320 self.validate(value)
/Library/Python/2.7/site-packages/pyraml/fields.pyc in to_python(self, value)
399 elif isinstance(value, dict):
400 # Value is JSON object, convert it to `ref_class`
--> 401 value = self.ref_class.from_json(value)
402 elif value is None:
403 # Value empty, just instantiate empty `ref_class`
/Library/Python/2.7/site-packages/pyraml/model.pyc in from_json(cls, json_object)
113 errors[model_field_name] = unicode(e)
114 if errors:
--> 115 raise ValidationError(errors)
116
117 return rv
ValidationError:
This library only supports RAML 0.8. Are there any plans to support RAML 1.0 at any point? Or is this project abandoned?
Following example script fails because yaml.add_representer
function which is called in initialization of pyraml module (pyraml/__init__.py
) changes behavior of PyYAML module. Modified PyYAML module will parse dictionary structure in YAML to UniqOrderedDict, so re-assignment to the data parsed by PyYAML will raise exception.
import pyraml
import yaml
if __name__ == '__main__':
with open("test.yml") as f:
data = yaml.load(f)
print(data)
data['key'] = "new_value" # <= raise ValidationError
print(data)
Result:
UniqOrderedDict([('key', 'value')])
Traceback (most recent call last):
File "example.py", line 10, in <module>
data['key'] = "new_value"
File "/usr/local/lib/python2.7/site-packages/pyraml/__init__.py", line 24, in __setitem__
raise ValidationError("Property already used: {0}".format(key))
pyraml.ValidationError: 'Property already used: key'
The exception is raised from the line below.
pyraml-parser/pyraml/__init__.py
Line 24 in 6d9f474
This problem is caused by design of yaml.add_representer
in PyYAML module, but it may be better to modify pyraml-parser because PyYAML module is used widely and is hard to be changed.
Hi,
To fix the security warning using old pyyaml version, can you update to latest version? Let me know if you need help with that.
Ref: https://nvd.nist.gov/vuln/detail/CVE-2017-18342
Thanks.
Hi,
I have tryed you parser on one of my RAML project and I have noticed a pretty annoying issue: Even if the resources are provided in the form of an OrderedDict, the order of the items is not conserved between the RAML file and the resulting OrderedDict.
It seems to be linked to the YAML parser itself that does not keep the order...
Do you think there is any simple solution to fix this issue?
Thanks :)
Is there a plan for adding docs or a tutorial for using this package? I am interested in seeing if it would fit some needs for parsing raml, and testing an API with it. I don't mind perusing code, but seems fairly easy to setup something like readthedocs
.
Hello @an2deg,
I really like the library that you have created here.
I just wanted to ask, based on your code if you have any specific usage scenario in mind.
Are you using this library to export services (like mulestudio does for raml) or create a client?
Thank you very much in advance!
example of an RAML put where the body of the put is not in the parse tree:
title: putexample
version: v1.0
/putexample:
put:
body:
application/json:
schema: |
{ "$schema": "http://json-schema.org/schema",
"value" : { "type": "boolean" }
}
responses:
200:
body:
application/json:
schema: |
{ "$schema": "http://json-schema.org/schema",
"value" : { "type": "boolean" }
}
example: |
{
"value": "true"
}
output from the parser:
(it sees the body, but does not fill an body object in the resulting tree, since the expected body at tree level is "None", Hence the parsing of the body fails ??).
parse_body: <pyraml.parser.ParseContext object at 0x02C088F0> {'application/json': {'schema': '{ "$schema": "http://json
-schema.org/schema",\n "value" : { "type": "boolean" }\n} \n'}}
None
parse_body: {'body': None, 'is_': None, 'formParameters': None, 'headers': None, 'notNull': None, 'example': None, 'sche
ma': None}
Kind Regards,
Wouter van der Beek
Hi there!
I noticed that this library does not support uriParameters
per the RAML spec. Is there any intention to?
Thanks!
The first issue is that urlopen() does not exist in urllib, but rather urllib_request.
from six.moves import urllib_request as urllib2
Further, the relative path doesn't work when using urlparse.urljoin(). For example, if we load http://example.com/dir/test.raml
, the relative url is set to http://example.com/dir
. When it comes across an include such as !include schema.json, we get
urlparse.urljoin(relativeurl, schama.json) = http://example.com/schema.raml
instead of the expected /dir/schema.json.
I've made the fixes in pull request #25. I think these will do, but I'm not a python export.
p=pyraml.parser.load("C:\Python27\test.raml")
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\pyraml\parser.py", line 136, in load
c, _ = _load_network_resource(uri)
File "C:\Python27\lib\site-packages\pyraml\parser.py", line 451, in _load_netw
ork_resource
with contextlib.closing(urllib2.urlopen(url, timeout=60.0)) as f:
AttributeError: 'Module_six_moves_urllib' object has no attribute 'urlopen'
I understand that RAML doesn't define a standard for JSON payloads and, instead, falls back to JSONSchema for that. However, having to switch between RAML and JSONSchema is disorienting and can lead to a few typos down the line. I would rather just use RAML throughout my schemas if I can. I understand that I should probably post this in the RAML forums but I'm curious to know what are your thoughts on this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.