RAML + Elasticsearch / Postgres / Mongodb / Your Data Store™ + Pyramid = RESTful API
Home Page: http://ramses.tech
License: Apache License 2.0
Ramses is a framework that generates a RESTful API using RAML. It uses Pyramid and Nefertari which provides Elasticsearch / Posgres / MongoDB / Your Data Store™ -powered views.
Looking to get started quickly? You can take a look at the "Getting Started" guide.
pip install ramses gives:
(a part of the output)
Complete output from command //anaconda/envs/hackathon/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/g1/4_6qvpkd7871k5hrggy4dbcr0000gn/T/pip-build-0vkGwK/cryptacular/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/g1/4_6qvpkd7871k5hrggy4dbcr0000gn/T/pip-vT6JgR-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.5-x86_64-2.7
creating build/lib.macosx-10.5-x86_64-2.7/cryptacular
copying cryptacular/__init__.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular
creating build/lib.macosx-10.5-x86_64-2.7/cryptacular/bcrypt
copying cryptacular/bcrypt/__init__.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/bcrypt
copying cryptacular/bcrypt/test_bcrypt.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/bcrypt
creating build/lib.macosx-10.5-x86_64-2.7/cryptacular/core
copying cryptacular/core/__init__.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/core
copying cryptacular/core/test_core.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/core
creating build/lib.macosx-10.5-x86_64-2.7/cryptacular/crypt
copying cryptacular/crypt/__init__.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/crypt
copying cryptacular/crypt/test_crypt.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/crypt
creating build/lib.macosx-10.5-x86_64-2.7/cryptacular/pbkdf2
copying cryptacular/pbkdf2/__init__.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/pbkdf2
copying cryptacular/pbkdf2/test_pbkdf2.py -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/pbkdf2
running egg_info
writing requirements to cryptacular.egg-info/requires.txt
writing cryptacular.egg-info/PKG-INFO
writing namespace_packages to cryptacular.egg-info/namespace_packages.txt
writing top-level names to cryptacular.egg-info/top_level.txt
writing dependency_links to cryptacular.egg-info/dependency_links.txt
warning: manifest_maker: standard file '-c' not found
reading manifest file 'cryptacular.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'cryptacular.egg-info/SOURCES.txt'
copying cryptacular/bcrypt/_bcrypt.c -> build/lib.macosx-10.5-x86_64-2.7/cryptacular/bcrypt
running build_ext
building 'cryptacular.bcrypt._bcrypt' extension
creating build/temp.macosx-10.5-x86_64-2.7
creating build/temp.macosx-10.5-x86_64-2.7/crypt_blowfish-1.2
creating build/temp.macosx-10.5-x86_64-2.7/cryptacular
creating build/temp.macosx-10.5-x86_64-2.7/cryptacular/bcrypt
gcc -fno-strict-aliasing -I//anaconda/envs/hackathon/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I//anaconda/envs/hackathon/include/python2.7 -c crypt_blowfish-1.2/crypt_blowfish.c -o build/temp.macosx-10.5-x86_64-2.7/crypt_blowfish-1.2/crypt_blowfish.o
gcc -fno-strict-aliasing -I//anaconda/envs/hackathon/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I//anaconda/envs/hackathon/include/python2.7 -c crypt_blowfish-1.2/crypt_gensalt.c -o build/temp.macosx-10.5-x86_64-2.7/crypt_blowfish-1.2/crypt_gensalt.o
gcc -fno-strict-aliasing -I//anaconda/envs/hackathon/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I//anaconda/envs/hackathon/include/python2.7 -c crypt_blowfish-1.2/wrapper.c -o build/temp.macosx-10.5-x86_64-2.7/crypt_blowfish-1.2/wrapper.o
gcc -fno-strict-aliasing -I//anaconda/envs/hackathon/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I//anaconda/envs/hackathon/include/python2.7 -c cryptacular/bcrypt/_bcrypt.c -o build/temp.macosx-10.5-x86_64-2.7/cryptacular/bcrypt/_bcrypt.o
cryptacular/bcrypt/_bcrypt.c:129:19: warning: unused variable 'module' [-Wunused-variable]
PyObject *module = Py_InitModule("_bcrypt", _bcrypt_methods);
^
1 warning generated.
gcc -bundle -undefined dynamic_lookup -L//anaconda/envs/hackathon/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.5-x86_64-2.7/crypt_blowfish-1.2/crypt_blowfish.o build/temp.macosx-10.5-x86_64-2.7/crypt_blowfish-1.2/crypt_gensalt.o build/temp.macosx-10.5-x86_64-2.7/crypt_blowfish-1.2/wrapper.o build/temp.macosx-10.5-x86_64-2.7/cryptacular/bcrypt/_bcrypt.o -L//anaconda/envs/hackathon/lib -o build/lib.macosx-10.5-x86_64-2.7/cryptacular/bcrypt/_bcrypt.so
ld: library not found for -lgcc_s.10.5
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "//anaconda/envs/hackathon/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/g1/4_6qvpkd7871k5hrggy4dbcr0000gn/T/pip-build-0vkGwK/cryptacular/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/g1/4_6qvpkd7871k5hrggy4dbcr0000gn/T/pip-vT6JgR-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/g1/4_6qvpkd7871k5hrggy4dbcr0000gn/T/pip-build-0vkGwK/cryptacular/
If a Parent model is before a Child model in the RAML the Child model is not created at the time the Parent looks for it.
Example see /contacts and /emails here /addressbook/api.raml
As a work around, the issue does not occur when the Parent is placed after the Child, however this prohibits the ability to nest objects for a more intuitive API structure.
This error is produced
Starting subprocess with file monitor
2015-10-10 16:04:28,697 INFO [nefertari_sqla.signals][MainThread] signals.setup_es_signals_for: setup_sqla_es_signals_for: <class 'nefertari_sqla.documents.ESBaseDocument'>
2015-10-10 16:04:28,700 INFO [nefertari][MainThread] __init__.includeme: nefertari 0.6.0
2015-10-10 16:04:28,703 INFO [nefertari.json_httpexceptions][MainThread] json_httpexceptions.includeme: Include json_httpexceptions
2015-10-10 16:04:28,703 INFO [ramses][MainThread] __init__.includeme: Parsing RAML
2015-10-10 16:04:28,800 INFO [ramses][MainThread] __init__.includeme: Starting models generation
2015-10-10 16:04:28,801 INFO [ramses.generators][MainThread] generators.generate_models: Configuring model for route `contacts`
2015-10-10 16:04:28,806 DEBUG [ramses.models][MainThread] models.get_existing_model: Model `Contact` does not exist
2015-10-10 16:04:28,806 INFO [ramses.utils][MainThread] utils.resource_schema: Searching for model schema
2015-10-10 16:04:28,806 INFO [ramses.models][MainThread] models.setup_data_model: Generating model class `Contact`
2015-10-10 16:04:28,808 DEBUG [ramses.models][MainThread] models.get_existing_model: Model `Phone` does not exist
Traceback (most recent call last):
File "/removed-base-directory/bin/pserve", line 11, in <module>
sys.exit(main())
File "/removed-base-directory/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 58, in main
return command.run()
File "/removed-base-directory/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 328, in run
global_conf=vars)
File "/removed-base-directory/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
return loadobj(APP, uri, name=name, **kw)
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
return context.create()
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
**context.local_conf)
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/removed-base-directory/lib/python2.7/site-packages/paste/urlmap.py", line 31, in urlmap_factory
app = loader.get_app(app_name, global_conf=global_conf)
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
name=name, global_conf=global_conf).create()
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
return fix_call(context.object, context.global_conf, **context.local_conf)
File "/removed-base-directory/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/removed-base-directory/addressbook-api/addressbook/__init__.py", line 9, in main
config.include('ramses')
File "/removed-base-directory/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
c(configurator)
File "/removed-base-directory/lib/python2.7/site-packages/ramses/__init__.py", line 48, in includeme
generate_models(config, raml_resources=raml_root.resources)
File "/removed-base-directory/lib/python2.7/site-packages/ramses/generators.py", line 178, in generate_models
config, raml_resource, route_name)
File "/removed-base-directory/lib/python2.7/site-packages/ramses/models.py", line 209, in handle_model_generation
raise ValueError('{}: {}'.format(model_name, str(ex)))
ValueError: Contact: Model `Phone` used in relationship `phone_numbers` is not definedFirst, I'm just introducing myself to ramses and nefertari, so I might be making some obvious mistakes here.
The latest docs outline the following list of steps to kickstart new project:
$ virtualenv my_project
$ source my_project/bin/activate
$ pip install nefertari
$ pcreate -s nefertari_starter my_project
$ cd my_project
The issue is, pcreate generates project files in the same directory where the Python virtual environment resides. The leads to rather messy content of my_project:
(my_project) vagrant my_project $ ls -la
total 60
drwxr-xr-x 7 vagrant vagrant 4096 Jun 26 12:15 .
drwxr-xr-x 4 vagrant vagrant 4096 Jun 26 12:16 ..
-rw-r--r-- 1 vagrant vagrant 704 Jun 26 12:15 api.raml
drwxr-xr-x 2 vagrant vagrant 4096 Jun 26 12:15 bin
drwxr-xr-x 2 vagrant vagrant 4096 Jun 26 12:09 include
-rw-r--r-- 1 vagrant vagrant 706 Jun 26 12:15 items.json
drwxr-xr-x 3 vagrant vagrant 4096 Jun 26 12:09 lib
lrwxrwxrwx 1 vagrant vagrant 3 Jun 26 12:09 lib64 -> lib
-rw-r--r-- 1 vagrant vagrant 1479 Jun 26 12:15 local.ini
drwxr-xr-x 4 vagrant vagrant 4096 Jun 26 12:15 my_project
drwxr-xr-x 2 vagrant vagrant 4096 Jun 26 12:15 my_project.egg-info
-rw-r--r-- 1 vagrant vagrant 60 Jun 26 12:10 pip-selfcheck.json
-rw-r--r-- 1 vagrant vagrant 69 Jun 26 12:09 pyvenv.cfg
-rw-r--r-- 1 vagrant vagrant 93 Jun 26 12:15 README.md
-rw-r--r-- 1 vagrant vagrant 68 Jun 26 12:15 requirements.txt
-rw-r--r-- 1 vagrant vagrant 716 Jun 26 12:15 setup.py
According to the plain Pyramid project creation guide, the project content is outside of the virtual environment (see directory tree. Here, the virtual environment is completely separate, placed in export VENV=~/env.
Although I understand users are free to arrange the virtual environment and the project content freely, for newbies like myself, wouldn't it be better if the docs are more explicit about separating the two?
$ mkdir my_project
$ cd my_project
$ pyvenv venv
$ source venv/bin/activate
$ pip install ramses
$ pcreate -s ramses_starter .
(venv) vagrant my_project $ ls -la
total 44
drwxr-xr-x 5 vagrant vagrant 4096 Jun 26 12:18 .
drwxr-xr-x 4 vagrant vagrant 4096 Jun 26 12:16 ..
-rw-r--r-- 1 vagrant vagrant 707 Jun 26 12:18 api.raml
-rw-r--r-- 1 vagrant vagrant 706 Jun 26 12:18 items.json
-rw-r--r-- 1 vagrant vagrant 1492 Jun 26 12:18 local.ini
drwxr-xr-x 4 vagrant vagrant 4096 Jun 26 12:18 my_project
drwxr-xr-x 2 vagrant vagrant 4096 Jun 26 12:18 my_project.egg-info
-rw-r--r-- 1 vagrant vagrant 93 Jun 26 12:18 README.md
-rw-r--r-- 1 vagrant vagrant 68 Jun 26 12:18 requirements.txt
-rw-r--r-- 1 vagrant vagrant 719 Jun 26 12:18 setup.py
drwxr-xr-x 5 vagrant vagrant 4096 Jun 26 12:17 venv
In other words, the Htchhiker's guide to Python way.
Finally, what is the actual best practice recommended by the Ramses team?
Hi,
How can I use geography objects in ramses?
There is support for it in both postgresql and elasticsearch but I does not seem to be supported in ramses.
Do you have any idea how to use geography objects in combination with ramses? Not just storing them but also using them for search for example distance to point search.
Ty!
ramle-example repository provides an example.raml
item: |
allow g:admin all
allow everyone view,options
allow {{user_self}} updateOne would expect the last line to allow some callable to dynamically calculate a principal which will be allowed to update the item.
ramses_example/init.py really provides a function user_self:
@registry.add
def user_self(ace, request, obj):
""" Give 'update' permission to user that is being created. """
from pyramid.security import Allow
return [(Allow, str(obj.username), 'update')]What surprised me is the fact, the callable does not provide only name of currently authenticated user, but also defines set of ACEs for him.
Reading ramses code ramses/acl.py I came to conclusion, that the callable really replaces the original line from RAML file and is able to put in it's place one or more ACE "lines" dynamically calculated by the callable.
To me it seems a bit confusing, that there is an instruction in RAML file consisting from three parts and two of them ("allow" and "update") are silently ignored.
Thinking of possible solution, I would propose to require all ACEs containing a callable to use following syntax:
dynamic {{callable}} dynamic
(of course, if you come with better word than dynamic I would not complain).
Current code (ignoring action and permission part of the ACE) does not prevent one to use it for developing functional application. It is enough to realize, that any ACE line containing a callable simply ignores the action and permission word around it.
Anyway, implementing more intuitive way of expressing dynamic parts of ACLs would be welcome.
Not sure why ramlfications v 0.1.8 throws these errors when 0.1.5 works fine.
tarting subprocess with file monitor
2015-11-27 17:31:32,631 INFO [ramses][MainThread] __init__.includeme: Parsing RAML
Traceback (most recent call last):
File "/hom/user/.pyenv/versions/membership-system/bin/pserve", line 9, in <module>
load_entry_point('pyramid', 'console_scripts', 'pserve')()
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 58, in main
return command.run()
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 328, in run
global_conf=vars)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
return loadobj(APP, uri, name=name, **kw)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
return context.create()
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
**context.local_conf)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/urlmap.py", line 31, in urlmap_factory
app = loader.get_app(app_name, global_conf=global_conf)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
name=name, global_conf=global_conf).create()
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
return fix_call(context.object, context.global_conf, **context.local_conf)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/hom/user/myProjects/hg/membership-system/license-api/licenseapi/__init__.py", line 242, in main
config.include('ramses')
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
c(configurator)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramses/__init__.py", line 45, in includeme
raml_root = ramlfications.parse(Settings['ramses.raml_schema'])
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/__init__.py", line 68, in parse
loader = load(raml)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/__init__.py", line 31, in load
return load_file(raml_file)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/_helpers.py", line 19, in load_file
return RAMLLoader().load(raml)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 88, in load
return self._ordered_load(raml, yaml.SafeLoader)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 73, in _ordered_load
return yaml.load(stream, OrderedLoader)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/__init__.py", line 71, in load
return loader.get_single_data()
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 39, in get_single_data
return self.construct_document(node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 43, in construct_document
data = self.construct_object(node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 66, in construct_mapping
return OrderedDict(loader.construct_pairs(node))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 145, in construct_pairs
value = self.construct_object(value_node, deep=deep)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 66, in construct_mapping
return OrderedDict(loader.construct_pairs(node))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 145, in construct_pairs
value = self.construct_object(value_node, deep=deep)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 66, in construct_mapping
return OrderedDict(loader.construct_pairs(node))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 145, in construct_pairs
value = self.construct_object(value_node, deep=deep)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 66, in construct_mapping
return OrderedDict(loader.construct_pairs(node))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 145, in construct_pairs
value = self.construct_object(value_node, deep=deep)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 66, in construct_mapping
return OrderedDict(loader.construct_pairs(node))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 145, in construct_pairs
value = self.construct_object(value_node, deep=deep)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 66, in construct_mapping
return OrderedDict(loader.construct_pairs(node))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 145, in construct_pairs
value = self.construct_object(value_node, deep=deep)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 66, in construct_mapping
return OrderedDict(loader.construct_pairs(node))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 145, in construct_pairs
value = self.construct_object(value_node, deep=deep)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/yaml/constructor.py", line 88, in construct_object
data = constructor(self, node)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 38, in _yaml_include
return self._parse_json(file_name, os.path.dirname(file_name))
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/ramlfications/loader.py", line 54, in _parse_json
schema = jsonref.load(f, base_uri=base_path, jsonschema=True)
File "/hom/user/.pyenv/versions/membership-system/lib/python2.7/site-packages/jsonref.py", line 321, in load
json.load(fp, **kwargs),
File "/hom/user/.pyenv/versions/2.7.6/lib/python2.7/json/__init__.py", line 290, in load
File "/hom/user/.pyenv/versions/2.7.6/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/hom/user/.pyenv/versions/2.7.6/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/hom/user/.pyenv/versions/2.7.6/lib/python2.7/json/decoder.py", line 381, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 26 column 13 (char 709)#%RAML 0.8
---
title: memberapi
documentation:
- title: Home
content: |
Software Licensing for internal useage.
baseUri: http://{host}:{port}/{version}
version: v1
mediaType: application/json
protocols: [HTTP]
securitySchemes:
- x_token_auth:
description: Authorization header token policy
type: x-ApiKey
- x_ticket_auth:
description: Standard Pyramid Auth Ticket policy
type: x-Ticket
settings:
secret: auth_tkt_secret
hashalg: sha512
cookie_name: ramses_auth_tkt
http_only: true
- item_owner_acl:
description: ACL that allows everyone to read, authenticated to create and item owners to edit item
type: x-ACL
settings:
collection: |
allow g:admin all
allow everyone view,options
allow authenticated create
item: |
allow g:admin all
allow authenticated view,options
- user_self_acl:
description: ACL that allows everyone to read, authenticated to create and owner to edit
type: x-ACL
settings:
collection: |
allow g:admin all
allow authenticated view,options
item: |
allow g:admin all
allow authenticated view,options
- public_acl:
description: ACL that allows everyone to read, authenticated to create and owner to edit
type: x-ACL
settings:
collection: |
allow g:admin all
allow everyone view,options
item: |
allow g:admin all
allow everyone view,options
securedBy: [x_ticket_auth]
/identities:
securedBy: [user_self_acl]
displayName: Collection of identities
get:
description: Get all identities
post:
description: Insert a identities
body:
application/json:
schema: !include schemas/identities.json
/{id}:
displayName: A particular identity
get:
description: Get a particular identity
patch:
put:
description: Update a particular identity
delete:
description: Delete a particular identity
/emails:
displayName: Collection of emails
get:
description: Get all emails
post:
description: Insert a emails
body:
application/json:
schema: !include schemas/emails.json
/{id}:
displayName: A particular email
get:
description: Get a particular email
delete:
description: Delete a particular email
patch:
description: Update a particular email
/phones:
displayName: Collection of phones
get:
description: Get all phones
post:
description: Insert a phones
body:
application/json:
schema: !include schemas/phones.json
/{id}:
displayName: A particular phone
get:
description: Get a particular phone
delete:
description: Delete a particular phone
patch:
description: Update a particular phone
/destinations:
displayName: Collection of destinations
get:
description: Get all destinations
post:
description: Insert a destinations
body:
application/json:
schema: !include schemas/destinations.json
/{type}:
displayName: A particular destination
get:
description: Get a particular destination
delete:
description: Delete a particular destination
patch:
description: Update a particular destination
/places:
displayName: Collection of places
get:
description: Get all places
post:
description: Insert a places
body:
application/json:
schema: !include schemas/places.json
/{id}:
displayName: A particular place
get:
description: Get a particular place
delete:
description: Delete a particular place
patch:
description: Update a particular place
/destinations:
displayName: Collection of destinations
get:
description: Get all destinations
post:
description: Insert a destinations
body:
application/json:
schema: !include schemas/destinations.json
/{type}:
displayName: A particular destination
get:
description: Get a particular destination
delete:
description: Delete a particular destination
patch:
description: Update a particular destination
/orders:
displayName: Collection of orders
get:
description: Get all orders
post:
description: Insert a orders
body:
application/json:
schema: !include schemas/orders.json
/{id}:
displayName: A particular order
get:
description: Get a particular order
delete:
description: Delete a particular order
patch:
description: Update a particular order
/sales:
displayName: Collection of sales
get:
description: Get all sales
post:
description: Insert a sales
body:
application/json:
schema: !include schemas/sales.json
/{id}:
displayName: A particular sale
get:
description: Get a particular sale
delete:
description: Delete a particular sale
patch:
description: Update a particular sale
/emails:
displayName: Collection of emails
get:
description: Get all emails
post:
description: Insert a emails
body:
application/json:
schema: !include schemas/emails.json
/{id}:
displayName: A particular email
get:
description: Get a particular email
delete:
description: Delete a particular email
patch:
description: Update a particular email
/phones:
displayName: Collection of phones
get:
description: Get all phones
post:
description: Insert a phones
body:
application/json:
schema: !include schemas/phones.json
/{id}:
displayName: A particular phone
get:
description: Get a particular phone
delete:
description: Delete a particular phone
patch:
description: Update a particular phone
/places:
displayName: Collection of places
get:
description: Get all places
post:
description: Insert a places
body:
application/json:
schema: !include schemas/places.json
/{id}:
displayName: A particular place
get:
description: Get a particular place
delete:
description: Delete a particular place
patch:
description: Update a particular place
/contacts:
securedBy: [user_self_acl]
displayName: Collection of contacts
get:
description: Get all contacts
post:
description: Insert a contacts
body:
application/json:
schema: !include schemas/contacts.json
/{id}:
displayName: A particular contact
get:
description: Get a particular contact
patch:
put:
description: Update a particular contact
delete:
description: Delete a particular contact
/orders:
securedBy: [user_self_acl]
displayName: Collection of orders
get:
description: Get all orders
post:
description: Insert a orders
body:
application/json:
schema: !include schemas/orders.json
/{id}:
displayName: A particular order
get:
description: Get a particular order
delete:
description: Delete a particular order
patch:
description: Update a particular order
/sales:
displayName: Collection of sales
get:
description: Get all sales
post:
description: Insert a sales
body:
application/json:
schema: !include schemas/sales.json
/{id}:
displayName: A particular sale
get:
description: Get a particular sale
delete:
description: Delete a particular sale
patch:
description: Update a particular sale
/sales:
displayName: Collection of sales
get:
description: Get all sales
post:
description: Insert a sales
body:
application/json:
schema: !include schemas/sales.json
/{id}:
displayName: A particular sale
get:
description: Get a particular sale
delete:
description: Delete a particular sale
patch:
description: Update a particular sale
/charges:
securedBy: [user_self_acl]
displayName: Collection of charges
get:
description: Get all charges
post:
description: Insert a charges
body:
application/json:
schema: !include schemas/charges.json
/{id}:
displayName: A particular payment
get:
description: Get a particular charge
delete:
description: Delete a particular charge
patch:
description: Update a particular charge
/payments:
displayName: Collection of payments
get:
description: Get all payments
post:
description: Insert a payments
body:
application/json:
schema: !include schemas/payments.json
/{id}:
displayName: A particular payment
get:
description: Get a particular payment
delete:
description: Delete a particular payment
patch:
description: Update a particular payment
/payments:
displayName: Collection of payments
get:
description: Get all payments
post:
description: Insert a payments
body:
application/json:
schema: !include schemas/payments.json
/{id}:
displayName: A particular payment
get:
description: Get a particular payment
delete:
description: Delete a particular payment
patch:
description: Update a particular payment
/paymentmethods:
securedBy: [user_self_acl]
displayName: Collection of paymentmethods
get:
description: Get all paymentmethods
post:
description: Insert a paymentmethods
body:
application/json:
schema: !include schemas/paymentmethods.json
/{id}:
displayName: A particular paymentmethod
get:
description: Get a particular paymentmethod
delete:
description: Delete a particular paymentmethod
patch:
description: Update a particular paymentmethod
/destinations:
securedBy: [user_self_acl]
displayName: Collection of destinations
get:
description: Get all destinations
post:
description: Insert a destinations
body:
application/json:
schema: !include schemas/destinations.json
/{type}:
displayName: A particular destination
get:
description: Get a particular destination
delete:
description: Delete a particular destination
patch:
description: Update a particular destination
/entities:
securedBy: [user_self_acl]
displayName: Collection of entities
get:
description: Get all entities
post:
description: Insert a entities
body:
application/json:
schema: !include schemas/entities.json
/{id}:
displayName: A particular entity
get:
description: Get a particular entity
patch:
put:
description: Update a particular entity
delete:
description: Delete a particular entity
/products:
securedBy: [public_acl]
displayName: Collection of products
get:
description: Get all products
post:
description: Insert a products
body:
application/json:
schema: !include schemas/products.json
/{name}:
displayName: A particular product
get:
description: Get a particular product
delete:
description: Delete a particular product
patch:
description: Update a particular product
/currencies:
securedBy: [user_self_acl]
displayName: Collection of currencys
get:
description: Get all currencys
post:
description: Insert a currencys
body:
application/json:
schema: !include schemas/currencies.json
/{symbol}:
displayName: A particular currency
get:
description: Get a particular currency
delete:
description: Delete a particular currency
patch:
description: Update a particular destination
/intervals:
securedBy: [public_acl]
displayName: Collection of intervals
get:
description: Get all intervals
post:
description: Insert a intervals
body:
application/json:
schema: !include schemas/intervals.json
/{name}:
displayName: A particular interval
get:
description: Get a particular interval
delete:
description: Delete a particular interval
patch:
description: Update a particular interval
/contracts:
securedBy: [user_self_acl]
displayName: Collection of contracts
get:
description: Get all contracts
post:
description: Insert a contracts
body:
application/json:
schema: !include schemas/contracts.json
/{id}:
displayName: A particular contract
get:
description: Get a particular contract
delete:
description: Delete a particular contract
patch:
description: Update a particular contract
/audits:
displayName: Collection of audits
get:
description: Get all audits
post:
description: Insert a audits
body:
application/json:
schema: !include schemas/audits.json
/{id}:
displayName: A particular audit
get:
description: Get a particular audit
delete:
description: Delete a particular audit
patch:
description: Update a particular audit
/audits:
displayName: Collection of audits
get:
description: Get all audits
post:
description: Insert a audits
body:
application/json:
schema: !include schemas/audits.json
/{id}:
displayName: A particular audit
get:
description: Get a particular audit
delete:
description: Delete a particular audit
patch:
description: Update a particular audit
/tasks:
displayName: Collection of tasks
get:
description: Get all tasks
post:
description: Insert a tasks
body:
application/json:
schema: !include schemas/tasks.json
/{id}:
displayName: A particular task
get:
description: Get a particular task
delete:
description: Delete a particular task
patch:
description: Update a particular taskThink there may be a similar matchdict problem with the AUTH procedure for GET requests for nested items at 4 levels.
A schema example can be found here https://github.com/adam-codeberg/ramses-templates/tree/master/memberships
For example, the API response forgets to include those nested items in the call back, despite _nested_depth:6 and the items can be accessed via a more direct call URL.
Included some info below.
GET :6543/api/contacts/1
"orders": [ { "_pk": "1", "_type": "Order", "_version": 0, "contact": 1, "fk_contact_id": 1, "fk_product_id": "Top hits of 200BC", "id": 1, "payments": [], "price_change": 0 } ],
x2 payments are missing.
GET :6543/api/contacts/1/orders
"data": [
{
"_pk": "1",
"_score": 1.0,
"_self": "http://localhost:6543/api/contacts/1/orders/1",
"_type": "Order",
"_version": 0,
"contact": 1,
"fk_contact_id": 1,
"fk_product_id": "Top hits of 200BC",
"id": 1,
"payments": [
{
"_pk": "1",
"_type": "Payment",
"_version": 0,
"amount": "10",
"fk_order_id": 1,
"id": 1,
"order": 1
},
{
"_pk": "2",
"_type": "Payment",
"_version": 0,
"amount": "10",
"fk_order_id": 1,
"id": 2,
"order": 1
}
],
"price_change": 0
}
],
And calling the item at level 4 fails all together
GET :6543/api/contacts/1/orders/1
{
"explanation": "The server could not comply with the request since it is either malformed or otherwise incorrect.",
"message": null,
"request_url": "http://localhost:6543/api/contacts/1/orders/1",
"status_code": 405,
"timestamp": "2015-10-20T11:23:36Z",
"title": "Method Not Allowed"
}
For now I've capped our JSON design at x3 levels, but would be nice to fix this at some point.
Heres the traceback
2015-10-20 22:27:32,267 DEBUG [txn.139968636479232][Dummy-3] _transaction.__init__: new transaction
2015-10-20 22:27:32,268 DEBUG [urllib3.util.retry][Dummy-3] retry.from_int: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
2015-10-20 22:27:32,269 DEBUG [urllib3.connectionpool][Dummy-3] connectionpool._make_request: "GET /licenseapi/Order/1/_source HTTP/1.1" 200 349
2015-10-20 22:27:32,269 INFO [elasticsearch][Dummy-3] base.log_request_success: GET http://localhost:9200/licenseapi/Order/1/_source [status:200 request:0.001s]
2015-10-20 22:27:32,269 DEBUG [elasticsearch][Dummy-3] base.log_request_success: > None
2015-10-20 22:27:32,269 DEBUG [elasticsearch][Dummy-3] base.log_request_success: < {"price_change": 0, "_version": 0, "fk_product_id": "Top hits of 200BC", "_pk": "1", "contact": 1, "fk_contact_id": 1, "id": 1, "payments": [{"_type": "Payment", "fk_order_id": 1, "_version": 0, "order": 1, "amount": "10", "_pk": "1", "id": 1}, {"_type": "Payment", "fk_order_id": 1, "_version": 0, "order": 1, "amount": "10", "_pk": "2", "id": 2}]}
2015-10-20 22:27:32,270 DEBUG [nefertari_sqla.documents][Dummy-3] documents.get_collection: Get collection: AuthUser, {'username': u'system', '_item_request': True, '_limit': 1, '__raise_on_empty': True}
2015-10-20 22:27:32,277 DEBUG [nefertari_sqla.documents][Dummy-3] documents.get_collection: get_collection.query_set: AuthUser (SELECT ramses_authuser._version AS ramses_authuser__version, ramses_authuser.username AS ramses_authuser_username, ramses_authuser.email AS ramses_authuser_email, ramses_authuser.password AS ramses_authuser_password, ramses_authuser.groups AS ramses_authuser_groups FROM ramses_authuser WHERE ramses_authuser.username = %(username_1)s LIMIT %(param_1)s OFFSET %(param_2)s)
2015-10-20 22:27:32,279 ERROR [nefertari.json_httpexceptions][Dummy-3] json_httpexceptions.create_json_response: 405 METHOD NOT ALLOWED: {"status_code": 405, "explanation": "The server could not comply with the request since it is either malformed or otherwise incorrect.", "message": null, "timestamp": "2015-10-20T11:27:32Z", "title": "Method Not Allowed"}
2015-10-20 22:27:32,279 DEBUG [txn.139968636479232][Dummy-3] _transaction.abort: abort
2015-10-20 22:27:32,280 ERROR [nefertari.json_httpexceptions][Dummy-3] json_httpexceptions.create_json_response: 405 METHOD NOT ALLOWED: {"request_url": "http://localhost:6543/api/contacts/1/orders/1", "timestamp": "2015-10-20T11:27:32Z", "title": "Method Not Allowed", "status_code": 405, "explanation": "The server could not comply with the request since it is either malformed or otherwise incorrect.", "message": null}
2015-10-20 22:27:32,280 DEBUG [nefertari.tweens][Dummy-3] tweens.timing: GET (http://localhost:6543/api/contacts/1/orders/1) request took 0.013090133667 seconds
As for new sqlalchemy supports JSON data type, I'm wondering if ramses supports this type. A quick test shows an error:
/ramses/models.py", line 212, in handle_model_generation
raise ValueError('{}: {}'.format(model_name, str(ex)))
ValueError: User: Unknown type: json
in my schema file I have:
"data": {
"type": ["string", "null"],
"_db_settings": {
"type": "json",
"required": false
}
}
When interacting with nested endpoints they appear to have trouble identifying the parents ID
The result is that the data gets inserted into elasticsearch but not postgre.
Use RAML and Schemas provided here /addressbook/
http --session user1 POST :6543/api/contacts name_title='Mr' name_first='Fred' name_last='Flinstone'
http --session user1 POST :6543/api/destinations type='Work'
http --session user1 POST :6543/api/contacts/1/emails address='[email protected]' fk_destination_type='Work' fk_contact_id='1'2015-10-10 19:46:58,396 DEBUG [txn.140440433747712][Dummy-3] _transaction.__init__: new transaction
2015-10-10 19:46:58,396 DEBUG [nefertari_sqla.documents][Dummy-3] documents.get_collection: Get collection: AuthUser, {'username': u'system', '_item_request': True, '_limit': 1, '__raise_on_empty': True}
2015-10-10 19:46:58,405 DEBUG [nefertari_sqla.documents][Dummy-3] documents.get_collection: get_collection.query_set: AuthUser (SELECT ramses_authuser._version AS ramses_authuser__version, ramses_authuser.username AS ramses_authuser_username, ramses_authuser.email AS ramses_authuser_email, ramses_authuser.password AS ramses_authuser_password, ramses_authuser.groups AS ramses_authuser_groups FROM ramses_authuser WHERE ramses_authuser.username = %(username_1)s LIMIT %(param_1)s OFFSET %(param_2)s)
2015-10-10 19:46:58,409 DEBUG [nefertari_sqla.documents][Dummy-3] documents.get_collection: Get collection: Email, {'_item_request': True, '_limit': 1, '__raise_on_empty': True, 'id': 1}
2015-10-10 19:46:58,418 DEBUG [nefertari_sqla.documents][Dummy-3] documents.get_collection: get_collection.query_set: Email (SELECT email._version AS email__version, email.id AS email_id, email.address AS email_address, email.fk_contact_id AS email_fk_contact_id, email.fk_destination_type AS email_fk_destination_type FROM email WHERE email.id = %(id_1)s LIMIT %(param_1)s OFFSET %(param_2)s)
2015-10-10 19:46:58,422 DEBUG [urllib3.util.retry][Dummy-3] retry.from_int: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
2015-10-10 19:46:58,426 DEBUG [urllib3.connectionpool][Dummy-3] connectionpool._make_request: "POST /_bulk HTTP/1.1" 200 121
2015-10-10 19:46:58,426 INFO [elasticsearch][Dummy-3] base.log_request_success: POST http://localhost:9200/_bulk [status:200 request:0.004s]
2015-10-10 19:46:58,426 DEBUG [elasticsearch][Dummy-3] base.log_request_success: > {"index": {"_type": "Email", "_id": "1", "_index": "licenseapi"}}
{"fk_destination_type": "Work", "_version": 0, "_pk": "1", "contact": null, "fk_contact_id": null, "address": "[email protected]", "id": 1}
2015-10-10 19:46:58,426 DEBUG [elasticsearch][Dummy-3] base.log_request_success: < {"took":3,"errors":false,"items":[{"index":{"_index":"licenseapi","_type":"Email","_id":"1","_version":1,"status":201}}]}
2015-10-10 19:46:58,427 INFO [nefertari.elasticsearch][Dummy-3] elasticsearch._bulk_body: Successfully executed 1 Elasticsearch action(s)
2015-10-10 19:46:58,431 DEBUG [txn.140440433747712][Dummy-3] _transaction.abort: abort
2015-10-10 19:46:58,432 ERROR [nefertari.json_httpexceptions][Dummy-3] json_httpexceptions.create_json_response: 400 BAD REQUEST: {"status_code": 400, "explanation": "The server could not comply with the request since it is either malformed or otherwise incorrect.", "message": "Bad or missing param 'contacts_id'", "timestamp": "2015-10-10T08:46:58Z", "title": "Bad Request"}
STACK BEGIN>>
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/waitress/task.py", line 74, in handler_thread
task.service()
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/waitress/channel.py", line 337, in service
task.service()
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/waitress/task.py", line 173, in service
self.execute()
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/waitress/task.py", line 392, in execute
app_iter = self.channel.server.application(env, start_response)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/paste/urlmap.py", line 216, in __call__
return app(environ, start_response)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/pyramid/router.py", line 242, in __call__
response = self.invoke_subrequest(request, use_tweens=True)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/pyramid/router.py", line 217, in invoke_subrequest
response = handle_request(request)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/nefertari/tweens.py", line 21, in timing
return handler(request)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/nefertari/tweens.py", line 52, in get_tunneling
return handler(request)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/nefertari/tweens.py", line 101, in cache_control
response = handler(request)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/pyramid/tweens.py", line 46, in excview_tween
response = view_callable(exc, request)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/pyramid/config/views.py", line 385, in viewresult_to_response
result = view(context, request)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/nefertari/view.py", line 375, in key_error_view
return JHTTPBadRequest("Bad or missing param '%s'" % context.args[0])
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 78, in __init__
create_json_response(self, **kw)
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 58, in create_json_response
msg += '\nSTACK BEGIN>>\n%s\nSTACK END<<' % add_stack()
File "/home/username/.pyenv/versions/project-name/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 26, in add_stack
return ''.join(traceback.format_stack())
STACK END<<
2015-10-10 19:46:58,434 DEBUG [nefertari.tweens][Dummy-3] tweens.timing: POST (http://localhost:6543/api/contacts/1/emails) request took 0.0384120941162 secondsHave been mashing face against keyboard for most of the day without success.
I've been creating a project using ramses (SQLAlchemy as the backend engine) by following the example project - https://github.com/brandicted/ramses-example - and the tutorial published on realpython.com - https://realpython.com/blog/python/create-a-rest-api-in-minutes-with-pyramid-and-ramses - although I'm stuck with something I see as a simple problem: I've been able to define one to may relationships using the schemas, but I haven't managed to do the same for many to many relationships. Is it possible to define many to many relationships using the schemas in ramses?
I've you've like to check the schemas I want to be related, here they are: https://github.com/Shemahmforash/ThisDayInMusicAPI/blob/develop/thisdayinmusic/schemas/tracks.json and https://github.com/Shemahmforash/ThisDayInMusicAPI/blob/develop/thisdayinmusic/schemas/playlists.json
Thanks a lot for your help.
Quick question,
Is there any existing tooling to filter json views before output with the existing tooling?
Useage example; instances where you have a collection with items and those items have user ownership and you want to hide the items from non-owners when the collection is queried via a GET /api/collection
If so great, but if not I'll look into monkey patching the views.py functions in ramses or something.
reference : ramses-example
step1:
create a user
step2:
create a story for this user
*look at the request
as a result the doc in mongo
step2:
delete this story delete: /stories/{id}
now , although the story is deleted , its still showing up when queried GET: /users/{username}
but the doc in mongodb is updated
database engine: mongodb
cheers
kalyan
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "array",
"items": {
"type": "object",
"properties": {
"orderId": {
"type": "integer"
}
}
}
}
ramses not supporting these type of schemas
Hi I was hoping to get ramses example up and running quickly; but had some problems following the how to. Help greatly appreciated.
python --version
Python 3.5.1
(venv) #( 02/11/16@12:48am )( tim@Tims-MacBook-Pro-2 ):~/Workspace/projects/hansard/api/testapi@master✗✗✗
pserve local.ini
2016-02-11 00:48:20,358 INFO [nefertari_sqla.signals][MainThread] signals.setup_es_signals_for: setup_sqla_es_signals_for: <class 'nefertari_sqla.documents.ESBaseDocument'>
2016-02-11 00:48:20,363 INFO [nefertari][MainThread] __init__.includeme: nefertari 0.6.1
2016-02-11 00:48:20,366 INFO [nefertari.json_httpexceptions][MainThread] json_httpexceptions.includeme: Include json_httpexceptions
2016-02-11 00:48:20,367 INFO [ramses][MainThread] __init__.includeme: Parsing RAML
2016-02-11 00:48:20,375 INFO [ramses][MainThread] __init__.includeme: Starting models generation
2016-02-11 00:48:20,376 INFO [ramses.generators][MainThread] generators.generate_models: Configuring model for route `items`
2016-02-11 00:48:20,380 INFO [ramses.utils][MainThread] utils.resource_schema: Searching for model schema
2016-02-11 00:48:20,380 INFO [ramses.models][MainThread] models.setup_data_model: Generating model class `Item`
2016-02-11 00:48:20,383 INFO [nefertari_sqla.signals][MainThread] signals.setup_es_signals_for: setup_sqla_es_signals_for: <class 'ramses.models.Item'>
2016-02-11 00:48:20,387 INFO [nefertari.elasticsearch][MainThread] elasticsearch.setup: Including Elasticsearch. {'enable_aggregations': 'false', 'chunk_size': 500, 'index_name': 'testapi', 'sniff': 'false', 'enable_polymorphic_query': 'false', 'index.disable': 'false', 'enable_refresh_query': 'false', 'hosts': 'localhost:9200'}
2016-02-11 00:48:20,387 INFO [urllib3.connectionpool][MainThread] connectionpool._new_conn: Starting new HTTP connection (1): localhost
2016-02-11 00:48:20,393 WARNI [elasticsearch][MainThread] base.log_request_fail: HEAD /testapi [status:404 request:0.006s]
2016-02-11 00:48:20,393 ERROR [nefertari.elasticsearch][MainThread] elasticsearch.perform_request:
2016-02-11 00:48:20,435 INFO [elasticsearch][MainThread] base.log_request_success: PUT http://localhost:9200/testapi [status:200 request:0.041s]
2016-02-11 00:48:20,435 INFO [ramses][MainThread] __init__.includeme: Starting server generation
2016-02-11 00:48:20,435 INFO [ramses.generators][MainThread] generators.generate_server: Server generation started
2016-02-11 00:48:20,436 INFO [ramses.generators][MainThread] generators.generate_resource: Configuring resource: `items`. Parent: `root`
2016-02-11 00:48:20,436 INFO [ramses.generators][MainThread] generators.generate_resource: Generating ACL for `items`
2016-02-11 00:48:20,437 INFO [ramses.generators][MainThread] generators.generate_resource: Generating view for `items`
2016-02-11 00:48:20,437 INFO [ramses.generators][MainThread] generators.generate_resource: Creating new resource for `items`
2016-02-11 00:48:20,465 INFO [ramses][MainThread] __init__.includeme: Running nefertari.engine.setup_database
2016-02-11 00:48:20,478 INFO [nefertari_sqla][MainThread] __init__.setup_database: Database does not exist. Creating database at postgresql://localhost:5432/testapi
2016-02-11 00:48:20,852 INFO [nefertari.elasticsearch][MainThread] elasticsearch.setup_mappings: Setting up ES mappings for all existing models
2016-02-11 00:48:20,858 WARNI [elasticsearch][MainThread] base.log_request_fail: PUT /testapi/_mapping/Item [status:400 request:0.006s]
2016-02-11 00:48:20,858 ERROR [nefertari.elasticsearch][MainThread] elasticsearch.perform_request: illegal_argument_exception
2016-02-11 00:48:20,861 ERROR [nefertari.json_httpexceptions][MainThread] json_httpexceptions.create_json_response: 400 BAD REQUEST: b'{"status_code": 400, "message": null, "title": "Bad Request", "explanation": "b\'illegal_argument_exception\'", "extra": {"data": "TransportError(400, \'illegal_argument_exception\', \'Field [_version] is defined twice in [Item]\')"}, "timestamp": "2016-02-11T00:48:20Z"}'
STACK BEGIN>>
File "/Users/tim/Workspace/projects/hansard/api/venv/bin/pserve", line 11, in <module>
sys.exit(main())
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 58, in main
return command.run()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 328, in run
global_conf=vars)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
return loadobj(APP, uri, name=name, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
return context.create()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
**context.local_conf)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/urlmap.py", line 31, in urlmap_factory
app = loader.get_app(app_name, global_conf=global_conf)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
name=name, global_conf=global_conf).create()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
return fix_call(context.object, context.global_conf, **context.local_conf)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/Users/tim/Workspace/projects/hansard/api/testapi/testapi/__init__.py", line 6, in main
config.include('ramses')
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/config/__init__.py", line 755, in include
c(configurator)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/ramses/__init__.py", line 66, in includeme
ES.setup_mappings()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 239, in setup_mappings
es.put_mapping(body=model_cls.get_es_mapping())
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 255, in put_mapping
**kwargs)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/client/indices.py", line 291, in put_mapping
'_mapping', doc_type), params=params, body=body)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/transport.py", line 329, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 60, in perform_request
extra=dict(data=e))
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/json_httpexceptions.py", line 67, in exception_response
return STATUS_MAP[status_code](**kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/json_httpexceptions.py", line 78, in __init__
create_json_response(self, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/json_httpexceptions.py", line 58, in create_json_response
msg += '\nSTACK BEGIN>>\n%s\nSTACK END<<' % add_stack()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/json_httpexceptions.py", line 26, in add_stack
return ''.join(traceback.format_stack())
STACK END<<
Traceback (most recent call last):
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 49, in perform_request
resp = super(ESHttpConnection, self).perform_request(*args, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/connection/http_urllib3.py", line 106, in perform_request
self._raise_error(response.status, raw_data)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, 'illegal_argument_exception', 'Field [_version] is defined twice in [Item]')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 239, in setup_mappings
es.put_mapping(body=model_cls.get_es_mapping())
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 255, in put_mapping
**kwargs)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/client/indices.py", line 291, in put_mapping
'_mapping', doc_type), params=params, body=body)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/elasticsearch/transport.py", line 329, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 60, in perform_request
extra=dict(data=e))
nefertari.json_httpexceptions.JHTTPBadRequest
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/tim/Workspace/projects/hansard/api/venv/bin/pserve", line 11, in <module>
sys.exit(main())
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 58, in main
return command.run()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 328, in run
global_conf=vars)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
return loadobj(APP, uri, name=name, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
return context.create()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
**context.local_conf)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/urlmap.py", line 31, in urlmap_factory
app = loader.get_app(app_name, global_conf=global_conf)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
name=name, global_conf=global_conf).create()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 710, in create
return self.object_type.invoke(self)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
return fix_call(context.object, context.global_conf, **context.local_conf)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/paste/deploy/util.py", line 55, in fix_call
val = callable(*args, **kw)
File "/Users/tim/Workspace/projects/hansard/api/testapi/testapi/__init__.py", line 6, in main
config.include('ramses')
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/pyramid/config/__init__.py", line 755, in include
c(configurator)
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/ramses/__init__.py", line 66, in includeme
ES.setup_mappings()
File "/Users/tim/Workspace/projects/hansard/api/venv/lib/python3.5/site-packages/nefertari/elasticsearch.py", line 241, in setup_mappings
raise Exception(ex.json['extra']['data'])
Exception: TransportError(400, 'illegal_argument_exception', 'Field [_version] is defined twice in [Item]')
Hi.
I have interest to your project. And I want to suggest one thing.
I find your project via elasticsearch blog (this article)
And I try to understand what the project do and how can I get some advantage using the project.
However it's very hard to find docs about the advantage of ramses.
How about describe some reasons why many people should use ramses in README file.
Maybe more people can use ramses more easily.
I went through the tutorials, everything went fine. Great work.
I am wondering if there is an easy way to define a relationship between items of the same schema, something like item and subitems where subitems are also type "item"? Trying this according to the tutorial (and using Postgres) gives an ValueError:
raise ValueError('{}: {}'.format(model_name, str(ex)))
ValueError: Person: Model Item used in relationship is not defined
I am not able to create a scaffold on Python 3.5, OS X 10.11.3.
When I issue the following commands:
$ virtualenv my_project
$ source my_project/bin/activate
$ pip install ramses
$ pcreate -s ramses_starter my_project
It says:
Unavailable scaffolds: ramses_starter
Hi,
A boolean event.fields['field'].new_value is the wrong type in after_create (string instead of boolean).
event.fields['field'].new_value is a boolean in after_create.
http://localhost:6543/api/ressource/1?_m=PATCH&field=true
Field definition:
"field": {
"_db_settings": {
"type": "boolean",
"default": false
}
}@registry.add
def complete_task_if_everything_is_delegated(event):
if 'is_completed' in event.fields:
am_i_boolean = isinstance(event.fields['is_completed'].new_value, bool) # false, should be true
type_of_field = type(event.fields['is_completed'].new_value) # string, should be booleanpcreate is deprecated and all examples i find in the web uses it. is there a way to install ramses_starter using cookiecutter or any other alternative method?
I'm trying to find out if it's possible to use Ramses without using elasticsearch.
I see that both Ramses and Nefertari have alternaitve code paths (e.g. using BaseDocument or ESBaseDocument) but I can't see a high level way to switch.
Thanks!
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.