Code Monkey home page Code Monkey logo

guillotina's Introduction

Introduction

image

image

Test Coverage

Python Versions

image

License

Chat

Docker Cloud Build Status

Please read the detailed docs

This is the working project of the next generation Guillotina server based on asyncio.

Dependencies

  • Python >= 3.7
  • PostgreSQL >= 9.6

Quickstart

We use pip

pip install guillotina

Run PostgreSQL

If you don't have a PostgreSQL server to play with, you can run one with Docker.

Download and start the Docker container by running

make run-postgres

Run the server

To run the server

g

Then...

curl http://localhost:8080

Or, better yet, use Postman to start playing with API.

You can also navigate in your Guillotina server with its built-in web admin interface by visiting http://localhost:8080/+admin/.

Deploy on Heroku

Read more Guillotina-Heroku.

image

Getting started with development

Using pip (requires Python > 3.7)

git clone [email protected]:plone/guillotina.git
cd guillotina
python3.7 -m venv .
./bin/pip install -r requirements.txt
./bin/pip install -r contrib-requirements.txt
./bin/pip install -e '.[test]'
./bin/pre-commit install

Run tests

We're using pytest

./bin/pytest guillotina

and for test coverage

./bin/pytest --cov=guillotina guillotina/

With file watcher...

./bin/ptw guillotina --runner=./bin/py.test

To run tests with cockroach db

USE_COCKROACH=true ./bin/pytest guillotina

Default

Default root access can be done with AUTHORIZATION header : Basic root:root

Docker

You can also run Guillotina with Docker!

First, run PostgreSQL

docker run --rm \
    -e POSTGRES_DB=guillotina \
    -e POSTGRES_USER=guillotina \
    -p 127.0.0.1:5432:5432 \
    --name postgres \
    postgres:9.6

Then, run Guillotina

docker run --rm -it \
    --link=postgres -p 127.0.0.1:8080:8080 \
    plone/guillotina:latest \
    g -c '{"databases": [{"db": {"storage": "postgresql", "dsn": "postgres://guillotina:@postgres/guillotina"}}], "root_user": {"password": "root"}}'

This assumes you have a config.yaml in your current working directory

Chat

Join us to talk about Guillotina at https://gitter.im/plone/guillotina

guillotina's People

Contributors

ableeb avatar bloodbare avatar cdevienne avatar datakurre avatar davisagli avatar dependabot[bot] avatar ebrehault avatar ericof avatar esteele avatar frapell avatar gforcada avatar hirokiky avatar inakip avatar jordic avatar jotare avatar karannaoh avatar ksuess avatar lferran avatar masipcat avatar nazrulworld avatar nilbacardit26 avatar pfreixes avatar psanlorenzo avatar qiwn avatar rboixaderg avatar sneridagh avatar svx avatar tisto avatar vangheem avatar waghanza avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

guillotina's Issues

Unable to start guillotina==1.5.2 with default configuration:asyncpg.exceptions.InvalidAuthorizationSpecificationError: role "guillotina" does not exist

  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/guillotina/commands/__init__.py", line 118, in make_app
    return make_app(settings=settings, loop=self.get_loop())
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/guillotina/factory/app.py", line 212, in make_app
    loop.run_until_complete(future)
  File "/opt/buildout.python/parts/opt/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/guillotina/db/factory.py", line 56, in PGDatabaseConfigurationFactory
    return await _PGConfigurationFactory(key, dbconfig, app)
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/guillotina/db/factory.py", line 44, in _PGConfigurationFactory
    await aps.initialize(loop=app.loop, **connection_options)
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/guillotina/db/storages/pg.py", line 428, in initialize
    **kw)
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/asyncpg/pool.py", line 339, in _async__init__
    await first_ch.connect()
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/asyncpg/pool.py", line 123, in connect
    **self._connect_kwargs)
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/asyncpg/connection.py", line 1378, in connect
    max_cacheable_statement_size=max_cacheable_statement_size)
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/asyncpg/connect_utils.py", line 298, in _connect
    connection_class=connection_class)
  File "/Users/ajung/src/guillotina/lib/python3.6/site-packages/asyncpg/connect_utils.py", line 274, in _connect_addr
    await asyncio.wait_for(connected, loop=loop, timeout=timeout)
  File "/opt/buildout.python/parts/opt/lib/python3.6/asyncio/tasks.py", line 352, in wait_for
    return fut.result()
asyncpg.exceptions.InvalidAuthorizationSpecificationError: role "guillotina" does not exist

Quickstart documentation issue

http://guillotina.readthedocs.io/en/latest/quickstart.html

(guillotina) ajung@Andreass-MBP ~/src/guillotina $ ./bin/guillotina create configuration
Could not find the configuration file config.yaml. Using default settings.
usage: guillotina [-h] [-c CONFIGURATION] [--debug] --template
                  {configuration,application} [-w] [-n] [-o OUTPUT]
                  command
guillotina: error: the following arguments are required: --template

Changing security without any catalog engine should not reindex

Exception on writing execution
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/guillotina/async.py", line 58, in initialize
view_result = await view()
File "/usr/local/lib/python3.6/site-packages/guillotina/api/search.py", line 93, in call
await search.reindex_all_content(self.context, self._security_reindex)
AttributeError: 'NoneType' object has no attribute 'reindex_all_content'

finish off zope.*

All we have left is zope.(configuration|component). We'll leave zope.interface in.

Implement swagger API explorer

Additionally, for improved docs experience...

  • sass package
  • auth with google, facebook, etc
  • be able to create temporary container
  • guillotina_sass
  • then, use guillotina_swagger with that container to explore API for developers to easily be able to explore the api.

Cockroach support - tid conflicts

We are experiencing lots of conflicts errors without the rediscache module. The tid allocation does seems to be enough accurate or fast:

  File "/usr/local/lib/python3.6/site-packages/guillotina/factory/app.py", line 84, in _handle
    return await super()._handle(request)
  File "/usr/local/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/local/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/local/lib/python3.6/site-packages/guillotina/traversal.py", line 229, in handler
    await commit(request, warn=False)
  File "/usr/local/lib/python3.6/site-packages/guillotina/transactions.py", line 22, in commit
    await get_tm(request).commit(request)
  File "/usr/local/lib/python3.6/site-packages/guillotina/db/transaction_manager.py", line 100, in commit
    return await shield(self._commit(request=request, txn=txn))
  File "/usr/local/lib/python3.6/site-packages/guillotina/db/transaction_manager.py", line 109, in _commit
    await txn.commit()
  File "/usr/local/lib/python3.6/site-packages/guillotina/db/transaction.py", line 276, in commit
    return await self._commit()
  File "/usr/local/lib/python3.6/site-packages/guillotina/db/transaction.py", line 290, in _commit
    await self.tpc_commit()
  File "/usr/local/lib/python3.6/site-packages/guillotina/db/transaction.py", line 345, in tpc_commit
    await self._store_object(obj, oid)
  File "/usr/local/lib/python3.6/site-packages/guillotina/db/transaction.py", line 331, in _store_object
    await self._manager._storage.store(oid, serial, writer, obj, self)
  File "/usr/local/lib/python3.6/site-packages/guillotina/db/storages/cockroach.py", line 262, in store
    oid, txn, old_serial, writer)
guillotina.exceptions.TIDConflictError: Mismatch of tid of object being updated. This is likely caused by a cache invalidation race condition and should be an edge case. This should resolve on request retry..```

Cockroach version: 1.0.3
Guillotina version: 2.3.17

@items pagination is broken

The @items service filters out the result based on permissions.
The pagination is made prior to filtering, leading to exotic paging information in the final result.

doc generator

Be able to generate documentation on the api automatically

Error on cors not defined properly

Unhandled exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 398, in start
    await resp.prepare(request)
AttributeError: 'HTTPUnauthorized' object has no attribute 'prepare'```

DB api improvements proposals

This issue gather the ideas we talked about during the Costa-Brava sprint.

Transactions

Simple transactions

Simplify the transaction opening api. It should always be a one line operation.

txn = await tm.begin()
await txn.commit()

If the tm is associated with a request, the it should be passed to the
transaction, which removes any need to explicitely pass the request.

Request default transaction

It the transaction is to become the default transaction of a request, the
eventual container of the request should be moved to the new transaction,
or refetched, and the transaction be set as the request._txn:

txn = await request.begin()
await txn.commit()

Also, if the transaction is the default request transaction, commiting it
should call execute_futures() automatically.

Context manager

A transaction manager for handling transactions could be provided (it
might already be the case):

async with transaction(tm):
    pass

Flush

In some cases it is needed to apply all changes to the database without closing
the transactions. There is currently no way to do that with the db api.

The proposal is to add a flush() function to Transaction, that does what
tpc_commit does (except for commit itself of course). This function would be
called automatically on commit.

async txn.flush()

Path-based index

Problems

When a object is added, or its path changed, the navigation api (async_get,
navigate_to) will not 'see' it until the changes are flushed to the database.

Implementing flush somehow fix the problem but at a double cost:

  • the user must remember to flush before navigating
  • a same object my be loaded/modified/flushed many times in a single transaction.

Another problem with the current implementation is that a same object can be
loaded several times as different instances, and loose all consistency between
those instances.

Proposal

The idea is to add a path-based index of all the objects that were loaded.

This index would be used by the content api (async_get etc.), and achieve
complete consistency of the objects loaded in a transaction.

The index would use weakref so that the memory footprint is as low as possible
(we don't want to keep an object loaded when its not used anywhere).

Unify command interface

Instead of multiple bin scripts, use one script to kick off registered commands(like django).

Example: bin/guillotina serve
Example: bin/guillotina shell
etc

Improve docs

  • use type hinting for programming API, auto docs
  • fix documenting programming api(read the docs)
  • docs for interfaces, auto doc
  • touch up addon docs
  • document tus API
  • document all endpoints(like swagger but not needing swagger)
  • auth framework, how to extend, configure and use
  • document how to write your own configurator
  • document component features(getUtility, getAdapter)
  • document extending cloud storage
  • document useful applications
  • document serialization and validation(un-included field validation) framework
  • document available schema fields
  • document security functions
  • document request object
  • document annotations
  • document registry
  • document renderers
  • document testing
  • document transaction helper functions
  • utils
  • directives

Marketing

"The full stack data framework"

  1. Fast and scalable out of the box: scalability/performance/asyncio/micro services
  2. Simplicity with batteries included, "the right amount of batteries", cors, es, etc, extensible
  3. Securely organized data

Why Guillotina

  • Performance: Traditional python web servers limit the number of simultaneous
    requests to the number of threads running the server. With AsyncIO, you are
    able to server many simultaneous requests at the same time.
  • Front-end friendly: Guillotina is designed to make your
    JavaScript engineers happy. With things like automatic swagger documentation
    for endpoints, OOTB CORS and websockets, your front-end team will be happy
    to work with Guillotina. We speak JSON but can adapt with any Content-Type
    payload request/response bodies.
  • AsyncIO: With AsyncIO, websockets are simple. More interestingly, AsyncIO
    is an ideal match with micro-service architectures.
  • Object model: Guillotina uses a hierarchial object model. This hierarchy
    of objects then maps to URLs. The hierarchy model is perfect for managing
    a large number of objects.
  • Security: Guillotina has a granular, hierarchial, multi-demensional
    security system that allows you to manage the security of your content
    at a level not available to other frameworks.
  • Scale: With integrations like Redis, ElasticSearch and Cockroach, you
    have the tools to scale.

no body in DELETE call

The HTTP spec does not (really) allow to post a body in a DELETE call.

So it might be blocked by some HTTP servers, or might not be supported by client/frameworks http implementation.

  • fix DELETE in the behaviors endpoint
  • fix DELETE in the addons endpoint

Wrong Content-Type with a typical browser Accept header

With the following 'Accept' request header :

Accept: text/html,*/*

The response Content-Type is "text/html". It think its wrong, it should be application/json:

  • The actual content remains application/json, it is not transformed to valid html
  • */* clearly allow the server to return something else than text/html
  • Even if application/json is not in Accept (and maybe in the absence of text/javascript), the returned content-type should still be application/json, unless of course the content can be rendered as one of the accepted content types.

PUT on resource

At the moment (as far I can see), to modify an existing resource, the only option is a PATCH.

The problem is we cannot delete existing attributes or behaviors by patching.

So I guess PUT would be useful.

InMemory cache. Scale Down

Right now, guillotina is super stable and scales up well.
But there are use cases where you only want to deploy a single guillotina service, and with this there are additional missing features, like the object cache. (We know, we can use the redis one, but doesn't make sense to deploy a new service neither all the invalidations pubsub feature).
The main idea is to provide just an in-process lru-cache, that fills this gap.
I'm not sure if this should go to an additional package or just implement a new InMemory cache strategy.
Ideas, thoughts?

Version conflict error

I am trying to run buildout from master branch after pull latest.
Unfortunately got version conflict error, coming from buildout.

............................................................
...............................................................
Version and requirements information containing urllib3:
  Requirement of aio_etcd: urllib3>=1.7.1
  Requirement of requests!=2.11.0,!=2.12.2,!=2.18.0,>=2.5.2: urllib3<1.22,>=1.21.1
  Requirement of python-etcd: urllib3>=1.7.1
While:
  Installing pytest.
Error: There is a version conflict.
We already have: urllib3 1.22
but requests 2.18.1 requires 'urllib3<1.22,>=1.21.1'.

I could fix this problem by pinning urillib3 version of 1.21.1 into buildout.cfg, but may be that is not the best solution.
Looking forward :)

guillotina user api service for POST

I don't see any api service for user to create new (user), may I am wrong?
What is the plan about this, will it get new package i.e guillotina.user.

pytest auto reloader

would be really nice if dev environment gave you a pytest auto reloader when code changed

Permissions based on groups are inconsistent

When a user has multiple groups and permissions on different levels are assigned to different groups, some permissions get ignored.

Given a user with the following groups: 'group1', 'group2'.
If group1 is assigned AccessContent on the container with AllowSingle, and group2 is assigned AccessContent with Allow on a child node, the user will not be able to access the child node or the container node (randomly it seems).

The following test reproduces the issue:
orus-io@ed87c4c

ASyncUtility constructor is called before app_settings is ready

When a ASyncUtility constructor is called, the app_settings global variable is not yet updated with the application settings.

This is because it is called when the components of an application are loaded (which is done at https://github.com/plone/guillotina/blob/master/guillotina/factory/app.py#L207), the application settings are not yet applied to the global app_settings (L219 in the same function).

I raised the issue on gitter and @vangheem concluded that load_application should be splitted in two parts:

  • part 1, load settings
  • part 2, load components

The global app_settings update would be done between the two parts.

Supporting versions of an API

How would I implement several versions (aspects) of an API with the containerish architecture of Guillotina?

Let's say I have a container with content and put a service on it. This is straight forward: containter/@service

Now what I want is container/v1/@service and container/v2/@otherservice.

Putting the version name in the service name (like container/@v1_service) does not work in all situations because there might be legacy names I can't/want to change.

Does sound adding a noop Container with the version as the name and access the items from its parent sound feasable, or this there another recommended way to do it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.