Maybe this is a good idea since generic resources are now quite opinionated and one might want to utilise serializers even without generic resources.

This should be obviously a string:
https://github.com/swistakm/graceful/blob/master/src/graceful/fields.py#L183

There is mixed code style:

• some classes inherit explicitly from object and some not when in python3 there is no such need for explicit object inheritance
• use of old-style super(SomeClass, self) call when new simpler super() is available

So decide if what to use and why. If decision is to keep things similar to py2 style consider changing your mind and making this package python2.6/2.7 compatible!

Verify this

There are few major issues with current validation error representation.

1. Validation on bulk resource creation will not tell which object in whole batch did not pass validation.
2. Validation of fields with many=True quits on first invalid value and does not allow to find out if all values were incorrect or only few of them. Also if not all, we don't know which ones.
3. ValueError exceptions are treated equally with ValidationError exceptions. This is great because allows for easier integration with built-in or custom types but on the other hand can leak some implementation details to the user agent. It is equally bad as presenting stack trace to user agent. We should treat ValidationError exceptions as explicit errors that can be safely presented to the user, but ValueError exceptions should be replaced with generic and safe information in type of "Could not parse" or "Invalid format".

Now it is markdown but should be converted to reStructuredText. Fortunately this does not require additional release AFAIK.

This is an enchancement that will surely break compatibility but it will be a solution to problem with deciding which method should have access to which of this basic data structures.

Because there is always single resource instance on given route no state can be stored in this object between method calls. This means that all required data must be always passed to each method that requires it. Because of that graceful needs to make very opinionated decision about what is available on each step of processing the request. Although this helps developer to focus on what is important in specific method handler (e.g., retrieve, list, create in generic views) it is very hard to extend if something non usual needs to be done - often it requires overriding of many methods.

We could also consider if this context could be passed to serialisers so more complex things could be done like: field serialisation that depends on query string parameters.

This will ensure that additional validation on whole resource level can be easily added to serialiser.

When this is done there should be also documentation extended with information how to perform such validation.

The best way I think is to add similar as_bad_request method to ValidationError

In some situations when we have many=True option on parameter provided it might be useful to provide a custom container type/class or method for grouping multiple instances.

Now every param class has only one method handler for single value param. Thanks to this implementation of parameter is short and consistent regardless of the many option setting. If multiple parameters are allowed then we always use list to contain provided parameters. Custom containers would be useful for additional preprocessing of params.

Here is example how this would work on real world example that allows joining multiple occurrences of solrq.Q object with specified operator (solrq is example external package):

from graceful.params import StringParam

import operator
from functools import reduce

class FilterQueryParam(StringParam):
    """
    Param that represents Solr filter queries logically 
   joined together depending on value of `op` argument
    """
    def __init__(
            self,
            details,
            solr_field,
            op=operator.and_,
            **kwargs
    ):
        if solr_field is None:
            raise ValueError("{} needs a `field` param cannot be None".format(
                self.__class__.__name__)
            )

        self.solr_field = solr_field
        self.op = op

        super(FilterQueryParam, self).__init__(
            details, **kwargs
        )

    def value(self, raw_value):
        return Q({self.solr_field: raw_value})

    def container(self, values):
        return reduce(self.op, values) if len(values) > 1 else values[0]

With such API whenever many is set to true then .container() will be fed with list of values got from multiple calls to .value() method handler. Additionally there can be an additional container_class keyword argument stored as Param attribute on initialisation (default container=list) that will be used by default implementation of container method:

class BaseParam(object):
    ...

    def container(self, values):
       return self.container_class(values)

In this way this feature can be used even on existing basic Param classes without the need of creating custom param definitions.

It is very useful feature to allow bulk creation and/or update of objects represented by resources. Initial idea is to add PATCH handler to ListCreateAPI class. Maybe also CreateMixin should be modified to allow bulk creation.

The rationale behind using PATCH is that this method is used to partially modify resource. List resources represent collections of objects, so passing a list of new representations to such resource may be considered as intention to add/modify them within existing collection.

The PUT method could be understood as intention to replace whole collection in list resource. Usefulness of that is questionable but it may be explored in future.

The important thing that needs to be discussed is the question if we should separate the bulk creation from bulk update or leave that to the user. In my opinion default "upsert-like" behaviour is more flexible. Developer may easily decide how to handle his case when writing his storage integration code. Anyway this should be decided before implementing because this will affect API naming and documentation.

Generally having access to falcon's req and resp parameters is a nice feature to have when you want to read/manipulate headers or provide some custom per-request context value. Typical use case for per-request context variables are database session/connection objects that cannot be stored as resource instance attributes (needs to be unique for every request).

These are usually implemented as middlewares that are able to update req.context context dictionary. Unfortunately when using generic API resource classes (e.g. ListCreateAPI, PaginatedListAPI) user is expected to provide only basic resource manipulation methods like list(), retrieve(), create() etc. These methods accept only following set of arguments:

• params: dictionary of deserialised keyword parameters (defined using resource-level param classes)
• meta: dictionary with additional metadata that will be included in output response
• validated (only on create/update handlers): deserialised resource representation provided by client in request body
• **kwargs: additional keyword arguments retrieved from falcon's URI template associated to resource route.

Because basic resource manipulation methods (list/retrieve/create etc.) accept only specific set of arguments and none of them represents falcon.Request object. The only place where custom context can be provided is **kwargs dictionary that was primarily supposed to only hold keyword arguments retrieved from URI template.

Currently available approach - overriding HTTP method handlers

Right now, the simplest way to provide custom context to every request is to directly override falcon's HTTP method handlers in custom resource classes (inherited from generic and basic resources classes) and pass them further in **kwargs dictionary using super(). Example:

# assume Session is some db layer abstraction

def MyResource(ListCreateAPI):
    def on_get(req, resp, **kwargs):  # <= overrides default HTTP  GET handler
        super().on_get(req, resp, db_session=Session(), **kwargs)

   def list(req,req, db_session, **kwargs):  # <= called by super().on_get()
       return db_session.all()

    def on_post(req, resp, **kwargs):  # <= overrides default HTTP  POST handler
        super().on_post(req, resp, custom_var="some_value", **kwargs)

    def create(req,req, validated, db_session, **kwargs):  # <= called by super().on_patch()
        return db_session.insert(validated)

Of course this is very impractical because every method handler that requires such additional context needs to be overridden. Also **kwargs are now only expected to hold only values from URI template and their purpose is documented exactly as URI template keyword values.

Currently available approach - using middleware/hooks

Overriding every HTTP method handler in exactly the same way is not a clean approach if specific context is required by every API endpoint. This will require a lot of redundant and messy code. One could reduce amount of boilerplate by providing custom base classes (based on graceful generics) with HTTP method handlers overridden to provide additional context. This will still require a lot of code duplication and will be hard to maintain in the long term.

In many frameworks (including falcon) the custom context is usually provided using two methods:

• use global middleware that is able to provide custom context for every request/response in the application: it is fine if same context is needed in every API endpoint. Usually great for things like db sessions/connections, or user sessions.
• use per-method or per-resource hooks: implemented in similar fashion to middleware but offer more fine-grained control and can be attached to specific resources or resource's HTTP methods (usually using decorators).

Both middleware and hooks from falcon can be used in graceful resources but their usage is very limited. The only parts of above features that could be used to provide context are:

• process_resource(self, req, resp, resource, params) method of middleware class: the params dictionary is the save object unpacked as **kwargs keyword arguments. We cannot use anything else because req & resp are unavailable in basic resource manipulation handlers (list/create/retrieve etc.) and the resource instance can be shared between multiple worker threads.
• action argument of falcon.before(action) decorator with signature of action(req, resp, resource, params): same as for middlewares -- only params parameter that translates directly to **kwargs can be used as a container for new context values.

Additionally usage of hooks in graceful is limited even further. They can be attached to whole resource:

@falcon.before(context_action)
def MyResource(ListCreateAPI):
    pass

But cannot be easily attached to specific resource manipulation method. The falcon.before() expects decorated function to have signature of a HTTP method handler (i.e. on_get(req, resp, **kwargs), on_post(req, resp, **kwargs) and so on). Due to this the only way to attach falcon.before hooks right now is through following boilerplate.

def MyResource(ListCreateAPI):
    @falcon.before(action)
    def on_get(req, resp, **kwargs):
        super().on_get(req, resp, **kwargs)

    @falcon.before(action)
    def on_post(req, resp, **kwargs):
        super().on_post(req, resp, **kwargs)

So it is too verbose and also counterintuitive. Note that compatibility with falcon hooks is another feature we would like to have so we could support any falcon contrib package that provides hooks. This anyway should be discussed as a separate issue (see #31 )

Summary

Existing ways of handling custom context values are too verbose, require too much boilerplate and generally exploit **kwargs dictionary of list/retrieve/update/create methods that has completely different purpose in both falcon and graceful.

I in my opinion the best approach would be to expose somehow the Request.context object in the basic resources. The best solution should:

• Solve all the problems of existing approaches (preferably by supporting every method explained earlier)
• Allow to provide context in the most convenient way to the user. So it should not favour any of the existing approaches.
• Do not introduce any backwards incompatibility (best) or at least be an opt-in feature with only minor backwards incompatibility when toggled (still fine).

Simply rethink how this should be handled. Maybe simply creating new param classes is a way to go or we should follow the same concept as it is done in fields.

Research how other frameworks handle that. Mostly Django REST Framework

Restrict "many" field handling to lists in similar way we handle serialization.

Since we are breaking backwards compatibility in 1.0.0 anyway, it is time to revisit some class and function names.

• make param names for basic Python types consistent
• make field names for basic Python types consistent
• remove Base suffixes from class names of serializers, resources, fields etc.

It would simplify some usecases like allowing CORS preflights on authorized resources.

While, I have been searching around contract-first development approach, I encountered OpenApi Generator. Basically, it can create client and server boilerplate codes for a given api spec. After generation process, only thing that needs to be done is implement the controller methods themselves.

Do we think of contributing to OpenApi Generator project for a new generator of graceful?

I did not analyze it deeply, but what I understand after a quick look is that, it can be pretty straightforward to generate serializers and controllers, as their structure will already be defined in the contract.

This may move the developer experience of graceful to an another level?

What do you guys think about it? Is it worth to give a shot?

Cheers.

I run on this from time to time. In most cases nested resource can be handled by using custom fields but they cannot be trivially validated/serialized/deserialized. Also those new fields classes cannot be reused as unique resource types so in some cases it would be useful to have support for nesting resource definitions.

I see two approaches:

1. wrapper field class (like SerializerField) that can work like a single field and delegates validation and (de)serialisation to inner serialised stored as a attribute. Pros: more explicit, allows both types of objects to have different apis. Cons: feels like more complex and also will require more coding to define API that uses nested resources.
2. make BaseSerializer a descendant of BaseField class. This should also work but will require serialisers and fields to have the same api. I'm afraid that this will make auto-describing of API more painful. Pros: shorter API descriptions. Cons: fields and serialisers have slightly different purposes so such coupling may introduce difficulties in future.

TestBase from falcon.testing is now deprecated. We should rewrite the tests to pytest style in order to make graceful more future-proof and all tests more consistent.

Decide which approach is better:

• additional classes for fields that support null/blank
• additional parameter(s) arguments(s) in BaseField class that support null/blank
• additional field class wrapper that adds such possibility (like: field = AllowNull(StringField("foo'))

Because we do not plan to use Python2 we can mark all keyword arguments as keyword-only (PEP 3102). This will make easier to extend existing base field functionality and custom field classes.

Wrapping them could be easier then.

Falcon development is pretty fast recently. Make sure we are up to date. Make testing a real matrix with different versions of python and different versions of falcon.

Edit: also Python 3.5 is available fro quite long time so make sure we cover it in tests too.

Usage of mixin classes may be a bit confusing. Extend the guide/resources.rst to provide introduction to mixins and how to use them with BaseResource class.

Additional tasks related with this issue:

• If #33 gets merged, add some brief information about context-aware classes and point to proper document for more detailed description on this topic.
• Decide on how to properly call all the retrieve/update/list/etc. methods and do cleanup in the docs to unify this terminology
• Put a small glossary for the users and doc writers somewhere

I defined a on_put method, 50% of the time the method is not allowed.

This is where I add the routes:

class EndpointExpositor(object):
    """Exposes the endpoints, divided in endpoints for data and metadata. The
    endpoints for data are those defined in the challenge description. Endpoints
    for metadata are all others.
    """

    def __init__(self, falcon_api, titulo_tesouro_crud):
        self.falcon_api = falcon_api
        print()
        print(dir(falcon_api))
        print()

        titulo_tesouro_request_handler = TituloTesouroRequestHandler(titulo_tesouro_crud)

        self.endpoint_mapping = {
            '/': None,
            '/titulo_tesouro': titulo_tesouro_request_handler,
            '/titulo_tesouro/{titulo_id}': titulo_tesouro_request_handler
        }

        endpoints = list(self.endpoint_mapping.keys())
        self.endpoint_mapping['/'] = HelpRequestHandler(endpoints)

    def expose(self):
        for (endpoint, handler) in self.endpoint_mapping.items():
            self.falcon_api.add_route(endpoint, handler)
            logging.info('Endpoint "{}" exposed.'.format(endpoint))

        logging.info('All endpoints exposed.')

This is the base handler:

class RequestHandler(object):
    """Superclass for all request handlers.
    """

    def on_get(self, req, resp):
        logging.info('GET request received at endpoint "{}"'.format(req.path))

    def on_post(self, req, resp):
        logging.info('POST request received at endpoint "{}"'.format(req.path))

    def on_delete(self, req, resp):
        logging.info('DELETE request received at endpoint "{}"'.format(req.path))

    def on_put(self, req, resp):
        logging.info('PUT request received at endpoint "{}"'.format(req.path))

And this is the child class:

class TituloTesouroRequestHandler(RequestHandler):
    """Handler for POST in endpoint "titulo_tesouro"
    """

def on_put(self, req, resp, titulo_id):
        super().on_put(req, resp)

        print('test')

I am using Gunicorn 19.7.0. Am I doing something wrong? Shouldn't the method be allowed once I have a on_put defined?

This is also true for forbidden and invalid fields

Now it's always JSON but we should be able to support more in a convenient way.

It happens in forbidden section.

Now graceful shamelessly states that this is "documentation-centered falcon REST toolkit" but it actually does not have any tool to generate such documentation.
It is true that everything that needs to be documented has describe() method and resources always respond to OPTIONS requests with some resource definition dict. It makes very easy to put such definition into some html template.

I already use my own documenter that does it's work. It is quite opinionated so I need to consider how it should be shared. Most likely this will be a separate repo. I think about using namespace package inside of graceful like 'graceful.extras'. This will be also a convenient way for future developers to create their own extensions in future so there will be no mess with naming like this is in some mature frameworks like django where many of packages have different install/import names and different naming conventions.

todo: more detailed description

We should allow parameters to be specified muliple times in query string like:

/some/endpoint/?q=something&q=else

This should then create list of q strings in params dict

Example header value:

Content-Type: application/json; charset=UTF-8

Hi,

I have sent both an empty email and a normal email to graceful at librelist.com but nothing happens

Hi, recently I found this repository searching some code about authentication and authorization of requests. Please tell me if you are going to support this feature in the future. Some code about it could help me a lot.

Consider following example:

class CategoriesList(PaginatedListAPI()):
    def list(params, meta, uri_template_variable, **kwargs):
        return [uri_template_variable] * 5

api.add_route("/v0/categories/{uri_template_variable}", ListResource())

Will raise exception on accessing /v0/categories/{ui_template_variable} and this is design inconsistency with other endpoint classes:

TypeError: list() missing 1 required positional argument: 'uri_template_variable'

This is very common feature in many similar serialization solutions. I see two possible approaches:

1. Providing decorator function that can turn serialize'rs method to dynamic field class object.
2. Providing new field class that is bound to the serializer instance.

Decorator approach

Decorator approach is the simplest to implement because does not affect serializers code and does not introduce any backwards incompatible changes.

def unbound_method_field(method):
    class CallField(fields.BaseField):
        def __init__(self):
            super().__init__(
                method.__doc__,
                label=method.__name__,
                read_only=True,
                source='*',
            )

        def to_representation(instance):
            return method(instance)

    return CallField()

class MySerializer(serializers.BaseSerializer):
    @unbound_method_field
    def my_field_times_10(instance):
        return instance['my_field'] * 10

The problem is that we cannot use real methods but only unbound functions defined in class namespace. This is due to how serializer's metaclass works. This creates a bit counterintuitive definitions within class instance. We could use a bit more complex implementation:

def bound_method_field(method):
    class CallField(fields.BaseField):
        def __init__(self):
            super().__init__(
                method.__doc__,
                label=method.__name__,
                read_only=True,
                source='*',
            )

        def to_representation(self, instance):
            return method(self, instance)

    return CallField()

class MySerializer(serializers.BaseSerializer):
    @bound_method_field
    def my_field_times_10(instance):
        return instance['my_field'] * 10

But this will be even more counter intuitive because such method will be "bound" to the field instance and not serializer instance.

Pros:

• ✅ Simple implementation without any backwards incompatible changes.

Cons:

• ❌ Such fields could be "read only" only.
• ❌ Will not work properly with methods defines with classmethod and staticmethod dictionaries
• ❌ Counterintuitive method definitions
• ❌ Overriding other field options like source, name, label and validators etc. would require more complex decorator
• ❌ Need to use star-like sources in 0.x graceful version
• ❌ It will make serializers harder to read and understand because some fields will be defined as "methods".

New field class type

We could take an example from Django REST Framework and Marshmallow projects and make all field instances bound to the serializer instance on BaseSerializer.fields property access. In order to ensure proper binding of inherited fields from base serializer classes we would have also to perform deep copy of all that fields. This would not have a negative impact on performance once we cache fields dictionary items (see: #54).

Binding fields to the parent is a bit easier in DRF because their serializers are initialized per-object instance. Serializers in graceful are more static and stateless object translators. Still, I believe it can be implemented in similar way.

Pros:

• ✅ It allows for more complex handling of custom serializer methods.
• ✅ Serializer method fields can be both "source aware" or work on whole resources.
• ✅ Once fields are bound to serializer class we can accept methods both for writes and reads.
• ✅ Minimal implementation would only consist of obligatory dynamic binding the fields to serializer instance. Once it is done user can implement their own serializer-bound method field implementations.
• ✅ It will work seamlessly with both staticmethod and classmethod.

Cons:

• ❌ It requires careful fields property handling on first access to ensure no performance impact.
• ❌ More complex change that requires changes in serializer metaclass.
• ❌ It requires a copy.deepcopy() call. Also dynamic field manipulation once serializer is initialized will be limited.
• ❌ Parent serializer will be added to the field dynamically. This will make code more complex and harder to read.
• ❌ We will have to take some extra precautions in order to ensure proper method is called in complex inheritance scenarios.

Related work:

• Django REST Framework: method that deep-copies serializer's fields
• Django REST Framework: property that caches serializer's field on first access
• Django REST Framework: BindingDict class that ensures fields are always bound to the serializer instance
• Marshmallow: deepcopy() during schema initialization
• Marshmallow: fiels to schema binding
• Marshmallow: actual binding implementation

Side note: the other advantage of approach based on Marsmallow's code is that we could finally pass fields name in the serializer during binding. Then fields would know their names and serializers would not have to wonder if field has its own source specified (see 1.x code). Still I believe that current API of read_instance makes instance processing a bit more performant and it is worth leaving as it is. We could even try to cache field sources inside of serializers namespace.

Summary

It seems that field binding is currently the best approach even if it results in more complex serializer's code. It should land in 1.0.0 release if we want to implement this. It is not that invasive change but at implementing it in 0.x would later require major changes in actual new field classes. We can optionally introduce this change gradually:

• implement obligatory field binding in 0.x
• add new field classes in 1.0.0 or in later minor releases of 1.x

self-explanatory

We need to define how this should be handled. The easiest approach is to assume that fields with source="*" argument should return dictionary that will be added to the object_dict instance. Anyway this will make sense only with custom serializer fields. So maybe it would be better completely remove "*" option from built-in parameters and maybe delegate set/get-attr/key responsibility to custom fields?

Server SHOULD include such header if this is applicable (see: RFC 2616 Section 9.2)

As noted in #27 PR default values are deserialised and validated every time they are retrieved and this could be obviously improved. Consider getting rid of that limitation or making default value deserialised only once on init.

Let's assume we have a resource as implemented below:

import falcon
from graceful.resources.generic import PaginatedListCreateAPI
from graceful.serializers import BaseSerializer
from graceful.fields import StringField


class TestSerializer(BaseSerializer):
    testStringField = StringField("Test String Field", source='test')


class TestList(PaginatedListCreateAPI):
    serializer = TestSerializer()

    def create(self, params, meta, validated, **kwargs):
        if validated.get('test') == "None":
            print("Is it a bug?")


app = application = falcon.API()

app.add_route("/v1/test", TestList())

When client sends a POST request with the following body:

{
  "testStringField": null
}

Then the test field in the validated become "None" instead of None.

Proposal:

It can be a solution to check if data is None in the from_representation
method of the StringField.

def from_representation(self, data):
        """Convert representation value to ``str`` if it is not None."""
        return str(data) if data is not None else None

We could use annotations feature in some parts of framework:

• annotating field value type
• annotating param value type

Instead of using class level attributes. This is only idea to consider.

Main focus of this issue is the reference documentation (docstrings in the source code).

This should be done using Sphinx paragraph level markup:

.. versionchanged:: version

.. versionadded:: version