Integration sample with SqlAlchemy ? about pytest-factoryboy HOT 15 CLOSED

Here is what ended up working for me with pytest-flask-sqlalchemy:

Session = scoped_session(
    lambda: current_app.extensions["sqlalchemy"].db.session,
    scopefunc=lambda: current_app.extensions["sqlalchemy"].db.session,
)

class BaseFactory(factory.alchemy.SQLAlchemyModelFactory):
    class Meta:
        abstract = True
        sqlalchemy_session = Session

I suggest adding a new subclass of SQLAlchemyModelFactory that contains this setting and plugs automatically into pytest-flask-sqlalchemy without any extra configuration.

from pytest-factoryboy.

hello @olegpidsadnyi , sorry took me a while to respond, i was hooked up in my day job. But for what it's worth.

The primary reason behind the whole session fixture is for me to be able to rollback my transaction (even for session.commit() ) for every test and not needing to re-create and drop the whole database every test. For that i followed alex's blog post, but as mentioned in his blog, it has some caveats, for Flask-SQLAlchemy v2.0 has some issues with it's SignallingSession, see this stackoverflow post and pull request 168. To resolve this problem I did alex's suggestion and sub-classed the SignallingSession, see code below:

# database.py

from flask.ext.sqlalchemy import SQLAlchemy, SignallingSession, SessionBase


class SessionWithBinds(SignallingSession):
    """This extends the flask-sqlalchemy signalling session so that we may
    provide our own 'binds' argument.

    See https://github.com/mitsuhiko/flask-sqlalchemy/pull/168
    Also http://stackoverflow.com/a/26624146/2475170

    """
    def __init__(self, db, autocommit=False, autoflush=True, **options):
        #: The application that this session belongs to.
        self.app = db.get_app()
        self._model_changes = {}
        #: A flag that controls whether this session should keep track of
        #: model modifications.  The default value for this attribute
        #: is set from the ``SQLALCHEMY_TRACK_MODIFICATIONS`` config
        #: key.
        self.emit_modification_signals = \
            self.app.config['SQLALCHEMY_TRACK_MODIFICATIONS']
        bind = options.pop('bind', None) or db.engine

        # Our changes to allow a 'binds' argument
        binds = options.pop('binds', None)
        if binds is None:
            binds = db.get_binds(self.app)

        SessionBase.__init__(
            self, autocommit=autocommit, autoflush=autoflush,
            bind=bind, binds=binds, **options
        )


class TestFriendlySQLAlchemy(SQLAlchemy):
    """For overriding create_session to return our own Session class"""
    def create_session(self, options):
        return SessionWithBinds(self, **options)


db = TestFriendlySQLAlchemy()

and then I reference this in my app's __init__.py

# app/__init__.py
from flask import Flask

from config import config

# instead of using the default `from flask.ext.sqlalchemy import SQLAlchemy()`
# we use an extended version which is more friendlier to tests
from .database import db as _db

db = _db


def create_app(config_name):
    """Create application using the given configuration.

    Application factory which takes as an argument the name of the
    configuration to use for the application. It then returns the
    created application.

    Args:
      config_name (string): name of the configuration.

    Returns:
      Flask: the created application.

    """
    app = Flask(__name__)
    app.config.from_object(config[config_name])
    config[config_name].init_app(app)

    db.init_app(app)  # <--- as you've mentioned. =)
    ...
    return app

Although this whole SignallingSession problem has been fixed already with flask-sqlalchemy/pull/249 , it wont probably be released yet untill the whole 2.1 milestone has been completed.

I tried your first recommendation, but using pytest.set_trace(), I can see that the session fixture uses a different sqlachemy session instance as with the one factory boy uses (i checked using model_factory._meta.sqlalchemy_session).

I also tried your second recommendation, but I always encounter an error with using outside the application/request context. Probably a problem on where I placing the get_session function.

But nevertheless, somehow I was able to make factory boy use the same session instance with my session fixture by encapsulating each factory class inside a pytest fixture which injects my session fixture. see example below:

@pytest.fixture(scope='function')
def model_factory(session):
    Class ModelFactory(factory.alchemy.SQLAlchemyModelFactory):
        Class Meta:
            model = model
            sqlalchemy_session = session
        name = factory.Sequence(lambda n: "Model %d" % n)
   return ModelFactory


def test_session_instance(session, model_factory):
   assert session == model_factory._meta.sqlalchemy_session

Don't know though if this is a good solution.

EDIT: I retried your first recommendation, I think problem lies on how or on when I import factories, If I import the factories inside the test function itself, it will have the same session instance.

# app/factories/__init__.py

import factory
from factory.alchemy import SQLAlchemyModelFactory

from app import models, db

class SomeModelFactory(SQLAlchemyModelFactory):
    class Meta:
        model = models.SomeModel
        sqlalchemy_session = db.session

    name = factory.Sequence(lambda n: "SomeModel %d" % n)

# tests/test_some_model.py

def test_session_instance(session):
   from app.factories import SomeModelFactory
   assert session == SomeModelFactory._meta.sqlalchemy_session  # this will pass

of course to avoid having to do imports inside the test function, I made a fixture which imports the whole factories modules itself and return it.

@pytest.fixture(scope='function')
def factories(session, request):
    import app.factories as _factories

    return _factories

EDIT: Looks like the above import strategy will only be true for the first time or test. I did some re-reading on a few blogs and stack posts, and I ended up with the second recommendation on this stack post, I probably didn't pay attention to it that much before because I was using in-memory or sqlite database for the tests.

@pytest.fixture(scope='function')
def session(db, request):
    db.session.begin_nested()

    def teardown():
        db.session.rollback()
        db.session.close()

    request.addfinalizer(teardown)
    return db.session

then i just declare and import my factories as just with the documentation.

from pytest-factoryboy.

To improve on @revmischa solution, I just used db.create_scoped_session provided by Flask-SQLAlchemy:

class FlaskSQLAlchemyModelFactory(factory.alchemy.SQLAlchemyModelFactory):
    """Connects factory meta session to a pytest-flask-sqlalchemy scoped session."""

    class Meta:  # noqa: D106
        abstract = True
        sqlalchemy_session = db.create_scoped_session()

Turns out db.create_scoped_session would be always the same -- leading to all sort of errors on a second run. I decided to follow the suggested solution. I did, however, use pytest-flask-sqlalchemy mocked-sessions feature:

from selfsolver.models import db

class FlaskSQLAlchemyModelFactory(factory.alchemy.SQLAlchemyModelFactory):
    """Connects factory meta session to a pytest-flask-sqlalchemy scoped session."""

    class Meta:  # noqa: D106
        abstract = True
        sqlalchemy_session = scoped_session(
            lambda: db.session, scopefunc=lambda: db.session
        )
        sqlalchemy_session_persistence = "commit"

Then in pytest.ini:

[pytest]
mocked-sessions = selfsolver.models.db.session

And it works perfectly. I did try @asduj approach and it also works. Since I can't really tell the difference, I went with the smaller solution.

from pytest-factoryboy.

Hi!
If I understand right what you are trying to achieve: You initialize the database in the fixture because you want a special test db?
I don't think you need to do all this db.create_scoped_session(options=options) etc. There's a package that provides SQLAlchemy integration into Flask - https://pythonhosted.org/Flask-SQLAlchemy/ I hope you are using it.
You should have lazy db somewhere globally defined.
db = SQLAlchemy() - Not yet attached to the app.
You can use it to define your models and it won't break the import.

Finally when you create your app you can initialize that db with your app, assuming all db settings are in the application config.

def create_app():
    app = Flask(__name__)
    db.init_app(app)  # <--------HERE
    return app

It will create internally all the necessary things like scoped session. See - https://github.com/mitsuhiko/flask-sqlalchemy/blob/master/flask_sqlalchemy/__init__.py#L735

This means you can import your globally defined lazy "db" and use db.session - which is a scoped session (threading local scoped session).

factory-boy has integration with SQLAlchemy. It understands scoped session in the Meta class of the factory. We use our own base class to attach all our factory classes to the right scoped session.

from factory.alchemy import SQLAlchemyModelFactory
from myproject.db import db

class ModelFactory(SQLAlchemyModelFactory):

    """Base model factory."""

    class Meta:
        abstract = True
        sqlalchemy_session = db.session

Then you can subclass it and in theory it should work.

There's also a trick with any function that returns session.
Even in your case you still can use current_app accessor from Flask, since your fixture pushes your app into the context.

So you can define some function like

from flask import current_app

def get_my_session():
    return current_app.extensions['sqlalchemy'].db.session

Then you can wrap this function into scoped_session and feed it to factory-boy. It will still be lazy, won't break your imports and will let you setup the database connection.

from sqlalchemy.orm.scoping import scoped_session
from factory.alchemy import SQLAlchemyModelFactory

class ModelFactory(SQLAlchemyModelFactory):

    """Base model factory."""

    class Meta:
        abstract = True
        sqlalchemy_session = scoped_session(get_my_session)

from pytest-factoryboy.

@chavz Did it work for you?

from pytest-factoryboy.

If FactoryBoy/factory_boy#391 is merged we could integrate with pytest-sqlalchemy and provide instances that initialize models from the session provided in the fixture.

from pytest-factoryboy.

FYI

def test_foo(dbsession, foo_facotry):
    foo_facotry._meta.sqlalchemy_session = dbsession
    org = organization_facotry.create()

Also works just fine.

from pytest-factoryboy.

@thedrow i don't particularly like the idea that _meta parameter is in the model's attribute values, just with the funny name that you probably won't use as an attribute name for your model.

Why did you need another session per factory in the first place? Could you describe your use case?
Are you using 2 sessions in your test code at the same time and why?

from pytest-factoryboy.

No I want to share the same session I created from the pytest fixture with the factories. In order to do that I need to override the session factoryboy uses.

from pytest-factoryboy.

@thedrow in this case you didn't need any pull request to factoryboy itself. I think it made it worse because of the parameter pollution.
The _meta.sqlalchemy_session is a lazy callable in the first place.
You could just create an autouse fixture to setup your session as a fixture, that will modify some global variable that is returned by that lazy callable (or defaults to your default settings).

I'd revert that PR in this case. Pytest shouldn't affect how factoryboy works IMO

from pytest-factoryboy.

I don't need or want a global session object.

from pytest-factoryboy.

sessionmaker is a factory. so it can return an object that is "global" within your pytest request session.
Which is the case in case of the dbsession fixture.

from pytest-factoryboy.

@thedrow are you defining the base factory class?

from sqlalchemy.orm.scoping import scoped_session
from factory.alchemy import SQLAlchemyModelFactory

class ModelFactory(SQLAlchemyModelFactory):
    """Base model factory."""

    class Meta:
        abstract = True
        sqlalchemy_session = scoped_session(<Some callable here that returns a result of your fixture>)

from pytest-factoryboy.

No I haven't done that.

from pytest-factoryboy.

Thank you all for valuable comments.

As ScopedRegistry use scopefunc as a key:

    def __call__(self):
        key = self.scopefunc()
        try:
            return self.registry[key]
        except KeyError:
            return self.registry.setdefault(key, self.createfunc())

It would be better to pass the default implementation:

Session = scoped_session(
    lambda: current_app.extensions["sqlalchemy"].db.session(),
    scopefunc=lambda: current_app.extensions["sqlalchemy"].db.session().has_key,
)

from pytest-factoryboy.