Unable to run unit tests using tortoise-orm initializer() with a postgres database about tortoise-orm HOT 6 CLOSED

I think to understand error you first need to understand what exactly initialiser doing:

1. It connects to default database
2. Runs create database for given database name (if there is {} in database name - it will format it with random uuid)
3. Populate this DB with schemas generated from your models
4. Connects to newly created db

Your error occurs because it's connection to default db failed, as it not exists. For most of postgres clients default db equals to user name. If your user was postgres it would connect to postgres db that usually exist, but in your case there no db with same name as user

So you have few options to workaround:

1. create database your_postgres_user
2. use user postgres
3. override PGDATABASE with another database name
4. do not use initializer/finalizer at all, and manage database yourself

Note that default database can't be equal to database you want to run your tests against, as you can't create/delete database you are already connected to

from tortoise-orm.

Thanks for the pointers! The fix I made is different though so I will try to explain here (maybe it will help someone else using Fastapi with tortoise-orm and trying to write unit tests like me)

Finally got it to work changing the way I used the FastApi TestClient.
Earlier I was using it as described here: https://fastapi.tiangolo.com/tutorial/testing/#using-testclient

client = TestClient(app)

This gave me the exception:

self = <tortoise.router.ConnectionRouter object at 0x7f4c3d82a380>, model = <class 'src.models.Account'>, action = 'db_for_read'

    def _router_func(self, model: Type["Model"], action: str):
>       for r in self._routers:
E       TypeError: 'NoneType' object is not iterable

Now I switched to using it so that startup and shutdown events get triggered as described in https://fastapi.tiangolo.com/advanced/testing-events/

with TestClient(app) as client:

This helped and my tests are running now. They run fine whether I use RegisterTortoise with lifespan or register_tortoise but with the deprecation at FastApi, the better approach is to use RegisterTortoise with lifespan as mentioned by @abondar

In 0.21 version we introduced new way to register Tortoise with fastapi -
tortoise.contrib.fastapi.RegisterTortoise
You can try it - may be it will change something

from tortoise-orm.

tortoise.exceptions.DBConnectionError: Can't establish connection to default database. Verify environment PGDATABASE. Exception: database "your_postgres_user" does not exist

Does your environment has PGDATABASE variable?

from tortoise-orm.

no

from tortoise-orm.

Thanks for the explanation @abondar . I have changed the user to postgres but now I get asyncpg.exceptions.ObjectInUseError: cannot drop the currently open database even though nothing else is using the database.

Can you explain the 4th option? "do not use initializer/finalizer at all, and manage database yourself"

My fastapi application is set up using register_tortoise() function and I create a TestClient for it to run unit tests. But when any of the routes being tested read/write using a tortoise Model, I get this error:

/usr/local/lib/python3.10/site-packages/tortoise/models.py:1273: in get
    db = using_db or cls._choose_db()
/usr/local/lib/python3.10/site-packages/tortoise/models.py:1010: in _choose_db
    db = router.db_for_read(cls)
/usr/local/lib/python3.10/site-packages/tortoise/router.py:36: in db_for_read
    return self._db_route(model, "db_for_read")
/usr/local/lib/python3.10/site-packages/tortoise/router.py:31: in _db_route
    return connections.get(self._router_func(model, action))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <tortoise.router.ConnectionRouter object at 0x7f4c3d82a380>, model = <class 'src.models.Account'>, action = 'db_for_read'

    def _router_func(self, model: Type["Model"], action: str):
>       for r in self._routers:
E       TypeError: 'NoneType' object is not iterable

/usr/local/lib/python3.10/site-packages/tortoise/router.py:18: TypeError

This is the config dict I am passing to register_tortoise:

{'connections': {'master': {'engine': 'tortoise.backends.asyncpg', 'credentials': {'host': 'localhost', 'port': '5432', 'user': 'postgres', 'password': 'your_postgres_password', 'database': 'your_postgres_db'}}, 'slave': {'engine': 'tortoise.backends.asyncpg', 'credentials': {'host': 'localhost', 'port': '5432', 'user': 'postgres', 'password': 'your_postgres_password', 'database': 'your_postgres_db'}}}, 'apps': {'models': {'models': ['src.models', 'aerich.models'], 'default_connection': 'master'}}, 'routers': ['src.models.Router'], 'use_tz': False, 'timezone': 'UTC'}

Router definition:

from tortoise.models import Model
from typing import Type

class Router:
    def db_for_read(self, model: Type[Model]):
        print(model, "db_for_read")
        return "slave"

    def db_for_write(self, model: Type[Model]):
        print(model, "db_for_write")
        return "master"

These problems are all faced during unit test runs only. When I run the application and test each endpoint via swagger manually everything works perfectly with no changes in code or configuration. However, I cannot run manual tests every time so it is imperative for me to get unit tests running or else I will have to switch back to other ORM.

from tortoise-orm.

I have changed the user to postgres but now I get asyncpg.exceptions.ObjectInUseError: cannot drop the currently open database even though nothing else is using the database.

Can you try changing your database in to something like
asyncpg://postgres:your_postgres_password@localhost:5432/test_{}?
To be sure, that db is always freshly created and not reused by anyone

That how we run tests internally and it works, at least here

My fastapi application is set up using register_tortoise()

In 0.21 version we introduced new way to register Tortoise with fastapi -
tortoise.contrib.fastapi.RegisterTortoise
You can try it - may be it will change something

Here how we use it in example
https://github.com/tortoise/tortoise-orm/blob/develop/examples/fastapi/main.py#L25

There is also simple test configuration for fast api in example
https://github.com/tortoise/tortoise-orm/blob/develop/examples/fastapi/_tests.py

But overall error you are getting indicates that for some reason tortoise wasn't properly initialised, because if it was - tortoise.router.ConnectionRouter.init_routers should have been called and you wouldn't be having None here.

For diagnostics - you can check in debugger if Tortoise._init is set to true before your failing .get() call

Can you explain the 4th option? "do not use initializer/finalizer at all, and manage database yourself"

It just means that you create database and migrate it to needed state beforehand, using some external tools or manually, and then you run tests
tortoise.contrib.fastapi.RegisterTortoise also provides option generate_schemas which may help preparing test env

from tortoise-orm.