Code Monkey home page Code Monkey logo

Comments (12)

simonw avatar simonw commented on June 10, 2024

All three of the cancelled ones reported the same almost-complete status:

platform linux -- Python 3.11.1, pytest-7.2.1, pluggy-1.0.0 -- /opt/hostedtoolcache/Python/3.11.1/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/db-to-sqlite/db-to-sqlite
collecting ... collected 19 items

tests/test_db_to_sqlite.py::test_db_to_sqlite[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [  5%]
tests/test_db_to_sqlite.py::test_db_to_sqlite[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 10%]
tests/test_db_to_sqlite.py::test_db_to_sqlite[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 15%]
tests/test_db_to_sqlite.py::test_index_fks[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 21%]
tests/test_db_to_sqlite.py::test_index_fks[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 26%]
tests/test_db_to_sqlite.py::test_index_fks[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 31%]
tests/test_db_to_sqlite.py::test_specific_tables[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 36%]
tests/test_db_to_sqlite.py::test_specific_tables[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 42%]
tests/test_db_to_sqlite.py::test_specific_tables[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 47%]
tests/test_db_to_sqlite.py::test_sql_query[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 52%]
tests/test_db_to_sqlite.py::test_sql_query[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 57%]
tests/test_db_to_sqlite.py::test_sql_query[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 63%]
tests/test_db_to_sqlite.py::test_postgres_schema PASSED                  [ 68%]
tests/test_docs.py::test_readme_contains_latest_help PASSED              [ 73%]
tests/test_fixtures.py::test_fixture_mysql PASSED                        [ 78%]
tests/test_fixtures.py::test_fixture_postgresql PASSED                   [ 84%]
tests/test_redact.py::test_redact[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 89%]
tests/test_redact.py::test_redact[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 94%]
Error: The operation was canceled.

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

Deduped the tests that pass are:

test_db_to_sqlite.py::test_db_to_sqlite
test_db_to_sqlite.py::test_index_fks
test_db_to_sqlite.py::test_postgres_schema                  
test_db_to_sqlite.py::test_specific_tables
test_db_to_sqlite.py::test_sql_query
test_docs.py::test_readme_contains_latest_help              
test_fixtures.py::test_fixture_mysql                        
test_fixtures.py::test_fixture_postgresql                   
test_redact.py::test_redact

When I run locally there's only one test that isn't in that list:

test_postgres_schema

This test here:

db-to-sqlite/tests/test_db_to_sqlite.py

Lines 108 to 122 in a2ae06e

@pytest.mark.skipif(psycopg2 is None, reason="pip install psycopg2")
def test_postgres_schema(tmpdir, cli_runner):
db_path = str(tmpdir / "test_sql.db")
connection = POSTGRESQL_TEST_DB_CONNECTION
result = cli_runner(
[connection, db_path, "--all", "--postgres-schema", "other_schema"]
)
assert result.exit_code == 0
db = sqlite_utils.Database(db_path)
assert db.tables[0].schema == (
"CREATE TABLE [other_schema_categories] (\n"
" [id] INTEGER PRIMARY KEY,\n"
" [name] TEXT\n"
")"
)

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

The SQL for creating that other schema looked odd to me:

db-to-sqlite/tests/conftest.py

Lines 133 to 139 in a2ae06e

CREATE SCHEMA other_schema
CREATE TABLE IF NOT EXISTS other_schema_categories (
id int not null primary key,
name varchar(32) not null
);
INSERT INTO other_schema.other_schema_categories (id, name) VALUES (1, 'Other junk');

Shouldn't that have a semicolon after CREATE SCHEMA other_schema?

The tests have been passing for a couple of years so apparently that worked OK before though.

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

Weird: this workaround got 3.10 to pass but still seems to hang for 3.9 and 3.11.

image

https://github.com/simonw/db-to-sqlite/actions/runs/4109240953/jobs/7090852122

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

Still hanging for 3.9 and 3.11: https://github.com/simonw/db-to-sqlite/actions/runs/4109280459/jobs/7090938290

It gets this far still:

tests/test_redact.py::test_redact[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 89%]
tests/test_redact.py::test_redact[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 94%]

It looks like the last of the series of test_redact tests might be hanging - that's the second PostgreSQL one that uses postgres:// as the connection string instead of postgresql://:

db-to-sqlite/tests/shared.py

Lines 39 to 45 in e722d7e

# Make sure it works with postgres:// connection strings too
pytest.param(
POSTGRESQL_TEST_DB_CONNECTION.replace("postgresql://", "postgres://"),
marks=pytest.mark.skipif(
psycopg2 is None, reason="pip install psycopg2"
),
),

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

Maybe more likely though is that the code that tears down the PostgreSQL test servers is hanging for some reason.

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

Urgh. Removing that postgres:// thing didn't fix the problem either.

https://github.com/simonw/db-to-sqlite/actions/runs/4109349112/jobs/7091083066

platform linux -- Python 3.8.16, pytest-7.2.1, pluggy-1.0.0 -- /opt/hostedtoolcache/Python/3.8.16/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/db-to-sqlite/db-to-sqlite
collecting ... collected 14 items

tests/test_db_to_sqlite.py::test_db_to_sqlite[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [  7%]
tests/test_db_to_sqlite.py::test_db_to_sqlite[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 14%]
tests/test_db_to_sqlite.py::test_index_fks[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 21%]
tests/test_db_to_sqlite.py::test_index_fks[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 28%]
tests/test_db_to_sqlite.py::test_specific_tables[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 35%]
tests/test_db_to_sqlite.py::test_specific_tables[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 42%]
tests/test_db_to_sqlite.py::test_sql_query[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 50%]
tests/test_db_to_sqlite.py::test_sql_query[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 57%]
tests/test_db_to_sqlite.py::test_postgres_schema PASSED                  [ 64%]
tests/test_docs.py::test_readme_contains_latest_help PASSED              [ 71%]
tests/test_fixtures.py::test_fixture_mysql PASSED                        [ 78%]
tests/test_fixtures.py::test_fixture_postgresql PASSED                   [ 85%]
tests/test_redact.py::test_redact[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 92%]
Error: The operation was canceled.

There's still something which is causing the tests to hang towards the end of the run.

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

Still the same problem: https://github.com/simonw/db-to-sqlite/actions/runs/4109380335/jobs/7091148981

Even skipping the test_redact() test - so it's clearly not that test that is causing the problem:

platform linux -- Python 3.9.16, pytest-7.2.1, pluggy-1.0.0 -- /opt/hostedtoolcache/Python/3.9.16/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/db-to-sqlite/db-to-sqlite
collecting ... collected 19 items

tests/test_db_to_sqlite.py::test_db_to_sqlite[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [  5%]
tests/test_db_to_sqlite.py::test_db_to_sqlite[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 10%]
tests/test_db_to_sqlite.py::test_db_to_sqlite[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 15%]
tests/test_db_to_sqlite.py::test_index_fks[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 21%]
tests/test_db_to_sqlite.py::test_index_fks[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 26%]
tests/test_db_to_sqlite.py::test_index_fks[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 31%]
tests/test_db_to_sqlite.py::test_specific_tables[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 36%]
tests/test_db_to_sqlite.py::test_specific_tables[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 42%]
tests/test_db_to_sqlite.py::test_specific_tables[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 47%]
tests/test_db_to_sqlite.py::test_sql_query[mysql://[email protected]:3306/test_db_to_sqlite] PASSED [ 52%]
tests/test_db_to_sqlite.py::test_sql_query[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 57%]
tests/test_db_to_sqlite.py::test_sql_query[***127.0.0.1:5432/test_db_to_sqlite] PASSED [ 63%]
tests/test_db_to_sqlite.py::test_postgres_schema PASSED                  [ 68%]
tests/test_docs.py::test_readme_contains_latest_help PASSED              [ 73%]
tests/test_fixtures.py::test_fixture_mysql PASSED                        [ 78%]
tests/test_fixtures.py::test_fixture_postgresql PASSED                   [ 84%]
tests/test_redact.py::test_redact[mysql://[email protected]:3306/test_db_to_sqlite] SKIPPED (https://github.com/simonw/db-to-sqlite/issues/47) [ 89%]
tests/test_redact.py::test_redact[***127.0.0.1:5432/test_db_to_sqlite] SKIPPED (https://github.com/simonw/db-to-sqlite/issues/47) [ 94%]
Error: The operation was canceled.

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

https://pybit.es/articles/pytest-timeout/ suggests using pytest-timeout and running it with these options in order to get a detailed traceback:

pytest --timeout=3 --timeout_method=thread

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

One of the MySQL tests failed with the timeout!

https://github.com/simonw/db-to-sqlite/actions/runs/4109415108/jobs/7091221231

  File "/home/runner/work/db-to-sqlite/db-to-sqlite/tests/conftest.py", line 171, in setup_mysql
    cursor.execute("DROP DATABASE {};".format(bits.database))
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)

+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++
Error: Process completed with exit code 1.

Looks like it failed during an attempt to drop the database.

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

The PostgreSQL fixture doesn't actually bother trying to drop the database - I've removed that from the MySQL one too.

from db-to-sqlite.

simonw avatar simonw commented on June 10, 2024

Yup, the MySQL teardown change fixed it.

My best guess is there was a transaction left open which was blocking the DROP DATABASE from running - absolutely no idea why that would only affect some versions of Python and not others though.

from db-to-sqlite.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.