Code Monkey home page Code Monkey logo

Comments (3)

simonw avatar simonw commented on June 12, 2024

image

from datasette-upload-csvs.

simonw avatar simonw commented on June 12, 2024

The problem is that any exceptions occur inside this block, which is run by db.execute_write_fn(insert_docs, block=False) and hence can't interrupt a current page request.

datasette-upload-csvs/datasette_upload_csvs/__init__.py

Lines 92 to 125 in 8127afc

def insert_docs(conn):
database = sqlite_utils.Database(conn)
# TODO: Support other encodings:
reader = csv_std.reader(codecs.iterdecode(csv.file, "utf-8"))
headers = next(reader)
docs = (dict(zip(headers, row)) for row in reader)
i = 0
def docs_with_progress():
nonlocal i
for doc in docs:
i += 1
yield doc
if i % 10 == 0:
database["_csv_progress_"].update(
task_id,
{
"rows_done": i,
"bytes_done": csv.file.tell(),
},
)
database[filename].insert_all(docs_with_progress(), alter=True, batch_size=100)
database["_csv_progress_"].update(
task_id,
{
"rows_done": i,
"bytes_done": total_size,
"completed": str(datetime.datetime.utcnow()),
},
)
return database[filename].count

Instead, it should write something to the _csv_progress_ table such that the polling JavaScript can detect that an error has occurred.

from datasette-upload-csvs.

simonw avatar simonw commented on June 12, 2024

Now this happens, and after you click OK the page reloads.

CleanShot 2022-06-09 at 12 19 55@2x

from datasette-upload-csvs.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.