Code Monkey home page Code Monkey logo

flask-excel's Introduction

Flask-Excel - Let you focus on data, instead of file formats

image

image

image

image

image

image

image

image

image

image

Support the project

If your company has embedded pyexcel and its components into a revenue generating product, please support me on github, patreon or bounty source to maintain the project and develop it further.

If you are an individual, you are welcome to support me too and for however long you feel like. As my backer, you will receive early access to pyexcel related contents.

And your issues will get prioritized if you would like to become my patreon as pyexcel pro user.

With your financial support, I will be able to invest a little bit more time in coding, documentation and writing interesting posts.

Known constraints

Fonts, colors and charts are not supported.

Nor to read password protected xls, xlsx and ods files.

Introduction

Here is a typical conversation between the developer and the user:

User: "I have uploaded an excel file"
      "but your application says un-supported file format"
Developer: "Did you upload an xlsx file or a csv file?"
User: "Well, I am not sure. I saved the data using "
      "Microsoft Excel. Surely, it must be in an excel format."
Developer: "OK. Here is the thing. I were not told to support"
           "all available excel formats in day 1. Live with it"
           "or delay the project x number of days."

Flask-Excel is based on pyexcel and makes it easy to consume/produce information stored in excel files over HTTP protocol as well as on file system. This library can turn the excel data into a list of lists, a list of records(dictionaries), dictionaries of lists. And vice versa. Hence it lets you focus on data in Flask based web development, instead of file formats.

The idea originated from the common usability problem: when an excel file driven web application is delivered for non-developer users (ie: team assistant, human resource administrator etc). The fact is that not everyone knows (or cares) about the differences between various excel formats: csv, xls, xlsx are all the same to them. Instead of training those users about file formats, this library helps web developers to handle most of the excel file formats by providing a common programming interface. To add a specific excel file format type to you application, all you need is to install an extra pyexcel plugin. Hence no code changes to your application and no issues with excel file formats any more. Looking at the community, this library and its associated ones try to become a small and easy to install alternative to Pandas.

The highlighted features are:

  1. excel data import into and export from databases
  2. turn uploaded excel file directly into Python data structure
  3. pass Python data structures as an excel file download
  4. provide data persistence as an excel file in server side
  5. supports csv, tsv, csvz, tsvz by default and other formats are supported via the following plugins:
A list of file formats supported by external plugins
Package name Supported file formats Dependencies

pyexcel-io

csv, csvz1, tsv, tsvz2

pyexcel-xls

xls, xlsx(read only), xlsm(read only)

xlrd, xlwt

pyexcel-xlsx xlsx openpyxl

pyexcel-ods3

ods

pyexcel-ezodf, lxml

pyexcel-ods ods odfpy
Dedicated file reader and writers
Package name Supported file formats Dependencies
pyexcel-xlsxw xlsx(write only) XlsxWriter
pyexcel-libxlsxw xlsx(write only) libxlsxwriter
pyexcel-xlsxr xlsx(read only) lxml
pyexcel-xlsbr xlsb(read only) pyxlsb
pyexcel-odsr read only for ods, fods lxml
pyexcel-odsw write only for ods loxun
pyexcel-htmlr html(read only) lxml,html5lib
pyexcel-pdfr pdf(read only) camelot

Plugin shopping guide

Since 2020, all pyexcel-io plugins have dropped the support for python version lower than 3.6. If you want to use any python verions, please use pyexcel-io and its plugins version lower than 0.6.0.

Except csv files, xls, xlsx and ods files are a zip of a folder containing a lot of xml files

The dedicated readers for excel files can stream read

In order to manage the list of plugins installed, you need to use pip to add or remove a plugin. When you use virtualenv, you can have different plugins per virtual environment. In the situation where you have multiple plugins that does the same thing in your environment, you need to tell pyexcel which plugin to use per function call. For example, pyexcel-ods and pyexcel-odsr, and you want to get_array to use pyexcel-odsr. You need to append get_array(..., library='pyexcel-odsr').

Other data renderers
Package name Supported file formats Dependencies Python versions

pyexcel-text

write only:rst, mediawiki, html, latex, grid, pipe, orgtbl, plain simple read only: ndjson r/w: json

tabulate

2.6, 2.7, 3.3, 3.4 3.5, 3.6, pypy

pyexcel-handsontable handsontable in html handsontable same as above

pyexcel-pygal

svg chart

pygal

2.7, 3.3, 3.4, 3.5 3.6, pypy

pyexcel-sortable sortable table in html csvtotable same as above

pyexcel-gantt

gantt chart in html

frappe-gantt

except pypy, same as above

Footnotes

This library makes information processing involving various excel files as easy as processing array, dictionary when processing file upload/download, data import into and export from SQL databases, information analysis and persistence. It uses pyexcel and its plugins:

  1. to provide one uniform programming interface to handle csv, tsv, xls, xlsx, xlsm and ods formats.
  2. to provide one-stop utility to import the data in uploaded file into a database and to export tables in a database as excel files for file download.
  3. to provide the same interface for information persistence at server side: saving a uploaded excel file to and loading a saved excel file from file system.

Tested Flask Versions

Since 2020 Auguest, latest pyexcel-io version 0.6.0 forced Flask-Excel to use Python 3.6+

image

image

For older Flask versions and lower Python version, please use pyexcel-io<=0.5.20.

Installation

You can install Flask-Excel via pip:

$ pip install Flask-Excel

or clone it and install it:

$ git clone https://github.com/pyexcel-webwares/Flask-Excel.git
$ cd Flask-Excel
$ python setup.py install

Usage

Here are some example codes:

from flask import Flask, request, jsonify
import flask_excel

app=Flask(__name__)
flask_excel.init_excel(app)

@app.route("/upload", methods=['GET', 'POST'])
def upload_file():
    if request.method == 'POST':
        return jsonify({"result": request.get_array(field_name='file')})
    return '''
    <!doctype html>
    <title>Upload an excel file</title>
    <h1>Excel file upload (csv, tsv, csvz, tsvz only)</h1>
    <form action="" method=post enctype=multipart/form-data>
    <p><input type=file name=file><input type=submit value=Upload>
   </form>
    '''

@app.route("/export", methods=['GET'])
def export_records():
    return excel.make_response_from_array([[1,2], [3, 4]], "csv",
                                          file_name="export_data")

if __name__ == "__main__":
    app.run()

Development guide

Development steps for code changes

  1. git clone https://github.com/pyexcel/Flask-Excel.git
  2. cd Flask-Excel

Upgrade your setup tools and pip. They are needed for development and testing only:

  1. pip install --upgrade setuptools pip

Then install relevant development requirements:

  1. pip install -r rnd_requirements.txt # if such a file exists
  2. pip install -r requirements.txt
  3. pip install -r tests/requirements.txt

Once you have finished your changes, please provide test case(s), relevant documentation and update CHANGELOG.rst.

Note

As to rnd_requirements.txt, usually, it is created when a dependent library is not released. Once the dependecy is installed (will be released), the future version of the dependency in the requirements.txt will be valid.

How to test your contribution

Although nose and doctest are both used in code testing, it is adviable that unit tests are put in tests. doctest is incorporated only to make sure the code examples in documentation remain valid across different development releases.

On Linux/Unix systems, please launch your tests like this:

$ make

On Windows systems, please issue this command:

> test.bat

Before you commit

Please run:

$ make format

so as to beautify your code otherwise travis-ci may fail your unit test.

License

New BSD License


  1. zipped csv file

  2. zipped tsv file

flask-excel's People

Contributors

antsiferovbogdan avatar bmwasaru avatar chfw avatar fuhrysteve avatar howl-anderson avatar luismsoares avatar waynetech avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flask-excel's Issues

How can I make the query result collection incrementally join the file stream of the flask-excel interface

How can I make the query result collection incrementally join the file stream of the flask-excel interface or other methods to solve the error situation that the query result is too large and the memory is insufficient?

When I query Mysql for a number of hundreds of thousands, tens of columns of data by flask-sqlalchemy, I will be in the query.all () due to insufficient memory, the process is terminated

code 137 (interrupted by signal 9 sigkill)

I find some ways:
https://stackoverflow.com/questions/1145905/sqlalchemy-scan-huge-tables-using-orm
https://github.com/sqlalchemy/sqlalchemy/wiki/WindowedRangeQuery
There are some methods that can be tried, but I have not found a function that flask-excel can handle incrementally.

If I don’t describe enough, I can add details.

Flasky integration

Hi jaska,
(i am flask beginner) and i have one question:

after your fix (bad request) works your scripts fine...

But when i can integrate it to my flask application based on Miguel Grinberg`s Flasky (https://github.com/miguelgrinberg/flasky/commits/master?page=2), after upload CSV file, app show error :"AttributeError: 'Request' object has no attribute 'get_array'"

(Maybe its trivial probem, but with my skill of Flask, i dont know, how i doing wrong...)

Can you help me?

Thank you.
Lukas

Export Template

Does this package support exporting HTML tables (jinja template) to excel? It would be a nice feature to have. Sometimes we might just want to design the table layout and it is easy to design in HTML.

Interoperability with uWSGI?

I'm filing this issue to see if others have successfully used this package with uWSGI as I ran into some issues using this package with uWSGI. Specifically, excel.make_response_from_array(....) would return None. Responses were fine when using the default flask development server.

It's entirely possible that this is due to some server configuration issues on my end. Sadly I didn't have much time to look into the issue as the associated feature development is on a deadline and since flask excel was just exporting a csv, I removed the package entirely and went with a raw python implementation.

Bad Request

Hi jaska,
thanks for your Flask Extension: Flask-Excel.

When i try your example app: "tiny_example.py" and upload file: /examples/example_for_upload.csv, response from flask in browser is: "Bad Request, The browser (or proxy) sent a request that this server could not understand."

Can you help me with solution?
(when you need more information, pls contact me)

Thanks.
Lukas
(Python 3.4)

How to ignore blank lines with flask-excel when import from xlsx?

I used flask-excel to import data from xlsx file. When blank lines exist in xlsx file, pyexcel gives '' for empty cell and inserts '' in every row into database. When I have a row unique, code will raise error.

Is there a simple way to ignore blank lines? Thanks.

Only in wsgi environment, occur response type error

Hi,

[My environment]
Windows 10
Python 3.6
IIS 7.5 or Apache 2.4.37 (I checked both environment with fastcgi, mod_wsgi)

[Condition]
Below code run well in flask cmd mode(ex> python excel_test.py)
http://localhost:5000/export
But when run in IIS or Apache meet TypeError(maybe 'None' Returned).

[Sample Code]
@app.route("/export", methods=['GET'])
def export_records():
return excel.make_response_from_array([[1, 2], [3, 4]], "xlsx", file_name="export_data")

[Error]
Error occurred:
Traceback (most recent call last):
File "c:\python\lib\site-packages\flask\app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "c:\python\lib\site-packages\flask\app.py", line 1816, in full_dispatch_request
return self.finalize_request(rv)
File "c:\python\lib\site-packages\flask\app.py", line 1831, in finalize_request
response = self.make_response(rv)
File "c:\python\lib\site-packages\flask\app.py", line 1957, in make_response
'The view function did not return a valid response. The'
TypeError: The view function did not return a valid response. The function either returned None or ended without a return statement.

[Workaround]
I had googling and find this code well worked in IIS, Apache.

@app.route("/export2", methods=['GET'])
def export_records2():
csvlist = [[1, 2], [3, 4]]
df = pd.DataFrame(csvlist, columns=["Team", "Player"])

print(df)

output = io.BytesIO()
writer = pd.ExcelWriter(output)
df.to_excel(writer, 'Tab1')
writer.save()

resp = make_response(output.getvalue())
resp.headers['Content-Disposition'] = 'attachment; filename=output.xlsx'
resp.headers["Content-type"] = "text/csv"
return resp

Thanks for your help

the response of make_response_from_array(array,file_type,file_name) is None! Different project ,same array, but different response.One is None. One is <Response 87649 bytes [200 OK]>.WHY?

The response of make_response_from_array(array,file_type,file_name) is None!
Different project ,same array, same logic, but different response. One is None. One is <Response 87649 bytes [200 OK]>.
Could you please tell what happened ? and what's wrong with my project?
I don't know how to describe this question!!!
Hope you can see it. Thanks you very much!!!

import is success , But the Data is not stored

import is success , and it return "Saved" ,But the Data is not stored , I think something is wrong ,but don't konw where..

image
image
image
image
127.0.0.1 - - [06/Aug/2017 22:05:35] "�[37mPOST /import HTTP/1.1�[0m" 200 -

Sheet: csv does not match any given tables

Hello! I can't understand what i'm do wrong
my route

@bp.route('/warehouse/', methods=['GET', 'POST'])
@login_required
def warehouse(id):

if request.method == 'POST':
    def article_init_func(row):
        a = Article()
        a.sku=row['sku']
        a.description=row['description']
        a.brand=row['brand']
        a.quantity=row['quantity']
        a.city=warehouse.city
        a.price=row['price']
        return a

    request.save_book_to_database(
        field_name='file', session=db.session,
        initializers = [article_init_func],
        tables=[Article])
    return redirect(url_for('/shop'), code=302)

my Model

class Article(Entity, db.Model):
pricelist_id = db.Column(db.Integer, db.ForeignKey('pricelist.id'), nullable=False)
brand = db.Column(db.String(64))
sku = db.Column(db.String(64))
description = db.Column(db.String(254))
quantity = db.Column(db.Integer)
city = db.Column(db.String(64))
delivery_terms = db.Column(db.String(64))
price = db.Column(db.Integer)

my CSV

brand,sku,description,quantity,price
br,qw3234,s sdf sd ,4,23.5,
br,qw2324,sd sdf sd ,4,23.5,
br,qw3324,sdf sdf sd ,4,23.5,
br,q32324,sdfs sdf sd ,4,23.5,

traceroute

127.0.0.1 - - [28/Sep/2018 02:37:51] "POST /warehouse/1 HTTP/1.1" 500 -
Traceback (most recent call last):
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1997, in call
return self.wsgi_app(environ, start_response)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/flask_login/utils.py", line 228, in decorated_view
return func(*args, **kwargs)
File "/home/g1ps0n/Dev/4auto/app/main/routes.py", line 54, in warehouse
tables=[Article])
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel_webio/init.py", line 237, in save_book_to_database
pe.save_book_as(**params)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel/core.py", line 113, in save_book_as
return sources.save_book(book, **dest_keywords)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel/internal/core.py", line 51, in save_book
return _save_any(a_source, book)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel/internal/core.py", line 55, in _save_any
a_source.write_data(instance)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel/plugins/sources/db_sources.py", line 110, in write_data
**self._keywords
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel/plugins/renderers/sqlalchemy.py", line 63, in render_book_to_stream
save_data(importer, to_store, file_type=self._file_type, **keywords)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel_io/io.py", line 127, in save_data
**keywords
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel_io/io.py", line 144, in store_data
writer.write(data)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel_io/book.py", line 218, in write
sheet_writer = self.create_sheet(sheet_name)
File "/home/g1ps0n/Dev/4auto/venv/lib/python3.6/site-packages/pyexcel_io/database/importers/sqlalchemy.py", line 98, in create_sheet
+ "Please be aware of case sensitivity."
Exception: Sheet: csv does not match any given tables.Please be aware of case sensitivity.

Version on pip not the latest one

When we do pip install Flask-Excel the version installed is 0.0.4.

On https://pypi.python.org/pypi they only have the 0.0.4 version.

I'm not sure if it's related, but when we do a
from flask.ext import excel
or
import flask.ext.excel

We have a warning that excel is an unresolved reference.

issue in import function execution

I have changed the table details and I am facing issue in import function execution. First of all, I had created the different table named GL_master but after the execution of my model table created in DB was named with "two underscore __" i.e. GL__master. why this unusual behavior I have no idea!!! Moreover I have change the sheet name to 'glmaster' and model name to 'Glmaster', but still facing issue in import functionality giving '500 Internal Server Error'. For better understanding here is my code.
capture

csv utf8-sig

Hey ~ I am using office to open the CSV file and the reason may be that utf8sig is not used, so I am in response. Charset = 'utf8-sig' setting is still garble. Can you tell me what is the solution

Flask-Excel + Flask-Cookiecutter

Hi jaska,

your module works great with all my projects, but now i need integrate it to project based on Flask-Cookiecutter.

When i use Flask-Excel like "traditional way", flask (like example: database_example.py), after upload i see error message:

TypeError: init() missing 12 required positional arguments: 'first_name', 'last_name', 'before_name', 'after_name', 'deleted', 'district_id', 'company_id', 'specialization_pri', 'specialization_sec', 'address', 'city', and 'psc'

Can you help me with Flask-Cookiecutter?
Really thanks.

Lukas

Flask-Excel did not work in my Ubuntu server

When I did

pip install Flask-Excel

on Windows, it worked as expected but when I did same on my Linux server, it did not.

I can the installed packages pyexcel_webio and others, but it did not work.

IndexError: Index out of range

Hi, when i use the save_book_to_database almost keeping with your example i kept getting the Index out of range error, but with other functions such as get_dict it works fine, what could be the problem?

AttributeError: 'module' object has no attribute 'make_response_from_query_sets'

I always get this error trying to use the documented 'make_response_from_query_sets

I use the exemple in the doc to try to run it

@admin.route('/exportcsv')
def export_csv():
query_sets = Classes.query.filter_by(id=1).all()
column_names = ['id', 'name','participants']
return excel.make_response_from_query_sets(query_sets, column_names, "xls")

New Line

How can we add a new line in data? Any idea, anyone?

make_response_from_records returns None

Package version:

Python 3.6.1
Flask (0.12.2)
Flask-Excel (0.0.6)
pyexcel (0.5.1.1)
pyexcel-io (0.4.2)
pyexcel-ods (0.4.0)
pyexcel-webio (0.1.2)
pyexcel-xls (0.4.0)

In my code:

return make_response_from_records(records, file_type='xls', file_name=fn)

This raises an exception:

  File "/miniconda/envs/flecom/lib/python3.6/site-packages/flask/app.py", line 1725, in make_response
    raise ValueError('View function did not return a response')
ValueError: View function did not return a response

I stepped into make_response_from_records(...) and found this code which returns None:

# python3.6/site-packages/pyexcel_webio/__init__.py

def dummy_func(content, content_type=None, status=200, file_name=None):
    return None


__excel_response_func__ = dummy_func


def _make_response(content, file_type,
                   status=200, file_name=None):
    if hasattr(content, "read"):
        content = content.read()
    if file_name:
        if not file_name.endswith(file_type):
            file_name = "%s.%s" % (file_name, file_type)
    return __excel_response_func__(
        content,
        content_type=FILE_TYPE_MIME_TABLE[file_type],
        status=status, file_name=file_name)

Multiple sheets?

Is it possible to have different sheets? I cannot find any example / documentation that says so.

Thanks and congratulations for the extensions.

No suitable database adapter found! pyexcel/Flask-Excel

We have a file, we have created the tables with the same column names and everything seems to be fine. We get the error: No suitable database adapter found! When we try to upload any excel file.

When we try to upload any excel file. We have tried this with a basic excel file mimicing the same columns in case there was an issue with the originating file. This has not resolved the issue. We have also created a new file with the same column names and entered some basic data and this still returns the same error.

AttributeError: 'Request' object has no attribute 'get_array'

I have the following function that works perfectly on my local environment with flask 1.1.2 on python 3.8.

@app.route("/upload", methods=['GET', 'POST'])
def upload_file():
    if request.method == 'POST':
        column_num = [x.lower() for x in request.get_array(field_name='file')[0]].index('address')
        rs = [P.upsert(process_address(Address(x[column_num])), ['address']) for x in request.get_array(field_name='file')[1:]]
        return '''
            <!doctype html>
            <title>File Uploaded</title>
            <h1>Excel File Uploaded</h1>
            <p><a href="./admin/property/"> View records... </a></p>'''
    return '''
    <!doctype html>
    <title>Upload an excel file</title>
    <h1>Excel file upload (csv, tsv, csvz, tsvz only)</h1>
    <form action="" method=post enctype=multipart/form-data><p>
    <input type=file name=file><input type=submit value=Upload>
    </form>
    '''

When I move it over to my pythonanywhere account I get the following error on flask 1.1.1 / python 3.8.


2020-07-29 02:57:16,407: Exception on /upload [POST]
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/jcrowe/mysite/flask_app.py", line 143, in upload_file
    column_num = [x.lower() for x in request.get_array(field_name='file')[0]].index('address')
  File "/usr/lib/python3.8/site-packages/werkzeug/local.py", line 348, in __getattr__
    return getattr(self._get_current_object(), name)
AttributeError: 'Request' object has no attribute 'get_array'

Can anyone help with this? I don't understand why the get_array function isn't there.

auto_commit=False in save_to_database and save_book_to_database to allow rollback

Hello,

I suggest telling about the auto_commit parameter in the documentation for save_to_database and save_book_to_database methods. In my need, when uploading a big table, if any row goes wrong (e.g.: foreign key constraint fails) I'd like to roll back the whole process. With auto_commit=False (default, omitted) I cannot, but when passing it as True and then committing the db.session, if any row goes wrong, an exception can be raised and then I perform the rollback, like below:

try:
    request.save_to_database(
        field_name='file',
        session=db.session,
        table=model,
        initializer=init_func,
        auto_commit=False)
    db.session.commit()
except:
    db.session.rollback()

Update existing rows in database

Hi,

Is there any option to overwrite/update the whole row in the table when the record with the same value eg. in field "name" already exists?
For example, I'd like to modify this snippet from the documentation:

@app.route("/import", methods=['GET', 'POST'])
def doimport():
    if request.method == 'POST':

        def category_init_func(row):
            c = Category(row['name'])
            c.id = row['id']
            return c

        def post_init_func(row):
            c = Category.query.filter_by(name=row['category']).first()
            p = Post(row['title'], row['body'], c, row['pub_date'])
            return p
        request.save_book_to_database(
            field_name='file', session=db.session,
            tables=[Category, Post],
            initializers=[category_init_func, post_init_func])
        return redirect(url_for('.handson_table'), code=302)
    return '''
    <!doctype html>
    <title>Upload an excel file</title>
    <h1>Excel file upload (xls, xlsx, ods please)</h1>
    <form action="" method=post enctype=multipart/form-data><p>
    <input type=file name=file><input type=submit value=Upload>
    </form>
    '''

I assume that I should probably write the proper init function.

Thanks.

View function did not return a response

With a simple example I get an error (View function did not return a response):

return flask_excel.make_response_from_array([[1,2], [3, 4]], "csv", file_name="example")

Python 3.5.3

Where does Flask-Excel save uploaded file?

Usually when you upload a file it gets saved to memory in your server.

-Where does Flask-Excel save files?
-How does the file is deleted after saving?
-If the file is saved in memory will this affect the memory or speed of my Flask application or web server?

Not working with Flask-Restful

Hello,

I came across your extension yesterday and I was unable to integrate your example code with flask excel. This is how i was trying to use it:

class Users(restful.Resource):
def post(self):
return jsonify({"result": request.get_array(field_name='file')})

But all is got was an array: {"result" : []}

It is not the worse thing in the world but it would be nice to have a way to integrate flask-restful with flaks-excel. Let me know what you think and if there is anyway i can help.

Usman

ImportError: No module named flask_excel

Hey there. Here is what I have:

main.py
import flask_excel

requirements.txt
Flask-excel

pip list
Flask-Excel 0.0.7

And I get ImportError... I have uninstalled, re-installed, leave the virtual env, re-join the virtual env. Nothing seems to work...

AttributeError: 'Request' object has no attribute 'save_book_to_database'

@app.route("/import", methods=['GET', 'POST'])
def doimport():
    if request.method == 'POST':

        def category_init_func(row):
            c = Category(row['name'])
            c.id = row['id']
            return c

        def post_init_func(row):
            c = Category.query.filter_by(name=row['category']).first()
            p = Post(row['title'], row['body'], c, row['pub_date'])
            return p
        request.save_book_to_database(
            field_name='file', session=db.session,
            tables=[Category, Post],
            initializers=[category_init_func, post_init_func])
        return "Saved"
    return '''
    <!doctype html>
    <title>Upload an excel file</title>
    <h1>Excel file upload (xls, xlsx, ods please)</h1>
    <form action="" method=post enctype=multipart/form-data><p>
    <input type=file name=file><input type=submit value=Upload>
    </form>
    '''

Return Book

Hi, I'm just trying pyexcel for creating xls file using python. I have created book data. But when I was trying to make response with make_response_from_array, the return file to client was not multi sheet / book ? is there anything that wrong with my simple example ? This is my example code:

`#Get Campaigns Blocking by HTTP GET
@app.route('/xlsheet', methods=['GET'])
def xlsheet():

two_dimensional_array = [
    [1,2,3,4,5],
    [6,7,8,9,0],
    [11,22,23,12,33]
]

multi = {
    'Sheet 1': [
        ['a',1],
        ['b',2],
        ['c',3]
    ],
    'Sheet 2': [
        ['a',1],
        ['b',2],
        ['c',3]
    ],
    'Sheet 3': [
        ['a',1],
        ['b',2],
        ['c',3]
    ],

}


#sheet = pe.Sheet(two_dimensional_array)
#sheet.save_as("/home/adsback/QAA-API/output1.xls")

#book = pe.get_book(bookdict=multi)
#book.save_as("/home/adsback/QAA-API/output2.xls")

return fe.make_response_from_array(multi, "xls", file_name="export_data")`

TypeError: invalid file: None

The example from the Quick Start works perfectly but when I try to use the get_dict() or get_array() functions when responding to requests made from Flask-Admin, I get the following error:

Here is the traceback: https://gist.github.com/af22b27762543a62709ea89897bc0539 .

I call it from here:

@pre_save(sender=BulkMessage)
def on_bulk_message_save_handler(model_class, instance, created):
    instance.file_fields = request.get_array(field_name='file')

The request object does have the correct file name at the point of calling get_array():

dump(request.files)
Contents of werkzeug.datastructures.ImmutableMultiDict object at 0x7fc8e6daf3b8
file    

<FileStorage: 'SMS_dummy_data.xlsx' ('application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')>

But the file contents are lost somewhere in get_array():

File "/home/tuk/.virtualenvs/bulksms/lib/python3.4/site-packages/pyexcel_webio/__init__.py", line 93, in get_dict

return pe.get_dict(**params)

[console ready]
>>> dump()
Local variables in frame
params  

{'file_content': b'', 'name_columns_by_row': 0, 'file_type': 'xlsx'}

keywords    

{'name_columns_by_row': True, 'field_name': 'file'}

self    

<ExcelRequest 'http://localhost:5000/admin/bulkmessage/new/?url=%2Fadmin%2Fbulkmessage%2F' [POST]>

Any ideas?

Sheet: Category does not match any given tables.Please be aware of case sensitivity.

It is a question.
It's the example from docs I copy the code and download the sample-data.
It works but when I rename the sheet in the sample file, like changing category into Category, It complains like this.

Sheet: Category does not match any given tables.Please be aware of case sensitivity.

And I check the code and manual carefully and still don't know how this "match" works. Without understanding this I can't use save_book_to_database. Could you explain how it works?

Thanks, and sorry for my poor English.

KeyError in initializer

Hello! I need help , again)
if i use mapdict parameter for CSV file i'ts raise

builtins.KeyError
KeyError: 'sku'

but for xslx files, with same content, its work prefectly

My Route

@bp.route('/warehouse/', methods=['GET', 'POST'])
@login_required
def warehouse(id):

if request.method == 'POST':
    def article_init_func(row):
        a = Article()
        a.sku=row['sku']
        a.description=row['description']
        a.brand=row['brand']
        a.quantity=row['quantity']
        a.city=warehouse.city
        a.price=row['price']
        return a

    map_row = ['brand', 'sku', 'description', 'quantity', 'price']

    request.save_to_database(
        field_name='file', session=db.session,
        initializers = [article_init_func],
        tables=[Article],
        mapdict=map_row)
    return redirect(url_for('/shop'), code=302)

my Model

class Article(Entity, db.Model):
pricelist_id = db.Column(db.Integer, db.ForeignKey('pricelist.id'), nullable=False)
brand = db.Column(db.String(64))
sku = db.Column(db.String(64))
description = db.Column(db.String(254))
quantity = db.Column(db.Integer)
city = db.Column(db.String(64))
delivery_terms = db.Column(db.String(64))
price = db.Column(db.Integer)

my CSV

brand,sku,description,quantity,price
br,qw3234,s sdf sd ,4,23.5,
br,qw2324,sd sdf sd ,4,23.5,
br,qw3324,sdf sdf sd ,4,23.5,
br,q32324,sdfs sdf sd ,4,23.5,

traceroute

Traceback (most recent call last):
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1997, in call
return self.wsgi_app(environ, start_response)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask_compat.py", line 33, in reraise
raise value
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask_compat.py", line 33, in reraise
raise value
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask\app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "c:\users\admin\dev\4auto\venv\lib\site-packages\flask_login\utils.py", line 261, in decorated_view
return func(*args, **kwargs)
File "C:\Users\Admin\Dev\4auto\app\main\routes.py", line 88, in warehouse
mapdict=map_row)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel_webio_init_.py", line 170, in save_to_database
pe.save_as(**params)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel\core.py", line 83, in save_as
return sources.save_sheet(sheet, **dest_keywords)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel\internal\core.py", line 43, in save_sheet
return _save_any(a_source, sheet)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel\internal\core.py", line 55, in _save_any
a_source.write_data(instance)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel\plugins\sources\db_sources.py", line 64, in write_data
**self._keywords
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel\plugins\renderers\sqlalchemy.py", line 35, in render_sheet_to_stream
**keywords
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel_io\io.py", line 127, in save_data
**keywords
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel_io\io.py", line 144, in store_data
writer.write(data)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel_io\book.py", line 220, in write
sheet_writer.write_array(incoming_dict[sheet_name])
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel_io\sheet.py", line 174, in write_array
self.write_row(row)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel_io\database\importers\sqlalchemy.py", line 45, in write_row
self._write_row(new_array)
File "c:\users\admin\dev\4auto\venv\lib\site-packages\pyexcel_io\database\importers\sqlalchemy.py", line 56, in _write_row
obj = self._native_sheet.row_initializer(row)
File "C:\Users\Admin\Dev\4auto\app\main\routes.py", line 74, in article_init_func
a.sku=row['sku']
KeyError: 'sku'

make_response_from_records with large data sets (xlsx) crashes

If I call make_response_from_records() to convert to CSV with 1M rows, it does it no problem. However if I do the same thing to XLSX, it runs memory up until the whole flask application comes crashing down and dies.

Obviously an Excel spreadsheet can't normally have 1M rows in it anyhow.

I can check the number of rows before I call make_response_from_records() to mitigate this problem.

I was opening this issue to see if that function could be updated to throw an exception if too many rows are passed instead of happily ingesting them until it dies.

Multiple Sheet using Flask Excel

I was wondering what the best way is for creating a multiple sheet Excel book. I have data that is being generated live (nothing from SQL).

My idea was to create a pyexcel_instance, use 'make_response()'. However, I was slightly confused with the best way to use PyExcel. Should I save my Pyexcel book when I make one? If so, would I have to delete the saved object to avoid memory leaks?

Update/Skipping row during data import into database

Current, it was only possible to import data into an empty database. However, it is more often that the database is not empty and hence needs a bit extra logic to cope with existing data: 1) update existing items 2) skipping them. Let's give the choice to the developer.

will not import

Hi, I've installed Flask Excel via pip, no problem within my virtualenv.

I see it

pip list
Flask (0.11.1)
Flask-Excel (0.0.5)

But when I add
import flask_excel as excel
I get
ImportError: No module named flask_excel

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.