Code Monkey home page Code Monkey logo

csv-grid's Introduction

CSV Data Export extension for Yii2


This extension provides ability to export data to CSV file.

For license information check the LICENSE-file.

Latest Stable Version Total Downloads Build Status

Installation

The preferred way to install this extension is through composer.

Either run

php composer.phar require --prefer-dist yii2tech/csv-grid

or add

"yii2tech/csv-grid": "*"

to the require section of your composer.json.

Usage

This extension provides ability to export data to CSV file. Export is performed via \yii2tech\csvgrid\CsvGrid instance, which provides interface similar to \yii\grid\GridView widget.

Example:

<?php

use yii2tech\csvgrid\CsvGrid;
use yii\data\ArrayDataProvider;

$exporter = new CsvGrid([
    'dataProvider' => new ArrayDataProvider([
        'allModels' => [
            [
                'name' => 'some name',
                'price' => '9879',
            ],
            [
                'name' => 'name 2',
                'price' => '79',
            ],
        ],
    ]),
    'columns' => [
        [
            'attribute' => 'name',
        ],
        [
            'attribute' => 'price',
            'format' => 'decimal',
        ],
    ],
]);
$exporter->export()->saveAs('/path/to/file.csv');

\yii2tech\csvgrid\CsvGrid allows exporting of the \yii\data\DataProviderInterface and \yii\db\QueryInterface instances. Export is performed via batches, which allows processing of the large data without memory overflow.

In case of \yii\data\DataProviderInterface usage, data will be split to batches using pagination mechanism. Thus you should setup pagination with page size in order to control batch size:

<?php

use yii2tech\csvgrid\CsvGrid;
use yii\data\ActiveDataProvider;

$exporter = new CsvGrid([
    'dataProvider' => new ActiveDataProvider([
        'query' => Item::find(),
        'pagination' => [
            'pageSize' => 100, // export batch size
        ],
    ]),
]);
$exporter->export()->saveAs('/path/to/file.csv');

Note: if you disable pagination in your data provider - no batch processing will be performed.

In case of \yii\db\QueryInterface usage, CsvGrid will attempt to use batch() method, if it present in the query class (for example in case \yii\db\Query or \yii\db\ActiveQuery usage). If batch() method is not available - yii\data\ActiveDataProvider instance will be automatically created around given query. You can control batch size via \yii2tech\csvgrid\CsvGrid::$batchSize:

<?php

use yii2tech\csvgrid\CsvGrid;

$exporter = new CsvGrid([
    'query' => Item::find(),
    'batchSize' => 200, // export batch size
]);
$exporter->export()->saveAs('/path/to/file.csv');

While running web application you can use \yii2tech\csvgrid\ExportResult::send() method to send a result file to the browser through download dialog:

<?php

use yii2tech\csvgrid\CsvGrid;
use yii\data\ActiveDataProvider;
use yii\web\Controller;

class ItemController extends Controller
{
    public function actionExport()
    {
        $exporter = new CsvGrid([
            'dataProvider' => new ActiveDataProvider([
                'query' => Item::find(),
            ]),
        ]);
        return $exporter->export()->send('items.csv');
    }
}

Splitting result into several files

While exporting large amount of data, you may want to split export results into several files. This may come in handy in case you are planning to use result CSV files with program, which have a limit on maximum rows inside single file. For example: 'Open Office' and 'MS Excel 97-2003' allows maximum 65536 rows per CSV file, 'MS Excel 2007' - 1048576.

You may use \yii2tech\csvgrid\CsvGrid::$maxEntriesPerFile to restrict maximum rows in the single result file. In case the export result produce more then one CSV file - these files will be automatically archived into the single archive file. For example:

<?php

use yii2tech\csvgrid\CsvGrid;

$exporter = new CsvGrid([
    'query' => Item::find(),
    'maxEntriesPerFile' => 60000, // limit max rows per single file
]);
$exporter->export()->saveAs('/path/to/archive-file.zip'); // output ZIP archive!

Note: you are not forced to receive multiple files result as a single archive. You can use \yii2tech\csvgrid\ExportResult::$csvFiles to manually iterate over created CSV files and process them as you like:

<?php

use yii2tech\csvgrid\CsvGrid;

$exporter = new CsvGrid([
    'query' => Item::find(),
    'maxEntriesPerFile' => 60000, // limit max rows per single file
]);
$result = $exporter->export();

foreach ($result->csvFiles as $csvFile) {
    /* @var $csvFile \yii2tech\csvgrid\CsvFile */
    copy($csvFile->name, '/path/to/dir/' . basename($csvFile->name));
}

Archiving results

Export result is archived automatically, if it contains more then one CSV file. However, you may enforce archiving of the export result via \yii2tech\csvgrid\ExportResult::$forceArchive:

<?php

use yii2tech\csvgrid\CsvGrid;

$exporter = new CsvGrid([
    'query' => Item::find(),
    'resultConfig' => [
        'forceArchive' => true // always archive the results
    ],
]);
$exporter->export()->saveAs('/path/to/archive-file.zip'); // output ZIP archive!

Heads up! By default \yii2tech\csvgrid\ExportResult uses PHP Zip extension for the archive creating. Thus it will fail, if this extension is not present in your environment.

You can setup your own archive method via \yii2tech\csvgrid\ExportResult::$archiver. For example:

<?php

use yii2tech\csvgrid\CsvGrid;

$exporter = new CsvGrid([
    'query' => Item::find(),
    'resultConfig' => [
        'forceArchive' => true,
        'archiver' => function (array $files, $dirName) {
            $archiveFileName = $dirName . DIRECTORY_SEPARATOR . 'items.tar';

            foreach ($files as $fileName) {
                // add $fileName to $archiveFileName archive
            }

            return $archiveFileName;
        },
    ],
]);
$exporter->export()->saveAs('/path/to/items.tar');

While sending file to the browser via \yii2tech\csvgrid\ExportResult::send() there is no need to check if result is archived or not as correct file extension will be append automatically:

<?php

use yii2tech\csvgrid\CsvGrid;
use yii\data\ActiveDataProvider;
use yii\web\Controller;

class ItemController extends Controller
{
    public function actionExport()
    {
        $exporter = new CsvGrid([
            'dataProvider' => new ActiveDataProvider([
                'query' => Item::find(), // over 1 million records
            ]),
            'maxEntriesPerFile' => 60000,
        ]);

        return $exporter->export()->send('items.csv'); // displays dialog for saving `items.csv.zip`!
    }
}

Customize output format

Although CSV dictates particular data format (each value quoted, values separated by comma, lines separated by line break), some cases require its changing. For example: you may need to separate values using semicolon, or may want to create TSV (tabular separated values) file instead CSV. You may customize format entries using \yii2tech\csvgrid\CsvGrid::$csvFileConfig:

<?php

use yii2tech\csvgrid\CsvGrid;

$exporter = new CsvGrid([
    'query' => Item::find(),
    'csvFileConfig' => [
        'cellDelimiter' => "\t",
        'rowDelimiter' => "\n",
        'enclosure' => '',
    ],
]);
$exporter->export()->saveAs('/path/to/file.txt');

csv-grid's People

Contributors

kesselb avatar klimov-paul avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

csv-grid's Issues

Calling unknown method: yii\grid\SerialColumn::renderHeaderCellContent()

I get this error when I try to pass a columns array with a SerialColumn.

The renderHeaderCellContent method is a protected method in the parent class Column, so it's not accessible outside the child class SerialColumn.

...
 'columns' => [
                ['class' => SerialColumn::className(), 'header' => 'S/N'],
                [
                    'attribute' => 'attribute_name',
                ]
            ],
...

This code snippet also causes the same error

$serialColumn = new SerialColumn();
$serialColumn->renderHeaderCellContent();

I think a solution to this would be to create a SerialColumn that exposes the renderHeaderCellContent method used in the composeHeaderRow in the CsvGrid class and I have created a pull request here - #3

PHP Version: 5.5.36
Great library you have here 👍

Crashes when query returns no results and csv headers are enabled

When Query returns no results and CSV export show header is enabled it crashes because columns are not initialized yet.

The while loop never runs, columns are never initialized (but a CSV file has already been created)

while (($data = $this->batchModels()) !== false) {

Thus this line allows the execution to continue (file is allocated but no data exist)

if (empty($result->csvFiles)) {

Tries to render header rows while columns have not been initialized.

$csvFile->writeRow($this->composeHeaderRow());

Export CSV as a Row wise header data instead of column wise.

I am doing export of csv file using Yii2 PHP Framework.
By using CSV-grid package i can able to do exporting of csv file as a column wise.
I want to export CSV as a Row wise using "Pivot Table" options into CSV.
I am unable to do that so, please help me.

Not compatible with GridView

What steps will reproduce the problem?

Use something like contentOptions in the columns.

What is the expected result?

That I would be able to use the same config I do for yii\web\GridView so that I could easily output a CSV with the same columns and formatting.

What do you get instead?

Error, Setting unknown property: yii2tech\csvgrid\DataColumn::contentOptions

Additional info

Why are you defining your own DataColumn instead of extending the yii\grid\DataColumn?

Q A
This Package Version 1.0.5
Yii Framework Version 2.0.42.1
PHP version 8.0.5
Operating system Linux

In case of empty query results error with non-existing results file

Steps to reproduce:

  1. Use ActiveDataProvider + query, that return 0 results.
  2. Try to save results $exporter->export()->saveAs('file.csv');

Expecting to get empty file.

PHP Warning 'yii\base\ErrorException' with message 'rename(,file.csv): No such file or directory'

in /vagrant/vendor/yii2tech/csv-grid/ExportResult.php:173

Stack trace:
#0 [internal function]: yii\base\ErrorHandler->handleError(2, 'rename(,file.cs...', '/vagrant/vendor...', 173, Array)
#1 /vagrant/vendor/yii2tech/csv-grid/ExportResult.php(173): rename('', 'file.csv')
#2 /vagrant/vendor/yii2tech/csv-grid/ExportResult.php(187): yii2tech\csvgrid\ExportResult->move('file.csv')
#3 /vagrant/console/controllers/InvoiceController.php(78): yii2tech\csvgrid\ExportResult->saveAs('file.csv')
#4 [internal function]: console\controllers\InvoiceController->actionDailyBackup()
#5 /vagrant/vendor/yiisoft/yii2/base/InlineAction.php(55): call_user_func_array(Array, Array)
#6 /vagrant/vendor/yiisoft/yii2/base/Controller.php(154): yii\base\InlineAction->runWithParams(Array)
#7 /vagrant/vendor/yiisoft/yii2/console/Controller.php(108): yii\base\Controller->runAction('daily-backup', Array)
#8 /vagrant/vendor/yiisoft/yii2/base/Module.php(454): yii\console\Controller->runAction('daily-backup', Array)
#9 /vagrant/vendor/yiisoft/yii2/console/Application.php(167): yii\base\Module->runAction('invoice/daily-b...', Array)
#10 /vagrant/vendor/yiisoft/yii2/console/Application.php(143): yii\console\Application->runAction('invoice/daily-b...', Array)
#11 /vagrant/vendor/yiisoft/yii2/base/Application.php(375): yii\console\Application->handleRequest(Object(yii\console\Request))
#12 /vagrant/yii(27): yii\base\Application->run()
#13 {main}

Directory not empty

What steps will reproduce the problem?

What is the expected result?

What do you get instead?

Additional info

Q A
This Package Version latest
Yii Framework Version 2.0.38
PHP version 7.4.2
Operating system windows 8
public function actionExportCsv(){
        $query = User::find()->select(['user.id', 'user.firstname'])
                ->joinWith(['auth_assignment'])
                ->where(['auth_assignment.item_name' => 'user'])
                ->asArray();

        $dataProvider = new ActiveDataProvider([
            'query' => $query,
        ]);

        $request = Yii::$app->request;

        if ($request) {
            $exporter = new CsvGrid([
            'dataProvider' => $dataProvider,
            'csvFileConfig' => [
                    'enclosure' => '',
                ],
            'columns' => [
                    [
                        'label' => 'Customer Id',
                        'value' => function ($model){
                            return $model['id'];
                        },
                    ],
                    [
                        'label' => 'Customer Name',
                        'value' => function ($model){
                            return Yii::$app->account->formatted_name($model['id']);
                        },
                    ],
                    [
                        'label' => 'Customer Mobile',
                        'value' => function ($model){
                            return Yii::$app->account->formatted_phone($model['id']);
                        },
                    ],
                    [
                        'label' => 'Customer Email',
                        'value' => function ($model){
                            return $model['email'];
                        },
                    ],
                    [
                        'label' => 'Status',
                        'value' => function ($model){
                            return $model['status'] == '1' ? 'Active' : 'Blocked';
                        }
                    ],
                    [
                        'label' => 'Active Since',
                        'value' => function($model){
                            return Yii::$app->account->format_date($model['created_at']);
                        }
                    ],
                ],
            ]);
        }

        return $exporter->export()->save('file.csv');
    }

I am just exporting csv from query in new project it run perfectly in my old project

Question: feedback to user

There is a way to give to user a progress feedback of exporting job.
With large data export may take several minutes and the user doesn't know how long does it take.
Thank you
Jack

Issue with multiple sheet in single csv

I am unable to add the multiple sheet in single csv.

$file_name ='all_'.date('Y-m-d h:i:s').'.csv';
$exporter = new CsvGrid([
'dataProvider' => new ArrayDataProvider([
'allModels' =>$newUserData['all'],
]),
]);
$exporter->export()->send($file_name);

is their any other way to add multiple sheet

Send exported file to browser

Hello Paul,

Is there a way to send the downloaded file to the browser, so the user can decide where to save it?
I hope you can provide me some help.

Thanks in advance.

Setting 200 as pagesize in ActiveDataProvider results in duplicated CSV records

Hi, i found a problem.

searchModel:

        $dataProvider = new ActiveDataProvider([
            'query' => Stock::find(),
            'pagination' => ['pageSize' => 200,]
        ]);

Controller:

 $searchModel = new StockSearch();
        $dataProvider = $searchModel->search(Yii::$app->request->queryParams);
        $exporter = new CsvGrid([
            'dataProvider' => $dataProvider,
            'columns' => [
                'PART_NUMBER',
                'QTY',
            ],
        ]);
        $filename = 'Stock';
        $date = new \DateTime();
        $date->modify("+2 hours");
        $date = Yii::$app->formatter->asDatetime($date,'yyyy-MM-dd_HH-mm-ss');
        return $exporter->export()->send($filename.'_'.$date.'.csv');

If I set pagesize to 200 I found duplicated and overwritten records in exported CSV.
The number of rows is the same of the table in database (5800 records more or less) so some rows get overwritten in CSV.
If I remove pagesize the export is correct (but takes a long time)
If I put 100 as pagesize it's everything ok.
I think this is very dangerous! Any hint?
Thank you!

Time to export 3200 records

Hi,
is it normal it gets at least 4 minutes to export 3200 rows?
Pagesize is set to 2000, resulting CSV filesize is 989KB
What can I do to speed up the process?

long number result for export csv and read by MS Excel

test code

$exporter = new CsvGrid([
    'dataProvider' => new ArrayDataProvider([
        'allModels' => [
            [
                'name' => 'length14',
                'number' => '01234567890123',
            ],
            [
                'name' => 'length15',
                'number' => '012345678901234',
            ],
            [
                'name' => 'length16',
                'number' => '0123456789012345',
            ],
            [
                'name' => 'length17',
                'number' => '01234567890123456',
            ],
            [
                'name' => 'length18',
                'number' => '012345678901234567',
            ],
        ],
    ]),
    'columns' => [
        [
            'attribute' => 'name',
        ],
        [
            'attribute' => 'number',
        ],
    ],
]);
return $exporter->export()->send('test.csv');

result csv

"Name","Number"
"length14","01234567890123"
"length15","012345678901234"
"length16","0123456789012345"
"length17","01234567890123456"
"length18","012345678901234567"

read by MS Excel

image

error

Records is right in csv file, but error in MS Excel.
When number length is larger than 16, MS Excel will truncation it. You can see it in length17 and length18, they end of 0

add option to output attribute names instead of labels

Can you please add an option so that resulting csv file would have attributes as headers, instead of labels or "converted" to regular words.
For example:
attributes are: id, first_name, last_name
currently the output csv will have headers as: Id, First Name, Last Name
desired headers: id, first_name, last_name

CSV Grid causes error upon export()->saveAs()

Occurs when a csv grid is completely empty (empty dataprovider) and headers are set to hidden.

What steps will reproduce the problem?

Load dataprovider with empty set, load this provider into a csvgrid, and set the showHeader property to false.

What is the expected result?

Empty csvgrid

What do you get instead?

Error 500: no such file or directory

Memory Exhausted on only 6000 rows

I'm receiving a memory exhausted on about 6000 rows, 2000 seems to work ok (have about 20 columns in the csv). I have 128mb of php memory and the batch commands are executing successfully when I look through the log.

Should this be happening? The docs seem to suggest I could export 30k, even 60k records with no issues and the only reason to be using document splitting would be if it exceeded Excel's max rows. (Thanks for the great extension by the way)

Yii2 csvgrid not supporting special character such as ">" ,"<" ,": "

What steps will reproduce the problem?

I simply fetch the result from database write into csv file

What is the expected result?

Expected result: > 5
upcoming result: <5

What do you get instead?

Additional info

Q A
This Package Version 1.0.1
Yii Framework Version 2.13
PHP version 7.0.33
Operating system Ubuntu 16

CSV is deleted before downloaded

Homestead (Vagrant-Box from Laravel) with PHP7.1 running HumHub.

$exporter = new CsvGrid(['dataProvider' => $dataProvider, 'columns' => ['id', 'username']]);
return $exporter->export()->send('my.csv');

I would like to send the generated csv instead of saving it. Unfortunately this exceptions happens: unlink(/code/humhub/protected/runtime/csv-grid/15157521565a588adcbf03e8.01521473/data-001.csv): Text file busy

yii\web\Response->sendFile open the csv with 'rb' and locks the file until the generate response is sent. But before the response is sent (and the file is unlocked) ExportResult->__destruct is called and tries to delete the locked file which leads to this exception.

Maybe this issue is related to my configuration. I could fix it by changing the ExportResult->send like this:
$content = file_get_contents($this->getResultFileName());
return Yii::$app->getResponse()->sendContentAsFile($content, $name, $options);

array 2 csv

It can write from array data into csv file ? Thanks

Issue in writing HTML data to CSV sheet

I have a textarea field for capturing content that accept HTML.

On printing array value (using die) this field render cell content as HTML very well (with p , br tags) [Attached Screenshot]

Now when this content data written in CSV this will be writing in HTML_ENCODE Format . There is no such code added to do the formatting. I need the complete HTML written in content field to get exported to sheet.

Controller Code :

 $productData= [];
 $comments = $dataItems->comments;			
 $productData['comments'] = $comments;

  $exporter = new CsvGrid([
            /* 'csvFileConfig' => [
              'cellDelimiter' => "\t",
              'rowDelimiter' => "\n",
              'enclosure' => '',
              ], */


            'dataProvider' => new ArrayDataProvider([
                'allModels' => $csvdata,
                    ]),
            'columns' => [			
                [ 'attribute' => 'comments', 'label' => 'comments'],
            ],
        ]);
		
        $source_directory = Yii::$app->basePath . \Yii::$app->params['web.upload.path'] . "/uploads/";
        $status = $exporter->export()->saveAs($source_directory . $file);

Any one facing same ? Any formatting need to add?
bug

Call to a member function renderHeaderCellContent() on array

Error Exception occurs when no data record against the query.

PHP Fatal Error – yii\base\ErrorException

Call to a member function renderHeaderCellContent() on array

391392393394395396397398399400401402403404405406407408409
/**
* Composes header row contents.
* @return array cell contents.
*/
protected function composeHeaderRow()
{
$cells = [];
foreach ($this->columns as $column) {
$cells[] = $column->renderHeaderCellContent();
}
return $cells;
}

/**
 * Composes header row contents.
 * @return array cell contents.
 */
protected function composeFooterRow()

Breaks pagination operations when used before GridView

a word of warning, this cannot be used before GridView, as it does not leave the yii\data\Pagination object is not left in the starting state.

This will cause your page to forced to the last page every time if used before GridView.

Until fixed, place in code after anything that uses pagination is rendered.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.