commercetools / sphere-order-export Goto Github PK

Support reference number

SUPPORT-3190

Description

Currently we do not support exporting fields with path order.variant.attributes.

Here in this mapping function we, for example, call a special mapping "formatImages" for "lineItems.variant.images" but nothing is being done for mentioned "lineItems.variant.attributes".

To make this work we would have to:

• add a mapping for all possible attribute types (number, text, ltext, enum, lenum...) which can be taken from product export
• handle mapping for both CSV and XML
• add tests for all possible attribute types

Use Case

Customer perspective:
The main reason why we need to obtain the product's attributes in the export of the orders is because some of them provide us with important information about the product at the time of sale, such as the cost price, the sale price to the optician, the brand, etc.
All these data provide us with very useful information to perform data analysis from our business and marketing areas.

Support reference number

SUPPORT-3190

Description

It is a nice to have feature, to give link to the example templates for exporting an order, which helps the first time users.

Expected Behavior

A link (https://github.com/sphereio/sphere-order-export/tree/master/data) next to the text "Orders export template as CSV or XML" on Impex order export page

Current Behavior

At present no link is there.

Context

Customers write to support asking for example templates

npx: installed 5 in 1.039s
ENOENT: no such file or directory, open './package-lock.json'
npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":"0.8.x || 0.10.x"} (current: {"node":"10.21.0","npm":"6.14.4"})
npm WARN notsup Not compatible with your version of node/npm: [email protected]

Currently this package fails becuase its one of the dependency [email protected] only works with node 0.10 or 0.8. So we have to update to the latest so that we can update package in express impex

The "deliveries" in "shippingInfo" on Order level should be exported to csv file

npx: installed 5 in 1.039s
ENOENT: no such file or directory, open './package-lock.json'
npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":"0.8.x || 0.10.x"} (current: {"node":"10.21.0","npm":"6.14.4"})
npm WARN notsup Not compatible with your version of node/npm: [email protected]

Currently this package fails becuase its one of the dependency [email protected] only works with node 0.10 or 0.8. So we have to update to the latest so that we can update package in express impex

Description

I'm consistently getting an "Insufficient scope" error when trying to export the order data using the CLI export tool. The command I'm using is:

order-export --projectKey <project-key> --accessToken <access-token> --targetDir c:/tmp --logDir c:/tmp --csvTemplate c:/tmp/header.txt

This is working fine for the customer-export using the same project-key and access token. The scope is configured correctly in Commercetools: the credentials have read-only access to all entities (including customer and order).

Expected Behavior

Create a local order export for the specified project key.

Current Behavior

Receiving an Insufficient scope error. This is the case when providing only an access key, only client-id and client-secret, and when providing all three. The same behavior occurs when using credentials that have manage-orders access. Also specifying the sphereHost and/or sphereAuthHost do not help. Exporting the orders using the Impex tool also works fine.

Stacktrace

{
	"name": "sphere-order-export-2.2.3",
	"hostname": "<hostname>",
	"pid": 17576,
	"level": 50,
	"data": {
		"body": {
			"statusCode": 403,
			"message": "Insufficient scope",
			"errors": [{
				"code": "insufficient_scope",
				"message": "Insufficient scope"
			}],
			"error": "insufficient_scope",
			"error_description": "Insufficient scope",
			"originalRequest": {
				"endpoint": "/channels",
				"payload": {
					"key": "OrderXmlFileExport",
					"roles": ["OrderExport"]
				},
				"options": {
					"config": {
						"project_key": "<project-key>"
					},
					"host": "api.sphere.io",
					"protocol": "https",
					"timeout": 60000,
					"rejectUnauthorized": true,
					"headers": {
						"User-Agent": "sphere-order-export - 2.2.3",
						"Authorization": "Bearer <token>"
					},
					"uri": "https://api.sphere.io/<project-key>",
					"gzipEnable": true
				}
			}
		},
		"message": "Forbidden",
		"name": "HttpError",
		"stack": "HttpError: Forbidden\n    at <root>\\Roaming\\npm\\node_modules\\sphere-order-export\\node_modules\\sphere-node-sdk\\lib\\services\\base.js:518:22\n    at ChannelService.BaseService._wrapResponse (<root>\\Roaming\\npm\\node_modules\\sphere-order-export\\node_modules\\sphere-node-sdk\\lib\\services\\base.js:523:11)\n    at <root>\\Roaming\\npm\\node_modules\\sphere-order-export\\node_modules\\sphere-node-sdk\\lib\\services\\base.js:400:40\n    at Request._callback (<root>\\Roaming\\npm\\node_modules\\sphere-order-export\\node_modules\\sphere-node-sdk\\lib\\connect\\rest.js:166:14)\n    at Request.self.callback (<root>\\Roaming\\npm\\node_modules\\sphere-order-export\\node_modules\\request\\request.js:185:22)\n    at Request.emit (events.js:203:13)\n    at Request.<anonymous> (<root>\\Roaming\\npm\\node_modules\\sphere-order-export\\node_modules\\request\\request.js:1161:10)\n    at Request.emit (events.js:203:13)\n    at Gunzip.<anonymous> (<root>\\Roaming\\npm\\node_modules\\sphere-order-export\\node_modules\\request\\request.js:1083:12)\n    at Object.onceWrapper (events.js:291:20)\n    at Gunzip.emit (events.js:208:15)\n    at endReadableNT (_stream_readable.js:1168:12)\n    at processTicksAndRejections (internal/process/task_queues.js:77:11)",
		"code": 403,
		"statusCode": 403
	},
	"project_key": "<project-key>",
	"msg": "Oops, something went wrong!",
	"time": "2019-09-23T09:01:36.048Z",
	"v": 0
}

Using sphere-order-export version 2.2.3.

When exporting orders if there is no order an empty file is generated and uploaded to sftp server.

A pain point when there is a cron job to process exported orders and the cron job processes empty files.

Support reference number

https://jira.commercetools.com/browse/SUPPORT-5385

Description

Order export does not support customLineItems

Expected Behavior

Order export should support customLineItems

Current Behavior

Order export does not support customLineItems

Support reference number

https://jira.commercetools.com/browse/SUPPORT-4137

Description

When using paymenInfo it returns a value of [object Object]

For the header discountCodes the CSV cell should be filled with all applied discount codes at this cart, separated with the multi value delimiter.

Super-Duper;10PERCENT;FREESHIPPING

Description

Order export should export zeros. It should also print out value in moneyFormat instead of [Object object] when exporting common fields.

In the order export, we should offer an option to fill all rows of an order with the general order information, instead of just the first row of an order.

Background: Currently, the 1st row in the order export contains general information such as address; in the 2nd row per order are all the line items of the order. The general rows are not filled because they are redundant, but some of our customers would like them to always be filled.

We would like to support the update actions "add deliveries".
(see https://dev.commercetools.com/http-api-projects-orders.html#add-delivery)

Due to the complexity of the delivery object, we are not able to structure an easy-to-use csv format. We have suggested a new structure for this (see possible solution).

GroupByDelivery, ItemID, quantity and parcel.id are mandatory fields.

Current Behavior

Not supported currently

Possible Solution

There could be a GroupByDeliveryID which is only there to separate the deliveries from each other. So all line items with the same GroupByDeliveryID belongs to the same Delivery.
Inside this, the parcelID is available to separate the parcels from each other.
Clarify: Dies the ID have to be unique over all parcels or just inside the delivery?

Support reference number

SUPPORT-16165

Description

/bigcustomer-prod/orders?sort=id%20asc&where=createdAt%3E%20%22[REDACTED]%22%20AND%20createdAt%3C%3D%20%22[REDACTED]%22&limit=100&expand=customLineItems%5B*%5D.state%5B*%5D.state&expand=lineItems%5B*%5D.state%5B*%5D.state&expand=lineItems%5B*%5D.supplyChannel&expand=discountCodes%5B*%5D.discountCode&expand=customerGroup&withTotal=false&where=id%20%3E%20%22[REDACTED]%22

Expected Behaviour

export orders without error for project with more than 20k orders per pay

Current Behaviour

times out even with 1 day interval

more efficient query

To avoid request times out

Query must sort by createdAt and id and increment the fromDate and the Id in the loop

Hey guys,

I just realized that sphere-order-export is failing to install at elastic.io because the production branch is missing. What is the reason you removed it?

Description

The Order Exporter creates very long directory names, including spurious JavaScript code, which exceed the 255-character filename limit; this causes it to fail.

Expected Behavior

Temporary directories should have names of the form '/tmp/tmp-7191EAcLrQidQ/'.

Current Behavior

Temporary directory names currently have JavaScript callback code embedded in them; actual example:

/tmp/tmp-7191EAcLrQidQ,function _cleanupCallback(next) {
    if (!called) {
        var index = _removeObjects.indexOf(_cleanupCallback);
        if (index >= 0) {
          _removeObjects.splice(index, 1);
        }

        called = true;
        removeFunction(arg);
    }
    if (next) next(null);
  }/

Context

This breaks my team's CSV Connector order export jobs on iron.io.

I don't understand why the problem is only making itself manifest now.

Possible Solution

I have tracked down the problem to l. 57 in ensureExportDir() in src/coffee/run.coffee. The call to tmp.dirAsync is being treated, when it is used at l. 129 onwards, as if it returned (a promise of?) a directory name, but in fact it returns (a promise of?) a two-element list, consisting of the directory name and a callback. When this list is used as a string, the string representation of the callback is included in it. Returning only the first element of the list would fix the problem.

I have not verified whether this affects the return values at lines 63 or 66 too, but I don't think it does.

Description

Using the order export with a csv template. I think the export is working correctly but if you want to download the file, you get an error message "Not found".

Expected Behavior

It should be possible to download the order export

Current Behavior

Error message not found is shown

Context

Support ticket

Possible Solution

Description

When executing an order export with a template, the export-csv will have missing line breaks every 99 lines

Expected Behavior

There should be a linebreak

Current Behavior

Due to missing of linebreak, the 100th line is joined along with 99th line.

Support reference number

SUPPORT-3370, SUPPORT-3575

Description

When a user with insufficient permissions is exporting the order, the export is in running state indefinitely.

Expected Behavior

Error message stating the scope of permissions.

Current Behavior

Never ending running state

Context

No appropriate error message which confuses the customer.

Description

Currently order export does not export custom fields. Please add export of custom fields based on export templates both for order as well as lineItems.

v3 throws error while installing in yarn workspaces

current output:
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

desired output:
http://images/example1.jpg;http://images/example2.jpg;http://images/example3.jpg;http://images/example4.jpg;http://images/example5.jpg;http://images/example6.jpg;

Description

When I try to export the order with discount code to a CSV file using the 'discountCodes' field in the CVS template

Expected Behavior

I want to see the list of code used on the order

Current Behavior

I get the following error:

{ data: {}, project_key: 'trw-prod' } 'Exporting orders as stream.'
.{ data: 
   { message: 'Cannot read property \'code\' of undefined',
     name: 'TypeError',
     stack: 'TypeError: Cannot read property \'code\' of undefined\n    at /app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:119:34\n    at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n    at formatDiscountCodes (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:117:14)\n    at CsvMapping._getValue (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:70:25)\n    at /app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:42:22\n    at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n    at CsvMapping._mapOrder (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:40:17)\n    at /app/node_modules/sphere-order-export/lib/orderexport.js:132:39\n    at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n    at _this.client.orders.expand.expand.expand.expand.perPage.process.accumulate (/app/node_modules/sphere-order-export/lib/orderexport.js:131:22)\n    at /app/node_modules/sphere-order-export/node_modules/sphere-node-sdk/lib/services/base.js:268:20\n    at tryCatcher (/app/node_modules/sphere-order-export/node_modules/bluebird/js/main/util.js:26:23)\n    at Promise._settlePromiseFromHandler (/app/node_modules/sphere-order-export/node_modules/bluebird/js/main/promise.js:489:31)\n    at Promise._settlePromiseAt (/app/node_modules/sphere-order-export/node_modules/bluebird/js/main/promise.js:565:18)\n    at Promise._settlePromises (/app/node_modules/sphere-order-export/node_modules/bluebird/js/main/promise.js:681:14)\n    at Async._drainQueue (/app/node_modules/sphere-order-export/node_modules/bluebird/js/main/async.js:123:16)\n    at Async._drainQueues (/app/node_modules/sphere-order-export/node_modules/bluebird/js/main/async.js:133:10)\n    at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/sphere-order-export/node_modules/bluebird/js/main/async.js:15:14)\n    at processImmediate [as _immediateCallback] (timers.js:396:17)',
     code: undefined,
     signal: undefined },
  project_key: 'trw-prod' } 'Oops, something went wrong!'

Context

I need to export the orders with promotion code. It seems that the order needs to be expanded with discountCodes[*].discountCode but this is not possible from the web

As Node 8 is reaching end-of-life, we should run tests on 10/12 pairings and drop support for node < 10

It's somewhat unclear without trying out whether the CSV export is exporting a line per order line item or a line per order or dynamically depending on whether a lineItems.* column is requested or not.

Export of discountCodes fails if discountCode is deleted. The solution that I propose here #64 does simply remove the discountCode from the export.
Perhaps it is better to export something else instead, like code.id or string "code_deleted" . ?

{ body: undefined, message: 'Cannot read property \'code\' of undefined', name: 'TypeError', stack: 'TypeError: Cannot read property \'code\' of undefined\n at /app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:119:34\n at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n at formatDiscountCodes (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:117:14)\n at CsvMapping._getValue (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:70:25)\n at /app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:42:22\n at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n at CsvMapping._mapOrder (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:40:17)\n at /app/node_modules/sphere-order-export/lib/orderexport.js:132:39\n at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n at _this.client.orders.expand.expand.expand.expand.perPage.process.accumulate (/app/node_modules/sphere-order-export/lib/orderexport.js:131:22)\n at /app/node_modules/sphere-node-sdk/lib/services/base.js:267:20\n at tryCatcher (/app/node_modules/sphere-node-sdk/node_modules/bluebird/js/main/util.js:26:23)\n at Promise._settlePromiseFromHandler (/app/node_modules/sphere-node-sdk/node_modules/bluebird/js/main/promise.js:503:31)\n at Promise._settlePromiseAt (/app/node_modules/sphere-node-sdk/node_modules/bluebird/js/main/promise.js:577:18)\n at Promise._settlePromises (/app/node_modules/sphere-node-sdk/node_modules/bluebird/js/main/promise.js:693:14)\n at Async._drainQueue (/app/node_modules/sphere-node-sdk/node_modules/bluebird/js/main/async.js:123:16)\n at Async._drainQueues (/app/node_modules/sphere-node-sdk/node_modules/bluebird/js/main/async.js:133:10)\n at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/sphere-node-sdk/node_modules/bluebird/js/main/async.js:15:14)\n at processImmediate [as _immediateCallback] (timers.js:396:17)', code: undefined, statusCode: undefined, signal: undefined }, project_key: '***' } 'Oops, something went wrong!'

Add a CLI parameter which will work as a predicate to filter only selected orders.

similar to https://github.com/sphereio/sphere-node-product-csv-sync#usage-3

Add support of custom line items

https://github.com/commercetools/express-impex/issues/688

Implement id and slug on priority as export options.
Others:- name, quantity, price

@hisabimbola sphere-order-export creates more than one update actions for each order, it creates number_of_line_items + 1 update actions however we only need 1 update action per order regardless of the line items.

Sample: orders_sync_1486046489898.txt

@MelNiermann

Remove github vulneribility warnings

current direct dependencies has some github vulnerability because of the Transitive dependencies. So we need to update below packages.

sphere-node-sdk update to 3.2.2
sphere-node-utils update to 2.0.1
handlebars update to 4.7.6
loadash update to 4.17.15

https://github.com/sphereio/sphere-order-export/blob/320103142285b7e7526cfe0c2814f1d3c36384d8/src/coffee/orderexport.coffee#L139-L143

This and the following lines seems to contain some project-specific business logic where there was an additional customObject containing payment info for every cart required. This is not the case in all other projects. Remove it?

Support reference number

SUPPORT-4978

Description

Deleted discount code that has been used on a completed order crashes order export.

{ data: {}, project_key: 'xxxxxxxxxxxxx' } 'Created output dir at /tmp/tmp-26iwVN9wz1f6DK'
{ data: {}, project_key: 'xxxxxxxxxxxxx' } 'Exporting orders as a stream.'
.{ data:
   { body: undefined,
     message: 'Cannot read property \'code\' of undefined',
     name: 'TypeError',
     stack:
      'TypeError: Cannot read property \'code\' of undefined\n    at /app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:153:35\n    at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n    at CsvMapping.formatDiscountCodes (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:151:14)\n    at CsvMapping._getValue (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:84:25)\n    at /app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:50:22\n    at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n    at CsvMapping._mapOrder (/app/node_modules/sphere-order-export/lib/mapping-utils/csv.js:48:17)\n    at /app/node_modules/sphere-order-export/lib/orderexport.js:132:39\n    at Function._.map._.collect (/app/node_modules/underscore/underscore.js:172:24)\n    at _this.client.orders.expand.expand.expand.expand.perPage.process.accumulate (/app/node_modules/sphere-order-export/lib/orderexport.js:131:22)\n    at /app/node_modules/sphere-node-sdk/lib/services/base.js:267:20\n    at tryCatcher (/app/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:512:31)\n    at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:569:18)\n    at Promise._settlePromise0 (/app/node_modules/bluebird/js/release/promise.js:614:10)\n    at Promise._settlePromises (/app/node_modules/bluebird/js/release/promise.js:694:18)\n    at _drainQueueStep (/app/node_modules/bluebird/js/release/async.js:138:12)\n    at _drainQueue (/app/node_modules/bluebird/js/release/async.js:131:9)\n    at Async._drainQueues (/app/node_modules/bluebird/js/release/async.js:147:5)\n    at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/bluebird/js/release/async.js:17:14)\n    at runCallback (timers.js:705:18)\n    at tryOnImmediate (timers.js:676:5)\n    at processImmediate (timers.js:658:5)',
     code: undefined,
     statusCode: undefined,
     signal: undefined },
  project_key: 'xxxxxxxxxxxxx' } 'Oops, something went wrong!'


Process exited with code (1)

Expected Behavior

Should export order without error.

Current Behavior

Orders associated with discount codes that have been deleted are not exportable.

Context

IMPEX(Order export).

Hi,
Could you please allow exporting the following attributes on a LineItem level? In case you can't implement the export for all the attributes at once, they are listed already prioritized, you can start with customerGroup and we step by step cross them out of the "to do list".

• customerGroup
• shippingMethod
• mastersku
• taxonomyId
• targetGroup
• brandName
• color
• events
• locations
• channel ( any information that identifies the Merchant project to which the order belongs )

When we fire the export request on impex using headers that refer to LineItem level, the the entire information on the .csv should be on LineItem level.
For Order # 140721PDE5200005908, for example, we should have 3 rows on the exported csv, as we have 3 items inside the order, but instead we have only one. We don't really get any information from the LineItem. Can you add the LineItem level precision to the Order Export?

Thank you!

Add createdFrom and createdTo parameters with a UTC format for a date range selection.
If createdTo is not set, use current timestamp.

Intro: We have the requirements from our US customer to be able to export attributes within the syncInfo object.

Template: orderNumber,syncInfo.externalId

Thanks