starsprung / amazon-ynab-sync Goto Github PK

View Code? Open in Web Editor NEW

42.0 42.0 5.0 1.21 MB

License: MIT License

JavaScript 4.23% TypeScript 95.77%

amazon-ynab-sync's People

Contributors

Stargazers

Watchers

Forkers

yanowitz cwspear twarne cwilby dlardo

amazon-ynab-sync's Issues

add a split-transaction charge-centric mode

(I'm happy to do this work, I just wanted to open an issue first to make sure it was friendly to the project).

All my Amazon spend is on a single credit card. What I want to do is have a transaction for the credit card charge, not the individual order items. The transaction can have a split for every transaction associated with the charge (right now, if there's a charge with three items and a coupon, we get 4 transactions. in this mode, you have one transaction with four splits sub-transactions.

I think the schema of the two files (Items and Orders & Shipments) allows for this, though I need to check various edge cases.

Gift Wrap is not accounted for

I ran the sync for the last 30 days and it got some of my Christmas gifts in the mix but the gift wrap was not added in anywhere. I had to go back and add separate line items to make my accounts reconcile.

I don't know if this is possible to add given that it does not show up in the order export from amazon, but it would be a great to have before next Christmas!

Submitting XX Transactions, Submitted 0 to YNAB

Hi, I think I got it working, but not sure what to do next. It says it has grabbed 59 transactions from Amazon, but 0 are submitted to YNAB

Where do I even start to figure this out?

I have silly level debugging turned on and it really does not show me anything

ynab-sync_1  | {"level":"debug","message":"Submitting batch of 59 transactions","metadata":{},"timestamp":"2022-09-26T15:03:03.151Z"}
ynab-sync_1  | {"level":"debug","message":"Reading XXXXXXXXXX-YYYYYY,Amazon from cache","metadata":{},"timestamp":"2022-09-26T15:03:03.151Z"}
ynab-sync_1  | {"level":"debug","message":"Path for XXXXXXXXXX-YYYY,Amazon is /home/ynabsync/.cache/amazon-ynab-sync/a0c4c1f60758c0a7c8f47b8940a1c98c4b66debbe27782e9590d9bd7c140b043.cache","metadata":{},"timestamp":"2022-09-26T15:03:03.151Z"}
ynab-sync_1  | {"level":"debug","message":"No cache value found for XXXXXXXXXX-YYYYYYY,Amazon","metadata":{},"timestamp":"2022-09-26T15:03:03.151Z"}
ynab-sync_1  | {"level":"debug","message":"No cached budget ID found","metadata":{},"timestamp":"2022-09-26T15:03:03.151Z"}
ynab-sync_1  | {"level":"info","message":"Submitted 0 transations to YNAB","metadata":{},"timestamp":"2022-09-26T15:03:03.304Z"}

Investigate providing the ability to map Amazon categories to budget categories automatically

"Subscribe & Save" (and maybe other) discounts not showing up

I have an item that I order monthly. Since I have multiple subscriptions, I get a discount. The item got shipped/charged alone. Here's a (cropped) screenshot:

The item shows up in YNAB as $10.02 which is the item's value + tax. The "subscribe and save" discount of $1.42 does not show up.

I checked the Items report, and it shows up as follows (redacted, but I attempted to keep CSV structure):

12/17/20,XXX,"XXX",XXX,"XXX","XXX",Amazon.com,,new,Amazon.com,,$9.99,$9.44,1,"XXX",,,[email protected],12/21/20,XXX,XXX,,XXX,XXX,1234-1234,Shipped,AMZN_US(XXX),$9.44,$0.58,$10.02,false,,false,XXX,USD,

The Orders and Shipments row is as follows:

12/17/20,XXX,"XXX",Amazon.com,,[email protected],12/21/20,XXX,XXX,,XXX,XXX,1234-1234,Shipped,AMZN_US(XXX),$9.44,$0.00,$0.58,$1.42,$0.58,$8.60,XXX,

Note that the discount of $1.42 shows up there (under the "Total Promotions" column).

How to handle multiple Amazon accounts?

My spouse and I have separate Amazon accounts, but all purchases for both accounts are charged to the same credit card account.

It looks like in order to use this tool I would need to run it twice, once for my Amazon account and once for hers. Are there any gotchas I should look out for?

How to run on a mac?

Puppeteer keeps looking for chromium at /usr/bin/chromium-browser and no config flag or env variable seems to be able to override it. (/usr/bin can't be written to on a mac, even with SIP disabled)

Any tips?

This no longer works

Since Amazon has canceled the option to create Order History Reports (https://www.amazon.com/b2b/reports), this no longer works. Such a bummer! This no longer works as of March 2023.

Thank you for @starsprung for making this happen. Hopefully there'll be a way to achieve this again in the future.

Pupeteer Error - Evaluation failed: Error: Element is not a <select> element.

I was able to sync on 05/22/2022 but today I get the same error on multiple installations (multiple computers).

Error: Evaluation failed: Error: Element is not a element. at :3:23 at ExecutionContext._evaluateInternal (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:217:19) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async ExecutionContext.evaluate (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:106:16) at async ElementHandle.evaluate (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer/lib/cjs/puppeteer/common/JSHandle.js:102:16) at async DOMWorld.select (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:295:24) at async AmazonOrderReportsApi._fillReportForm (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/amazon-order-reports-api/lib/index.js:232:9) at async AmazonOrderReportsApi._getReport (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/amazon-order-reports-api/lib/index.js:267:28) at async AmazonOrderReportsApi.getItems (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/amazon-order-reports-api/lib/index.js:124:9) at async Object.getAmazonTransactions (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/lib/amazon.js:72:26) at async Object.batch [as default] (/home/MYUSERNAME/.nvm/versions/node/v16.15.1/lib/node_modules/amazon-ynab-sync/node_modules/it-batch/index.js:20:20)

Option have Payee always be Amazon.com or other default

I like to be able to track how much I spend on Amazon with the reports and looking at how much per payee.

Add an option to distribute shipping/promotions proportionally to items in order

As discussed in #11

This is easy for single-item orders, but harder to do accurately for multi-item orders.

Support international users

This may be possible via "Request my Data" reports, though with some limitations.

How to integrate with Amazon/Chase credit card?

We have an Amazon credit card that is used exclusively at Amazon. I'm not sure how to integrate this with the amazon-ynab-sync tool. If I set up a new cash account for the sync, it'll duplicate my charges I think with the ones showing up from the Amazon card. Is there a way to do this? I'd love to be able to categorize the charges on the card... right now I just lump them into a single "household" category.

TimeoutError after 30-50 seconds

I used to get this occasionally, but it would work properly when I tried again, so I assumed it was just amazon servers being slow.

Now it errors out every time (maybe 10 tries), and in less than 60 seconds (usually about 35), so I'm thinking it's a bug.

Or maybe there's just a 30 second timeout that needs to be higher with current server load? [ ETA: YES. See comments two weeks later. Don't bother reading more of this entry. ]

DEBUG=* shows:

------- lots of bursts of puppeteer debugging info cut --------
...
  puppeteer:protocol:RECV ◀ {"method":"Network.dataReceived","params":{"requestId":"85537.975","timestamp":197556.927511,"dataLength":89,"encodedDataLength":0},"sessionId":"65696E4FF54A1FE685C48DB4"} +0ms
  puppeteer:protocol:RECV ◀ {"method":"Network.dataReceived","params":{"requestId":"85537.975","timestamp":197556.928222,"dataLength":0,"encodedDataLength":131},"sessionId":"65696E4FF54A1FE685C48DB4"} +1ms
  puppeteer:protocol:RECV ◀ {"method":"Network.loadingFinished","params":{"requestId":"85537.975","timestamp":197556.925519,"encodedDataLength":446,"shouldReportCorbBlocking":false},"sessionId":"65696E4FF54A1FE685C48DB4"} +0ms
------- this is the last pause before termination, for 1-2 seconds --------
  puppeteer:protocol:SEND ► {"method":"Browser.close","id":301} +28s
  puppeteer:protocol:RECV ◀ {"id":301,"result":{}} +2s
  puppeteer:protocol:RECV ◀ {"method":"Target.detachedFromTarget","params":{"sessionId":"65696E4FF54A1FE685C48DB4","targetId":"A5D2F3249B49396AA7921E4D"}} +0ms
  puppeteer-extra-plugin:user-data-dir onClose +49s
  puppeteer-extra-plugin:user-data-dir removeUserDataDir +0ms
(node:85487) [DEP0147] DeprecationWarning: In future versions of Node.js, fs.rmdir(path, { recursive: true }) will be removed. Use fs.rm(path, { recursive: true }) instead
(Use `node --trace-deprecation ...` to show where the warning was created)
  puppeteer-extra-plugin:user-data-dir onClose +13ms
  puppeteer-extra-plugin:user-data-dir removeUserDataDir +0ms
  puppeteer-extra-plugin:user-data-dir Error: ENOENT: no such file or directory, stat '/tmp/puppeteer_dev_profile-kFpxUA'
  puppeteer-extra-plugin:user-data-dir     at Object.statSync (node:fs:1526:3)
  puppeteer-extra-plugin:user-data-dir     at __node_internal_ (node:internal/fs/utils:767:8)
  puppeteer-extra-plugin:user-data-dir     at Object.rmdirSync (node:fs:1146:15)
  puppeteer-extra-plugin:user-data-dir     at Plugin.deleteUserDataDir (/home/sandro/.npm-global/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer-extra-plugin-user-data-dir/index.js:72:11)
  puppeteer-extra-plugin:user-data-dir     at Plugin.onClose (/home/sandro/.npm-global/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer-extra-plugin-user-data-dir/index.js:114:12)
  puppeteer-extra-plugin:user-data-dir     at process.emit (node:events:406:35)
  puppeteer-extra-plugin:user-data-dir     at process._fatalException (node:internal/process/execution:177:19)
  puppeteer-extra-plugin:user-data-dir     at processPromiseRejections (node:internal/process/promises:246:11)
  puppeteer-extra-plugin:user-data-dir     at processTicksAndRejections (node:internal/process/task_queues:97:32) +0ms
/home/sandro/.npm-global/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer/lib/cjs/puppeteer/common/helper.js:120
            rejectCallback(new Errors_js_1.TimeoutError('Timeout exceeded while waiting for event'));
                           ^

TimeoutError: Timeout exceeded while waiting for event
    at Timeout.<anonymous> (/home/sandro/.npm-global/lib/node_modules/amazon-ynab-sync/node_modules/puppeteer/lib/cjs/puppeteer/common/helper.js:120:28)
    at listOnTimeout (node:internal/timers:557:17)
    at processTimers (node:internal/timers:500:7)
Command exited with non-zero status 1
3.45user 0.98system 0:49.33elapsed 8%CPU (0avgtext+0avgdata 108328maxresident)k

UnhandledPromiseRejectionWarning

Hi there -

Thanks for this tool, it is exactly what I need! I've not been able to get it to work successfully yet. I'm getting emails from Amazon about "order history report is ready", seems like I get two of those per run of the tool, so I know it's authenticating to Amazon and getting the report. However, this is the output from the tool:

$ amazon-ynab-sync --trace-warnings
{"level":"info","message":"Retrieving reports from Amazon","metadata":{},"timestamp":"2021-01-05T20:00:38.759Z"}
(node:13423) UnhandledPromiseRejectionWarning: Error: Invalid Record Length: columns length is 18, got 1 on line 2
    at Parser.__onRow (/snapshot/amazon-ynab-sync/node_modules/csv-parse/lib/index.js:762:9)
    at Parser.__parse (/snapshot/amazon-ynab-sync/node_modules/csv-parse/lib/index.js:635:40)
    at Parser._flush (/snapshot/amazon-ynab-sync/node_modules/csv-parse/lib/index.js:453:22)
    at Parser.prefinish (_stream_transform.js:142:10)
    at Parser.emit (events.js:315:20)
    at prefinish (_stream_writable.js:618:14)
    at finishMaybe (_stream_writable.js:626:5)
    at Parser.Writable.end (_stream_writable.js:570:5)
    at ReadStream.onend (_stream_readable.js:677:10)
    at Object.onceWrapper (events.js:421:28)
(Use `amazon-ynab-sync --trace-warnings ...` to show where the warning was created)
(node:13423) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:13423) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.