t-fowl / everyday-rewards-receipts Goto Github PK

View Code? Open in Web Editor NEW

6.0 1.0 0.0 32 KB

Download all receipts from your everyday rewards account

License: MIT License

Rust 100.00%

cli download downloader groceries receipt rust scraper transactions woolworths everyday-rewards woolies

everyday-rewards-receipts's Introduction

everyday-rewards-receipts

Simple tool to download all receipts from everyday rewards for later processing.

This tool will download the pdf receipt as well as store the api response of the transaction alongside as json.
This sidecar file is useful as it contains the itemisation for the receipt - negating the need to strip text from the pdfs.

Installation

If you are a Rust developer you can install from crates.io

cargo install everyday-rewards-receipts

Usage

> everyday-rewards-receipts --help
Usage: everyday-rewards-receipts [OPTIONS] --token <TOKEN>

Options:
      --token <TOKEN>  
  -o, --output <PATH>  [default: ./receipts]
  -h, --help           Print help information
  -V, --version        Print version information

Retrieving your authentication token:

Add the following bookmark

javascript:(function(){alert("Access Token: " + JSON.parse(sessionStorage.authStatusData).access_token)})();

Go to everyday rewards
Login
Click on the bookmark and note down the presented value

Output structure:

output/
    # Derived from the Activity feed in everyday rewards
    This_Month/
        id0.pdf
        id0.json
        ...
    August_2021/
        id1.pdf
        id1.json
        ...
    August_2022/
        id2.pdf
        id2.json
        ...
    ...

Licence

MIT License

Copyright (c) [2022] [Thomas Fowler]

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

everyday-rewards-receipts's People

Contributors

Stargazers

Watchers

everyday-rewards-receipts's Issues

Don't Downloading duplicate receipts

When running old receipts appear to be re-downloaded because they are served with a different filename.

Should be tackled at the same time as #4

Build and release binaries

Use GHA to test, build and release binaries

Add tests

unable to download all receipts - some months get skipped

Issue:

No receipt for online transactions

Working Transaction ID: 1732416902 (In-store)
Not Working Transaction ID: 1738356113 (Online)

Output Reference:

In-store transaction works fine:
Item { id: "1732416902", receipt: Some(...) }

Online transaction gets skipped due to no receipt:
Item { id: "1738356113", receipt: None }
Skipping 1738356113 as there is no associated receipt.

JSON for Working Transaction (In-store):

 {'id': '1732416902', 'displayDate': 'Fri 23 Feb', 'description': '$37.49 at Neutral Bay', 'message': '', 'displayValue': '+ 38 pts', 'displayValueHandling': 'NORMAL', 'icon': 'woolworths', 'iconUrl': 'https://prod.mobile-api.woolworths.com.au/zeus/mnemosyne/v1/public/activity/supermarkets_division_logo.png', 'transaction': {'origin': 'Neutral Bay', 'amountAsDollars': '$37.49'}, 'highlights': [], 'receipt': {'receiptId': 'U2FsdGVkX1+UnHRzqGXiA963dcd+pflUSI4kgLphDirnVa1EOINXPj1zeXR/u4crP/8CkhBbOXE3TBrZ+7BMj/7igzJ+24jsRa5Me+Fx3Bw=', 'receiptSource': 'INSTORE'}, 'transactionType': 'purchase', 'actionURL': 'com.woolworths.rewards://activityDetails?id=eyJhY3Rpdml0eUlkIjoiMTczMjQxNjkwMiIsInJlY2VpcHRJZCI6IlUyRnNkR1ZrWDErVW5IUnpxR1hpQTk2M2RjZCtwZmxVU0k0a2dMcGhEaXJuVmExRU9JTlhQajF6ZVhSL3U0Y3JQLzhDa2hCYk9YRTNUQnJaKzdCTWovN2lnekorMjRqc1JhNU1lK0Z4M0J3PSIsInJlY2VpcHRTb3VyY2UiOiJJTlNUT1JFIn0=', 'showChevron': True}

JSON for failing Transaction (Online):

{'id': '1738356113', 'displayDate': 'Mon 26 Feb', 'description': '$138.51 online at Woolworths Supermarket', 'message': "Here's $10 Everyday Rewards Dollars", 'displayValue': '+ 336 pts', 'displayValueHandling': 'NORMAL', 'icon': 'woolworths', 'iconUrl': 'https://prod.mobile-api.woolworths.com.au/zeus/mnemosyne/v1/public/activity/supermarkets_division_logo.png', 'transaction': {'origin': 'Woolworths Supermarket', 'amountAsDollars': '$140.51'}, 'highlights': [{'iconUrl': 'https://prod.mobile-api.woolworths.com.au/zeus/mnemosyne/v1/public/activity/activity_list_piggybank.png', 'description': "Here's $10 Everyday Rewards Dollars", 'value': None, 'style': 'HIGHLIGHT'}, {'iconUrl': 'https://prod.mobile-api.woolworths.com.au/zeus/mnemosyne/v1/public/activity/activity_list_booster.png', 'description': '198 Bonus points collected', 'value': None, 'style': 'STANDARD'}], 'receipt': {'receiptId': 'U2FsdGVkX1+4EvL26v4unJN+NUs2VswNOPM7k/iL84lyviokW1TKyV2cD4HocL2WRAlufbTDzU5Mr7RsqVJY6l5kehxac5EiEdyYiXeKQ4g=', 'receiptSource': 'ONLINE'}, 'transactionType': 'purchase', 'actionURL': 'com.woolworths.rewards://activityDetails?id=eyJhY3Rpdml0eUlkIjoiMTczODM1NjExMyIsInJlY2VpcHRJZCI6IlUyRnNkR1ZrWDErNEV2TDI2djR1bkpOK05VczJWc3dOT1BNN2svaUw4NGx5dmlva1cxVEt5VjJjRDRIb2NMMldSQWx1ZmJURHpVNU1yN1JzcVZKWTZsNWtlaHhhYzVFaUVkeVlpWGVLUTRnPSIsInJlY2VpcHRTb3VyY2UiOiJPTkxJTkUifQ==', 'showChevron': True}

Major Differences:

receiptSource: The working transaction is INSTORE, whereas the problematic one is ONLINE.
receiptId: Both have valid receiptId values, but the application fails to process the online transaction's receipt.

Hope this helps pinpoint the problem. Let me know if you need more info!

Figure out how to get a long lived token

Find out how the apps do it

Publish to crates.io

Set create time / modtime for receipts to the transaction datetime

Recognise duplicate activity items one with a receipt and one without

Everyday rewards often has duplicate activity items in their feed one having a receipt and one without (Guessing it is to do with bonus rewards points etc).

Enhancement: Instead of logging that an item has been skipped because it doesn't have a receipt, recognise if it is a duplicate of an item which has a receipt and subsequently ignore

Flag to retroactively go back and rename old receipts to match the latest naming scheme

Since first developed item ids have changed numerous times, including ddMMyyyy, yyyyMMddHHmmss....$STORE, and now base-64 encoded OpenSSL encrypted values which don't lend themselves to being on a filesystem path.

In ba68fc3 the output path was changed to the generated human-readable filename provided when downloading receipts, and I think it would be beneficial to retroactively apply this (and future) output paths to old downloads.

Thankfully this is possible due to saving the raw json response as sidecar files.