Code Monkey home page Code Monkey logo

Comments (13)

jameskupke avatar jameskupke commented on May 29, 2024 1

Oh no, still using 0.3.25. Will run again with newer version.

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024 1

Looks good: let's close this, thanks.

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024

I'm guessing it's probably because Coinbase repports a native_amount == 0: could you paste the DaLI debug log line corresponding to that transaction? You can find it as follows:

  • open the DaLI-generated RP2 input file,
  • look for the transaction with spot_price set to 0,
  • copy its unique_id field
  • open the DaLI debug log
  • search for the line with the copied unique_id

Then paste here the line and I'll take a look.

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024

I guess, you already pasted a similar transaction in issue #16:

2022-03-26 08:27:11,229/Coinbase/name/DEBUG: Transaction: {"id": "ZZZZZZZZ-ZZZZ-ZZZZ-ZZZZ-ZZZZZZZZZZZZ", "type": "inflation_reward", "status": "completed", "amount": {"amount": "0.000166", "currency": "ALGO"}, "native_amount": {"amount": "0.00", "currency": "USD"}, "description": null, "created_at": "2021-XX-XXT14:55:27Z", "updated_at": "2021-XX-XXT14:55:27Z", "resource": "transaction", "resource_path": "/v2/accounts/135c87f3-44a8-5c13-ae9c-14ac0d182a79/transactions/ZZZZZZZZ-ZZZZ-ZZZZ-ZZZZ-ZZZZZZZZZZZZ", "instant_exchange": false, "from": {"id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "resource": "user", "resource_path": "/v2/users    /XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "currency": "ALGO"}, "details": {"title": "Algorand reward", "subtitle": "From Coinbase", "header": "Received 0.000166 ALGO ($0.00)", "health": "positive"}, "hide_native_amount": false}

DaLI computes the spot_price here: if native_amount/amount is 0 (which it is here, I imagine due to a rounding error), then spot_price will end up being zero. It doesn't look like there is any other data that Coinbase is giving us to figure the correct spot_price. I have an idea about how to resolve this: I'll try it out tomorrow and report back here.

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024

@jameskupke, I just uploaded a new version that should fix the problem: however you need to pass the -s CLI option. This option reads spot_price information from the Web if it's not available (or if it's 0, like in this case). The switch was available in previous versions, too, but it was using Yahoo Finance, which had 1-day time granularity. The new verison uses the Coinbase Pro API which has 1 min granularity. Let me know if this solves the problem, thanks.

from dali-rp2.

jameskupke avatar jameskupke commented on May 29, 2024

Hi @eprbell:

I just got around to pasting a transaction. Looks like the spot price may be really small? "0E+4"

2022-03-27 18:48:53,963/dali/DEBUG: Self-contained transaction: InTransaction:
  plugin=Coinbase
  unique_id=AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA
  raw_data={"id": "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA", "type": "inflation_reward", "status": "completed", "amount": {"amount": "0.000166", "currency": "ALGO"}, "native_amount": {"amount": "0.00", "currency": "USD"}, "description": null, "created_at": "2021-XX-XXT15:40:22Z", "updated_at": "2021-XX-XXT15:40:22Z", "resource": "transaction", "resource_path": "/v2/accounts/BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB/transactions/AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA", "instant_exchange": false, "from": {"id": "CCCCCCCC-CCCC-CCCC-CCCC-CCCCCCCCCCCC", "resource": "user", "resource_path": "/v2/users/CCCCCCCC-CCCC-CCCC-CCCC-CCCCCCCCCCCC", "currency": "ALGO"}, "details": {"title": "Algorand reward", "subtitle": "From Coinbase", "header": "Received 0.000166 ALGO ($0.00)", "health": "positive"}, "hide_native_amount": false}
  timestamp=2021-XX-XX 15:40:22 +0000
  asset=ALGO
  exchange=Coinbase
  holder=James Kupke
  transaction_type=Income
  spot_price=0E+4
  crypto_in=0.000166
  fiat_fee=0
  fiat_in_no_fee=0.00
  fiat_in_with_fee=0.00
  notes=Algorand reward

For reference here's the Coinbase raw CSV file for some of the transactions: (All the transaction IDs are removed, but they're normally in the first column)

Transaction ID,Transaction Type,Date & time,Asset Acquired,"Quantity Acquired (Bought, Received, etc)",Cost Basis (incl. fees paid) (USD),Data Source,"Asset Disposed (Sold, Sent, etc)",Quantity Disposed,Proceeds (excl. fees paid) (USD)
,Interest,2021-04-29T12:15:39Z,ALGO,0.000242,0.0003323386,Coinbase,,,
,Interest,2021-04-30T14:07:58Z,ALGO,0.000242,0.0003394897,Coinbase,,,
,Interest,2021-05-01T12:24:16Z,ALGO,0.000242,0.000336985,Coinbase,,,
,Interest,2021-05-02T07:42:50Z,DAI,0.00005544,0.000055545336,Coinbase,,,
,Interest,2021-05-02T10:59:38Z,ALGO,0.000242,0.0003308261,Coinbase,,,

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024

Can you verify for me the following:

  • are you using DaLI v 0.3.26?
  • are you passing the -s switch on the command line?

The spot prices of last year should be between $0.4 and $3, based on: https://www.coinbase.com/price/algorand

Can you check the following in the generated ODS file:

  • go to the ALGO tab
  • look for transactions that have the following in the Notes column: "spot_price read from Coinbase Pro"
  • check if the spot prices of these transactions are all bad, or only a few

The new spot_price-reading code is working ok for me, but I don't have Algorand transactions. Strange...

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024

No worries, the new version should output the correct value (but remember to use the -sswitch.

from dali-rp2.

jameskupke avatar jameskupke commented on May 29, 2024

Reran with the latest version with the -s switch.

Spot prices are now non zero values, but "USD In No Fee" and "USD In No Fee" are $0.00

This causes this error when running RP2:

INFO: Country: us
INFO: Accounting Method: fifo
INFO: Configuration file: output/crypto_data.config
INFO: Input file: output/crypto_data.ods
INFO: Processing ALGO
ERROR: Fatal exception occurred:
Traceback (most recent call last):
  File "/home/james/.local/lib/python3.10/site-packages/rp2/rp2_main.py", line 93, in _rp2_main_internal
    input_data: InputData = parse_ods(configuration=configuration, asset=asset, input_file_handle=input_file_handle)
  File "/home/james/.local/lib/python3.10/site-packages/rp2/ods_parser.py", line 128, in parse_ods
    _create_and_process_transaction(
  File "/home/james/.local/lib/python3.10/site-packages/rp2/ods_parser.py", line 168, in _create_and_process_transaction
    transaction: AbstractTransaction = _create_transaction(configuration, current_table_type, internal_id, row_values)
  File "/home/james/.local/lib/python3.10/site-packages/rp2/ods_parser.py", line 277, in _create_transaction
    transaction = InTransaction(**argument_pack)
  File "/home/james/.local/lib/python3.10/site-packages/rp2/in_transaction.py", line 79, in __init__
    self.__fiat_in_no_fee = configuration.type_check_positive_decimal("fiat_in_no_fee", fiat_in_no_fee, non_zero=True)
  File "/home/james/.local/lib/python3.10/site-packages/rp2/configuration.py", line 279, in type_check_positive_decimal
    raise RP2ValueError(f"Parameter '{name}' has zero value")
rp2.rp2_error.RP2ValueError: Parameter 'fiat_in_no_fee' has zero value
INFO: Log file: ./log/rp2_2022_03_29_20_54_35_442552.log
INFO: Generated output directory: output/
INFO: Done

If I set "USD In" fields to Spot Price * Crypto In, it seems to process those fields correctly.

Not sure if this is still part of the same problem now, since Spot Price technically now shows up. I can close and cause separate issue if that makes sense.

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024

Sure, let's open a new issue. I think I know what the problem is: it's probably because Coinbase incorrectly reports the native_amount as 0 (same root cause as this issue) and DaLI uses that value to fill in the fiat_in fields. It should be easy to fix.

Before closing this issue: can you confirm the Algo spot price is between $0.4 and $3?

Thanks for providing all the data to help fix the bug!

from dali-rp2.

jameskupke avatar jameskupke commented on May 29, 2024

Sure, here's the timestamps and spot prices from the ods file:

Timestamp Spot Price
2021-04-27 11:19:48 +0000 $1.28
2021-04-28 12:22:48 +0000 $1.27
2021-04-29 12:15:39 +0000 $1.37
2021-04-30 14:07:58 +0000 $1.40
2021-05-01 12:24:16 +0000 $1.39
2021-05-02 10:59:38 +0000 $1.37
2021-05-03 14:01:51 +0000 $1.43
2021-05-04 10:37:18 +0000 $1.44
2021-05-05 11:07:01 +0000 $1.39
2021-05-06 12:12:58 +0000 $1.56
2021-05-07 13:06:17 +0000 $1.64
2021-05-08 14:17:02 +0000 $1.54
2021-05-09 14:03:53 +0000 $1.49
2021-05-10 12:44:16 +0000 $1.48
2021-05-11 11:05:05 +0000 $1.42
2021-05-12 12:04:09 +0000 $1.42
2021-05-13 12:57:51 +0000 $1.36
2021-05-14 11:22:14 +0000 $1.40
2021-05-15 11:26:04 +0000 $1.42
2021-05-16 13:23:07 +0000 $1.48
2021-05-17 12:41:04 +0000 $1.37
2021-05-18 12:29:58 +0000 $1.38
2021-05-19 12:48:05 +0000 $0.96
2021-05-20 12:24:49 +0000 $1.07
2021-05-21 12:20:53 +0000 $1.08
2021-05-22 13:27:50 +0000 $0.98
2021-05-23 12:54:41 +0000 $0.79
2021-05-24 11:34:18 +0000 $0.96
2021-05-25 10:54:07 +0000 $0.90
2021-05-26 12:06:57 +0000 $1.05
2021-05-27 11:32:26 +0000 $1.01
2021-05-28 12:59:07 +0000 $0.90
2021-05-29 11:17:31 +0000 $0.86
2021-05-30 13:03:10 +0000 $0.87
2021-05-31 12:03:17 +0000 $0.90
2021-06-01 11:24:31 +0000 $0.91
2021-06-02 14:03:05 +0000 $0.94
2021-06-03 11:00:31 +0000 $1.07
2021-06-04 13:55:14 +0000 $1.03
2021-06-05 14:33:58 +0000 $1.00
2021-06-06 13:05:42 +0000 $1.01
2021-06-07 13:03:21 +0000 $1.09
2021-06-08 14:16:36 +0000 $0.89
2021-06-09 11:40:28 +0000 $1.09
2021-06-10 13:59:52 +0000 $1.01
2021-06-11 11:42:10 +0000 $1.02
2021-06-12 13:50:37 +0000 $0.98
2021-06-13 11:22:35 +0000 $1.02
2021-06-14 10:44:41 +0000 $1.04
2021-06-15 11:41:35 +0000 $1.05
2021-06-16 13:32:37 +0000 $1.01
2021-06-17 13:41:44 +0000 $1.06
2021-06-18 10:42:12 +0000 $1.03
2021-06-19 14:04:54 +0000 $0.96
2021-06-20 13:22:57 +0000 $0.90
2021-06-21 13:20:18 +0000 $0.84
2021-06-22 12:09:02 +0000 $0.72
2021-06-23 11:43:23 +0000 $0.84
2021-06-24 13:52:50 +0000 $0.86
2021-06-25 11:59:26 +0000 $0.83
2021-06-26 12:51:35 +0000 $0.80
2021-06-27 14:23:04 +0000 $0.79
2021-06-28 14:00:12 +0000 $0.83
2021-06-29 13:59:17 +0000 $0.92
2021-06-30 11:38:16 +0000 $0.86
2021-07-01 12:26:56 +0000 $0.84
2021-07-02 12:30:26 +0000 $0.81
2021-07-03 11:35:48 +0000 $0.86
2021-07-04 13:45:53 +0000 $0.90
2021-07-05 14:32:54 +0000 $0.86
2021-07-06 12:57:37 +0000 $0.93
2021-07-07 13:25:32 +0000 $0.93
2021-07-08 12:43:31 +0000 $0.87
2021-07-09 14:43:02 +0000 $0.92
2021-07-10 11:16:17 +0000 $0.88
2021-07-11 12:54:12 +0000 $0.91
2021-07-12 14:19:44 +0000 $0.88
2021-07-13 13:43:56 +0000 $0.86
2021-07-14 12:59:00 +0000 $0.85
2021-07-15 14:29:54 +0000 $0.83
2021-07-16 13:13:17 +0000 $0.80
2021-07-17 11:50:51 +0000 $0.78
2021-07-18 13:33:44 +0000 $0.80
2021-07-19 13:14:14 +0000 $0.76
2021-07-20 13:49:30 +0000 $0.69
2021-07-21 13:17:58 +0000 $0.75
2021-07-22 18:42:14 +0000 $0.79
2021-07-23 15:22:51 +0000 $0.80
2021-07-24 15:00:54 +0000 $0.84
2021-07-25 17:44:25 +0000 $0.87
2021-07-26 11:54:28 +0000 $0.90
2021-07-27 17:58:10 +0000 $0.83
2021-07-28 18:19:44 +0000 $0.84
2021-07-29 15:14:12 +0000 $0.84
2021-07-30 17:17:24 +0000 $0.83
2021-07-31 17:19:15 +0000 $0.84
2021-08-01 17:39:26 +0000 $0.84
2021-08-02 16:43:20 +0000 $0.84
2021-08-03 14:57:18 +0000 $0.83
2021-08-04 15:06:41 +0000 $0.83
2021-08-05 14:07:54 +0000 $0.84
2021-08-06 13:38:52 +0000 $0.85
2021-08-07 13:09:26 +0000 $0.86
2021-08-08 14:07:17 +0000 $0.88
2021-08-09 16:21:17 +0000 $0.87
2021-08-10 12:25:52 +0000 $0.90
2021-08-11 13:29:58 +0000 $0.93
2021-08-12 12:25:35 +0000 $0.90
2021-08-13 16:27:40 +0000 $0.92
2021-08-14 12:24:21 +0000 $0.93
2021-08-15 13:44:21 +0000 $0.93
2021-08-16 12:36:56 +0000 $1.00
2021-08-17 14:30:36 +0000 $0.96
2021-08-18 12:51:31 +0000 $0.90
2021-08-19 11:05:16 +0000 $0.98
2021-08-20 13:52:44 +0000 $1.11
2021-08-21 14:28:13 +0000 $1.20
2021-08-22 15:00:28 +0000 $1.14
2021-08-23 13:42:31 +0000 $1.17
2021-08-24 17:42:13 +0000 $1.06
2021-08-25 11:35:11 +0000 $1.02
2021-08-26 16:06:48 +0000 $1.00
2021-08-27 12:25:52 +0000 $1.02
2021-08-28 11:15:05 +0000 $1.04
2021-08-29 13:36:01 +0000 $1.02
2021-08-30 13:21:57 +0000 $0.99
2021-08-31 18:08:52 +0000 $1.12
2021-09-01 17:31:58 +0000 $1.15
2021-09-02 17:12:58 +0000 $1.14
2021-09-03 15:19:40 +0000 $1.24
2021-09-04 12:26:40 +0000 $1.22
2021-09-05 14:48:29 +0000 $1.29
2021-09-06 13:52:59 +0000 $1.34
2021-09-07 11:42:26 +0000 $1.36
2021-09-08 15:33:28 +0000 $1.74
2021-09-09 15:39:42 +0000 $2.24
2021-09-10 13:04:01 +0000 $2.05
2021-09-11 18:43:02 +0000 $2.08
2021-09-12 15:18:12 +0000 $2.33
2021-09-13 16:47:12 +0000 $2.26
2021-09-14 17:54:29 +0000 $2.07
2021-09-15 17:29:59 +0000 $2.06
2021-09-16 11:44:49 +0000 $2.20
2021-09-17 18:32:00 +0000 $1.94
2021-09-18 15:30:13 +0000 $2.04
2021-09-19 15:07:00 +0000 $2.03
2021-09-21 11:55:13 +0000 $1.82
2021-09-22 14:49:44 +0000 $1.78
2021-09-23 15:56:31 +0000 $1.97
2021-09-24 09:21:34 +0000 $1.80
2021-09-24 11:15:38 +0000 $1.71
2021-09-25 13:53:23 +0000 $1.77
2021-09-26 17:17:24 +0000 $1.71
2021-09-27 14:55:27 +0000 $1.78
2021-09-28 14:23:26 +0000 $1.63
2021-09-29 13:50:07 +0000 $1.65
2021-09-30 17:59:50 +0000 $1.62
2021-10-01 16:09:29 +0000 $1.76
2021-10-02 16:42:03 +0000 $1.83
2021-10-03 10:48:18 +0000 $1.80
2021-10-04 12:29:49 +0000 $1.97
2021-10-05 15:46:49 +0000 $1.94
2021-10-06 17:17:14 +0000 $1.90
2021-10-07 11:03:25 +0000 $1.83
2021-10-08 12:51:02 +0000 $1.91
2021-10-09 17:42:40 +0000 $1.90
2021-10-10 10:52:34 +0000 $1.86
2021-10-11 17:21:59 +0000 $1.77
2021-10-12 12:27:10 +0000 $1.63
2021-10-13 12:10:55 +0000 $1.73
2021-10-14 14:43:14 +0000 $1.88
2021-10-15 11:49:18 +0000 $1.78
2021-10-16 13:58:29 +0000 $1.81
2021-10-17 18:42:34 +0000 $1.81
2021-10-18 12:18:44 +0000 $1.75
2021-10-19 16:58:35 +0000 $1.72
2021-10-20 18:43:57 +0000 $1.82
2021-10-21 12:13:30 +0000 $1.81
2021-10-22 13:10:59 +0000 $1.91
2021-10-23 14:08:33 +0000 $1.90
2021-10-24 11:29:10 +0000 $1.86
2021-10-25 17:28:48 +0000 $1.96
2021-10-26 14:21:19 +0000 $1.98
2021-10-27 11:38:30 +0000 $1.82
2021-10-28 13:40:20 +0000 $1.85
2021-10-29 15:04:42 +0000 $1.85
2021-10-30 14:31:53 +0000 $1.81
2021-10-31 17:30:42 +0000 $1.84
2021-11-01 10:52:12 +0000 $1.84
2021-11-02 17:10:53 +0000 $1.89
2021-11-03 16:19:50 +0000 $1.91
2021-11-04 15:40:22 +0000 $1.86

Looks okay to close?

from dali-rp2.

eprbell avatar eprbell commented on May 29, 2024

The USD In set to 0 was caused by Coinbase returning low-precision fiat numbers (and rounding to 0 any amount < $0.01). I added a workaround in the latest version. @jameskupke, could you test it and let me know if it works? Thanks!

from dali-rp2.

jameskupke avatar jameskupke commented on May 29, 2024

Ok, running on 0.4.0

from dali-rp2.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.