Dynamo DB seems to be corrupting shared-local-instance.db file about dynamodb-admin HOT 9 OPEN

I think I managed to isolate the culprit to the option: -optimizeDbBeforeStartup...

from dynamodb-admin.

I've tried on the database I have handy and couldn't reproduce. I think this could depend on the structure of the database. If you have exact steps, including db structure then please share.

from dynamodb-admin.

I hope this helps:

This was a rather simple database, with only 5 tables, most of them having only a hash and range keys and no other indexes, except for one, which had a secondary global index with hash and range keys.

All the tables were created (locally) through DynamoDB admin, so the "workflow" was to run docker-compose up with the dynamodb-local instance, then jump into dynamodb-admin and create the tables. One of these tables was populated through a JS script, but that was only added later, and after the problem was already known to happen. The other ones were left empty and would only be populated through the testing of the API.

Writing to the database was done through js, using aws-sdk. Writes worked a-ok, and a first update worked ok as well, but as soon as the API tried reading an item that had been updated, dynamodb-local would crash. This also happened if we tried to update an item manually, as I described.

Since we stopped using dynamodb-admin we haven't had such a problem, despite not changing anything other than that (the code is the same, and the docker instance is the same as well).

Do you think there might be somewhere I can find logs for either dynamodb-admin or dynamodb-local? This way I might provide more/better info.

Other than that, I hope this helps.

from dynamodb-admin.

I have this issue as well. Any UpdateItem operation duplicates the item and then just the db stops working. I can literally replicate it with a single table.

  MyOwnTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: MyOwnTable
      TableClass: STANDARD
      AttributeDefinitions:
        - AttributeName: another_id
          AttributeType: S
        - AttributeName: this_id
          AttributeType: S
      KeySchema:
        - AttributeName: another_id
          KeyType: HASH
        - AttributeName: this_id
          KeyType: RANGE
      BillingMode: PAY_PER_REQUEST

If you use dynamodb-admin to go to this table, create the following item.

{
  "another_id": "123",
  "this_id": "abc",
  "myVal": 4
}

Click the Save button. Reopen the same item and then change "myVal" to 5.

The UI doesn't react, the log fills up with

ar 31, 2022 3:22:02 AM com.almworks.sqlite4java.Internal log
WARNING: [sqlite] SQLiteDBAccess$14@1991783d: job exception
com.amazonaws.services.dynamodbv2.local.shared.exceptions.LocalDBAccessException: Given key conditions were not unique. Returned: **details on the fields you just changed**
	at com.amazonaws.services.dynamodbv2.local.shared.access.LocalDBUtils.ldAccessFail(LocalDBUtils.java:799)
	at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccessJob.getRecordInternal(SQLiteDBAccessJob.java:224)
	at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess$14.doWork(SQLiteDBAccess.java:1555)
	at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess$14.doWork(SQLiteDBAccess.java:1551)
	at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.AmazonDynamoDBOfflineSQLiteJob.job(AmazonDynamoDBOfflineSQLiteJob.java:117)
	at com.almworks.sqlite4java.SQLiteJob.execute(SQLiteJob.java:372)
	at com.almworks.sqlite4java.SQLiteQueue.executeJob(SQLiteQueue.java:534)
	at com.almworks.sqlite4java.SQLiteQueue.queueFunction(SQLiteQueue.java:667)
	at com.almworks.sqlite4java.SQLiteQueue.runQueue(SQLiteQueue.java:623)
	at com.almworks.sqlite4java.SQLiteQueue.access$000(SQLiteQueue.java:77)
	at com.almworks.sqlite4java.SQLiteQueue$1.run(SQLiteQueue.java:205)
	at java.lang.Thread.run(Thread.java:748)

and finally, we can see that there are two items on the table. Clicking on them triggers more of these logs.

from dynamodb-admin.

I just found out something interesting. I downloaded NoSQL Workbench from Amazon and tried it again. I get the same thing. I think we can safely assume that this doesn't depend on the UI for the Dynamodb local instance, but rather the way the instance is setup. I'm currently using https://www.serverless.com/plugins/serverless-dynamodb-local, which COULD be the culprit. This plugin uses https://www.npmjs.com/package/dynamodb-localhost as an underlying dependency, and this one uses the official DynamoDBLocal.jar file... Interesting 🤔

from dynamodb-admin.

I added noStart:true for serverless-dynamodb-local, and decided to stand up my own Dockerized Local DynamoDB. It works. It does seem there's an issue on how serverless-dynamodb-local or dynamodb-localhost work.

@AugustoQueiroz By any chance, were you using any of those two libraries?

from dynamodb-admin.

Removing that option worked for me.

from dynamodb-admin.

Maybe it is related to saving different data comprised out of different Dynamo's ConversionSchema (V1, V2, V2_COMPATIBLE). This can potentially courrpt the data.

from dynamodb-admin.

Yes, UpdateItem is created a new record and cause duplicated key for me. It is also occur in my both MacOS and wsl Ubuntu.

from dynamodb-admin.