passren / pydynamodb Goto Github PK

Thank you for making this library!

If I issue:

SELECT * FROM "cache" WHERE "my-value" != ''

to both boto3 directly (pure PartiQL) and to pydynamodb, pydynamodb never returns (the table is large).

I see that the statement it pushes down is SELECT * FROM "cache".

I am guessing it is eagerly pre-fetching data and locally evaluating the WHERE.

I would expect the entire SELECT statement to be pushed down as it is to PartiQL, which is able to handle this.

This works in the DynamoDB PartiQL editor:

SELECT payload."size"
FROM "table"
WHERE pk='foo'

But when I do:

cursor.execute('SELECT payload."size" FROM "table" WHERE pk='foo')

I get this error:

dynamodb error: An error occurred (ValidationException) when calling the ExecuteStatement operation: Statement wasn't well formed, can't be processed: Invalid path dot component

An example of what my data looks like:

[
  {
    'pk': foo',
    'payload': {'size': 2, 'bla': 'bla'},
  },
  {
    'pk': 'bar',
    'payload': {'size': 2, 'bla': 'bla'},
  }
]

How may I go about selecting size from payload?

Note that I intend to use DynamoDB in Apache Superset. Was getting the same error there in the SQL Lab section. Digging around their repo led me here. Installed the package and ran the same query to confirm it wasn't a Superset thing.

Hi there,

I'm using pydynamodb with sqlalchemy in order to convert a simple app which previously used sqlite.

I made the change to the connection string, and I see that pynamodb can connect using my credentials to the dynamodb table.
But when inserting a record I get the error below:

sqlalchemy.exc.OperationalError: (pydynamodb.error.OperationalError) An error occurred (ValidationException) when calling the ExecuteStatement operation: Statement wasn't well formed, can't be processed: Unexpected keyword
[SQL: INSERT INTO urls (key, secret_key, target_url, is_active, clicks) VALUES (?, ?, ?, ?, ?)]
[parameters: ('KSCBE', 'YMYXBEAP', 'http://globo.com', None, None)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

This is my database model:

class URL(Base):
    __tablename__ = "urls"

    id = Column(Integer, primary_key=True)
    key = Column(String)
    secret_key = Column(String)
    target_url = Column(String)
    is_active = Column(Boolean)
    clicks = Column(Integer)

Any advice?
Thanks in advance.

Hello,

I'd like to raise a feature request for my use-case.

I have Superset running in k8s and I am working with ServiceAccount tokens to get AWS credentials.

In order to get AWS credentials, I would need to make a call to the API assume_role_with_web_identity with the token I provide.

Looking into the Connection class, this is not currently supported.

Related GitHub issue I opened in the Superset repo.

Thank you

interesting project, one thought would be to add support for a custom type for maps https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ql-reference.data-types.html

At present, connecting superset to DynamoDB via pyDynamoDB requires us to specify access_key and secret_key that adds to operational maintenance. It would be great if the driver can take iam credentials from the environment. This can be kept behind simple boolean flag similar to what pyathena or redshift-connector already provide. Thanks.

1. contains() and begins_with() are not available in PartiQL [SELECT statements].

The syntax below is executable.
SELECT * FROM UserTB WHERE user_name = 'tes0125' Limit 2

contains() or begins_with() don't work. [SELECT statements]
SELECT * FROM UserTB WHERE user_name = 'tes0125' AND begins_with(range_key, 'FRIEND#') Limit 2

2. Is there any way to handle paging?