passren / pydynamodb Goto Github PK
View Code? Open in Web Editor NEWPyDynamoDB is a Python DB API 2.0 (PEP 249) client for Amazon DynamoDB. A SQLAlchemy dialect is offered as well. Superset official database driver.
License: Other
PyDynamoDB is a Python DB API 2.0 (PEP 249) client for Amazon DynamoDB. A SQLAlchemy dialect is offered as well. Superset official database driver.
License: Other
Thank you for making this library!
If I issue:
SELECT * FROM "cache" WHERE "my-value" != ''
to both boto3 directly (pure PartiQL) and to pydynamodb, pydynamodb never returns (the table is large).
I see that the statement it pushes down is SELECT * FROM "cache"
.
I am guessing it is eagerly pre-fetching data and locally evaluating the WHERE.
I would expect the entire SELECT statement to be pushed down as it is to PartiQL, which is able to handle this.
This works in the DynamoDB PartiQL editor:
SELECT payload."size"
FROM "table"
WHERE pk='foo'
But when I do:
cursor.execute('SELECT payload."size" FROM "table" WHERE pk='foo')
I get this error:
dynamodb error: An error occurred (ValidationException) when calling the ExecuteStatement operation: Statement wasn't well formed, can't be processed: Invalid path dot component
An example of what my data looks like:
[
{
'pk': foo',
'payload': {'size': 2, 'bla': 'bla'},
},
{
'pk': 'bar',
'payload': {'size': 2, 'bla': 'bla'},
}
]
How may I go about selecting size
from payload
?
Note that I intend to use DynamoDB in Apache Superset. Was getting the same error there in the SQL Lab section. Digging around their repo led me here. Installed the package and ran the same query to confirm it wasn't a Superset thing.
Hi there,
I'm using pydynamodb with sqlalchemy in order to convert a simple app which previously used sqlite.
I made the change to the connection string, and I see that pynamodb can connect using my credentials to the dynamodb table.
But when inserting a record I get the error below:
sqlalchemy.exc.OperationalError: (pydynamodb.error.OperationalError) An error occurred (ValidationException) when calling the ExecuteStatement operation: Statement wasn't well formed, can't be processed: Unexpected keyword
[SQL: INSERT INTO urls (key, secret_key, target_url, is_active, clicks) VALUES (?, ?, ?, ?, ?)]
[parameters: ('KSCBE', 'YMYXBEAP', 'http://globo.com', None, None)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)
This is my database model:
class URL(Base):
__tablename__ = "urls"
id = Column(Integer, primary_key=True)
key = Column(String)
secret_key = Column(String)
target_url = Column(String)
is_active = Column(Boolean)
clicks = Column(Integer)
Any advice?
Thanks in advance.
Hello,
I'd like to raise a feature request for my use-case.
I have Superset running in k8s and I am working with ServiceAccount
tokens to get AWS credentials.
In order to get AWS credentials, I would need to make a call to the API assume_role_with_web_identity with the token I provide.
Looking into the Connection class, this is not currently supported.
Related GitHub issue I opened in the Superset repo.
Thank you
interesting project, one thought would be to add support for a custom type for maps https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ql-reference.data-types.html
At present, connecting superset to DynamoDB via pyDynamoDB requires us to specify access_key and secret_key that adds to operational maintenance. It would be great if the driver can take iam credentials from the environment. This can be kept behind simple boolean flag similar to what pyathena or redshift-connector already provide. Thanks.
The syntax below is executable.
SELECT * FROM UserTB WHERE user_name = 'tes0125' Limit 2
contains() or begins_with() don't work. [SELECT statements]
SELECT * FROM UserTB WHERE user_name = 'tes0125' AND begins_with(range_key, 'FRIEND#') Limit 2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.