Comments (5)
Thanks for opening this issue @ab41996! Can you confirm that you created a Snowflake Connector block prior to running this code and that you passed the name of the block into snowflake_flow
when running this example?
from prefect-snowflake.
Hi yes can confirm I used a block which I had named "personal-snowflake-conn" with credentials from my credential block "personal-snowflake"
from prefect-snowflake.
Hi @ab41996, can you share prefect version
and pip show prefect-snowflake
? I suspect you're using an older version of prefect-snowflake.
Mine is:
Version: 2.7.7
API version: 0.8.4
Name: prefect-snowflake
Version: 0.26.1
I was able to run this successfully:
from prefect import flow, task
from prefect_snowflake import SnowflakeConnector
@task
def setup_table(block_name: str) -> None:
with SnowflakeConnector.load(block_name) as connector:
connector.execute(
"CREATE TABLE IF NOT EXISTS customers2 (name varchar, address varchar);"
)
connector.execute_many(
"INSERT INTO customers2 (name, address) VALUES (%(name)s, %(address)s);",
seq_of_parameters=[
{"name": "Ford", "address": "Highway 42"},
{"name": "Unknown", "address": "Space"},
{"name": "Me", "address": "Myway 88"},
],
)
@task
def fetch_data(block_name: str) -> list:
all_rows = []
with SnowflakeConnector.load(block_name) as connector:
while True:
# Repeated fetch* calls using the same operation will
# skip re-executing and instead return the next set of results
new_rows = connector.fetch_many("SELECT * FROM customers2", size=2)
if len(new_rows) == 0:
break
all_rows.append(new_rows)
return all_rows
@flow
def snowflake_flow(block_name: str) -> list:
setup_table(block_name)
all_rows = fetch_data(block_name)
return all_rows
snowflake_flow("snowflake-connector")
from prefect-snowflake.
Version: 2.8.2
API version: 0.8.4
Name: prefect-snowflake
Version: 0.2.2
from prefect-snowflake.
Have updated my prefect-snowflake version and it is now running fine apologies for wasting your time :)
from prefect-snowflake.
Related Issues (20)
- SnowflakeCredentials requires password even when using private_key to authenticate
- Password is revealed in the connection_params when logging HOT 1
- `SnowflakeConnector.load()` snippet references the wrong import path
- add a poll_frequency_seconds to snowflake snowflake_query and snowflake_multiquery
- Rename SnowflakeConnector 'schema' to 'schema_name' HOT 1
- Add collection sync workflow using cruft
- Issues deleting from or inserting rows from temp table using SnowflakeConnector HOT 2
- leverage executemany to stream data HOT 4
- Snowflake Credentials Endpoint Issue HOT 11
- Snowpark Integration HOT 2
- Improve authentication experience
- Error "Do not provide both private_key and private_key_path; select one." HOT 1
- Multi-query sync HOT 3
- Can't choose what kind of cursor to use with snowflake
- Add in fetch_pandas_all & fetch_pandas_batches from Snowflake connector HOT 1
- Issues passing in a private key for snowflake credentials block
- Returning query_id in various methods
- Typo in code sample in README.md causes failure
- Connect params are overwritten when calling snowflake_connector.get_connection
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prefect-snowflake.