llooker / blocks_redshift_admin Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Could you please add a license to the project? In fact, most of the repos under this account don't have any license so I'm not sure if we can use it on our own* or in Looker.
*: Parsing the LookML and using it for something else.
Quick (Looker) view to check that performance issues aren't due to Redshift views
Saw this company: http://intermix.io/
Leaving a note to review to put together potential features for the redshift admin block.
What would be the recommended model to use here if your looker system connects to multiple redshift clusters??
Is there some preferred way to namespace things so that this could be used on one looker instance with multiple tenants (in essence).
And make it consistent between the table and in performance dashboard pie chart
DIST_NONE => vibrant green
DIST_ALL_NONE=> black/green
DIST_INNER => grey/green
BCAST_INNER => red
DIST_OUTER => orange
DIST_BOTH => orange/red
The tables which you need to grant privileges for are not up to date with the most recent views.
Something like this should work:
dimension: looker_query_context {
sql: REGEXP_SUBSTR(REGEXP_SUBSTR(${text},'(Query\\sContext\\s\'\\{)(.*)(\\}\')'),'(\\{)(.*)(\\})') ;;
}
dimension: looker_user_id {
sql:json_extract_path_text(${looker_query_context},'user_id') ;;
}
dimension: looker_history_id {
sql:json_extract_path_text(${looker_query_context},'history_id') ;;
}
dimension: looker_instance_slug {
sql:json_extract_path_text(${looker_query_context},'instance_slug') ;;
}
PDT in this view is failing:
view: redshift_tables {
derived_table: {
# Insert into PDT because redshift won't allow joining certain system tables/views onto others (presumably because they are located only on the leader node)
persist_for: "8 hours"
sql: select
"database"::varchar,
"schema"::varchar,
"Table_id"::bigint,
"table"::varchar,
"encoded"::varchar,
"diststyle"::varchar,
"sortkey1"::varchar,
"max_varchar"::bigint,
"sortkey1_enc"::varchar,
"sortkey_num"::int,
"size"::bigint,
"pct_used"::numeric,
"unsorted"::numeric,
"stats_off"::numeric,
"tbl_rows"::bigint,
"skew_sortkey1"::numeric,
"skew_rows"::numeric
from svv_table_info
;;
# http://docs.aws.amazon.com/redshift/latest/dg/r_SVV_TABLE_INFO.html
distribution_style: all
indexes: ["table_id","table"] # "indexes" translates to an interleaved sort key for Redshift
}
This is due to the "sortkey1" column, we see the error when we select just that column:
select "sortkey1"::varchar from svv_table_info
====>
The Amazon Redshift database encountered an error while running this query.
Invalid character data was found. This is most likely caused by stored data containing characters that are invalid for the character set the database was created in. The most common example of this is storing 8bit data in a SQL_ASCII database.
Use datagroups (for all PDTs, except Redshift slices since that's not in any dashboard, or ever joined)
Since strings may contain sensitive data, e.g. within filter values, it would be good to use a regex to redact all strings by default when creating the block's PDTs
To replace the hard-to-use table in the Redshift Query Inspection dashboard
https://blog.tanelpoder.com/posts/visualizing-sql-plan-execution-time-with-flamegraphs/
See discussion and recommendation here: https://looker.slack.com/archives/C6VC7QNG4/p1523579288000057
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.