dolthub / data-analysis Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
data-analysis's People
Contributors
Stargazers
Watchers
Forkers
aperrien snipesjr kulnor easylifewithr rl1987 danielchalef levinsamuel chefcurrywiththecode rafael-mora captainstabs drzeshankhan vbdevpro cashiuus wesleycheung0 laureano haneyr databytes64 paulzuradzki bjohnsonphpni superdave98008data-analysis's Issues
Museum Collection Coverage
By this query of all the Van Gogh's in our database, it looks like we have pretty poor coverage.
$ dolt sql -q "SELECT institution_name, institution_city, title FROM objects where maker_full_name like '%Van Gogh%'"
+----------------------------+------------------+------------------------------------------------------------+
| institution_name | institution_city | title |
+----------------------------+------------------+------------------------------------------------------------+
| Albertina | Vienna | Landschaft mit Haus und Bäumen |
| Albertina | Vienna | Friedhof im Regen |
| Orsay Museum | Paris | Eugène Boch |
| Orsay Museum | Paris | Portrait de l'artiste |
| Orsay Museum | Paris | Le Docteur Paul Gachet |
| Orsay Museum | Paris | Portrait de l'artiste |
| Orsay Museum | Paris | Vaches dans un pré |
| Orsay Museum | Paris | La Salle de danse à Arles |
| Orsay Museum | Paris | L'église d'Auvers-sur-Oise, vue du chevet |
| Orsay Museum | Paris | La Méridienne |
| Orsay Museum | Paris | L'Arlésienne |
| Orsay Museum | Paris | Roses et anémones |
| Orsay Museum | Paris | Mademoiselle Gachet dans son jardin à Auvers-sur-Oise |
| Orsay Museum | Paris | Chaumes de Cordeville à Auvers-sur-Oise |
| Orsay Museum | Paris | Dans le jardin du docteur Paul Gachet |
| Orsay Museum | Paris | Deux fillettes |
| Orsay Museum | Paris | Tête de paysanne hollandaise |
| Orsay Museum | Paris | La Chambre de Van Gogh à Arles |
| Orsay Museum | Paris | L'Italienne |
| Orsay Museum | Paris | Hôpital Saint-Paul à Saint-Rémy-de-Provence |
| Orsay Museum | Paris | La Nuit étoilée |
| Orsay Museum | Paris | Paysanne près de l'âtre |
| Orsay Museum | Paris | Fritillaires couronne impériale dans un vase de cuivre |
| Orsay Museum | Paris | La Guinguette à Montmartre |
| Orsay Museum | Paris | Le Restaurant de la Sirène à Asnières |
| Orsay Museum | Paris | Les Roulottes, campement de bohémiens aux environs d'Arles |
| Orsay Museum | Paris | Palette de Vincent Van Gogh |
| Orsay Museum | Paris | Portrait du Docteur Gachet |
| Rijksmuseum | Amsterdam | Brief aan Pieter Haverkorn van Rijsewijk |
| Victoria and Albert Museum | London | The fortifications of Paris |
+----------------------------+------------------+------------------------------------------------------------+
I think some sort of coverage report and a "Most desired collections" list would be a good thing. Maybe use it in another bounty. Like only pay for museums on the "most desired list".
Data structures changed
Hi!, I was trying to build a Data Analysis test using the data sets from the health insurance USA's companies, but all the data sources I tested did not work. I spent a few hours debugging and trying to get it working, but the data structure is pretty different from what it was in 2022. I tested all the downloaders from the repo, some of those didn't work, others did (with minor adjustments on the URLs) download the data json URLs, but the giant jsons structure that were parsed via those URLs didn't match with what the dialysis parser is expecting.
Any clues?
Most in storage objects in museums
There is a bit of a controversy about how many items are in storage at museums.
https://podcasts.apple.com/us/podcast/dragon-psychology-101/id1119389968?i=1000478450374
We could use our museum collections database:
https://www.dolthub.com/repositories/dolthub/museum-collections
to estimate the square footage required to display all the things in a museums collection. For the top X, we could get the actual square footage of the building. This could be thought provoking.
Hospital Prices V3 Outlier Analysis
https://www.dolthub.com/repositories/dolthub/hospital-price-transparency-v3
Find all the prices in this databases that are way outside the distribution on the high side. Trim to common procedures. Create a top X list. Investigate some of the findings. Are they still overcharging in their listed charge master? Call the hospital.
Can you infer housing construction from housing sales data?
NIMBY (Not in my backyard) vs YIMBY (yes in my backyard) is a current thing.
Can we use "first sale" data in the us-housing-price-v2 database:
https://www.dolthub.com/repositories/dolthub/us-housing-prices-v2
to gauge which places are building more houses, even relative to other places in the database.
Hospitals Code Analysis
https://www.dolthub.com/repositories/dolthub/hospital-price-transparency-v3
Try to join or group similar codes based on description and price. See if we can identify a set of "super codes" that describe the universe of procedures/prices better.
Can you see recent rising interest rates affect home sales? If so where?
https://www.dolthub.com/repositories/dolthub/us-housing-prices-v2
Latest housing price bounty data must contain recent sales. Mortgage interest rates have gone up massively in the past 6 months. Can you see an effect in the recent housing sale data, either in velocity or price? If so, where?
Overcrowded Prisons Analysis
We collected prisons data in this bounty database:
https://www.dolthub.com/repositories/dolthub/us-jails
In the jails table there is a num_inmates_rated_for and in the inmate_population_snapshots there are population snapshots.
This query is interesting:
$ dolt sql -q "select jails.facility_name, count(*) as periods_over_capacity from inmate_population_snapshots join jails on jails.id=inmate_population_snapshots.id where inmate_population_snapshots.total > jails.num_inmates_rated_for group by jails.id order by periods_over_capacity desc limit 20"
+--------------------------------------------+-----------------------+
| facility_name | periods_over_capacity |
+--------------------------------------------+-----------------------+
| York Correctional Institution | 269 |
| Robinson Correctional Institution | 269 |
| New Haven Correctional Center | 269 |
| Hartford Correctional Center | 269 |
| Osborn Correctional Institution | 269 |
| MacDougall-Walker Correctional Institution | 269 |
| Cheshire Correctional Institution | 269 |
| Brooklyn Correctional Institution | 269 |
| Corrigan Correctional Center | 269 |
| Bell County Central Jail | 268 |
| Irving City Jail | 268 |
| Hurst City Jail | 268 |
| Garner Correctional Institution | 268 |
| Aransas County Detention Center | 267 |
| Willard-Cybulski Correctional Institution | 267 |
| Anderson County Jail | 267 |
| Knox County Jail | 262 |
| Radgowski Correctional Institution | 262 |
| Northern Correctional Institution | 258 |
| El Paso County Detention Facility - Annex | 255 |
+--------------------------------------------+-----------------------+
We know there are some data quality issues in this data. For instance, one of the bounty hunters concatenated instead of summed male and female inmates to make total which is why the top of this list is all 269 periods over. But, there's a kernel of cool analysis here and it would make a good blog.
Weirdest objects in the museums database
It would be fun to write a "weirdness" detection algorithm for the museum collections and then find the weirdest objects and rate the weirdest collection.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.