Create an Apache Superset theme that follows the guidelines

Links:

• https://github.com/apache/incubator-superset
• design guidelines

Now it's possible to login at the same time on different machines using the same account

Feature Request

I suggest to specify which of the fields that should necessarily filled in. And those that have not an effect so far. It can simplify the process the user should go through.

In general, it might be useful to give a little definition of the fields - particular attention to the definition of Dominio and Sottodominio.

Feature request:
It's not possible to ingest and manage Shape - SHP datasets.
It would be useful in order to be aligned with some dataset defined in Task Force #DatiPubblici
http://elenco-basi-di-dati-chiave.readthedocs.io/it/latest/index.html

accedendo a “Storie” → “Le mie storie” vengono visualizzate delle storie che non sono state create dall’utente loggato. Sembra che vengano proposte tutte le data stories presenti.

Subject of the issue

There is a page titles "Le mie storie", those shown are not mine.

Your environment

• Google Chrome - Version 64.0.3282.167 (Official Build) (64-bit)

Steps to reproduce

1. Choose Storie in the side menù
2. Look at the page

Expected behaviour

From the title one infers that the page gathers the stories created by the user or the organization he belongs to.

Actual behaviour

The user is able stories that don't belong to him.

To describe a story of a dashboard would be useful to insert external images.

If I browse without using the full screen window, I'm not able to access the menù.

After creating a dashboard, title and subtitle are no longer editable

Subject of the issue

A user is able to delete stories he didn't produce.

Expected behaviour

I expect to be able to remove only the stories that belong to me.

Actual behaviour

I can remove stories I didn't create.

When you are creating a new dashboard in DAF, it would be very useful to filter slices created by superset. Now the system display all slices created in DAF and this job is very slow and cpu consumption. I suggest to generate a form filter that generate a list with these fields:
|| owner’s slice || title of slice || type of slice || screenshot of slice || link preview of slice ||

The form for stories creation can be improved. It would be nice if it adopted a more modern look. One of the inpirations in this regard is the medium.com editor, play with it and take a look at what you can do.

Links for inspiration:

• Medium.com

Le storie sono articoli scritti da noi e dalla nostra community di esperti: partendo dai dati, interpretiamo il mondo e aiutiamo la socità a prendere decisioni basate sui fatti

After a while, the search box stops working. you need to logout and login again to make it work.

Create an ansible recipe so that it's easy to deploy an instance of daf-dataportal on cloud platforms (Bluemix, Oracle, Azure, Openshift).

Links:

• Ansible

This anomaly, unfortunately does not always happen with the same steps, so I have difficulty in reproducing the exact sequence. However, the problem usually occurs during these activities:

• inserting new box in DAF dashboard;
• drag the slice from one side of the dashboard to another.

Maybe it's useful to add an info box explaining the meaning of every label in order to support the user

Subject of the issue

While compiling the ingestion form, one has the possibility of changing the type inferred by Kylo. In particular if we do it for a datetime. If Kylo doesn't recognize the format of date the feed is not created.

Your environment

• Google Chrome - Version 64.0.3282.167 (Official Build) (64-bit)

Steps to reproduce

This is the sample (sample_incidenti.txt) I used. In the ingestion form, I changed the type of the the date from string to timestamp.

Expected behaviour

I expected to be able to load data on edge1.

Actual behaviour

The data are not loaded (on edge1). Talking with Fabian it is because kylo doesn't recognise the column defined by the user as timestamp. In particular, the format is not the one that kylo uses.

As we discussed, in order to avoid the stop of the feed, the easiest thing to do, is to not allow the user to change the time of a column into timestamp whether it is not inferred by kylo.

Be creative!
There are several sources of data, create a dashboard which visualizes them in a creative way and can be useful for users.

You can find many different datasets on https://www.dati.gov.it
You can use Metabase or Superset or Jupyter for providing a visualization.

while using mapbox on superset (after one or two query)
full error:
An error occurred while rendering the visualization: Invariant Violation: Minified React error #37; visit http://facebook.github.io/react/docs/error-decoder.html?invariant=37 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.

Feature Request

It would be useful to add and remove a field/column to an existing dataset previously defined in the Metadata process.

Subject of the issue

The 'Go back' seems to not work properly.

Your environment

• Google Chrome - Version 64.0.3282.186 (Official Build) (64-bit)

Steps to reproduce

Start an Ingestion form, once you are at page 3, push 'Go back'.

Expected behaviour

I expect to be redirected to the previous page (in this case page 2).

Actual behaviour

I'm back to the initial page

Feature Request

It worked well with the previous release.

Subject of the issue

While a user fills in the module, if he needs to come back to previous stages he's asked to fill in again:

1. Concetto semantico
2. Tag

Your environment

• Google Chrome - Version 64.0.3282.167 (Official Build) (64-bit)

Steps to reproduce

Just try to go back after having completed the module.

Expected behaviour

I don't expect to insert again the info related to potentially many many columns.

Actual behaviour

I've to do enter again the info

Subject of the issue

There are no filters in the dashboard created on the dataportal.

Actual behaviour

Apparently the widgets are independent thus it's not possible to insert filters. For compleate dashboards having this feature is essential.

Create a loading form for updating metadata from DAF Catalog Manager.
You have to implement the "RUD" part (in the ingestion form) of the CRUD paradygm, so Read, Update and Delete.

Note: you will need to modify slightly also https://github.com/italia/daf-dataportal-backend for this to work.
The code you will have to modify is in this folder https://github.com/italia/daf-dataportal/tree/master/src/components/IngestionForm

Feature Request

We should understand exactly which are the Kylo requirements about the sample file the user has to upload at step 1.
It's not significant to express a max number of line to use.

Subject of the issue

The updated date of every dataset is hard coded

Your environment

Chrome/Safari

Steps to reproduce

1. select "Dataset" from the menu on the left side
2. click on a dataset that has been recently created
3. check the text-field: "Creato il "

Expected behaviour

The real date should be displayed

Actual behaviour

The date is hard coded, the same for every dataset.

Create a microservice which goes through the various plots, takes a screenshot, and caches them.

You can fork this repository as a starting point.

Create an API endpoint, linked to the ingestion form, which will save the data on an external PostgreSQL database.

After creating new Dashboard with many slices (pie, table, country map etc.) I insert the "Filter Box", but this not seem to filter anything. This filter work fine inside Superset Dashboard.

As a User
I want to be able to reset, update my password
Since I forget or I want to update it

We need a complete design of the feature taking into account:

1. approaches
2. behaviour of the feature
3. ui design and flow
4. frontend implementation
5. backend implementation

Note: These issues are for bugs and feature requests.

Subject of the issue

Datasets: filter by format doesn't work.

Your environment

• Google Chrome - Version 64.0.3282.167 (Official Build) (64-bit)

Steps to reproduce

1. Log in
2. Go to Datasets

Then I tested these three options:
3) If you filter by format
or
4) Filter by organization and format
or
5) Filter by category and format

Expected behaviour

3. I expect to see al the dataset labeled as those specified in the format filter
4. I expect to see all datasets related to the chosen organization whose format is the selected one
5. I expect to see all datasets related to the chosen category whose format is the selected one

Actual behaviour

In 3, 4 and 5 the system gives back 0 results.

Subject of the issue

In the ingestion form, add a properties to specify that CSV/JSON files for a feed will be uploaded compressed (ZIP) on the SFTP channel. A zip archive will not contain multiple csv/json files.

Expected behaviour

The new feed will read zip files and will decompress them before loading on hdfs

Actual behaviour

Now it's not possible to provide this information, although a feed can be manually changed (via the administration console) to support zip files

Wrong email address: [email protected]
https://dataportal-private.daf.teamdigitale.it

Subject of the issue

It's not possible to retrieve an existing dataset by the search function.

Your environment

Safari/Chrome

Steps to reproduce

1. Login into https://dataportal-private.daf.teamdigitale.it
2. Type "strutturericettivepiemonte" into the search box (the name of an existing dataset with access permission)
3. Press "Cerca"

Expected behaviour

The dataset has to be listed in the search list.

Actual behaviour

The dataset is not displayed.

Note

It's not a standard behaviour. For some dataset the function works properly.
The dataset exists and the table name is: REGI__europa.test_ingestion_o_test_strutturericettivepiemonte
A dashboard with Superset is available:
test_piemonte_strutture_ricettive

Subject of the issue

Jupyter gives a "Permission Denied" message when trying to access a private dataset uploaded by the user, or a private dataset that the user is allowed to access.

Your environment

The error looks to be independent of the adopted browser; for reference, it was seen with Chrome 64.0.3282.186 (official build)

Steps to reproduce

1. login into the dataportal
2. choose a private dataset you can explore with other tools (e.g. Superset).
3. fetch code snippet to load it in Jupyter
4. start Jupyter with PySpark3 kernel and initialise endpoint and session as per user manual
5. run code snippet from 3) and get "Permission Denied".

Notes:

• this problem is not seen with open data.
• by manually adding typos to the path found in 3), the message changes to “Path does not exist”, thus confirming the original target path existed but was prevented from access.

Expected behaviour

A pyspark DataFrame should be created.

Actual behaviour

One receives the following message:

An error occurred while calling o47.load.
: org.apache.hadoop.security.AccessControlException: Permission denied: user=d_mc, access=EXECUTE, inode="/daf/ordinary/test_ingestion":daf:daf:drwxrwx---
at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkAccessAcl(DefaultAuthorizationProvider.java:363)

To speed up the process, I suggest to play/start the slices only after clicking on an image to avoid the generation of multiple objects that may overload the client and server side cpu.

Feature Request

Add the created date field to the dataset page layout.

I tried to sign up. Received the mail with the activation link. I get "Errore durante l'attivazione riprovare più tardi".

Subject of the issue

Error 500 when you click on column names to order by Slice | Visualization Type | Datasource | Creator

Your environment

Chrome

Steps to reproduce

1_ Go to Superset
2_ Click on "Slicer" on the top bar
3_ Click on the column names (Slice | Visualization Type | Datasource | Creator) in order to order by the selected field

Actual behaviour

500 - Internal Server Error
Please see the screenshot attached.

When I resize the browser window to the half of my screen the content of the left bar is hidden.

Metabase is used to generate plots.
Theme it so it follows the design guidelines.

You can find the current reference implementation of the style guidelines here:
https://github.com/italia/design-web-toolkit

You are not required to use the existing web components, just to follow the user interface guidelines

Implement the new look and create a pull request

Subject of the issue

"Errore durante il caricamento. Si prega di riprovare più tardi."
It's not possible to load the csv sample attached

Original dataset:
http://opencup.gov.it/documents/21195/100257/Report+Incentivi+CSV/76d83867-b82c-4e48-9041-d952a19b67a9

Your environment

Chrome Version 64.0.3282.186 (Official Build) (64-bit)

Steps to reproduce

1. download the new dataset from OpenCup
2. create a csv sample for the metadata process (ex. first 10 lines)
3. start the metadata process
4. load the csv sample at the first step: Passo 1: Carica file e descrivi le colonne

Expected behaviour

The sample should be loaded and the fields displayed in order to proceed with the metadata informations.

The sample is parsed correctly in Python with Pandas using the code:

import pandas as pd
data = pd.read_csv("incentivisample.csv", sep="|")

79 columns

Actual behaviour

Error message is displayed: "Errore durante il caricamento. Si prega di riprovare più tardi."

In the ingestion form is write in the right way, but after calling backend service i noticed the typo.

Feature request:

It's not possible to ingest and manage XML datasets.
It would be useful in order to be aligned with some dataset defined in Task Force #DatiPubblici
http://elenco-basi-di-dati-chiave.readthedocs.io/it/latest/index.html

In some cases, after Chrome crash, the Dashboard resulted corrupted (almost all blocks have been lost), forcing me to start the work again.

Could be useful add information about how access to the dataset using jupyter.

For example, we can add the following information:

path_dataset = "/daf/opendata/alsia_o_atti_d_di_d_concessione1_0"
df = (spark.read.format("parquet")
     .option("inferSchema", "true")
     .option("header", "true")
     .option("sep", "|")     
     .load(path_dataset)
)

The front end seems to have unlimited time for session by which the website never get log-in process again.

It points to homepage instead of the next (or numbered) page.