dsfsi / covid19za Goto Github PK

Is your feature request related to a problem? Please describe.
Not a problem but an enhancement.

Describe the solution you'd like
Use the ministry releases as an automated way to suggest new additions to the data.

Describe alternatives you've considered
Currently manually populated.

Describe the solution you'd like
There have been comparisons across different groups to track spread since first case. See example here

Can we create this for South Africa vs rest of world
South Africa vs African countries.

Currently the data is in CC 4.0 SA NC and wanted to discuss if we just leave it as CC 4.0 SA?

We want to build what is called a line list – a table summarizing information about people who are infected, dead, recovered etc. The table would include demographic and location information. Such a dataset would help us understand how COVID19 transmission in Africa is similar or different to what’s being observed in other regions of the world.

This would be part of a broader effort focused on collating COVID19 data globally. We need volunteers ASAP to help in this effort. The data will be made publicly available and volunteers will be acknowledged.

[email protected]

Describe the solution you'd like
Add an endpoint to the api that exposes testing timeline data.

See the pdf here. https://sacoronavirus.co.za/wp-content/uploads/2020/03/Latest-confirmed-cases-21-March-.pdf.pdf.pdf

DoH has now created a sheet inside the PDF that has some patient demographics. Can we validate these with our confirmed list.

Which Dataset

covid19za_timeline_confirmed.csv

Error Description

The 32 year old FEMALE was incorrectly made a MALE in the PDF. In the raw data she is a FEMALE. Blame the GOV.

Suggested fixes

1. Change the last Free State Entry for 20032020 to a FEMALE

Describe the bug
The column YYYYMMDD and date in covid19za_timeline_testing.csv does not agree in all cases. See e.g.
07-03-2020 | 20200306
I'm not sure what the significance of the two columns are?
Don't see the same problem in covid19za_timeline_confirmed.csv

To Reproduce
Steps to reproduce the behavior:
Go to https://github.com/dsfsi/covid19za/blob/master/data/covid19za_timeline_testing.csv

Describe the bug
Retrieving data from the API programmatically from localhost throws CORS exceptions.

Access to XMLHttpRequest at 'https://covid-za-api.herokuapp.com/cases/confirmed' from origin 'http://localhost:8100' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

To Reproduce
Steps to reproduce the behavior:

1. Attempt to retrieve data programmatically
2. Data retrieval from API fails because of CORS

Expected behavior
Expect to be able to retrieve data from the API.

Screenshots

Hi @vukosim, can you make all the date format to be the same, you are using two different format, you are using YYYY-MM-DD and YYYY-DD-MM on the same column, I have explored the data and I have spotted that.

Add graph showing how many new cases there are each day. The actual value instead of me subtracting the values of the main graph.

Hi,

I'm busy using the data on my own side project but wanted to contribute some of my time back to the project to help out.

What tech should I learn to help out?

I saw Go on the API?

Describe the solution you'd like
Notebooks and scripts to recreate the analysis from Day 0 across Africa and the rest of the world vs SA.

Last night, the President announced that there has been local transmission. It would be good to track these cases separately.

We can now start attempting to infer western cape district breakdowns. See statements by premier here https://www.westerncape.gov.za/department-of-health/news

At least at this stage, one interesting thing about the SA data is how much local transmission there has been, compared to people arriving from abroad, already infected. I could try to parse the "transmission type" column.

Could you split the "transmission type" column in data/covid19za_timeline_confirmed.csv to be
a) imported/local/unknown
b) countries visited if imported

I'm happy to do the split and PR if this makes sense

For example,

case_id,date,YYYYMMDD,country,province,geo_subdivision,age,gender,transmission_type
1,05-03-2020,20200305,South Africa,KZN,ZA-KZN,38,male,imported,Italy
2,07-03-2020,20200307,South Africa,GP,ZA-GP,39,female,imported,Italy
13,11-03-2020,20200311,South Africa,WC,ZA-WC,36,male,imported, Germany; Austria; Switzerland 
168,20-03-2020,20200320,South Africa,GP,ZA-GP,47,female,imported,pending travel history
169,20-03-2020,20200320,South Africa,GP,ZA-GP,23,male,local,

Negative Cases
It would also be useful to add negative cases in the dashboard

Investigate getting a custom domain for the dashboard that is easy

maybe https://coronavirusza.co.za

Is your feature request related to a problem? Please describe.
This is just an idea.

Describe the solution you'd like
Would be insteresting to see the transmission type by gender and age

Describe alternatives you've considered

Additional context

Is your feature request related to a problem? Please describe.
The dashboard may be more informative if one can quickly see the daily change in numbers of infected, recovered, tests, etc with one glance without having to look at the plots.

Describe the solution you'd like
Show the change since the previous day in the blocks at the top of the dashboard

Describe alternatives you've considered
The graphs on https://coronamap.co.za/ show the change indicated with a + or - at the top of each plot.
The info blocks on the Hong Kong dashboard shows it with little triangles pointing up or down - https://chp-dashboard.geodata.gov.hk/covid-19/en.html

Additional context
Add any other context or screenshots about the feature request here.

Describe the bug
The word cases is misspelled as casis on page 2 of the dashboard

To Reproduce
Steps to reproduce the behavior:

1. Go to 'page #2' on the Daily Positive Casis graph

Expected behavior
Casis should be spelled as Cases

Screenshots

Desktop (please complete the following information):

• OS: [e.g. iOS]: MacOS
• Browser [e.g. chrome, safari] : Firefox
• Version [e.g. 22]: 74

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
We would like to monitor occurrences on a municipality level

Describe the solution you'd like
A new column in the confirmed dataset (at a minimum) stating municipality

Describe alternatives you've considered
Contacting the NICD directly. We'll still pursue this, but they are probably swamped.

Hi,

I'm a paediatrician working in the UK. We need data on the pre-existing medical conditions of children who catch COVID, so that we can start looking for patterns.

When you're collecting your data, please please include individual patient data on people with co-morbidities, especially children but I figure it would be useful for adults too! We'd need their ages as well ;)

I'm happy to be contacted on the email in my profile, and then we can move over to my official NHS account if needs be.

All the best

Dr Jonathan Fisher
NIHR Clinical Lecturer in Paediatrics
UCL-GOS Institute of Child Health

Thanks for your work to help the people in need! Your site has been added! I currently maintain the Open-Source-COVID-19 page, which collects all open source projects related to COVID-19, including maps, data, news, api, analysis, medical and supply information, etc. Please share to anyone who might need the information in the list, or will possibly contribute to some of those projects. You are also welcome to recommend more projects.

https://weileizeng.github.io/Open-Source-COVID-19/

Cheers!

on the api, the entry with case_id of 7 and timestamp of 20200309, has a transmission_type as "r".

Currently, there are fragmented and limited consolidated public data regarding the healthcare systems in South Africa. This extends beyond just knowing about a particular hospital but also relates to functions of the hospital such as specialised care, location, the estimated population in the district, and which hospitals can potentially assist in the COVID-19 pandemic.

To combat this, hospitals with specialised staff/units are available in South Africa, but this knowledge is not always accessible to the general public. Furthermore, this data seems to be in different formats, depending on the province/district and is not consolidated on one platform which makes data analysis challenging related to resource management.

I propose that we collate all the hospital data within South Africa so that we can have a clear visualisation of maximum capacity, and have data available so that other data scientists can perform assessments related to resource management, projections per district and identify what COVID-19 hotspots mean in terms of hospital capacity.

To do this, we need to start by collating data per province, related to the location of the hospital, and capability of the hospital.

1. Location of hospital
   a. Name of hospital
   b. GPS coordinates
   c. Province
   d. District
   e. Subdistrict
   f. Estimated population within the district
2. Capacity and capability
   a. Description of the hospital (each type of hospital has its own detail)
   b. Size (differs per hospital type for example small district is 50 – 150 beds, small tertiary is 400)
   c. Operational functions (services which must be included within the type of hospital)
   d. Type of hospital (district, regional, tertiary, central or specialized)
   e. Maximum capacity
   f. Speciality services (what specialised services are at the hospital, for example, internal medicine and infectious disease units etc.)
   g. Can the hospital currently respond to COVID-19?
   h. Source confirming the hospital can currently respond to COVID-19
   i. CEO of hospital
   j. Postal address of hospital
   k. The physical address of the hospital
   l. Contact number
   m. Email

The last question “Can the hospital currently respond to COVID-19?” is challenging to answer unless public statements have been made. We know that the following hospitals are/were highlighted as quarantine hospitals.

Province HosptName GPSCoordinant TotalHospitalBeds
Limpopo Polokwane Hospital 23.8968° S, 29.4574° E 450
Mpumalanga Rob Ferreira Hospital 25.4769° S, 30.9709° E 461
Gauteng Charlotte Maxeke Hospital 26.1760° S, 28.0439° E 1088
Gauteng Steve Biko Academic Hospital 25.7303° S, 28.2044° E 832
Gauteng Tembisa Hospital 25.9830° S, 28.2382° E 840
KwaZulu Natal Grey's Hospital 29.5795° S, 30.3643° E 530
North West Klerksdorp Hospital 26.8789° S, 26.6633° E 890
Northern Cape Kimberely Hospital 28.7462° S, 24.7730° E 287
Free State Pelonomi Hospital 29.1396° S, 26.2447° E 469
Eastern Cape Livingstone Hospital 33.9254° S, 25.5697° E 541
Western Cape Tygerberg Hospital 33.9106° S, 18.6123° E 1384

We also know, based on articles written about each one of these hospitals, that they experience a heavy load of patients throughout the year – irrespective of COVID-19.

The data that is consolidated can be used as a dashboard for the general public to see where they can go in terms of public healthcare, and additionally, other scientists can use this data to make contextually relevant inferences if they have this data and add it to their analysis.

Error Description

according to news24 and timeslive the numbers are 27 and 74 not 26 and 75 respectively.

Is your feature request related to a problem? Please describe.
The dashboard is doing a really good job of providing a visual overview of the underlying data, but I am concerned that there is a lot of emphasis on how things are escalating (which is the honest truth and important to see).

I think people might increasingly be looking at the data for signs that lockdown and other measures are working and slowing spread down, but we know that this dashboard will debunk that idea over the next few weeks as more cases are detected due to spread (delayed because of incubation period) and increased testing capabilities, and as tally of casualties grow.

The dashboard may play a role in due time to help people comply with implemented measures to curb the spread of COVID or it may play a role in causing people to lose faith (based on data) in the implemented measures.

Describe the solution you'd like
Can we organise the dashboard to emphasise the positives a bit more? Recoveries, active cases vs closed cases, severe vs mild cases. (We mustn't let people lose hope as the pandemic numbers get worse).

Is it possible to add annotations such as when the state of disaster was implemented, lockdown started, other measures put in place (e.g. when the high-throughput testing equipment goes online) to explain some of the data we're seeing. Also to help people who may not be that familiar with this dataset and data in general to get context and adjust expectations?

Maybe show a shaded graph where we can show that we expect cases to grow drastically for the next few weeks at least before there could be a bit of a plateau.

Visualisations can drive anxiety (and the situation and data will cause anxiety regardless of how we visualise it), but I wonder if people have ideas to present the data as non-sensational but honestly as possible.

Additional context
The broader public may look at the data with different contexts than someone who knows the data. I've heard from a number of people that the red map from Harvard and seeing the numbers ticking over is creating massive anxiety for them.

Is it possible to get a RSS feed or perhaps a Twitter bot linked up that everytime you folks update the data it sends out a summaritive tweet? I can then plug that to a Discord Intel channel using MEE6 Mod bot. Keep up the good work![Feature]

As per 13 March 2020, there are supposed to be 24 confirmed cases. Missing 4.

I came across this list CSSEGISandData/COVID-19#576.

Wanted to see what others are doing and how they are displaying various pieces of information. Maybe useful resource if people want to get ideas for displaying data. Most look more or less the same at this stage.

Nobody else displays information about # of tests or # negative cases (see #17, #26, #38) yet

Is your feature request related to a problem? Please describe.
Not relating to an issue, just an idea.

Describe the solution you'd like
Overlay the number of infections with the number of recoveries.

Describe alternatives you've considered

Additional context
Currently the DB indicates that the mortality rate in SA is 0. I am concerned that people will read it in context of the number of infections (not the number of closed cases). Providing the number of recoveries will provide more context and give viewers an idea of the reliability of the proposed mortality rate.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
See reports here. We need the numbers per day in csv format.

Is your feature request related to a problem? Please describe.
Currently, we are only getting numbers from the NICD/DoH in terms of final numbers. One place we can get this data is their Whatsapp information service that then gives daily numbers after the update.

We need a solution to check 2 Whatsapp updates, calculate the difference and create the CSV.

The Whatsapp messages are now stored in data/doh_whatsapp/ as .txt files

Describe the solution you'd like
Process 2 consecutive Whatsapp .txt files and then output the CSV that has the

confirmed.csv template. Similar to the scraper.

Example from the Whatsapp line

extracted into .txt file example below

Current Status of Cases of COVID-19 in South Africa
24 MARCH 2020 - 11:28am

Total cases: 554
153 New cases
2 Full recovery (Confirmed Negative and cleared for returning home)
0 Deaths

The breakdown per province of total infections is as follows:
302 Gauteng
130 Western Cape
80 KwaZulu Natal
18 Free State
5 North West
9 Mpumalnaga
4 Limpopo
2 Northern Cape
2 Eastern Cape

Current projections estimate that the virus could effect 60% of South Africa's citizens at some point, but not at the same time.
Most South Africans will only experience mild symptoms and humans are capable of developing immunity to the virus.

The National Department of Health will now be releasing results as they are submitted by both private and public laboratories. In instances where NDOH confirmatory tests yield different results, the public will be duly informed.

TEST RESULTS OF CITIZENS REPATRIATED FROM WUHAN:
All the citizens from Wuhan were tested and their results came back negative for COVID-19.
They will continue to be kept in quarantine for the prescribed period and will thereafter be reunified with the community.

Describe the solution you'd like
See https://www.offerzen.com/blog/sa-covid-19-project-vulnerable-communities-map

If there are interesting datasets, please do make them available to Offerzen as well as make them available here.

Describe the solution you'd like
We now have hospital data available, it will be updated in the next few days. We need to map all of the hospitals. On hover, there must be some information on the district size, bed capacity etc.

Which Dataset

covid19za_provincial_cumulative_timeline_confirmed

Error Description

In the latest release of cases data, DIRCO eleased all the province data with the addition of an unknown field since the province isn't verified. See below:

Suggested fixes

1. Add unknown field and add relevant data

Which Dataset

Error Description

Two-year-old case here states that it is from travelling from New Zealand yet the government and NICD statements state 2-year-old male with no international travel history.

Suggested fixes

Hi,

I saw that a citation for correctly citing the dataset has been added to the repos readme file.

I'm building a dashboard with the data and would like to check if I'm correctly citing the dataset on my Data Sources page?

Is your feature request related to a problem? Please describe.
Currently the dashboard does not allow the user to drill down and explore the cased within their immediate locality (region/district)

Describe the solution you'd like
There could be an additional field in the confirmed_cases model to represent the region in which the case was detected.

Describe alternatives you've considered
None

Additional context
I've noted that the City of Joburg is now publishing this data.

Describe the bug
Cannot scrape NICD or SACoronavirus.co.za

http://www.nicd.ac.za/covid-19-update-23/

Screenshots

Expected behavior
Should create csv

Hey @cishiv

The number of confirmed positive cases is a function of how many tests have been done. In order to understand the extent of Covid-19's penetration, we need to know # of positive cases and # of people that have been tested relative to the total population, which is currently estimated as 59M (https://www.worldometers.info/world-population/south-africa-population/)

Hi Vukosi,

I'm thinking it may be useful to display the number of tests done against the number of cases confirmed on the barchart on page 2 of the dashboard. Or showing positives vs negatives. It will give us an idea of how many tests are essentially wasted because we're testing unnecessary due to not being able to prioritize patients and highlighting the symptoms that are required for testing? But at the same time realising that people will be tested if they've been in touch with positives even if they aren't displaying symptoms.

Not wasting tests on the wrong patients seems like an important aspect of successful response?

Thanks,
Anelda

Which Dataset

Confirmed cases timeline; provincial cumulative timeline

Error Description

I would expect the latest number of confirmed cases on the dashboard to be the same as the latest number in provincial cases or timeline confirmed

But right now the repo cumulative cases is 554, and the dashboard says 709. Which suggests the dashboard does not pull from the repo

Suggested fixes

Either
a. make the dashboard feed from the repo for common data source,
b. Feed the repo from the data source behind the dashboard
c. Document that they have different data sources in the README

At any rate I'd like come commentary on this discrepancy as it is tripping us up when ingesting the repo data.

Confirmed Cases that requires hospitalization or NOT.
Is it possible to have another column/measurement that would list if the patient required hospitalization or not?

Hospitalized or Self-Isolated feature
A column that would indicate if the patient was hospitalized that could (and i could be very wrong) serve as proxy to the serious of the infection

Is your feature request related to a problem? Please describe.
Once recovered and case fatality data is available the api should be updated to allow for this data to be retrieved.

Dependent on #83

Statements are supposed to be the same but it seems NICD sometimes changes their statement when case data becomes available. See these two statements

http://www.nicd.ac.za/covid-19-update-21/

and

https://sacoronavirus.co.za/2020/03/19/latest-confirmed-cases-of-covid-19-19th-march-2020/

In the first 1 it says the first 2 patients when to the DRC, the second statement is different.

Also,

See that the last cases in Gauteng are different. 1 is pending and the other has " no contact details on lab form, information being obtained from the private doctor"

Describe the solution you'd like
API to access the data easily.

@vutlhari is working on this.

Is your feature request related to a problem? Please describe.
First off thank you for all the great work you guys have been doing. Currently recovered cases are not represent in the csv files. Do you plan to record this information? Assuming you do do you think it will be possible to map the recovered case to the covid19za_timeline_confirmed.csv case_id? In other words will we be able to determine which case_id from timeline_confirmed has recovered?

Out of scope of this issue but worth noting is that I am curious whether it is going to be possible to do the same with case fatalities?

There seems to be data access issue with the nicd and doh. Have they been contacted to request csv dumps or some alternative to scraping their reports?

Describe the solution you'd like
I would like for a file identical in structure to covid19za_timeline_deaths.csv to be populated for recovered case data.

@vukosi we have new 10 cases confirmed but not yet out on gov.za (total should now be 61)