Code Monkey home page Code Monkey logo

covid19tracker-api's Introduction

COVID-19 Tracking API

This is a Laravel-powered API source for tracking COVID-19 cases in a region. It is currently used by the COVID-19 Tracker Canada project.

See it in action at https://api.covid19tracker.ca

Getting Started

Prerequisites

A hosting environment configured for Laravel 8.x should have no issues. Please see Laravel's Server Requirements for up-to-date information.

MySQL Database

Data is stored in a MySQL database. You will need to create one for this project to use.

Cache

A few routes use caching and has been tested with file and redis cache drivers.

Homestead

Laravel Homestead is also supported out of box for local development.

Important In order to use the CSV Seeders, you must have PHP 7.4 installed. Most deployments typically install version 7.2.

Installing

1. Clone project

git clone [email protected]:andrewthong/covid19tracker-api.git
cd covid19tracker-api

2. Install Dependencies

composer install

3. Configure Environment

cp .env.example .env

Update the .env with database credentials.

4. Generate Key

php artisan key:generate

5. Run Migrations

php artisan migrate

6. Data Management Support

Required if you plan to run the C19T-Manager companion as well.

php artisan passport:install

From the resulting output, copy personal access to PERSONAL_CLIENT_SECRET and password grant to PASSWORD_CLIENT_SECRET in your .env file.

Update MANAGE_URL in your .env file with the URL where C19T-Manager is deployed.

Then create your first admin user:

php artisan tinker
$user = new \App\User
$user->name = 'Fname';
$user->email = '[email protected]';
$user->password = Hash::make('some_password_123');
$user->assignRole('admin');
$user->save();

Roles are currently simplified to admin and editor. This project uses laravel-permission to manage roles.

Deployment

You can now launch. Instructions vary depending on your setup (LEMP server, Homestead...). When ready, browse to the URL as configured and you should see the API documentation.

Seeding Data

The database seeders are specific to Canada, but can be tweaked for another region as needed.

Provinces and Territories

Running this seed inserts all 13 provinces and territories of Canada.

php artisan db:seed --class=ProvinceSeeder

Health Regions

Running this seed inserts health regions in Canada.

php artisan db:seed --class=HealthRegionsSeeder

Reports

This is a sample of report data. Be sure to adjust the province to match the province data accordingly.

php artisan db:seed --class=ReportSeeder

Cases and Fatalities

These do not have a sample CSV, but the seeder classes are available.

php artisan db:seed --class=CaseSeeder
php artisan db:seed --class=FatalitySeeder

Postal Districts

Used to determine province from rapid test submissions.

php artisan db:seed --class=PostalDistrictSeeder

Data Management

C19T-Manager is a companion app to facilitate reporting.

The URL where C19T-Manager is hosted will need to be whitelisted for CORS. Update MANAGE_URL in .env accordingly. Use a comma for multiple URLs.

MANAGE_URL=http://localhost:1234,http://localhost:5678

Open Submissions

Currently only rapid tests are supported.

All open submissions can use reCAPTCHA to minimize abuse. To enable this, sign-up and add your secret key to the .env for RECAPTCHA_SECRET_KEY.

Rapid Tests

POST /collect/rapid-test

Example payload:

{
  "test_result": "negative",
  "test_date": "2022-01-23",
  "postal_code": "V3A",
  "age": "20-29",
  "g-recaptcha-response": "{if enabled}"
}

Processing Reports

Configure cron to run Task Scheduling for Laravel and report changes done via the C19T-Manager will trigger processing automatically.

Manual Processing

Processing reports is accessible through the CLI. Not necessary if you have configured the task scheduler. This is needed if not using the C19T-Manager and updating report tables directly.

php artisan report:process

The prompt should be straightforward. Out-of-box, the processing function:

  • Sums the daily case and fatality totals for each province
  • Calculates the day-to-day change in tests, hospitalizations, criticals (intensive care), recoveries and vaccinations from the reports table.
  • Stores this summarized data, which is used by the reports API

Health Region (HR) Reports

Support for health regions was added in late 2020. Health regions have a separate reports table (hr_reports) and can be processed from the CLI as well:

php artisan report:processhr
  • unlike standard reports, health region reports calculates case and fatality from the hr_reports table.

Caching

A few select routes have caching enabled to minimize database load and speed performance of the API.

The cache is flushed when a report processing operation finishes. The cache can also be manually flushed from the C19T-Manager companion app.

Tests

Configure testing env

By default, tests will use the APP_ENV defined in phpunit.xml. Create .env.testing with the following template and configure it as needed:

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=test
DB_USERNAME=test
DB_PASSWORD=secret
# used for partner-specific route
PARTNER01=

Running tests

php artisan test

Built With

Authors

  • Andrew Thong - Initial work
  • Noah Little - Concept, data consultation

See also: COVID19Tracker.ca acknowledgements.

License

This project is licensed under the MIT License.

Acknowledgments

covid19tracker-api's People

Contributors

andrewthong avatar dependabot[bot] avatar noahlittle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

covid19tracker-api's Issues

Quebec Data - Total tests are actually number of people tested

Hello folks,

First of all, thanks a lot for publicly providing that API. It's very well documented and great to work with. :)

I was having a look at the API and went over the data in Quebec to look how it compares to the reported numbers. In that regard, I noticed that the change_tests and total_tests do not expose the values that I would expect.
I think the reason is that they do not refer to the number of tests, but rather the number of people tested (which can be quite different as someone might be tested multiple times).
This can be seen quite well on https://www.inspq.qc.ca/covid-19/donnees where the number your API provides correspond to the 4.1 section which describes the number of people tested positive and negative. However, the plot at 4.2, which shows the number of tests, is quite different from the API values.
This can also be seen at https://www.quebec.ca/en/health/health-issues/a-z/2019-coronavirus/situation-coronavirus-in-quebec/#c51841 where the total number of tests stated on the website for August 10 is 1,394,431 (see below). This is much more than the number exposed by the API (926,538 for August 12).
image

Would it be possible to integrate the total number of tests instead or in addition to the number of people tested? I think this number is very valuable as it describes the current capabilities of laboratories much better.

Best,
Georg

health region data should include population and other demographic data

health region data from stats canada should be included in the /hr results. This data can be picked up from:
https://www12.statcan.gc.ca/rest/census-recensement/CR2016Geo.json?geos=HR . Each Health Region ID can be used to call https://www12.statcan.gc.ca/rest/census-recensement/CPR2016.json?dguid=${hr[0]}&topic=13 to collect the last census population values.

health regions don't match current health region definitions

It appears that the health region data uses a hybrid of 2016 census health region/unit definition and current configurations. For example, Saskatchewan used to have 13 health regions, but currently has only one health unit. Yet, the health region data breaks it down by 6 sub divisions. the hr_uid for these itemsdoesn't appear to match the official hr_uid.

Peel data for 2021-06-12 recorded 1million extra new cases

The data entry for June 12 included an extra 1million new cases - which is nearly impossible given the population of that health region is only 1.4million.

{
            "date": "2021-06-10", "change_cases": 96, "change_fatalities": 0, "change_recoveries": 95, "change_vaccinated": 0, "change_vaccinations": 0, "total_cases": 110527, "total_fatalities": 771,
            "total_recoveries": 108941, "total_vaccinated": 88848, "total_vaccinations": 1212884
          },
          {
            "date": "2021-06-11", "change_cases": 67, "change_fatalities": 1, "change_recoveries": 138, "change_vaccinated": 41005, "change_vaccinations": 55545, "total_cases": 110594,
            "total_fatalities": 772, "total_recoveries": 109079, "total_vaccinated": 129853, "total_vaccinations": 1268429
          },
          {
            "date": "2021-06-12", "change_cases": 1000087, "change_fatalities": 2, "change_recoveries": 99, "total_cases": 1110681, "total_fatalities": 774, "total_recoveries": 109178,
            "total_vaccinated": 129853, "total_vaccinations": 1268429
          },
          {
            "date": "2021-06-13", "change_cases": 64, "change_fatalities": 1, "change_recoveries": 151, "total_cases": 1110745, "total_fatalities": 774, "total_recoveries": 109329,
            "total_vaccinated": 129853, "total_vaccinations": 1268429
          }
        ],
        "id": 3553,
        "population": 1480214,
        "province": "ON",
        "short": "Peel",
        "updated_at": "2021-06-13T23:42:02.000Z"

It looks like there was a superfluous extra 1. You can see this when calculating the 7day average of new cases:
image

cf: https://covid19.colinbendell.dev/

hr_uids are incorrect in BC and now inconsistent with CovidTimelineCanada

Hi!

This is an issue to track the problem discovered by jeanpaulrsoucy in ccodwg/CovidTimelineCanada#23 (comment)

Per his note:

The health region IDs for BC used by covid19tracker.ca are wrong: https://api.covid19tracker.ca/province/BC/regions
...but this traces back to the health region IDs for BC used by us in Covid19Canada being wrong: https://github.com/ccodwg/Covid19Canada/blob/master/other/hr_map.csv
...but this traces back to the health region IDs for BC used by ESRI (for their map file) being wrong: https://resources-covid19canada.hub.arcgis.com/datasets/covid19canada::health-region-summaries

However, [CovidTimelineCanada's] current HRUIDs for BC are now correct, according to both StatCan and BC itself.

He notes that he has reached out to @noahlittle on this. This issue is currently blocking microCOVID.org from being able to report at the health-region level: microCOVID/microCOVID#1452

No stats by city

I see that there's city info in the individual cases, but I don't see any endpoints to get totals by city. If this isn't added yet, it may prove useful.

Any data to differentiate first and second dose?

Hi
I would need the number of people "fully" vaccinated with two doses.
I haven't found the definition for change_vaccinations and change_vaccines_distributed, maybe the answer is there but i'm not sure.
Thank you for your good work

Adding split health region reports

Hi! I'm trying to add more fine-grained Canadian data to https://github.com/microcovid/microcovid and thought your API would be perfect!

For our intake, we need about 2 weeks worth of reports from each health region, a couple times a week. However, since I can't find a way to get all the health region reports together, I'm left making ~100 separate API calls for every single individual health region.

Would it be possible to allow split reports? I'd love to be able to use the following:

curl 'https://api.covid19tracker.ca/reports/regions/split'

All endpoints to return "last_updated"

The /summary endpoint returns a "last_updated" value. It would be great if others (especially /reports) returned that value, as well. Specifically, I'm pulling Alberta data and looking for null values or zero length arrays to judge if the content is added for the current day (and popping the array if it isn't). It would be helpful to have that field instead.

Cannot connect to https://api.covid19tracker.ca since Oct 1

Hello, thank you for making your API public. This is a very useful service. I have a Perl script that has been successfully making calls to the API, like https://api.covid19tracker.ca/regions, through the summer. On Oct 1 (between Sep 30 7a and Oct 1 7a EST), it stopped working. The error message is Can\'t connect to api.covid19tracker.ca:443.

I can access the API directly in my browser and I can connect to other SSL APIs from Perl without issue. Do you know if you made a proxy or firewall change around this time that affects access? I used the following SSL tests against your site to hopefully see if there was an obvious connectivity issue. They show something, but I'm not sure what your next diagnostic steps may be.

  • Digicert - unable to connect to covid19tracker.ca, no other details provided
  • SSL Labs - B grade, but I think that's because the server supports old TLS versions
  • SSL Shopper - no issues found

I know there's not a lot to go on here, but am hoping that you're aware of a change on Oct 1 that could have affected connectivity. Thank you.

Axios CORS issue

Hey,

I am trying a get request to the api but I keep getting the following error:

Access to XMLHttpRequest at 'https://api.covid19tracker.ca/provinces' from origin 'http://localhost:3002' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

My code:

axios.get(`https://api.covid19tracker.ca/provinces`,
      {  
        headers: {
          'Access-Control-Allow-Origin': '*',
          'Content-Type': 'application/json',
        }
      }

      ).then((res) => {
      console.log(res);
    })

Can someone tell me what am I doing wrong?

Data issue (Ottawa May 28)

I'm not sure if this is where to post this, but there's an issue with the vaccination data for Ottawa (region 3551) for today (May 28th). It shows change_vaccinations of 17771 (which seems correct), but change_vaccinated is 485840 (I'm sure it's supposed to be in the range of 4,000). I'm sure it's a problem with the Ontario source data, but it's worth noting.

BC Weekend Data Entry

Since Oct 16/21, BC's Saturday and Sunday change_cases have been entered as 0 and then Monday entered as the 3 day total. Previous to October/21 the data was entered by the day. Could we please go back to the daily figures?
Thanks for keeping up with this data all these years.
Doug Hull

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.