Code Monkey home page Code Monkey logo

covid-vaccine-spotter's Introduction

COVID-19 Vaccine Spotter


SHUTDOWN PLANNED: VaccineSpotter.org will wind down operations on September 23, 2021. See the shutdown notice for more details. This code will remain available if it's of interest, but I haven't been actively working on things in a few months now, so I think it's time to consider this chapter closed.


A tool to help you track down COVID-19 vaccine appointment openings at your state's pharmacies. The code behind https://www.vaccinespotter.org.

UI Development Setup

Requirements:

  • Node.js 14+
  • Yarn

This process is clunky right now, so my apologies.

  1. Clone the repo: git clone https://github.com/GUI/covid-vaccine-spotter.git
  2. Install dependencies (inside the repo): yarn install
  3. Fetch data from the API for the website to run: yarn setup
  4. To run the development server for the website: yarn dev. The development site should then be available at http://localhost:3000/.

DB and Backend Development Setup

docker-compose is used to bring up a local postgres DB. Then an image runs to bootstrap the database with states and postal codes. Finally, as an example of a backend task running, a container runs the refresh-website task.

docker-compose build
docker-compose up

To run additional tasks, you can use docker-compose run:

docker-compose run --rm --no-deps task_runner bin/refresh-cvs

Very Beta API

All of the data being collected is published as JSON files here: https://www.vaccinespotter.org/api/

Subscribe to this discussion for any announcement of API changes: #27

Website Language Translations

Contributing Translations

If you would like to help translate the website content into other languages, thank you! You can perform translations and add additional languages at Localazy.

Developer Translation Workflow

  1. If you're adding new text to the website, just wrap the English text with the $t helper, like $t('my text here...').
  2. In order to translate these strings into other languages, and you have permissions to Localazy, then the steps are:
    1. Run ./bin/run locale:extract to extract any new strings into the website/lang/messages.pot file.
    2. Run ./bin/run locale:upload to upload the updated website/lang/messages.pot file to Localazy.
    3. After translations are performed, run ./bin/run locale:download to download the resulting language files.

License

This project is licensed under the terms of the MIT license.

covid-vaccine-spotter's People

Contributors

ablyler avatar clottman avatar craig-cogdill avatar gui avatar jorupp avatar joshuapaup avatar lawrencek1992 avatar localyzerexpresslingoport avatar mkokotovich avatar mr0grog avatar mtlynch avatar nbelans avatar patrickdemers6 avatar shirubana avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

covid-vaccine-spotter's Issues

Some Walgreens locations you can't book despite claiming to have availability

This has been heavily affecting at least Colorado users the past several days causing lots of frustration.

Basically the issue is that Walgreens' data will claim to have availability for a specific store. Even if you specifically select that you need "First & Second Dose" it will let you proceed with booking at locations that have appointments in the next several days. However, once you get the step to book the second dose, Walgreens will inform you that there is no availability and therefore you can't book anything.

Many CO users have reported that they've called the pharmacies and the pharmacies have said they are not currently doing any first & second doses in CO, only second doses. However, I know I've heard from some users that have still had success in booking first and second doses at Walgreens (however rare), while I've added warnings, I've been hesitant to yank the data entirely.

So while I sort of consider this an issue on Walgreen's end with providing misleading information on the initial requests until you get further into the process, this is something I'd like to address to prevent more frustration on the part of users. After digging into this, in order to fix this, we need to check for future booking dates to simulate what the Walgreens does when you go to book the 2nd dose. The logic Walgreens seems to use for looking for 2nd dose appointments is:

  • Look for appointments starting 28 days ahead of the 1st dose time. This seems to happen regardless of vaccine time (Moderna of Pfizer).
  • Each appointment slot API request fetches 3 days in the future of the first date, so there has to be some time slot bookable within that 28 + 3 day window from the original appointment date.

We should then only consider the 1st appointments valid if there's a 2nd dose appointment open in the future according to these rules.

Scan "next" date range for Pharmaca

With the Pharmaca scanning, we're parsing the initial date range that gets returned by default. This seems to maybe be the next month-ish of appointments. But there's a link we could follow to also scan the "next" date range (it's not exactly month based). You can keep clicking "next" seemingly indefinitely, though. But maybe we should at least scan the next 1-2 date ranges to see if any other appointments are further out (right now, nothing seems to be scheduled that far out, so this is more of a hypothetical right now).

Albertsons/Safeway scanning broken

Around 4:30 today, the Albertsons scanning broke. As far as I can tell you now need to fill out the captcha before making any requests. They had added a captcha last week (I think), but the endpoints actually didn’t seem protected until today.

So far in trying to get this working again, I have code that can fill out the captcha (using a paid captcha service) in a headless browser, but the authenticated session is then only good for ~25 requests before the captcha is required again. So this could be rather slow and costly to support. I can push the captcha code up anyone’s interested, but I’m going to have to call it a night for today. I’ll see if I can figure anything out or find anyone at Albertsons to reach out to.

Add translations into other languages

Could start out by adding something like Google Translate to the page, but if there's demand and interest, I'd certainly be open to properly localizing the tool for other languages.

Switch to PostgreSQL for database storage

The current usage of Cosmos DB was sort of an early experiment while I was trying to figure out what cloud offerings could best host everything for free. But with the tasks running in AWS Lambda and the database over in Azure, this is all a bit funky and could use some cleanup.

And nothing necessarily against Cosmos DB, but I'd personally be more productive in PostgreSQL, so now that this tool seems like it's useful and I'd like to do more things in the database, I'd be fine paying a bit for a PostgreSQL service where I'd be more comfortable/productive. And hopefully this would also make everything easier to setup and run for others (so there wouldn't be the odd dependencies split between cloud services).

Instructions for running locally?

I'd like to contribute if I can, but I can't find instructions on how to build/run this locally so I can test my changes. Any help would be appreciated.

Some Rite Aid locations you can't book despite claiming to have availability

This doesn't seem to affect all Rite Aids, but it at least seems to affect several different ones I've checked in Maryland.

  • We currently hit the checkSlots API endpoint which returns a boolean for whether or not appointments are available. This seems to be what Rite Aid's site uses to determine whether or not to take the user to the calendar page.

  • For several of these locations I've checked, this checkSlots API returns true, indicating appointments should be available. However, if you navigate to Rite Aid's site and try to book an appointment at this location, you run into errors. You are let past the first page after selecting the location, which hits the checkSlots API, but when you land on the calendar page, I just get a spinner that spins forever, along with JavaScript errors being thrown in the console:

    Screen Shot 2021-03-08 at 5 01 54 PM

    In this case checkSlots returns:

    {
      "Data": {
        "slots": {
          "1": true,
            "2": false
        }
      },
      "Status": "SUCCESS",
      "ErrCde": null,
      "ErrMsg": null,
      "ErrMsgDtl": null
    }

    But when on the calendar page, an additional request is made to en.ragetavailableappointmentslots.json, which returns this for these problematic locations:

    {
      "Data": {
        "slots": {
          "1": [],
          "2": []
        }
      },
      "Status": "SUCCESS",
      "ErrCde": null,
      "ErrMsg": null,
      "ErrMsgDtl": null
    }
  • When locations are bookable on Rite Aid, this second en.ragetavailableappointmentslots.json API response returns something with data in the slots, like:

    {
      "Data": {
        "slots": {
          "1": [
            "2021-03-09T13:45:00",
            "2021-03-09T14:36:00",
            "2021-03-09T14:39:00",
            "2021-03-09T14:45:00",
            ...
          ],
          "2": []
        }
      },
      "Status": "SUCCESS",
      "ErrCde": null,
      "ErrMsg": null,
      "ErrMsgDtl": null
    }
  • Ideally we could hit this en.ragetavailableappointmentslots.json API endpoint to get the seemingly more accurate information as well as getting slot times, but unfortunately this API endpoint is protected by recaptcaha, so this is much more difficult/slower to hit.

  • This doesn't affect all locations, so some locations that return true in checkSlots are indeed bookable it would seem. I'm unsure of how many locations this affects, so I'm hesitant to yank any of the data.

So while this seems like maybe more of an issue on Rite Aid's side with a mismatch between checkSlots and en.ragetavailableappointmentslots.json (that even their web app fails to handle gracefully), I just wanted to compile the notes I had so far on this. I'm not really sure there's much that can be done about this without being able to access en.ragetavailableappointmentslots.json (which probably isn't feasible due to recaptcha). In the meantime, hopefully Rite Aid can fix things on their end.

Expand to other states?

I've had a number of requests from people looking for this type of service in other states. The code is all open source (albeit in need of cleanup and documentation), so theoretically others could run this stack for other states. However, if we can get things running smoothly in Colorado first and then if there hasn't been any activity from other people running similar things in other states, this is something I could maybe explore. While there's obviously some regional and local chains that will differ in other states, just having CVS, Walgreens, Walmart, Albertsons (and all its sub-brands), and Kroger (and all its sub-brands) could cover a lot of pharmacies in a lot of states right out of the gate. This would obviously be a lot more scraping (which I'm trying to do in a courteous way), so I'd need to think about how to best distribute the scraping (so stuff still gets updated frequently, but still throttling things to try and be courteous).

Time zone display can be wonky

After the latest updates, the time zone handling appears sporadically broken. I had previously been using time-elements to handle rendering a static time on the page, but having it update to the client's time zone. However, with the switch to Nuxt/Vue for the website, it seems like this doesn't always behave as expected. I suspect this might be fighting with Nuxt or Vue's own template rendering and refreshing. It sometimes works, but after refreshes, I can usually get some of the times on the page to end up being in EST instead of MST.

Improve Walgreens logic to try and better handle first/second dose issues

Currently we just get back whether or not each Walgreens has any appointments in the next several days. But people have found that even if there are appointments available, Walgreens won't let you schedule the appointment unless you can also schedule the second appointment at the same time. But often there are no appointment slots at the right point in the future, so this can still be frustrating. Ideally we'd be able to determine whether just single appointments are available (for the second dose) or whether first and second appointments are available.

From what I recall, the second dose scheduling just looked forward a month in the calendar to see if there were any slots there. It's possible we could do something similar with our lookups, I'd just want to try and verify the logic Walgreens performs for the second doses to see if it's always 4 weeks (or whatever the date range is), or if it varies by vaccine type, etc.

Refresh Walmart store metadata to detect new stores that carry the vaccine

While we're actively refreshing all Walmart locations for vaccine appointments, there's another piece of how Walmart works: each store's metadata (that's not part of the appointment details) declares whether or not each store can possibly carry the vaccine. So in order to catch appointments of new stores that start carrying the vaccine, we need to also refresh this store metadata on some type of regular basis.

Add filtering based on pharmacy chain

I've heard from some users wanting to be able to filter based on pharmacy chain, particularly with the new interface updates.

I think this mostly comes from frustration with some pharmacy chains that seem to have more current issues (eg, Walgreens in Colorado right now), but these issues don't seem to affect everyone, so I'm also not sure how prevalent this need is. I hadn't wanted to clutter up the UI too much, but it is something that we could perhaps consider (I think ideally you'd be able to select multiple pharmacy chains to view, so you could just deselect the ones you didn't want to view). The only corner case that comes to mind is that if you're refreshing this type of page with a filtered pharmacy view and I add a new pharmacy chain, you may miss out on the new ones.

Improve error message when you enter a zip code that can't be found

With the addition of ZIP code search, I've heard from one user that I think was rightfully confused that they couldn't search for zip codes in other states and get any results. I think in the error message when a ZIP code isn't found in your state, we could provide better guidance on this and maybe point towards the other states.

This could also perhaps be made better by improving the overall navigation so it's more obvious how to switch between states: #52

There's maybe a broader issue here of whether or not the state-centric search makes sense, but I'm sort of in favor of keeping it that way for the purposes of this tool. I'm doing a sort of dumb client-side ZIP code lookup so I don't have to pay for geocoding services, and while loading an entire state's worth of zip codes is semi-okay (still a bit big), loading the entire US would probably be too much. And given the state-centric nature of vaccinations, I think this state-based approach mostly makes sense, it's maybe just less convenient if you're helping friends in other states and needing to swap between states.

Adjust map pins to better deal with color blindness

I've heard from several users that the current map colors aren't great for red/green color blindness (which I know should have been obvious given the colors I picked :). I tried to mitigate this slightly by making the green ones bigger, but this could all use some better distinction and coloring to try and better deal with this (maybe just better colors, or also maybe consider different pin shapes).

Add ability to search and filter by location

This is increasingly a request I'm hearing from users. The current approach of dumping all the information in the state into one page obviously isn't too user friendly, particularly in states with lots of locations.

I've started work addressing this and hope to have some improvements on this front soon.

Add Prettier/ESLint

The code is a messy mess right now. I'd at least like to add some basic linting and standardize the formatting.

Support Kroger in states beyond CO

This is difficult due to their anti-bot technology. I sort of have something working for CO, but it's finicky. It might be possible to scale this to other states, though.

Improve UI navigation between different states

If you land on a state-specific page, it may not be obvious that the site has data for other states. Technically you can click on the header in the navbar (eg, "Colorado COVID-19 Vaccine Spotter"), and you'll land back on the home page, but this is not obvious (or perhaps even the correct action for that title) at all.

Having some better type of global navigation might be useful. Or alternatively, maybe just some state drop down in the nabber to switch between states (although on mobile, this would probably need to be collapsed in a menu due to space).

Walmart scanning needs additional inventory check

Currently I'm just hitting /pharmacy/v2/clinical-services/time-slots/{cid} for a specific store. However, it looks like we need to first hit /pharmacy/v2/clinical-services/inventory/store/{store_id}/{cid} to check for inventory first. I think this might be a new step, since I don't remember this inventory step from before (but it's also possible I just missed this ☹️). But this definitely seems to be required now, since otherwise there are stores that do return appointment slots despite having no inventory (in which case the Walmart site doesn't actually ever let you get as far as checking for slots). Oof. :( Hopefully this doesn't affect too many stores, since I hate giving false positives.

Expand Pharmaca scanning to other states

This is a smaller regional pharmacy, but they are in a number of other states. As part of this, also need to move this data into The PostgreSQL format to better standardize things.

Expand Walgreens scanning to all states

Also need to move it into the new Postgres structure. I think that will also allow for more intelligent scanning with PostGIS to reduce the number of requests due to the number or stores Walgreens has.

Push website hosting to S3 bucket

The website is currently hosted via GitHub pages. But pushing to the gh-pages branch every minute may not be a particularly viable long-term solution given the number of commits and frequency of pushes (there's also a lot of churn in the files, since all the JSON changes are recorded every minute). I've read about GitHub Pages build limits per day, although I'm not sure that affects us, since we're not actually building the site with GitHub pages (with Jekyll), just pushing an already built site to the branch. I received a couple GitHub Page build failure notifications last night, which is maybe related to all this. In any case, I guess it seems like just pushing to an S3 bucket might be a better plan so we don't abuse GitHub Pages.

Link to Meijer Pharmacy?

I've had requests to track Meijer, but I'm not sure that's possible, since they seem to operate with a registration process (so I'm not sure there's really appointments to scan): https://clinic.meijer.com I'm not sure if it's worth linking to other pharmacies like this that can't be scanned, just so people know what to do (although then that opens up a can of worms in linking to other possible places to get vaccines, which is something I've hoped to defer to official state resources). But anyway, just something to think about.

Add filtering based on vaccine type

This is tricky to do, since I'm not sure we have the data necessary to do this right now, but just noting a request I've heard. But if we can reliably normalize the data from pharmacy providers, I know some people are only looking for 1st or 2nd shots and then some people are looking for types of vaccines (eg, Moderna, Pfizer, J&J).

Right now I've started to keep track of the appointment "type" which does include some of this information, but this is usually just based on text we get back from the pharmacy, so it's not standardized across different pharmacies. And we also don't have this information for all chains (some we just have appointment times, and other we don't have any details).

Switch CVS to use their official API

CVS appears to actually have a COVID-19 Vaccine Availability API: https://developer.cvshealth.com/apis 👏 I've applied for a developer account, but haven't been approved yet.

I'm hoping this API could provide better details about the appointments available. Based on the endpoint we're scraping, we just know whether or not any appointments are available, but not details about the appointments. So this doesn't distinguish between first/second dose availability and doesn't tell us how many appointments are available. I'm hoping the API might provide more details.

Albertsons indicates appointments available when there are none

Example:

{
    "type": "Feature",
    "geometry": {
        "type": "Point",
        "coordinates": [
            -96.8306729,
            33.0430426
        ]
    },
    "properties": {
        "id": 1048028,
        "url": "https://www.albertsons.com/pharmacy/covid-19.html",
        "city": "Plano",
        "name": "Tom thumb 2570",
        "state": "TX",
        "address": "5968 West Parker Rd.",
        "provider": "albertsons",
        "time_zone": "America/Chicago",
        "postal_code": "75093",
        "appointments": null,
        "provider_brand": "tom_thumb",
        "carries_vaccine": true,
        "provider_brand_name": "Tom Thumb",
        "provider_location_id": "1600122449411",
        "appointments_available": null,
        "appointments_last_fetched": "2021-02-24T23:30:38.249+00:00"
    }
}

But checking through the provided URL shows no appointments available.

The README.md lacks a license

This is usually just the norm for Open-Source projects like this one (though it might not be necessary with the inclusion of a separate LICENSE.txt).

Integrate Weis, Wegman's Support

Hi! I'm one of the maintainers of VaccinatePA, a site that crowd-sources PA vaccine availability. Recently we added VaccineSpotter's to our site and have received glowing feedback for the Realtime Availability.

We recently were introduced to VaccCheck [0], which is similar to what VaccineSpotter does but for the large Northeast chains Wegmans and Weis Markets. However, the way VaccCheck works doesn't fit in our workflow (it requires the program to be run on a local machine).

Do you think it could be possible to integrate what they're currently doing into VaccineSpotter?

[0] https://github.com/dinsmoro/VaccCheck

Display when appointments are typically found

As a Vaccine Hunter, I want the system to display the days of the week and the times that appointments are typically newly found, so that I can target my vaccine hunting time to those days and times and more likely find an open spot for that location. (maybe right under each location a small message like “ appointments over the last week were typically published on: Monday at 6 PM, Tuesday at midnight, Thursday at 4 AM) (FYI I haven’t programmed in 10 years but I’m an agile coach now, so I can’t help with the programming unfortunately!). Keep up the good work!

Map/Zip Code not Working

Hello all-- I'm on the MN map, and typing in zip code and hitting search does not change anything on the page. On a whim, I tried a random (but real) Zip Code on the WI page as well, and the search did not update as well. I am in Chrome and Safari as well. The MN page was allowing me to search up to an hour ago. Data is still updating.

Add notifications

Letting people signup for notifications (either e-mail or text) would obviously be even better than making people refresh the page. Already in progress in #1.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.