Format the scraper output in accordance with the schema as outlined here.

Format the scraper output in accordance with the schema as outlined here.

Format the scraper output in accordance with the schema as outlined here.

• Preload DB with zip -> city -> county -> state mapping

• Load election office data from scrapers

• Prod DB Live
• Server Endpoints Working
• Handle case where we don't support the state
• Add this verbiage (or something similar) anywhere: "Absentee voting in-person requirements: Registered to vote in your state"
• Mobile friendly

Website to be scraped: https://www.maine.gov/sos/cec/elec/munic.html
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Format the scraper output in accordance with the schema as outlined here.

Format the scraper output in accordance with the schema as outlined here.

Format the scraper output in accordance with the schema as outlined here.

• Website to be scraped: https://dos.elections.myflorida.com/supervisors/
  The output of scrape should be formatted in accordance with the milestone goal of this issue.

Website to be scraped: https://www.sos.state.oh.us/elections/elections-officials/county-boards-of-elections-directory/
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Format the scraper output in accordance with the schema as outlined here.

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Difficulty: Medium

Pretty much see this TODO and see how you can add support. Should be straight-forward I think once you understand the code, but reach out to Omar with questions if not.

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Website to be scraped: https://www.sos.state.mn.us/elections-voting/find-county-election-office/
The output of scrape should be formatted in accordance with the milestone goal of this issue.

We decided on a solution and it has been implemented. For security reasons, specifics will not be mentioned in this issue.

Website to be scraped: https://www.sos.state.tx.us/elections/voter/county.shtml
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Website to be scraped: https://sos.nebraska.gov/elections/election-officials-contact-information
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Website to be scraped: https://elections.wi.gov/sites/elections.wi.gov/files/2020-08/WI%20County%20Clerks%20Updated%208-7-20.pdf
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Website to be scraped: https://mvic.sos.state.mi.us/Clerk
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Format the scraper output in accordance with the schema as outlined here.

Website to be scraped: https://sos.iowa.gov/elections/auditors/auditorslist.html
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Difficulty: Medium - Hard

Right now, for all states that have been imported into the DB handler module, each of their county election office info gets populated into the database. This isn’t horrible as MongoDB, if we tell it to, is smart enough to not write to the database if data already exists for the field we’re trying to insert into, but it still makes an unnecessary amount of I/O calls. If we can modify the get_election_office_info() function to allow for loading of targeted states, that’d make the application faster, decrease unnecessary I/O calls, and overall allow us to make targeted updates to the DB.

• Corresponding TODO

States affected by the problem

• NH
• ME
• WI

Difficulty: Easy - Medium

At the time of writing, the handler only supports two states. Here’s what’s needed to help support more

• Every scraper file must have an output formatted in accordance to the database schema as seen in our documentation.
• Every scraper file must have a “get_election_offices()” function that runs the scraper
• Every scraper must match this path naming scheme
  • scrapers.[LOWER_CASE_STATE_NAME].[LOWER_CASE_STATE_NAME]_scraper
• Every scraper must run asynchronously. All I/O blocking functions (http requests, file I/O, etc) must be run asynchronously.
• Import the scraper file to the wtb_db_handler script like so:
  • https://github.com/Acesonnall/WalkTheVote/blob/master/handler/wtv_db_handler.py#L25-L28
    If after completing these you run into problems let @Acesonnall know. If you’re trying to add support for a state that lists election offices by city, that’s not yet supported, see #52 for that below if you want to add that.

Completed

• Alabama
• Alaska
• Arizona
• Arkansas
• California
• Colorado
• Connecticut
• Delaware
• Florida
• Georgia
• Hawaii
• Idaho
• Illinois
• Indiana
• Iowa
• Kansas
• Kentucky
• Louisiana
• Maine
• Maryland
• Massachusetts
• Michigan
• Minnesota
• Mississippi
• Missouri
• Montana
• Nebraska
• Nevada
• New Hampshire
• New Jersey
• New Mexico
• New York
• North Carolina
• North Dakota
• Ohio
• Oklahoma
• Oregon
• Pennsylvania
• Rhode Island
• South Carolina
• South Dakota
• Tennessee
• Texas
• Utah
• Vermont
• Virginia
• Washington
• Washington, DC
• West Virginia
• Wisconsin
• Wyoming

Website to be scraped: https://vt.ncsbe.gov/BOEInfo/PrintableVersion/
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Website to be scraped: https://elections.sos.ga.gov/Elections/countyelectionoffices.do
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Difficulty: Hard - Very Hard

To do this you’ll have to refactor all code that makes calls to the database. But there’s potentially a huge speed boost benefit to completing this. Right now we use a combination of PyMongo (main db driver) and PyModm (synchronous ORM for PyMongo). With the way PyModm works, we can make calls asynchronously. The only other ORM for PyMongo that I found that is asynchronous is one called Motor, but its more complex to use the PyModm. umongo is also promising

• Corresponding TODO

Format the scraper output in accordance with the schema as outlined here.

Format the scraper output in accordance with the schema as outlined here.

Format the scraper output in accordance with the schema as outlined here.

Information registering pip modules.

Ideas for functions:

• formatAddressData to return a dict of a parsed address
• helper functions to check if certain fields exist
• helper function to register and edge case

Website to be scraped: https://azsos.gov/county-election-info
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Website to be scraped: https://www.votespa.com/Resources/Pages/Contact-Your-Election-Officials.aspx
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Website to be scraped: https://www.sos.arkansas.gov/uploads/elections/ARCountyClerks.pdf
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Format the scraper output in accordance with the schema as outlined here.

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted

Two ways forward:

• Short term: Make a separate collection in our DB specifically for WI
• Long term: Need to find a data source that includes all of WI's cities, towns, and villages and use that for our database mapping.

Resolves #29

Format the scraper output in accordance with the schema as outlined here.

Format the scraper output in accordance with the schema as outlined here.

Website to be scraped: https://app.sos.nh.gov/Public/Reports.aspx
The output of scrape should be formatted in accordance with the milestone goal of this issue.

Format the scraper output in accordance with the schema as outlined here.

Need a web hosting solutions as well as a server hosting solution, preferably one that would be easy to scale and load balance should the site attract large amount of traffic (i.e. site gets referenced by a news article or shared multiple times via social media).

Feel free to edit this issue to further generalize it/break it down into smaller tasks.

Scrape and format the scraper output in accordance with the schema as outlined here.

• Scraped
• Formatted