codeforpittsburgh / fishfryform Goto Github PK

Finish up the Open API Spec document and implement some more consistent, human-friendly API feedback messaging as part of that.

This would be a public option to get more input without Code for Pittsburgh doing the data entry. This should include option to upload (csv,txt,pdf). These files would have to be verified before updating the db.

Mark provided a list of venues where the menu is a url. Need to copy url over to proper url field and replace menu text with "See website for menu".

Need add'l logic and tests to make sure that the API methods can handle partial submissions, and elegantly fill in the blanks for non-mandatory fields. I think right now it might be rejecting anything less than a full fish fry feature.

Create a new filter option for ECO friendly. Of course who determines the ECO friendliness would be up to the location

Enable dates/times to be edited in the form, rather than deleted/re-entered

From @mhowe0422 on March 31, 2018 13:25

Instead of re-coding dates for the next five years, use a table of the dates from Ash Wednesday to Good Friday

Copied from original issue: CodeForPittsburgh/fishfrymap#23

At the very least, some code minification is in order. The rest of the site doesn't require much in the way of task-running.

@mhowe0422 has noted some issues with the way the Swagger UI is generating the curl commands from the test page: it appears to mishandling newlines, for example.

I might try switching to Swagger UI v2 (down from current v3) and see if that helps. Generally, however, #17 , #18 , and #19 should get us closer to a fully functioning API than chasing down this specific issue.

We know the date for Ash Wednesday and Friday's in Lent. We just need to list the dates, check if the locations is open on that date and add the time range. An additional box for locations that have two open times per date.

To support CodeForPittsburgh/fishfrymap#33 in FishFryMap

Data schema and form validation updates:

• Add a COVID-flag" text field, which would be for capturing any COVID related guidance that the venue is putting out (in addition to the general guidance.
• Add a boolean field to indicate if there is a drive-thru option.
• Add the Eco text field (#11), which would take the same approach as the COVID field.

To get this off the ground, I jammed everything on the server side into one file, app.py. It really needs to be split into logical components (e.g., map, form, API) using Blueprints for flexibility and maintainability.

On Good Friday, turn on the Good Friday filter.

API POST, PUT, and DELETE methods require credentials, but the API isn't currently actually expecting them! Need to add that so @mhowe0422 can work some data cleaning magic.

This FishFryForm class may just as well reflect exactly the structure of a geojson feature--i.e., geometry, properties. Plus the methods we have to interact with the form (geocode, read form, push to form, etc.)

Should have approached this whole thing using a TDD method, but c'est la vie. Still it'd be nice to have some tests for both the client and server side so future improvements are easier to manage.

From @mhowe0422 on March 31, 2018 13:28

Using API loop through the list and update the json data. Code is ready for running.

Copied from original issue: CodeForPittsburgh/fishfrymap#24

certain unicode characters, (http://www.fileformat.info/info/unicode/char/2019/index.htm), line breaks, carriage returns, tabs...etc. Cause errors in the submissions. Single apostrophes...ugh. Rookie mistakes!

Known high severity security vulnerability detected in uglify-js < 2.6.0 defined in package-lock.json.
package-lock.json update suggested: uglify-js ~> 2.6.0.

Known moderate severity security vulnerability detected in hoek < 5.0.3 defined in package-lock.json.
package-lock.json update suggested: hoek ~> 5.0.3.

Known moderate severity security vulnerability detected in pycrypto <=2.6.1defined in Pipfile.lock.

Known high severity security vulnerability detected in mime < 1.4.1 defined in package-lock.json.
package-lock.json update suggested: mime ~> 1.4.1.

Known high severity security vulnerability detected in mixin-deep <1.3.1 defined in package-lock.json.
package-lock.json update suggested: mixin-deep ~> 1.3.1.

...use PynamoDB in this part of the API, instead of boto3.

Really going to need to think hard about how this might be possible or even worth it. Much easier to clean form data than raw text...

I need to renew for the PythonAnywhere hosted-site that https://fishfry.codeforpgh.org points to.

I've currently got the FishFry data in DynamoDB. With that, the fishfryform web app's GUI forms and ReST API are carrying the burden of schema validation and gate-keeping for the database. While within the code base a thin "database interface layer" keeps the Dynamo-specific code at arms-length from everything else, everything still sort of bends towards accommodating unstructured data. Having the app need to enforce schema sort of cancels out the benefits of using a schema-less database; either I maintain schema enforcement logic in the app or the database does it natively. It's been a fun experiment.

It would be nice to get back to using a normal PostgreSQL database so we can do more fun spatial things with PostGIS, but maintain the schema flexibility inherent in something like DynamoDB. So...

...if there's time early in 2019, I might swap out DynamoDB for a hosted PostgreSQL database and use Grout, which is a relatively new thing from Azavea.

We'll see!

Code currently removes the older dates from the Upcoming Fish Frys list. What is going to happen on the last day for fish? You'll have Open today, but nothing to display for Upcoming.

Remove the Upcoming heading and just display the Open today. Then after the day/time has passed, the Open Today could change to "Thanks for your support, see you next year"

Instead of hardcoding venue types, it would be before have them in a db table and read in on the fly. This way we can add new types without html changes. I have two new types: Veterans and Market

CARTO's pricing model for their API access changed; consequently we won't be able to utilize CARTO as a database for the Fish Fry data. This breaks pretty much everything here. However...

We can migrate the database over to a PostgreSQL DB where the web application is currently living (on a pythonanywhere account), and implement some off-the-shelf Python Flask extensions to handle all the CRUD operations and UI stuff (that we otherwise had to custom build to make this work with CARTO).