ifrcgo / go-api Goto Github PK

This should speed up some larger queries with more relations.

There are actually two facets to this:

1. setting the BULK_IMPORT env variable inside of ingest_appeals doesn't actually work, it needs to be set before the script commences.

2. we shouldn't disable elasticsearch indexing during sync, since that's one of the primary ways that new appeals get created.

Running ingest_mdb the first time on a fresh database works great.

After it runs subsequent times, I've found that the user table gets weird. Currently there are just 7 users in the user table. When I ran it the first time, I confirmed that there were many more (I was able to search for myself through the API, and my user number is in the 13,000's).

Also I don't think the ingest logs are working, getting this working would really help debug this.

To start, index page titles for each data page (field report, appeal, event, etc) along with the type of page.

Figure out why appeals and field reports aren't appearing in search.

Match the updated actions here: https://github.com/IFRCGo/go-frontend/blob/50e12d3ec56cd1783c0e897eb3c52d464f432cde/app/assets/scripts/utils/field-report-constants.js#L1236

Ie, include contacts, etc.

Creations and edits of relevant objects should trigger email notifications to user who have subscribed to these notifications.

This will entail creating a separate app for notifications, and a model to hold these subscription preferences.

Two possible fixes:

1. default to less aggressive border-radius setting for both the bar and it's container
2. always render the bar full width, and use x-axis positioning to visually indicate the amount rather than width (and make the container overflow:hidden).

I think we shouldn't worry about this until next week, but making a note of it for now.

Populate existing subscriptions table from existing DMIS data

Initial sync can take a bit of time and we don't want to start another sync before the first has finished.

Ordering for lots of items in the admin are off. Make them alphabetical or otherwise logically ordered.

The base Django user model only has username, first and last name, and email address. The Django user model is easily extended to include additional information (called a User Profile).

The existing database has extra user info, such as additional contact info among other things. If we need to create user profiles we need to know which fields of the original database should be carried over.

Default tastypie cache is no-cache, which we should change down the road.

Docs

Need to figure out the possible choices here, but the field should definitely be an enum type field.

Including error responses.
Also, look into turning off the default Django HTML responses for 500 status codes, etc.

Currently using vanilla coverage.

For example, you could subscribe the creation of a new field report, appeal, or event.

We should take care to version the API, either individual routes or the entirety of Phase 1.

@matthewhanson field reports should have an owner who is a member of the User table, but this can and should be null for all of the automatically-ingested reports.

ERU types referenced in IFRCGo/go-frontend#32

Update the Country and DisasterType models to include additional fields as given in the fixture data.

Add fixture data for Countries and DisasterTypes to django api app

@matthewhanson these are the notes from Sasi on ingesting the GDACs data:

Info from GDACS (What we refer as emergency in Go is referred as event in GDACS)

http://gdacs.org/rss.aspx?profile=ARCHIVE&fromarchive=true&from=2017-12-05&to=2017-12-06

The above rss feed provides the archive of all the disasters from a specific date to specific date. So, you can consume the latest events on a given day. Some events might be active for few weeks. Each event in GDACS has an event ID. So, a new event will be created if the event ID do not exist.

If there are changes, the events will be found in the archive link below and needs to be updated weekly once.

http://gdacs.org/Xml/rssarchive.xml

Each event is embedded in the item tag

title - Text
description - Text
image - URL (from enclosure tag)
report - URL (This is coming from the link tag)
publishedDate - DateTime
year - Int (From the gdacs:year)
lat - (from geo:Point -> geo:lat)
long - (from geo:Point -> geo:long)
eventType - (from gdacs:eventtype)
alertLevel - (from gdacs:alertlevel)
alertScore - (from gdacs:alertscore)
eventID - (from gdacs:eventid)
severity - Text ( from gdacs:severity)
severityUnit ( from gdacs:severity --->unit )
severityValue ( from gdacs:severity ----> value )
populationUnit - (from gdacs:population ---> unit )
populationValue - (from gdacs:population ----> value)
vulnerability - (from gdacs:vulnerability ----> value)
country - (from gdacs:country)

ISO3 code - (from gdacs:resource id="jsonEvent", you can find the URL for JSON and ISO3 code lies with in it. This JSON is not well maintained. Otherwise, we might have got all the above information from this)

An emergency page will be generated only when alert level for the event is Orange or Red

Newest additions to models are no longer completely covered by tests. Use code coverage module to check % coverage

Update README with

• instructions on Docker image usage
• abstract API
• dev info about branches and CI deployment (pushes tag to github, pushes image to Docker hub)
• required environment variables
• deployment info (exposing port, using secret settings)

cc @matthewhanson

Create table structure so admins can modify their current ERU deployments and indicate available resources.

If possible I'd like to avoid the schema structure we used for field report sources, where there are a fixed universe of source 'types' but each new source is essentially a new record, since that makes it harder to create and modify in the admin.

Starting with the bare minimum for authentication.

This is because Django's is_authenticated scheme doesn't work for our api key authentication.

Ideally there is a proper method to call, otherwise we will just have to check the auth settings in the header against what we've stored.

We need to create an endpoint to upload documents and link them to existing objects (ie to an appeal).

@renecnielsen where do historic appeals documents and situation reports currently live?

Regions should not be editable as they are derived from countries.