Ingest appeals data about go-api HOT 6 CLOSED

Ingestion of Appeals data from the Appeals API has been added, which provides info on both Events (aka Disasters) and Appeals.

A record from the Appeals API looks like

    {
        "APP_Id": 6377,
        "APP_code": "MDRYE002",
        "APP_name": "Yemen - Flash Floods",
        "APP_startDate": "2008-10-29T00:00:00",
        "APP_endDate": "2009-11-30T00:00:00",
        "APP_status": "Closed",
        "ADT_name": "Floods",
        "APP_modifyTime": "2010-03-03T08:40:32.523",
        "OSR_name": "Middle East and North Africa",
        "OSS_name": "Yemen country office",
        "OSC_name": "Yemen",
        "Details": [
            {
                "APD_TYP_Id": 66,
                "APD_noBeneficiaries": 1750,
                "APD_amountCHF": 219800,
                "APD_code": "M08DR141",
                "ContributionAmount": null,
                "OB": null,
                "APD_startDate": "2008-10-29T00:00:00",
                "APD_endDate": "2008-11-09T00:00:00",
                "APD_modifyTime": "2008-11-11T07:51:56.973"
            },
            {
                "APD_TYP_Id": 64,
                "APD_noBeneficiaries": 4900,
                "APD_amountCHF": 1754011,
                "APD_code": "M08DR152",
                "ContributionAmount": 1267986.06,
                "OB": null,
                "APD_startDate": "2008-11-10T00:00:00",
                "APD_endDate": "2008-12-04T00:00:00",
                "APD_modifyTime": "2008-12-08T10:25:00.663"
            },
            {
                "APD_TYP_Id": 64,
                "APD_noBeneficiaries": 10500,
                "APD_amountCHF": 1997283,
                "APD_code": "M08DR160",
                "ContributionAmount": 1267986.06,
                "OB": null,
                "APD_startDate": "2008-12-05T00:00:00",
                "APD_endDate": "2009-11-30T00:00:00",
                "APD_modifyTime": "2010-03-03T08:40:08.007"
            }
        ]
    }

At the top level we have details on an Event, of which there could be multiple Appeals (under Details). The start and end date of the event is based on all the appeals under it.

In the current database an 'Event' is made up of the Appeals under it, and since appeals are regional (they have a Region, Country, and District or Sector (?, not sure what it's called, but OSS_name is some sort of subregion). So this means an Event is rather localized.

In Go, I put Region as part of the Event, and Country and Sector under the Appeals. This opens up the possibility in the future that an Appeals from different smaller areas could be part of the same overall Event, which would be Regional in scope, so some of the events created from ingested might be able to be manually merged.

Mapping from the Appeals API record to the new Go database is as follows:

Event:

• APP_Id -> eid (unique identifier for this event)
• APP_code -> code (if this is just a unique id, I'm not sure if both it and APP_Id are required)
• APP_name -> name
• ADT_name -> ForeignKey to a DisasterType (if match to existing one it uses that, otherwise it uses 'Other')
• APP_status -> status
• OSR_name -> region (although this is a free text field at the moment)

Appeals (1+ per Event):

• APD_code -> aid (unique id for this appeal)
• APD_startDate -> start_date
• APD_endDate -> end_date
• OSC_name -> country (looks up matching country from Country table)
• OSS_name -> sector (free text for now, subregion of country)
• APD_noBeneficiaries -> num_beneficiaries
• APD_amountCHF -> amount_requested
• ContributionAmount -> amount_funded

Note that in many cases the additional appeals just appear to be extensions of an earlier Appeal, although they have different APD_codes.

from go-api.

This opens up the possibility in the future that an Appeals from different smaller areas could be part of the same overall Event

👍

Note that in many cases the additional appeals just appear to be extensions of an earlier Appeal, although they have different APD_codes.

I'm assuming these are the actual appeals as opposed to DREFs. A DREF always comes first, and an appeal usually follows if the event is large enough to warrant it.

This PR is merged and I have a small fix in #56. After that's merged I think we can close this ticket.

from go-api.

I've also been assuming they are all Appeals - do they have DREFs in their API, and if they do, how do you tell the difference?

from go-api.

@matthewhanson Just learned that:

apd_typ_id == 66 means it's a DREF
apd_typ_id == 64 means it's an "Emergency Appeal"

There's a third categorization for conflict-related appeals, I'm checking what ID corresponds to that now.

from go-api.

Sso sounds like they can both remain in the same table as "Appeals", and just add in that type id

from go-api.

Oh yeah they should definitely stay as-is, they should just be aware of the type.

from go-api.