Create a mechanism to represent saved lists on a server. A user would create the list, or saved query for later access via the API.

And when there are inevitably duplicate records for the same person, what happens to the discarded GUID when the records are merged?

Some systems don’t store linked or embedded resources such as address separately. What would they put for a self link?
Proposed:
They could have a convention to get a partial resource that is proprietary via query params that would be encoded in the self links. Clients would treat this as opaque

I forgot to bring this up today on the call, but I'm a little concerned about the naming of collections for a person.

The three I'm concerned about are:

"phones" versus "phone_numbers"
"emails" versus "email_addresses"
"addresses" versus "postal_addresses"

I much prefer the ones on the right for clarity. Conversationally we always talk about "phone numbers", not "phones", which things you use to talk on. "Emails" could be a collection of mailings sent to or from a person when we really mean "email addresses." "Addresses" is less ambiguous, but I'd probably differentiate it from "email addresses" just for consistency.

Schema.org uses "Postal Address", "Email", and "Telephone", but don't ever seem to use them in the plural. The plurals are more problematic than the singular forms in my mind though.

Thoughts?

Some volunteers may prefer to be addressed by a nickname (ex: Richard is Dick, Robert is Rob, Morris is Monty, or less common like Steve is "Red"

Create a field (and set a standard format -- epoch? YYYY-MM-DD? YYYYMMDD?) for a Person's date of birth.

Some orgs have a need to maintain a list of targets like legislators. Including them in the people collection causes complexity in simple supporter queries for email blasts etc.

options

Separate targets collection

this would use the same schema as person, but be in a different collection

Lists

Create the concept of a list that people can be associated with. This could be used to associate legislators to a 'legislators' list.
It could also have lists for current supporters, past supporters, donors, as an example.
Some products create individual lists for volunteers to canvass

Tags

Use tags to associate people to a give virtual list

A few aesthetic points:

Address

• state should be region, which is the more commonly used term when handling data from many countries. For example, it's the term used by vCard and Schema.org.
• locality is a less urban-centric term than city. Used for example by vCard, Schema.org and xAL (an OASIS standard used by KML).
• Either country_code should be abbreviated to country or state should be expanded to state_code. Right now, the naming is not consistent. I prefer removing _code.
• Why prefix address_type and address_status with address_? Since these properties are on an Address document, then we already know they apply to the address.
• Not sure why lat and lng are abbreviated if all other properties are not.

Person

• For name components (first, last, etc.), if the intention is to handle data from many countries, terms like given_name, family_name, etc. are more appropriate - especially given that in many regions, the family name is written first, not the given name as is usually the case in the US. If you are looking to reuse standard terms, this part of the Popolo spec reuses the same terms as Schema.org, FOAF, vCard, etc. which are:
  • family_name
  • given_name
  • additional_name
  • honorific_prefix
  • honorific_suffix

W3C and DMCI provide some guidance on name representation.

Anyway, as I said, just aesthetic issues.

On addresses I propose renaming "lat" and "lng" to "latitude" and "longitude". We don't use abbreviations anywhere else for our fields and it would be nice to be consistent.

This issue arose with a request to add a 'notes' attribute to interactions to capture CRM like info.
This can also be useful on all objects.
An alternative is suggested to have attachments collections which could include text notes, files, etc as well as metadata like author, date

Does the group plan on publishing any standards on how to authenticate users?

It could certainly be left open as well, but if there are plans on supporting certain authentication types officially would be good to get some guidance on that.

What about having embedded resources contain a subset of attributes vs a whole resource?
How does this impact client conditional complexity?

Approaches

One approach is to have a consistent summary and complete view of a resource
This would provide consistency, but we need to make sure that the different
responses where a person object would be returned (attendance, donation) have the attributes they need

Only include 1:1 resources, not collections
The default embed policy would be to consistently only include resources that have a 1:1 association and exclude embedded collection associations

Collection of links / list of named resources only
Only attributes are name and link. After more discussion: "What about email? that is also almost always needed?"

Note: this reduces to summary view embedded resource
We could document this with a column in schema table for summary or extended

A few questions and comments regarding address attributes.

1. Would a "potential address" from TargetSmart or Catalist proximity matching have an address_type = "potential"? In our internal organizing software, we have collections of potential addresses which organizers will visit until they can determine a primary address.

2. What about bad addresses? It is useful in many cases to know if an address is bad. I'd suggest an additional attribute to reflect when an address is bad, such as if a person has moved. For union organizers, it is helpful because we know not to visit that house again.

3. Geocodes. It seems that "lat" and "lng" properties should optional attributes? Perhaps with some accuracy attribute such as "rooftop" or "approximate".

pick this up post beta

For a person there's a field called "party" that I think needs some minor tweaks. My suggestions:

First, I'd suggest renaming the the field to either "party_affiliation" or "party_identification". If you think about it, this is really the gender issue all over again. What data are we actually recording here? A person might be registered as a Republican, but identify as "Independent." It seems that both pieces of data would be useful. Renaming to either "party_affiliation" or "party_identification" allows us to add "party_registration" down the line.

Second, the options are currently "democrat", "republican, "independent", and "none". If we're naming the party, let's capitalize the options (as we do all other enumerations) and use the correct names: "Democratic", "Republican".

That leads us to "Independent", "None" and by association null. "Independent" should really mean "Unaffiliated" shouldn't it? And if so, shouldn't "Unaffiliated" mean the same as "None"? I suppose some people self-identify as "Independent" and are engaged in politics while apolitical people would say they don't have a party affiliation.

Thoughts?

Need to describe what compliance means.

Initial Thinking
AEP is required

all collections are optional as different products may only support certain kinds of scenarios. A system that does not do anything with donations would not need to support the donations collection.

If a product does support a collection, it must support the defined schema for that collection.
(modulo vendor extensions)

For an event, I suggest adding a "capacity" attribute. Our event system allows events to have a stated capacity when created and we shut off RSVPs when the capacity is reached (or slightly above).

Related to capacity, I notice that there is a guestsCanInviteOthers attribute. This makes sense, but you require each Attendance object to have a 1:1 ratio with a Person, which isn't necessarily something you'd have if guests can invite others.

For our system we have the concept of an Attendance that has a totalGuests count attribute on it for "anonymous" guests. We also have an array of simple strings to hold guest names.

I would definitely like to see a totalGuests or totalAttendees attribute on the Attendance type. What do others think?

One thing our documentation doesn't do well is note required/optional status. I think we should probably add a column to each table with this data.

For example, for the Phone Number type, we should require "number" and "primary" be sent over. The rest can be null, but these two should always be set.

The gender and sex fields can be confusing to some and may be seen as unecessary complexity.
However, with increasing frequency, orgs have a need to represent the emerging reality of transgender identity.

Needs further discussion and community feedback

I believe the pain and complexity we've encountered around updating embedded resources are the result of incorrect modeling. I propose that we simplify the people model to reduce the complexity of updating embedded resources. This should be done through the introduction of value types for several objects which are currently resources.

To best explain the issue with our current design, I'm first going to review HTTP verbs and the concept of a resource. Then I'll review entities and value types and propose a simplification to our current person model.

REST and HTTP Verbs

Since OSDI is taking a RESTful approach to building the API, I want to ensure everyone has a firm understanding of the HTTP verbs that are used to manage the data resources exposed by the API.

Generally speaking, the HTTP verbs that operate on a resource have the following definitions (taken from Wikipedia):

Given these definitions, there are a couple of points that should be highlighted:

• PUT requires the full resource be sent to the server and actually allows for the creation of a resource at a client-supplied URI. In many scenarios, and likely ours, there's no way a full resource will be able to be created solely on the client without some server interaction. For example, the API server will likely have to take part in a resource's creation by supplying server-side IDs and corresponding URIs. As a result, we should limit PUTs to replacing existing resources.
• PATCH doesn't define how partial modification of a resource is achieved. This will be up to the API and depends on the type of resource being updated. Since we are dealing with JSON, RFC 6902 is one method we could adopt. It is quite likely that we won't even need PATCH support to begin with.

Entities and Value Types

HAL and the HTTP verbs help us define the interaction and framework of our API. To better describe the models of our API, I'd like to define a couple additional terms that I've borrowed from Domain Driven Design:

Entities

An entity is something with an identity that does not depend on a parent context. An entity may have relationships with other entities. For example, a Donation entity might reference a Donor entity. Both the Donation and Donor exist as independent objects with their own identities and can be updated independently.

In the case of a HAL API, an entity is a resource that can be accessed at a URI. (For the DDD people out there, technically I'm proposing that a Person be treated as an aggregate, not just an entity.)

Value Types

A value type is an object that describes data but has no intrinsic identity. A great example of a value type we already use is date of birth. We describe a person's date of birth with the following value type:

date_of_birth: {
    month: 1,
    day: 15,
    year: 1978 
}

A date of birth is described entirely by its attributes. It has no identity and no meaning outside of the context, in this case a person, to which it is attached.

Similarly, a person could have an array of phone number value types:

phone_numbers: [
    {
        country_code: "1",
        area_code: "202",
        local_number: "555-1212",
        type: "Home"
    },
    {
        country_code: "1",
        area_code: "202",
        local_number: "555-1212",
        type: "Mobile"
    }
]

In the context of the OSDI data set, each phone number is essentially useless outside of a person and can be wholly identified from its values.

From the perspective of a RESTful API, value types are not resources. They cannot live on their own or be accessed independently and lacking identities they will not have URIs.

Proposal

I propose that we treat postal addresses, email addresses, and phone numbers as value types without any inherent identity of their own. These values would always be returned as intrinsic parts of the person entity. The result of this simplification is as follows:

• Atomic creation and modification of a resource and its values is now straightforward.
• Retrieving a resource would always include all its associated value types.
• The value types would not be in the _embedded section of a HAL resource.
• The value types would not be referenced in the _links section of a HAL resource.
• Updating, adding, or removing a value type of a person would require either a PUT of the entire modified person resource or a PATCH describing the modification.

This proposal does not directly address updating embedded elements using a HAL API. Rather, by adding value types to our model, we reduce the complexity of the API for both hosts and consumers.

Computed properties are generated by the system and read-only. Example: current attendee count on events.
When doing an update what happens if these are included in the representation?
Proposed:

They may be omitted. if they are present the values are ignored

Need description of rules for vendor extensions

Where can extensions be added and how should they be done?

Possibilities

new collections
new values for enums
new attributes on an existing resource

For address lines, does it make more sense to simple have an array instead of "address1", "address2", "addressN"? This seems hackish and doesn't map well for statically typed language.

I suggest either removing the addressN fields and leaving address1 and address2 or simply have one string array field:

{
    "address_lines" :  [ "line1", "line2" ]
}

I doubt we'd ever need more than 1 and 2 unless we're doing international addresses (in which case the field names should probably be different anyway).

Since many if not most systems include a name-prefix field (Ms., Mr., Rev.) and a name-suffix field (Jr., Ph.D.), it would make sense to include them as properties here. If the systems on both ends of the API support those fields but the API itself doesn't, then prefix and suffix have to be munged into the other name fields, or left out, which is messy and/or data-corrupting.

Minor suggestion: rename "middle_initial" to "middle_name". Oftentimes you see whole middle names and maiden names in data, and if the field is called "middle_initial" it can cause confusion as to whether the data will be truncated in transit.

Partial updates are a necessity, but should not be done via HTTP PUT as described here (https://github.com/wufm/osdi-docs#updating-a-resource). The semantics of PUT is to replace the entire resource with the body of the request. O'Reilly's "HTTP: The Definitive Guide" elaborates, saying: "POST is used to send data to a server [for processing]. PUT is used to deposit data into a resource on the server (e.g., a file)."

For partial updates, I propose:

• use POST
• fields having blank posted values ("") are left unchanged on the server
• null is explicitly submitted as the value for fields that are to be set to null. (javascript uses null, not nil)

The PATCH command is supposedly meant for partial data updates, but is patchily supported at the moment and is more effort to implement. I think POST is the best candidate verb.

Josh and I had a lengthy discussion on the HAL mailing group: https://groups.google.com/forum/?fromgroups#!searchin/hal-discuss/documentation/hal-discuss/lt0CnC3eev4/K3nry6KxJ6cJ

So it appears we're supposed to namespace the links/embeds on documents (a curie - http://www.w3.org/TR/curie/ ).

That would make a simplified person model look like thos:

{
  "_links": {
     "self": {
       "href": "http://host/path/to/self"
     },
     "curies": [{
         "name": "osdi"
        ,"href": "http://path/to/rel/docs/{rel}"
        ,"templated": true
        }
     ],
     "osdi:addresses": {
        "href": "http://path/to/addresses"
     }
  },
  "first_name": "name"

Where the osdi:addresses curie would translate into docs. What are everyone's thoughts on this?

Consider adding a "handles" collection to the person aggregate, to keep track of the many system IDs a person may have. E.g,,

handles = [
{
"service" : "linkedin",
"id" : "markpaquette1"
},
{
"service": "ngp_van",
"id": "123456798"
}, ...
]

This would remove the need for the dedicated properties "twitter_handle" and "guid", and make for a much more flexible system.

How do updates work with embedded resources? Can a client include an embedded resource be included in an update?

The twitter handle should not include the @. It can be safely inferred and there's no reason to include it in every record.

In a case like getting an event resource, if we wanted to return attendance as a collection of links to rsvps, what is the HAL representation?

Create a bullet list explaining the purpose and benefits of HAL

Originally raised on 10/24 dev call, but we could not come to consensus
deferred post beta

We agreed to split accuracy into it's own issue from #46.

The API documentation I mistakenly sent us to during the call today was Google's Geocoding API for Business that's intended to be used in mobile clients. That's why the accuracy was in meters and not in these general categories.

Here's a sampling of what some of the major geocoding APIs provide:

Google's Geocoding API v3:

• ROOFTOP indicates that the returned result is a precise geocode for which we have location information accurate down to street address precision.
• RANGE_INTERPOLATED indicates that the returned result reflects an approximation (usually on a road) interpolated between two precise points (such as intersections). Interpolated results are generally returned when rooftop geocodes are unavailable for a street address.
• GEOMETRIC_CENTER indicates that the returned result is the geometric center of a result such as a polyline (for example, a street) or polygon (region).
• APPROXIMATE indicates that the returned result is approximate.

Bing has a similar, but not identical field in their responses:

• Rooftop — The geocode point was matched to the rooftop of a building.
• Interpolation — The geocode point was matched to a point on a road using interpolation.
• InterpolationOffset — The geocode point was matched to a point on a road using interpolation with an additional offset to shift the point to the side of the street.
• Parcel — The geocode point was matched to the center of a parcel.

Yahoo tackles it in a completely different manner. They have an integer scale of 1-100, where 99 equals an exact coordinate, 50 equals a neighborhood, 0 is no match, and there are a ton of point values in between.

Proposal

Adopt Google's accuracy levels as an enum with some slightly modified descriptions.

In descending order of accuracy:

• rooftop — an exact match
• interpolated — a street match with an interpolated point for the building number
• geometric_center — the center of a polyline (street) or polygon (city, postal code, etc.)
• approximate — an accuracy of less than geometic_center

NOTE: Providers who use geocoding services which provide different accuracy categorization should provide the best match from the above enumerated options.

We should capture future scenarios even if they are out of scope

Remove Addresses from the AEP as a top level collection.

Add common attributes for created, modified to main part of spec

The spec currently defines the payment processor mechanisms as out of scope.
However, some vendors need the ability to track the status of a donation, pending, declined, etc.
This could be accommodated by status fields on donations

Is a HAL client supposed to transparently handle when a resource is linked vs embedded? If the caller of a client library API specifies a subordinate resource, is the client library supposed to go fetch the embedded resource if only a link is present?

pagination is set up for primary collections, eg when a client is specifically getting a collection
for subcollections, eg the addresses collection that is embedded in a person, pagination properties do not show up.

Is this right?
I think yes

Looking at the definition of Person, I see primary_phone with a type of string. Later though, I see a collection of a Phone type. Are we planning on having a Phone type? I think it might be useful at least so we can capture the type of phone (mobile, work, home) in addition to the number.

Union clients store information about an employer for every person. We should figure out where this belongs.

One of my main objections to having query string filters is that a client then needs to know how to look at a url and append information... so it breaks the JSON+HAL convention of "I don't really need to know about the API" to use it. At Amicus we've been able to get away from using any "filters" - until now. So we started thinking about the problem again...

http://tools.ietf.org/html/draft-kelly-json-hal-05
The JSON+HAL spec has a concept that could fix this though:

_links: {
  osdi:filter: [
    {  href: "/blah?tags={tagList}",
       name: 'tag',
       templated: true
    } ,
    {
      href: "/blah?state={state}",
      name: 'state',
      templated: true
    }
  ]
}

Now you can list out available filters and have a good way for clients to use them. One issue we haven't figured out is combining them (multiple filters)

From Ray
Geocodes. It seems that "lat" and "lng" properties should optional attributes? Perhaps with some accuracy attribute such as "rooftop" or "approximate".

From Mark
Hello and thank you for having me in the group. Regarding geocodes:

1. They are so ubiquitous and so useful for engaging supporters (find & map all supporters in an area, assign & query on legislative districts, etc). My preference would be that they be kept among the core fields and not be made optional attributes.

They should be named "latitude" and "longitude", as none of the other address fields (and no person fields) are abbreviated.

For accuracy, latitude and longitude should be served as text fields, not numeric. Numeric latitude and longitude fields are subject to corruption when moved from system to system, as rounding errors and differences in precision between systems cause the 4th, 5th and 6th decimal places to mutate. In most real-life applications, this kind of precision is not necessary, but in latitude and longitude it can mean the difference between assigning the right or wrong congressional district. Also, when viewed as text, it is clear how much precision* there is in the coordinates, as no digits have been artificially added or lopped off by numeric conversions. *precision != accuracy