gost / server Goto Github PK

CodeValue's are now hardcoded, load into memory from database on start

Implement the mandatory name attribute for the following entities:

thing.name
location.name
datastream.name
sensor.name
observedproperty.name
observation.name
featureofinterest.name

Function post observation without feature of interest should be improved by combining the queries to the database

Parse Observation Result to correct ObservationType.

ObservationType: "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement" Result "18" should return 18 so nor parsing is needed in clients.

In Crud test CreateEntities:
http post localhost:8080/v1.0/Observations
{
"phenomenonTime": "2015-03-01T00:00:00.000Z",
"resultTime": "2015-03-01T01:00:00.000Z",
"result": 100,
"Datastream":{"@iot.id": 1}
}

actual error: No FeatureOfInterest supplied or Location found on linked thing
expected: no error because Thing(1) has a Location

Implement Delete for Observed Property

Move create tables code from pgqueries.go to database script

Sample:
curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d '{
"description": "my backyard",
"encodingType": "application/vnd.geo+json",
"location": {
"type": "Point",
"coordinates": [-117.123,
54.123]
}
}' "http://localhost:8080/v1.0/Things(5)/Locations"

Actual: return HTTP 500 Thing(5) does not exist
Should: return HTTP 400

sample http://gost.geodan.nl/v1.0/Things(2)/description
requesting properties should be implemented

Combine delete methods into 1 function with parameter name of entity.

database.postgis.datastream.DeleteThing
database.postgis.featureofinterest.DeleteFeatureOfInterest
database.postgis.historicallocation.DeleteHistoricalLocation
database.postgis.location.DeleteLocation
database.postgis.observation.DeleteObservation
database.postgis.observedproperty.DeleteObservedProperty
database.postgis.sensor.DeleteSensor
database.postgis.thing.DeleteThing

same for the *Exist methods

from the latest spec: "The HistoricalLocation can also be created, updated and deleted. One use case is to migrate historical observation data from an existing observation data management system to a SensorThings API system"

Should have support for multiple locations in post/put/patch request to HistoricalLocation?

sample:
HTTP POST localhost:8080/v1.0/HistoricalLocations

{
"time": "2015-03-01T00:40:00.000Z",
"Thing":{"@iot.id": 1 },
"Locations": [{"@iot.id": 2},{"@iot.id": 3}]
}

Result: 2 records in HistoricalLocations
Als GET to HistoricalLocation needs to fixed.

Change the id from type string to interface{} in all entity structs

$ curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d '{

"phenomenonTime": "2016-05-09T11:04:15.790Z",
"resultTime" : "2016-05-09T11:04:15.790Z",
"result" : 38,
"Datastream":{"@iot.id":1}
}' "http://localhost:8080/v1.0/Observations"
{
"error": {
"status": "Internal Server Error",
"code": 500,
"message": [
"pq: insert or update on table "observation" violates foreign key constraint "fk_featureofinterest""
]
}
}

Remove "Exist" checks on database and just insert -> handle fk errors to display appropriate error to user

from the spec:

resultQuality: "Describes the quality of the result."
DQ_Element -Zero-to-many

validTime: "The time period during which the result may be used."
TM_Period (ISO 8601 Time Interval string)
Zero-to-one

parameters "Key-value pairs showing the environmental conditions during measurement. NamedValues in a JSON Array"
Zero-to-one

sample: http://gost.geodan.nl/v1.0/Things(9223372036854775807)
probably type of id field

observedproperty.description, featureofinterest.description, datastream.description, sensor.description, thing.description,

Try batch insert observations to increase speed

Columns:
. Location.Location
. FeatureOfInterest.Feature

add where statement on counters, for example:

/datastream(1)/observations
actual: counts all observations
expected: counts observations where datastream=1

Create an entity: "Upon successful completion, the response SHALL contain a HTTP location header that contains the selfLink of the created entity."

Sample header:
Location → http://gost.geodan.nl/v1.0/Things(1)

fix: "encodingtype" should be "encodingType"

sample:

curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d '{
"phenomenonTime": "2016-05-09T11:04:15.790Z",
"resultTime" : "2016-05-09T11:04:15.790Z",
"result" : 38,
"Datastream":{"@iot.id":1}
}' "http://gost.geodan.nl/v1.0/Observations"

Actual response: 500 internal server error
Expected response: 400 client error

Make HistoricalLocation.Time mandatory in post/put request and the value comes from the client
(so no time.Now() from the server)

Implement HTTP Put and Patch on entities

At the moment a secure service is missing. Most browsers don't accept data retrieval from non-secure servers when the original domain is secure (from https to http).

Add the supported encodingTypes for an entity to the returned error message when posting an entity (sensor, location or featuresofinterest) fails because of a wrong supplied encodingType.

While running 4 clients at the same time publishing a message every 16ms and requesting a GET, GOST crashes.

Request:
localhost:8080/v1.0/Datastreams
{
"unitOfMeasurement": {
"name": "Celsius",
"symbol": "degC",
"definition": "http://qudt.org/vocab/unit#DegreeCelsius"
},
"description": "test datastream.",
"observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
"Thing": { "@iot.id": 1 },
"ObservedProperty":{ "@iot.id":1},
"Sensor": { "@iot.id": 1 }
}

Response:
{
"error": {
"status": "Bad Request",
"code": 400,
"message": [
"Thing does not exist"
]
}
}

But http://localhost:8080/v1.0/Things(1) does exist

Make server stateless again (server should be in same state before/after request):
. fix counters
. fix FeatureOfInterest request

sample: http://localhost:8080/v1.0/Observations
{
"phenomenonTime": "2015-03-01T00:00:00.000Z",
"resultTime": "2015-03-01T01:00:00.000Z",
"result": 100,
"Datastream":{"@iot.id": 1}
}

result: panic

but works with:
{
"phenomenonTime": "2015-03-01T00:00:00.000Z",
"resultTime": "2015-03-01T01:00:00.000Z",
"result": 100,
"Datastream":{"@iot.id": "1"}
}

remove Version?

add Datastream - PhenomenonTime and ResultTime support

Change field metadata in entity sensor to interface and change in database from character varying(255) to jsonb

sample: http://gost.geodan.nl/v1.0/DataStreams(17)/Observations
actual response: All observations
expected response: 400 error

request: http://localhost:8080/v1.0/HistoricalLocations(1)
error: The entity does not have mandatory control information : @iot.id

at line long automatedFOIId = checkAutomaticInsertionOfFOI(obsId2, locationEntity, -1);

java.lang.AssertionError: ERROR: Automatic created FeatureOfInterest does not match last Location of that Thing.

expected [{"coordinates":[-114.05,51.05],"type":"Point"}]
actual [{"coordinates":[10,10],"type":"Point"}]

sample url: http://gost.geodan.nl/v1.0/Things/$ref
$ref should be implemented

For very large datasets it would be nice to have a smaller overhead in the key-value pairs.
Therefore I propose to add a format that sents the data as 1D arrays. I have added an example to give the idea but any other solution that gets rid of redundant 'keys' is also fine.

Current situation:

{
   "count": 200,
   "@iot.nextLink": "gost.geodan.nl/v1.0/Datastreams(8)/Observations/?$top=200&$skip=200",
   "value": [
      {
         "phenomenonTime": "2016-07-06T18:21:39.000Z",
         "result": 25.6
      },
      {
         "phenomenonTime": "2016-07-06T18:12:14.000Z",
         "result": 25.6
      },...

example desired situation:

{
   "count": 200,
   "@iot.nextLink": "gost.geodan.nl/v1.0/Datastreams(8)/Observations/?$top=200&$skip=200$format=condense",
  "values":{
      "phenomenonTime": ["2016-07-06T18:21:39.000Z","2016-07-06T18:12:14.000Z",...],
      "result":[25.6,25.6,...]
  }

Implement Datastream.ObservedArea, from the spec: "The spatial bounding box of the spatial extent of all FeaturesOfInterest that belong to the Observations ", return GeoJSON Polygon

for example: http://localhost:8080/v1.0/Things(1)/Datastreams(5)/Thing should return Things(1)

Occasionally we see Mosquitto broker crashes on the server:

kern.log:May 26 09:36:25 gost kernel: [313113.955286] mosquitto[708]: segfault at e3a1 ip 00007fdafab66a0c sp 00007fff16bd2b50 error 4 in libwebsockets.so.7[7fdafab5e000+1f000]

sample url: http://gost.geodan.nl/v1.0/Datastreams(2)
observationType is missing?

http post http://localhost:8080/v1.0/Things({[email protected]})/Locations
Content-Type: application/json
{
"description": "my backyard",
"name": "backyard",
"encodingType": "application/vnd.geo+json",
"location": {
"type": "Point",
"coordinates": [-117.123,
54.123]
}
}

url: http://gost.geodan.nl/v1.0/Things
body:
{
"description": "Office Building",
"properties": {
"reference": "Third Floor"
},
"Locations": [
{
"description": "West Roof",
"location": { "type": "Point", "coordinates": [-117.05, 51.05] },
"encodingType": "http://example.org/location_types#GeoJSON"
}
],
"Datastreams": [
{
"unitOfMeasurement": {
"name": "Lumen",
"symbol": "lm",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/unit/Instances.html#Lumen"
},
"description": "Light exposure.",
"observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
"ObservedProperty": {
"name": "Luminous Flux",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/quantity/Instances.html#LuminousFlux",
"description": "Luminous Flux or Luminous Power is the measure of the perceived power of light."
},
"Sensor": {
"description": "Acme Fluxomatic 1000",
"encodingType": "http://schema.org/description",
"metadata": "Light flux sensor"
}
}
]
}

actual: after post thing twice error 500:
pq: duplicate key value violates unique constraint

expected: 201 created

sample:
http://localhost:8080/v1.0/Observations
content-type: application/pdf
{
"phenomenonTime": "2015-03-01T00:00:00.000Z",
"resultTime": "2015-03-01T01:00:00.000Z",
"result": 102,
"Datastream":{"@iot.id": 1}
}

should not be accepted, 400 error

for example: http://localhost:8080/v1.0/Things(1)/description/$value

When no FOI posted or linked when posting an observation take the location of the linked thing, keep lookups in the database to a minimum by loading already added fois from thing in memory on startup so observation inserts are not slowed down . When a things location is updated make sure the next time the foi from thing is requested a new foi will be inserted in the database for the observation to use.