vexelon-dot-net / e-additives.server Goto Github PK

Hello @petarov.

Start by saying a big thank you for this API. It seems to be very promising and useful tool.

Know my question.

I just read the documentation, and I found this response:

{
    "id": "1",
    "code": "100",
    "last_update": "2013-09-08T20:16:36+0300",
    "name": "Curcuma (turmeric)",
    "status": "Safe",
    "veg": null,
    "function": "food coloring (yellow-orange)",
    "foods": "Fats and oils, curry, processed cheese and fish fingers.",
    "notice": "Safe in recommended doses. High doses should not be taken by people with gallstones, obstructive jaundice, acute bilious colic or toxic liver disorders.",
    "info": "Orange-yellow colouring extracted from the rhizomes of the turmeric plant.",
    "url": "http://hostname/additives/100"
}

Of course, most of the fields are self-explained based on the name you have assigned to them, but there are two fields with no meaningful names, and those two fields are the status and the veg.

I guess the veg field` it is used as a flag for vegetarians safe foods, but I am not really sure if that's the purpose of that field.

In addition, I am not sure if the status is always right, or, if it has a different meaning than what I understand. The reason I ask that is that when I access your website and see the additive E 120, you have set the status to empty string, while the warning says that is better to now consumed. In other cases, such as the E 120 you make it clear that this additive is not safe at all.

So what is the purpose of the attribute veg and status?

Thank you in advance! 😃

• Identify milestones until BETA version
• Create project architecture
• Set main tools and libraries to be used
• Create core database schema
• Create script to transform existing XML addtivies data to SQL import script
• Create script to create database schema and import additives data into SQL database
• Add server configuration file (config.php)
• Add REST API - C(reate)/R(ead)/U(pdate)/D(elete) of additives stored in database
• Add REST API documentation (markdown file)
• Add Redis data caching for READ requests

Add travis continuous integration script and setup build.

SQL queries must be optimized. There are too many sub-selects in MS01. These may possibly be replaced by JOIN-s.

• Remove sub-selects (Subqueries)
• Additive count must be cached in the AdditiveCategory table
• Refactor: SQL table names must be set from constants

For all data that has been cached on server side the server must return cache-control and/or expire HTTP headers.

User agents (browsers) would use such headers to understand when to cache responses from the the server side.

• Spec
• More Info
• Even more info

Current JSON results returns date-time without any timezones info, e.g., 2013-08-28 00:07:42

All [API calls] that return last_update date-time info must return that result as ISO 8601 formatted string.

Javascript client side can handle this, i.e.,

var date = new Date("2009-05-21T16:06:05.000Z");

Server must expect a proper Accept header from clients, e.g.,

GET /additives HTTP/1.1 
Accept: application/vnd.vexelon.eadditives-v1+json

HTTP API requests without Accept must be rejected with proper HTTP response code. Closest thing I found was:

406 Not Acceptable

There currently can be set only one API key on the server side. This functionality must be extended so that different authorization keys can be used.

Extend Developer API with support for modifying existing additives data: CUD (create,
update, delete) functionalities.

Database tables prefixes must be added. Changes would be required both in the database model and scripts and in the server backend code.

• Unit tests written with PHPUnit.
• Automated tests using PHPUnit and/or Qunit and javascript ajax calls.

Hi, this seems like a pretty good idea to me and I thought I might build a little app around it to try out some ideas. I managed to get a key from mashape and it works, but the mashape test buttons failed and a couple of the links in the docs (like to this GitHub page for example) 404ed. So given the age of the repo, I just wanted to check the general status of the API and check that it won't disappear any time soon. If I build something worth developing further maybe you would be open to PRs...?

When importing data from db_app_data.sql using MySQL Workbench the whole system crashes and the kernel spits out a message:

system crash:kernel: ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen

It must be noted that the db_app_data.sql file contains both EN and BG additive properties data. This would mean ~3200 SQL INSERT statements.

System spces: Arch Linux/3.9.9-1-ck, mysql 5.5.30-7 16gb ram, SSD hdd.

• Add REST API authorization / API keys.
• Deployment strategy based on Travis-CI
• Add Automated Tests - external JS or PHPUnit?
• Review/Optimize SQL queries

All API requests MUST include a valid User-Agent header. Requests without User-Agent header will be rejected with HTTP error 403 Forbidden.

User-Agent: <any string>

All API requests MUST include a valid X-Authorization header. The form of the header is:

X-Authorization: EAD-TOKENS apiKey="quoted-string"

Requests without this header will be rejected with 401 Unauthorized.

The config.php must contain a placeholder where API key must be specified. All clients must use a valid API key when sending requests to the server.

When the script is run under Windows the following errors occur:

> python xml2sql.py --bg ..\data\bg.xml --en ..\data\en.xml
Parsing elements ...
Found 366 XML items.
Writing 366 items to SQL file ..\data\bg.xml.sql

Traceback (most recent call last):
  File "xml2sql.py", line 208, in <module>
    toSQL(itemsList, fileName)
  File "xml2sql.py", line 132, in toSQL
    f.write(sql)
  File "E:\Python33\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 134-142: character maps to <undefined>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "xml2sql.py", line 210, in <module>
    print (inst.args)
  File "E:\Python33\lib\encodings\cp850.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 147-155: character maps to <undefined>

Authorization is required when attempting to access the main route resource, e.g., http://hostname/api/. This normally gives back a list of accessible resources:

{
    "additives_url": "http://hostname/api/additives",
    "additive_url": "http://hostnamet/api/additives/{code}",
    "additive_search_url": "http://hostname/api/additives/search",
    "categories_url": "http:/hostname/api/categories",
    "category_url": "http://hostname/api/categories/{id}"
}

There should not be authorization requirements for GET requests to the route.

This is not really an issue (yet) but an evaluation of possible cache slams must be done at some point if a cache server configuration is used.

The EAD server use of cache is currently quite primitive due to the low amount of data. However, when complexity of Data I/O increases so would the chance to get cache stampedes.

Some tests would be good to have as well.

More info on the subject:

• Slam defense
• Minimizing cache stampedes
• How to Dismantle an APC Bomb