conwetlab / ckanext-datarequests Goto Github PK

A plugin that allows users to request data that is not published yet

License: GNU Affero General Public License v3.0

Shell 0.51% Python 88.50% JavaScript 0.65% CSS 0.75% HTML 9.59%

ckan ckan-instance ckan-extension datarequest dataset ckanext

ckanext-datarequests's Introduction

CKAN Data Requests

CKAN extension that allows users to ask for datasets that are not already published in the CKAN instance. In this way we can set up a Data Market, not only with data supplies but also with data demands.

How it works

You have two ways for creating, updating, deleting, viewing and closing a datarequest: you can use the graphical interface or the programatic API.

User Interface

If you prefer to use the graphical interface, you should click on the "Data Requests" section that will appear in the header of your CKAN instance. In this section you'll be able to view the current data requests. In addition, there will be a button that will allow you to create a new data request. In the form that will appear, you will have to introduce the following information:

Title: a title for your data request
Description: a long description for your data request. You should include as much details as you can in order to allow others to understand you needs and upload a dataset that fulfil your requeriments.
Organization: in some cases, you want to ask specific data to an specific organization. If you are in such situation, you should complete this field.

Once that you have created your data request, you can view it by clicking on the link provided when you created it. When you are the owner of a data request, you will also be able to:

Close the data request if you consider that there is a new dataset that fulfil your needs
Update the data request if you can to add/remove some information
Delete the data request if you do not want it to be available any more

API

On the other hand, you can also use the API. To access this API, you should POST the following URL (as you do for other actions):

http[s]://[CKAN_HOST]:[CKAN_PORT]/api/action/[ACTION_NAME]

Here you have a brief description of all the implemented actions:

`create_datarequest(context, data_dict)`

Action to create a new data request. This function checks the access rights of the user before creating the data request. If the user is not allowed, a NotAuthorized exception will be risen.

In addition, you should note that the parameters will be checked and an exception (ValidationError) will be risen if some of these parameters are not valid.

Parameters (included in `data_dict`):

title (string): the title of the data request
description (string): a brief description for your data request
organization_id (string): The ID of the organization you want to asign the data request (optional).

Returns:

A dict with the data request (id, user_id, title, description,organization_id, open_time, accepted_dataset, close_time, closed, followers).

`show_datarequest(context, data_dict)`

Action to retrieve the information of a data request. The only required parameter is the id of the data request. A NotFound exception will be risen if the id is not found.

Access rights will be checked before returning the information and an exception will be risen (NotAuthorized) if the user is not authorized.

Parameters (included in `data_dict`):

id (string): the ID of the datarequest to be returned.

Returns:

A dict with the data request (id, user_id, title, description,organization_id, open_time, accepted_dataset, close_time, closed, followers).

`update_datarequest(context, data_dict)`

Action to update a data request. The function checks the access rights of the user before updating the data request. If the user is not allowed, a NotAuthorized exception will be risen

In addition, you should note that the parameters will be checked and an exception (ValidationError) will be risen if some of these parameters are not valid.

Parameters (included in `data_dict`):

id (string): the ID of the datarequest to be updated
title (string): the updated title of the data request
description (string): a updated brief description for your data request
organization_id (string): The ID of the organization you want to asign the data request (optional).

Returns:

A dict with the data request (id, user_id, title, description,organization_id, open_time, accepted_dataset, close_time, closed, followers).

`list_datarequests(context, data_dict)`

Returns a list with the existing data requests. Rights access will be checked before returning the results. If the user is not allowed, a NotAuthorized exception will be risen

Parameters (included in `data_dict`):

organization_id (string) (optional): to filter the result by organization
user_id (string) (optional): to filter the result by user
closed (string) (optional): to filter the result by state (True: Closed, False: Open)
offset (int) (optional) (default 0): the first element to be returned
limit (int) (optional) (default 10): The max number of data requests to be returned
q (string) (optional): to filter the result using a free-text.
sort (string) (optional) (default asc): desc to order data requests in a descending way. asc to order data requests in an ascending way.

Returns:

A dict with three fields: result (a list of data requests), facets (a list of the facets that can be used) and count (the total number of existing data requests)

`delete_datarequest(context, data_dict)`

Action to delete a new data request. The function checks the access rights of the user before deleting the data request. If the user is not allowed, a NotAuthorized exception will be risen.

Parameters (included in `data_dict`):

id (string): the ID of the datarequest to be deleted

Returns:

A dict with the data request (id, user_id, title, description,organization_id, open_time, accepted_dataset, close_time, closed, followers).

`close_datarequest(context, data_dict)`

Action to close a data request. Access rights will be checked before closing the data request. If the user is not allowed, a NotAuthorized exception will be risen

Parameters (included in `data_dict`):

id (string): the ID of the datarequest to be closed
accepted_dataset (string): The ID of the dataset accepted as solution for the data request

Returns:

A dict with the data request (id, user_id, title, description,organization_id, open_time, accepted_dataset, close_time, closed, followers).

`comment_datarequest(context, data_dict)`

Action to create a comment in a data request. Access rights will be checked before creating the comment and a NotAuthorized exception will be risen if the user is not allowed to create the comment

Parameters (included in `data_dict`):

datarequest_id (string): the ID of the datarequest to be commented
comment (string): The comment to be added to the data request

Returns:

A dict with the data request comment (id, user_id, datarequest_id, time and comment)

`show_datarequest_comment(context, data_dict)`

Action to retrieve a comment. Access rights will be checked before getting the comment and a NotAuthorized exception will be risen if the user is not allowed to get the comment

Parameters (included in `data_dict`):

id (string): The ID of the comment to be retrieved

Returns:

A dict with the following fields: id, user_id, datarequest_id, time and comment

`list_datarequest_comments(context, data_dict)`

Action to retrieve all the comments of a data request. Access rights will be checked before getting the comments and a NotAuthorized exception will be risen if the user is not allowed to read the comments

Parameters (included in `data_dict`):

datarequest_id (string): The ID of the datarequest whose comments want to be retrieved
sort (string) (optional) (default asc): desc to order comments in a descending way. asc to order comments in an ascending way.

Returns:

A list with all the comments of a data request. Every comment is a dict with the following fields: id, user_id, datarequest_id, time and comment

`update_datarequest_comment(context, data_dict)`

Action to update a comment of a data request. Access rights will be checked before updating the comment and a NotAuthorized exception will be risen if the user is not allowed to update the comment

Parameters (included in `data_dict`):

id (string): The ID of the comment to be updated
comment (string): The new comment

Returns:

A dict with the data request comment (id, user_id, datarequest_id, time and comment)

`delete_datarequest_comment(context, data_dict)`

Action to delete a comment of a data request. Access rights will be checked before deleting the comment and a NotAuthorized exception will be risen if the user is not allowed to delete the comment

Parameters (included in `data_dict`):

id (string): The ID of the comment to be deleted

Returns:

A dict with the data request comment (id, user_id, datarequest_id, time and comment)

`follow_datarequest(context, data_dict)`

Action to follow a data request. Access rights will be cheked before following a datarequest and a NotAuthorized exception will be risen if the user is not allowed to follow the given datarequest. ValidationError will be risen if the datarequest ID is not included or if the user is already following the datarequest. ObjectNotFound will be risen if the given datarequest does not exist.

Parameters (included in `data_dict`):

id (string): The ID of the datarequest to be followed

Returns:

True

`unfollow_datarequest(context, data_dict)`

Action to unfollow a data request. Access rights will be cheked before unfollowing a datarequest and a NotAuthorized exception will be risen if the user is not allowed to unfollow the given datarequest. ValidationError will be risen if the datarequest ID is not included in the request. ObjectNotFound will be risen if the user is not following the given datarequest.

Parameters (included in `data_dict`):

id (string): The ID of the datarequest to be unfollowed

Returns:

True

Installation

Install this extension in your CKAN instance is as easy as install any other CKAN extension.

Activate your virtual environment

. /usr/lib/ckan/default/bin/activate

Install the extension

pip install ckanext-datarequests

Note: If you prefer, you can also download the source code and install the extension manually. To do so, execute the following commands:
$ git clone https://github.com/conwetlab/ckanext-datarequests.git
$ cd ckanext-datarequests
$ python setup.py install

Modify your configuration file (generally in /etc/ckan/default/production.ini) and add datarequests in the ckan.plugins property.

ckan.plugins = datarequests <OTHER_PLUGINS>

Enable or disable the comments system by setting up the ckan.datarequests.comments property in the configuration file (by default, the comments system is enabled).

ckan.datarequests.comments = [true|false]

Enable or disable a badge to show the number of data requests in the menu by setting up the ckan.datarequests.show_datarequests_badge property in the configuration file (by default, the badge is not shown).

ckan.datarequests.show_datarequests_badge = [true|false]

Restart your apache2 reserver

sudo service apache2 restart

That's All!

Translations

Help us to translate this extension so everyone can create data requests. Currently, the extension is translated to English, Spanish, German and Brazilian Portuguese. If you want to contribute with your translation, the first step is to clone this repo and move to the develop branch. Then, create the locale for your translation by executing:

python setup.py init_catalog -l <YOUR_LOCALE>

This will generate a file called i18n/YOUR_LOCALE/LC_MESSAGES/ckanext-datarequests.po. This file contains all the untranslated strings. You can manually add a translation for it by editing the msgstr section:

msgid "This is an untranslated string"
msgstr "This is a itranslated string"

Once the translation files (po) have been updated, compile them by running:

python setup.py compile_catalog

This will generate the required mo file. Once this file has been generated, commit your changes and create a Pull Request (to the develop branch).

Tests

This sofware contains a set of test to detect errors and failures. You can run this tests by running the following command (this command will generate coverage reports):

python setup.py nosetests

Note: The test.ini file contains a link to the CKAN test-core.ini file. You will need to change that link to the real path of the file in your system (generally /usr/lib/ckan/default/src/ckan/test-core.ini).

Note 2: When creating a PR that includes code changes, please, ensure your new code is tested. No PR will be merged until the Travis CI system marks it as valid.

Changelog

v1.1.0

New: Compatibility with CKAN 2.8.0
New: Somali translation (thanks to @SimuliChina)

v1.0.0

New: Option to follow data requests.
New: Email notifications:
- An email will be sent to organization staff when a data request is created in a organization.
- An email will be sent to followers, people that commented, datarequest creator and organization staff when a comment in a datarequest is created.
- An email will be sent to followers, people that commented, datarequest creator and organization staff when a data request is closed.
New: Major API changes:
- datarequest_create ➡️ create_datarequest
- datarequest_show ➡️ show_datarequest
- datarequest_update ➡️ update_datarequest
- datarequest_index ➡️ list_datarequests
- datarequest_delete ➡️ delete_datarequest
- datarequest_close ➡️ close_datarequest
- datarequest_comment ➡️ comment_datarequest
- datarequest_comment_show ➡️ show_datarequest_comment
- datarequest_comment_list ➡️ list_datarequest_comments
- datarequest_comment_update ➡️ update_datarequest_comment
- datarequest_comment_delete ➡️ delete_datarequest_comment

v0.4.1

New: Brazilian Portuguese translation (thanks to @allysonbarros)

v0.4.0

New: Move CI to Travis
New: Compatibility with CKAN 2.7 (controller adapted by @owl17)

v0.3.3

New: German Translation (thanks to @kvlahrosch)

ckanext-datarequests's People

Contributors

Stargazers

Watchers

ckanext-datarequests's Issues

database initialization not working with CKAN 2.3 and postgresql 9.1.x

HI, We' ve tried to use this module but database generation failed since postgresql apparently does not support max length for field definition. I get the following exception:

Error - <class 'sqlalchemy.exc.ProgrammingError'>: (ProgrammingError) type modifier is not allowed for type "text"

Removing max length constraint from db.py and plugin.py did the job for me:

https://github.com/conwetlab/ckanext-datarequests/blob/master/ckanext/datarequests/db.py#L68

Spam

Hey. Thanks for this plugin. We're having a small issue of spam. Any suggestions on how to handle it?

Internal server error on redirect after create and close

Hi,
I am getting an internal server error on redirect after closing a request. The closing works.
The same thing happens on redirect after creating a request. The request is created, I get the error on the redirect that happens.

Anyone has ever had this problem?

typo in the new_comment templates

Email template reads:

An user has commented in the "Need dataset" data request. Check this new comment just in case you want to give it a reply.

You are receiving this notifications because:

You are the owner of the data request
You are a member of the organization where the data request has been created
You are following the data request
You have posted a comment in the data request

Should read:

"A" user

New translations not working

Hi, I wanted to add a translation for Macedonian directly on the instance of CKAN that we have.

I navigated to the ckanext-datarequests folder and I ran python setup.py init_catalog -l <YOUR_LOCALE> there.
Then I edited the mk/LC_MESSAGES/ckanext-datarequests.po file, and then I ran python setup.py compile_catalog, after which I got the following message:

97 of 97 messages (100%) translated in ckanext/datarequests/i18n/mk/LC_MESSAGES/ckanext-datarequests.po
compiling catalog ckanext/datarequests/i18n/mk/LC_MESSAGES/ckanext-datarequests.po to ckanext/datarequests/i18n/mk/LC_MESSAGES/ckanext-datarequests.mo

I then ran sudo supervisorctl reload. When I navigate to anything related to the datarequests while having chosen Macedonian as a language, I don't get the translations, and it's defaulted back to English.

Any idea on how to proceed?

Cannot choose Organization

I installed the extension with CKAN 2.6.6 and when creating a new datarequest I cannot add a specific organization. I start typing the org name but nothing happens, and even if I write the correct name I cannot choose it.
Thanks.

Also the creation date a new data request is -1 days ago

AttributeError: 'module' object has no attribute 'redirect'

Hi,
After I submit data request application, my webpage show server error.
Here is my apache2 error log.

I also look into base.py and I didn't found any redirect function also.
Please help, thank you

Data request display does not handle time zones properly

Data requests are displayed with inaccurate "Created" strings, eg "-1 days ago".

This appears to result from a lack of time zone handling. actions.py populates the creation timestamp without specifying a time zone, but when it is rendered, it uses a localisation method (formatters.localised_nice_date) that assumes a lack of time zone data means UTC.

Either the time zone should be specified when first creating the data request, or else the current server time zone (eg from helpers.get_display_timezone()) should be assumed when displaying it. The latter approach would also resolve existing data requests.

This is similar to ckan/ckanext-xloader#102

Prepare for Python 3

This codebase is currently not compatible with Python 3.

We've recently been updating the qld-gov-au fork to handle Python 3, but it has other changes mixed in (adding CircleCI testing, adding a "Closing Circumstances" field, etc), so it wouldn't be as simple as just merging it. It could serve as a reference, though.

Add sort

Would be nice to sort by date, age, requester and organization

Prepare for CKAN 2.9

Pylons routing is going away in CKAN 2.9, and should be replaced with Flask blueprints.

We're likely to build this into the qld-gov-au fork soon, but that has other changes mixed in. Could still serve as a reference.

SQalchemy error during installation

<class 'sqlalchemy.exc.InvalidRequestError'>: Table 'datarequests' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.

KeyError('ckanext') in site.py while processing ckanext_datarequests-0.2.13-py2.7-nspkg.pth

Out of the blue, I started getting this exception:

Traceback (most recent call last):
  File "/usr/lib/ckan/default/lib/python2.7/site.py", line 702, in <module>
    main()
  File "/usr/lib/ckan/default/lib/python2.7/site.py", line 682, in main
    paths_in_sys = addsitepackages(paths_in_sys)
  File "/usr/lib/ckan/default/lib/python2.7/site.py", line 281, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/usr/lib/ckan/default/lib/python2.7/site.py", line 203, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/usr/lib/ckan/default/lib/python2.7/site.py", line 172, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
KeyError: 'ckanext'

Tried to debug, the KeyError is from exec(line) where line is any line from '.../site-packages/ckanext_datarequests-0.2.13-py2.7-nspkg.pth'.

I don't know how to further investigate this, if I place a set_trace() in a try block around the problematic statement and exec it manually in the debugger, everything works.

interaction with ckanext-disqus

Hi,

Can you advise if comments left on data requests will be handled by ckanext-disqus if installed (https://github.com/ckan/ckanext-disqus), or is it via a different mechanism?

Thanks!

Make comment subsystem optional

For folks who have integrated a different commenting system (e.g. discourse, disqus).

Perhaps, make it a setting in the ckan INI file?

Add datarequests count

So it can be shown in box_stats? See https://github.com/ckan/ckanext-showcase/blob/master/ckanext/showcase/templates/home/snippets/stats.html

CKAN 2.8

Any plans to update this extension for CKAN 2.8?

datarequests plugin throws `UndefinedError: 'group_type' is undefined` in `ckan-2.8.0`

In ckan-2.8.0 the template organization/read_base.html has an extra context variable named group_type. This template is extended by organization/datarequests.html without providing the extra context variable which causes the exception UndefinedError: 'group_type' is undefined to be raised when this extension is used with ckan-2.8.0.

Delete comments and follows when Data Request deleted

When a request is deleted all the associated follows and comments must be deleted from the database.

Show age of requests on main datarequest page

And have the ability to sort it (#4)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

conwetlab / ckanext-datarequests Goto Github PK

ckanext-datarequests's Introduction

CKAN Data Requests

How it works

User Interface

API

create_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

show_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

update_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

list_datarequests(context, data_dict)

Parameters (included in data_dict):

Returns:

delete_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

close_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

comment_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

show_datarequest_comment(context, data_dict)

Parameters (included in data_dict):

Returns:

list_datarequest_comments(context, data_dict)

Parameters (included in data_dict):

Returns:

update_datarequest_comment(context, data_dict)

Parameters (included in data_dict):

Returns:

delete_datarequest_comment(context, data_dict)

Parameters (included in data_dict):

Returns:

follow_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

unfollow_datarequest(context, data_dict)

Parameters (included in data_dict):

Returns:

Installation

Translations

Tests

Changelog

v1.1.0

v1.0.0

v0.4.1

v0.4.0

v0.3.3

ckanext-datarequests's People

Contributors

Stargazers

Watchers

Forkers

ckanext-datarequests's Issues

Recommend Projects

Recommend Topics

Recommend Org

`create_datarequest(context, data_dict)`

Parameters (included in `data_dict`):

`show_datarequest(context, data_dict)`

Parameters (included in `data_dict`):

`update_datarequest(context, data_dict)`

Parameters (included in `data_dict`):

`list_datarequests(context, data_dict)`

Parameters (included in `data_dict`):

`delete_datarequest(context, data_dict)`

Parameters (included in `data_dict`):

`close_datarequest(context, data_dict)`

Parameters (included in `data_dict`):

`comment_datarequest(context, data_dict)`

Parameters (included in `data_dict`):

`show_datarequest_comment(context, data_dict)`

Parameters (included in `data_dict`):

`list_datarequest_comments(context, data_dict)`

Parameters (included in `data_dict`):

`update_datarequest_comment(context, data_dict)`

Parameters (included in `data_dict`):

`delete_datarequest_comment(context, data_dict)`

Parameters (included in `data_dict`):

`follow_datarequest(context, data_dict)`

Parameters (included in `data_dict`):

`unfollow_datarequest(context, data_dict)`

Parameters (included in `data_dict`):