opendatatrentino / jackan Goto Github PK
View Code? Open in Web Editor NEWJava client to access CKAN repositories
Home Page: http://opendatatrentino.github.io/jackan
License: Other
Java client to access CKAN repositories
Home Page: http://opendatatrentino.github.io/jackan
License: Other
Releasing with proper IT tests causes issues because of this.
See opendatatrentino/tod-commons#1
Hi,
I'm "novice" about Jackan/Ckan technologies. During these days I created several JUnit Test
to better understand how to use Jackan with Ckan.
I'm using Jackan 0.4.2 with Ckan (2.5.1) and seems that the method CkanClient.makeResourceUrl(CKAN_CLIENT.getCatalogUrl(), resourceCreated.getPackageId(), resourceCreated.getId())) returns a broken URL.
In particular the URL returned does not contain '/dataset/' in the path.
The method returns:
http://[hostname]/[datasetID or datasetName]/resource/[resourceID] but is broken
adding the string "/dataset/" in the path after [hostname]:
http://[hostname]/dataset/[datasetID or datasetName]/resource/[resourceID] is right
Is it a bug or a misconfiguration on CKAN server? Thx a lot.
Although CKAN has a plugin to expose data as DCAT RDF, not all portals support this plugin. Moreover, the plugin is marked as work-in-progress.
Couldn't find how to configure Jackson and Joda for custom formatting of dates. Anyway this is an issue only when uploading, which we don't support yet.
Query done and object structure don't actually allow to return the packages associated with an org.
Would be nice to know in tests which kind of server are we dealing with (CKAN / DKAN / other ?)
Exception in thread "main" eu.trentorise.opendata.jackan.JackanException: Error while getting dataset.
at eu.trentorise.opendata.jackan.ckan.CkanClient.getHttp(CkanClient.java:120)
at eu.trentorise.opendata.jackan.ckan.CkanClient.getDatasetList(CkanClient.java:130)
at eu.trentorise.opendata.ckanalyze.main.TestMain.main(TestMain.java:42)
Caused by: org.apache.http.client.HttpResponseException: Bad Request
at org.apache.http.client.fluent.ContentResponseHandler.handleResponse(ContentResponseHandler.java:47)
at org.apache.http.client.fluent.ContentResponseHandler.handleResponse(ContentResponseHandler.java:40)
at org.apache.http.client.fluent.Response.handleResponse(Response.java:85)
at org.apache.http.client.fluent.Response.returnContent(Response.java:92)
at eu.trentorise.opendata.jackan.ckan.CkanClient.getHttp(CkanClient.java:108)
... 2 more
Tests fails because Timestamp formatting is timezone dependant.
First spotted in #39
Some write tests don't pass with demo.ckan.org . In some cases it is just a matter of authorizations. But as a rule of thumb, we should keep demo.ckan.org as a reference and make sure most tests pass. In the wiki we will try to keep reviews of the server status
Maybe we can implement this workaround: Pragmatists/JUnitParams#52
we must turn Jackan to jackan
Implementing file uploads doesn't seem difficult (see http://docs.ckan.org/en/ckan-2.2/filestore.html), although it's not a priority for now.
Generated html table is very raw.
We could improve it with DataTables. Also, a CSV export would be nice.
Various methods like createCkanResource, createCkanDataSet
call printStackTrace, we need to rethrow exception wrapped in a JackanException
Adding extras or others in CkanBaseResources works but they are not uploaded
Some repository needs authorization also for get requests
Hey, are you actively developing it, if not, I can help, in our work we are using ckan for a webpage and we need to simplificate the developmen with an API, we can work with this
we need title (can be like 'Laghi monitorati Trento') and also license
Some travis continuous integration can help with testing pull requests
I have an issue reading certain records from demo.ckan.org which have "others" property populated with a simple string.
Example of such an record is:
http://demo.ckan.org/api/3/action/package_search?q=id:69c59a57-9ff3-43ff-915a-3a1c04bc4d80
You can see that there is a property others: "qui1hfv3qtm85lg2iokob6lg1m" within the very first resource entry.
It appears that Jackan fails to load this resource because it attempts to load a simple string into the private Map<String, Object> others bean property as defined in CkanResourceBase class.
Hi, I'm experiencing some issues with these domains:
http://www.dati.piemonte.it (API URL: http://www.dati.piemonte.it/rpapisrv/api/rest/package)
In the developer page there's written that their API are compatible with CKAN model, but Jackan doesn't recognize the domain: Couldn't interpret json returned by the server!
I saw that if I use as domain "http://www.dati.piemonte.it", it becomes: "http://www.dati.piemonte.it/api/3/action/package_list" and I think this is not correct.
Can you do something?
http://www.dati.gov.it/ (API URL: http://www.dati.gov.it/api/3/action/package_list)
Jackan answer: NullPointerException: Need a valid id or name!
I think that for dati.gov the parameter "limit" is mandatory.
Same question as above.
Thanks.
Hello
I'm trying to create a new dataset with a csv file as a resource. The send process runs with no errors, but in CKAN I can't access the csv content. My code is as follows:
dataset.setResources(new ArrayList<CkanResource>());
String fileName = "test.csv";
CkanResource resource = new CkanResource(CKAN.ckanDatasetUrl + "/" + fielName, fileName);
resource.setId(UUID.randomUUID().toString());
resource.setName(fileName);
resource.setFormat("csv");
resource.setMimetype("text/csv");
StringBody body = new StringBody(csvContent, ContentType.MULTIPART_FORM_DATA);
resource.putOthers("upload", body);
Any help will be very helpful.
Sincerely,
Alex
Hi guys, I'm currently using the Jackan version 0.4.2 and I'm wondering if there is a way of using the Visibility/Searchable fields in order to make (make not) the dataset under creation visible and/or searchable. The extended CKanDataset class you provide, seems to have setPriv() and setOpen() methods, but I do not know if they are the ones I was looking for.
However I tried to use them, but they seem to have no effect. When creating a dataset using the CKanDataset (which extends CkanDatasetBase), the call reaches this point
` public synchronized CkanDataset createDataset(CkanDatasetBase dataset) {
checkNotNull(dataset, "Need a valid dataset!");
checkToken("Tried to create dataset" + dataset.getName());
String json = null;
try {
json = getObjectMapperForPosting(CkanDatasetBase.class).writeValueAsString(dataset);
} catch (IOException e) {
throw new CkanException(COULDNT_JSONIZE + dataset.getClass()
.getSimpleName(),
this, e);
}
DatasetResponse response = postHttp(DatasetResponse.class, "/api/3/action/package_create", json,
ContentType.APPLICATION_JSON);
return response.result;
}`
Is getObjectMapperForPosting(CkanDatasetBase.class) method invocation right? Or we need to differenciate between CkanDatasetBase and CkanDataset?
Thanks :)
Exceptions in HTML reported by CkanTestReporter are not properly escaped.
Dears, I am getting the following error message while accessing http://datahub.io via cc.getDatasetList().
Error message --> Exception in thread "main" eu.trentorise.opendata.jackan.exceptions.CkanException: Error while performing GET. Request url was: https://datahub.io/api/3/api/3/action/package_list?limit=1000&offset=0 CkanClient{catalogURL=https://datahub.io/api/3, ckanToken=null} Caused by: javax.net.ssl.SSLException: hostname in certificate didn't match: <datahub.io> != <.ckan.io> OR <.ckan.io> OR <ckan.io>
Is there any workaround on this problem?
Thanks a lot.
Cannot create a dataset with a group because Ckan complains about a mysterious '__junk' field. This is because Ckan doesn't like the passed group to have users in object format inside. If users are represented as strings or not serializaed at all it works fine.
Hey guys,
I'm using Jackan 0.4.2
to connect to an SSL secured server using SNI (Server Name Identification). I think there's a problem with the httpclient version (currenlty 4.3.1
, above version shouldn't be affected) included by the shade plugin.
I tried with version 4.3.3
(httpclient has a version 4.3.6
but httpcore did not so I chose the common version) and this worked out. Probably I could also use an even higher version of httpclient but I don't know if you rely on other parts of the API, though.
PR is in progress ...
BTW: Is there a reason why you include all sources and ship the stuff? This led me checkout the sources and patch the pom.xml
. Wouldn't a plain dependency work out the same?
Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.