Code Monkey home page Code Monkey logo

lr-data's Introduction

LR-Data

This is a small utility to help pull the data from the Learning Registry into a datastore of your choice.

Dependencies

LR-Data requires:

Platform-Specific Requirements:

On OS X, you will also need libevent, which can be installed with homebrew: brew install libevent

#Setup Run pip install -U -r requirements.txt

#Configuration All configuration is done in the src/celeryconfig.py file. For information of configuring Celery please see their document. For lr-data configuration modify

`config = {

"lrUrl": "http://lrdev02.learningregistry.org/harvest/listrecords",

"mongodb":{	

	"database":"lr",

	"collection":"envelope",

	"host": "localhost",

	"port": 27017,

},

"couchdb":{

	"dbUrl":"http://localhost:5984/lr-data"

},

"insertTask":"tasks.save.insertDocumentMongo",

"validationTask":"tasks.validate.emptyValidate",

"redis":{

	"host":"localhost",

	"port":6379,

	"db":0

}

}`

set insertTask to be the celery task you wish to use to save the data and modify validationTask to be your validation task #Startup To start run celryd -B from the source directory. To run as a deamon follow these instructions

Contributing to the project

We welcome contributions to this project. Fork this repository, make changes, and submit pull requests. If you're not comfortable with editing the code, please submit an issue and we'll be happy to address it.

License

Copyright ©2016 Advanced Distributed Learning

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

lr-data's People

Contributors

creighton avatar pauliejes avatar wegrata avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lr-data's Issues

Publisher information not being indexed

  • I publish a resource to my LR node with publisher information.
  • I look at resource document in LR node and verify the publisher information.
  • LR-Data writes the document information to the elasticsearch index.
  • I query elasticsearch directly, and the publisher field for this new resource is null.
  • Data from LR resource document:
"publisher": {
"url": "http://www.starwars.com",
"@type": "Organization",
"name": "Star Wars"
},
  • Relevant LR-Data log entries:
[2014-06-12 19:50:47,478: WARNING/PoolWorker-1] ,
 'publisher':
[2014-06-12 19:50:47,478: WARNING/PoolWorker-1] None
[2014-06-12 19:50:47,478: WARNING/PoolWorker-1] ,
 'standards':

TypeError

I'm publishing with EasyPublish, and am seeing the following in [email protected] when running python start.py. Wondering if you've seen this before.

[2014-06-02 16:27:20,772: WARNING/PoolWorker-1] Traceback (most recent call last):
[2014-06-02 16:27:20,772: WARNING/PoolWorker-1] File "/mnt/LR-Data/src/tasks/save.py", line 521, in createRedisIndex
[2014-06-02 16:27:20,772: WARNING/PoolWorker-1] doc = process_json_ld(envelope, mapping)
[2014-06-02 16:27:20,772: WARNING/PoolWorker-1] File "/mnt/LR-Data/src/tasks/save.py", line 449, in process_json_ld
[2014-06-02 16:27:20,772: WARNING/PoolWorker-1] graph_data = process_json_ld_graph(payload_graph, mapping)
[2014-06-02 16:27:20,772: WARNING/PoolWorker-1] File "/mnt/LR-Data/src/tasks/save.py", line 400, in process_json_ld_graph
[2014-06-02 16:27:20,772: WARNING/PoolWorker-1] standards.extend(handle_standards_json_ld(node, mapping))
[2014-06-02 16:27:20,773: WARNING/PoolWorker-1] File "/mnt/LR-Data/src/tasks/save.py", line 378, in handle_standards_json_ld
[2014-06-02 16:27:20,773: WARNING/PoolWorker-1] for alignment in alignments:
[2014-06-02 16:27:20,773: WARNING/PoolWorker-1] File "/mnt/LR-Data/src/tasks/save.py", line 377, in <genexpr>
[2014-06-02 16:27:20,773: WARNING/PoolWorker-1] alignments = (n['targetName'] for n in node['educationalAlignment'] if 'targetName' in n)
[2014-06-02 16:27:20,773: WARNING/PoolWorker-1] TypeError: string indices must be integers

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.