Code Monkey home page Code Monkey logo

crce's Introduction

CRCE - Component Repository supporting Compatibility Evaluation

CRCE is an experimental repository, designed to support research into component-based and modular systems undertaken by ReliSA research group at the Faculty of Applied Sciences, University of West Bohemia (http://relisa.kiv.zcu.cz). The older project website is over at Assembla: https://www.assembla.com/spaces/crce/wiki .

To cite CRCE as a research result, please use the following citation:

P.Brada and K.Jezek, “Repository and Meta-Data Design for Efficient Component Consistency Verification,” Science of Computer Programming, vol. 97, part 3, pp. 349–365, 2015. Available: http://www.sciencedirect.com/science/article/pii/S0167642314002925

Prerequisities

  • JDK 11 set in JAVA_HOME environment variable before starting CRCE, tested on OpenJDK 11.0.4
  • MongoDB, tested on v2.6.10, v3.4.10
  • Maven 3, tested on 3.5.2

On linux, switching JDK version for development/build can be done via sudo update-java-alternatives (or, less ideal as it does not set all aspects of the environment, sudo update-alternatives --config java).

Mongo DB can either be run locally or in docker using sudo docker run -d -p 27017:27017 -v ~/data:/data/db mongo, where ~/data is directory on host machine to be used as a storage by Mongo.

Some CRCE plugins use a mathematical solver, for which you need to install lpsolve library to your computer. To do that, follow their guide step by step. Note that on Windows, you do not have to place the libs to \WINDOWS or \WINDOWS\SYSTEM32 as the guide states. Put it wherever you wish and add the directory to your Path.

Build

Build process consists of two parts. Compiling and building the code itself and building the docker image.

On linux or similar, the ./build-code.bash script in project root directory can be used to perform these build steps. See top of script source for parameters tweaking the build.

  1. crce-parent in /pom directory
  2. shared-build-settings in /build
  3. everything in /third-party (bash: .../third-party$ for d in * ; do cd $d; mvn clean install; cd .. ; done)
  4. crce-core-reactor in /core
  5. crce-modules-reactor in /modules
  6. provision-reactor in /deploy

In case of maven error "Received fatal alert: protocol_version", use mvn -Dhttps.protocols=TLSv1.2 ... after https://stackoverflow.com/a/50924208/261891. Forbidden hack to speed up build: -Dmaven.test.skip=true.

Build docker image

Docker image is placed in directory /deploy but before it can be used, bundles must be collected by running the following commands in /deploy directory:

mvn clean pax:directory
./prepare-bundles.sh

The prepare-bundles.sh script is needed due to the issues further described in the Issues section.

To finally build the image itself, execute the following command in /deploy directory:

docker build . -t ${image-tag}

Start up

To start on local machine, make sure the /deploy/conf folder exists and contains all important configuration (especially the cz.zcu.kiv.crce.repository.filebased-store.cfg. After that, you run CRCE using Maven plugin for pax in crce-modules-reactor module (i.e. /deploy directory):

mvn pax:provision

Running with docker

Assuming the image is build and Mongo DB is running at an accessible address, CRCE can be run in docker by this command:

docker run -it \
        -p 8080:8080 \
        --add-host mongoserver:172.17.0.1 \
        -v /felix/deploy:/felix/deploy \
        <image-tag>

The --add-host ... and -v ... parameters allow docker to connect to MongoDB (so use the correct IP address instead of the 172.17... above) and to install new bundles (from the provided directory). If MongoDB is running locally on 127.0.0.1, add --network="host" to make localhost accessible.

The -p parameter maps the port of docker virual machine to the real port on the computer. The --add-host allows docker to connect to the Mongo database running locally (Docker usually uses 172.17.0.1 ip to access localhost from the container). The app expects the database to be listening on port 27017 by default. The -v parameter maps directory in container to the real directory in the host machine so that it can be used to install new bundles via CLI.

For the simple case of starting the (above) freshly built image on localhost, use

docker run -it --network="host" --add-host mongoserver:127.0.0.1 <image-tag>

If everything works, the output log should write up some info about dependencies terminated by lines similar to the following:

Listening for transport dt_socket at address: 65505
____________________________
Welcome to Apache Felix Gogo

g! X 10, 2017 10:38:47 DOP. org.glassfish.jersey.server.ApplicationHandler initialize
INFO: Initiating Jersey application, version Jersey: 2.9.1 2014-06-01 23:30:50...

At the moment, a bunch of errors will probably come up:

[Fatal Error] :1:1: Content is not allowed in prolog.

The cause of the latter is a badly loaded binary of mathematical solver which does not affect common application run. Any other error/exception (typically OSGi complaining about a thing) is a problem that needs to be examined as such. However, it should not happen with this version.

Accessing the running CRCE

Started up, the application is accessible at:

Updated (more or less) REST WS documentation is available at Apiary. Basic endpoints: metadata/<id> (XML), resources (list all), resources/<id> (download one).

Debugging

Remote debugging is possible when running CRCE in Docker. To enable remote debug, application needs to be started with following flags:

-Xdebug -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005

And expose the port 5005 by adding -p 5005:5005 to the docker run command.

Check the deploy/Dockerfile to see the difference between running the app in normal and debug mode.

Code updates

After modifying a part of code, only the parental module needs to be rebuilt (no need to rebuild all). After that, the pax process must be restarted.

Configuration

Configuration is done via OSGI service called Configuration Admin. Details on how it's implemented in Felix can be found in Apache Felix Configuration Admin Service.

By default, Felix will look into the config directory for possible configuration (example of such directory can be found in deploy/conf.default)

Issues

Required runtime environment capability in some bundles

Some bundles (mostly those from the third parties) either require a particular version of OSGi runtime environment or require the capability osgi.ee to be set to a specific value. This may or may not work with newer version of Java and to avoid runtime problems, these requirements need to be removed.

To remove them, it is necessary to edit manifest (META-INF/MANIFEST.MF) of such bundle and remove lines containing: Require-Capability: osgi.ee; or Bundle-RequiredExecutionEnvironment: J2SE-. This is done in the deploy/prepare-bundles.sh script.

An ideal solution would be to either find newer versions of such bundles or replace them with different bundles with same functionality. This however may introduce compatibility issues (as some other bundles may depend on the current versions) or may not even be possible (replacement bundle does not exist).

An interim fix is to use /deploy/prepare-bundles.sh as described above which strips the requirements from the manifest files.

crce's People

Contributors

danekja avatar kalwi avatar pbrada avatar cajova-houba avatar fidransky avatar

Stargazers

 avatar  avatar Simon  Spero avatar  avatar

Watchers

edwardt avatar James Cloos avatar David Pejřimovský avatar  avatar  avatar Roman Pešek avatar  avatar Veronika Kutková avatar  avatar  avatar

crce's Issues

NPE when JaCC modules are used

Running both old and new web UI on CRCE with comparator modules gives NPE on web frontend when trying to list buffer/store contents.

move key modules into "core"

There are modules - in the modules/ folder - which better be in CRCE core, since they are part of the "must have" functionality. Analyze which are those, and re-structure the project to move them into core.

build issues: dependency versions, repositories

When trying to build:

  1. maven complains about missing "version" for hibernate-jpa-2.1-api.jar
  2. missing repository for jna-osgi.jar
  3. artefact jna-platform not available at appropriate time/step
  4. similarly lpsolve artefact

(As of 10/2017, when I first encountered these.)

dockerize CRCE basic deploy

It would be great to have CRCE run as Docker image, so that trying and using the (basic, vanilla) CRCE is made easier. As a basic decomposition, one image for MongoDB and one for CRCE vanilla.

make JaCC modules accessible

There are a few CRCE modules which use JaCC libraries/components, which -- for IPR reasons -- are kept in a department local Maven repository. The are needed for build, so it is important to either remove them from vanilla CRCE build or, better, move them somewhere accesible for every CRCE build.

Java 11

Make sure project can be compiled and run on Java 11.

Plan for spring 2018

Update 3/2018

  • práce na DP pokročila, ale nebude odevzdávat 5/2018 -> znovu zapsat KIV/DIP na 2018/2019
  • implementované je 3. a 4. kromě množin komponent, bude dokončeno cca v 6/2018

show current buffer contents while in upload

During upload session, it is curently not clear what has been uploaded to the buffer. It would be sufficient to add a read-only view of the Buffer contents under the upload pane.

security vulnerability in jackson-databind dependency

Automated message from github, 25.3.2019:

Known high severity security vulnerability detected in com.fasterxml.jackson.core:jackson-databind < 2.7.9.4 defined in pom.xml.

pom.xml update suggested: com.fasterxml.jackson.core:jackson-databind ~> 2.7.9.4.

Plan for 2017/2018

Ramcovy harmonogram:

  • do konce ledna 2017 hotove body 1 a 2 zadani DP, tj. overene fungujici vyvoj pro CRCE + analyzovane systemy pro inspiraci
  • v lednu schuzka - uzaverka predchoziho bodu a diskuse nad bodem 3 zadani
  • do konce dubna 2017 hotovy bod 4 tj. implementace, uzavreli bychom to prezentaci na ReliSA seminari (spolu s nacviky prezentaci diplomantu, ktere pravidelne delame cca v kvetnu)
  • do konce cervna overeni, doladeni, zhodnoceni

cleanup URL

The "crce-vaadin" is too technology dependent, the new Web UI should be accessed via "crce-webui".

merge "rpesek" into "dev"

Review the new module, the whole CRCE functionality on "ghessova" branch, cleanup, merge into "dev".

See also: #22

enable import of metadata

CRCE could support external "offline" indexers, if we had an service API to import metadata conforming to the model (and a description of the model).

Well formedness and consistency checking will have to be part of the import.

architecture documentation, examples, cleanup

Topic for KIV/SAR semester course project.

  • learn and analyse CRCE internals
  • create documentation of its architecture - modules, APIs, processes/flows, (use UML); data structures, persistent data formats (use whatever agreed upon)
  • update build so that internal API dev docs are generated and published
  • analyse, where internal APIs break rules of good API design, and create refactored versions of some (agreed upon ones)
  • create/update example of "how to build a CRCE plugin"

Former documentation (https://app.assembla.com/spaces/crce/wiki) can be used as a basis.

CRCE silently failing when no connection to Mongo

When no MongoDB connection is available at CRCE start, the Metadata service object is not created and consequently downstream modules fail. At WebUI, a NPE is signalled at BufferImpl.java:379 .

Suggested fix: report at WebUI that metadata repo is not available and disable all related functionality.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.