Code Monkey home page Code Monkey logo

fexum's Introduction

FEXUM Build Status

Framework for Exploring and Understanding Multivariate Correlations

This repository contains the backend of our big data framework for exploring and understanding:

  • Feature selection
  • Feature extraction
  • Multivariate correlations
  • Time series

The frontend and correlation algorithm are located in different repositories.

The basis for this application has been made into a paper, which was submitted to and accepted by ECML PKDD 2017, and published as part of the conference proceedings in the "Lecture Notes in Computer Science" (LNCS) series.

Contributors

Getting Started

  1. Run $ docker-compose build to build all containers (do that when you are changing dependencies)
  2. Run $ docker-compose up to start all containers. Put -d for headless mode
  3. Run $ docker-compose run web_wsgi python3 manage.py migrate to apply migrations
  4. Access the docker ip (mostly localhost) at port 80.

Python Shell

$ docker-compose run web_wsgi python3 manage.py shell

This is useful when you want to test and create database objects, e.g.:

>> from features.tasks import *
>> from features.models import *
>> for feature in Feature.objects.all() 
   ... a_test_name.delay(feature.name)

Task Monitoring

Open the docker host at port 5555

Running with docker in your own network environment

By default the network is configured for the docker containers as followed:

networks:
  fexum:
    ipam:
      driver: default
      config:
        - subnet: 10.151.100.0/24

If this does not work with your network setup, change the subnet.

Development & Contributing

Testing

For testing simply run:

$ docker-compose run web_wsgi py.test

Data Migrations

Create migrations after changing or creating models

$ docker-compose run web_wsgi python3 manage.py makemigrations

fexum's People

Contributors

danthe96 avatar juliuskunze avatar lichtso avatar louiskirsch avatar marcuspappik avatar nikriek avatar xasetl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

fexum's Issues

Restructure project

  • Create docker folder with files for credentials, docker-compose and Dockerfiles

Use Median for subsampling

median_32_sample = data.groupby(np.arange(len(data)) // 32).median()

Instead of taking every 20th value to subsample, we should take the median out of every 20 values. The example above does exactly that for every 32 values.

Fix custom target index (rar-mfs)

I'm opening this issue here so that it's all in one place.
This line in Runner.scala over in danthe96/rar-mfs explains that the target attribute for RaR can be set in idxOfClassAttribute. I have already added the implementation to supply a custom index from the command line, but somewhere deep in RaRSearch Tom did not actually implement using this value.
There is a specific test case in #23, test_calculate_rar_target0, that will pass once this issue is resolved.

Upload RaR results from JSON

Could you either build an endpoint or an admin function to upload (or paste) a JSON file to use as rar results for a specific target & dataset?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.