Code Monkey home page Code Monkey logo

dynamic_clustering's Introduction

A supervised Graph-based deep learning algorithm to detect and quantify clustered particles

In this repository, all software related with REFERENCE_TO_PAPER (including simulation software) can be found.

Installation

During the whole development of the project, we used different machines with different operating systems. Also, some of these have modern GPUs to train. However, some users may not have special hardware and not be allow to run the code. We provide different YML files to run the software on different machines, depending on the available hardware and software. To install this repository, you will need to have Conda installed. YML files descriptions are below:

  • environment-cpu.yml: Install with this file if you don't have GPUs.
  • environment-titan.yml:Install with this file if you use a GPU (this YML worked with TITAN V).
  • environment-m1-2021.yml:Install with this file if you use a Mac M1 2021 which includes a powerful GPU for Deep Learning.

Run the following command:

conda env create -n dynamic_clustering --file SELECTED_YML_FILE

Then, activate the created environment:

conda activate dynamic_clustering

How to simulate

In order to reproduce datasets, please see generate_datasets.py. This script works as an example and it is the script that we used to generated datasets in our work.

We strongly recommend to run the simulations with cythonbuilder. According to our experience, simulations are faster. Run the following commands to cythonize the code:

mv Cluster.py Cluster.pyx
mv Experiment.py Experiment.pyx
mv Particle.py Particle.pyx
mv hypo.py hypo.pyx
mv ImmobilizationProbabilities.py ImmobilizationProbabilities.pyx
mv TrajectoryDisplacementGenerator.py TrajectoryDisplacementGenerator.pyx
mv utils.py utils.pyx

cythonbuilder build

If you want to recover the original files, run the following commands:

mv Cluster.pyx Cluster.py
mv Experiment.pyx Experiment.py
mv Particle.pyx Particle.py
mv hypo.pyx hypo.py
mv ImmobilizationProbabilities.pyx ImmobilizationProbabilities.py
mv TrajectoryDisplacementGenerator.pyx TrajectoryDisplacementGenerator.py
mv utils.pyx utils.py

rm *.so *.pyi
rm -r ./ext

Docker Image for Dynamic Simulations

The proyect was executed on Windows to avoid Linux virtualizations. However, we noticed that these simulation run faster on Linux systems. We used Docker to run simulations and we used the following code to run it:

docker build -t dynamic_simulations .
docker run -i -v dynamic_simulation_files:/usr/src/app/datasets --name simulation_container -t dynamic_simulations bash

Once the shell is open, follow the instructions of the first part.

Then, to retrieve .csv files, run the following command:

docker cp simulation_container:/usr/src/app/datasets/ ./

dynamic_clustering's People

Contributors

lucassaavedra123 avatar

Watchers

 avatar

dynamic_clustering's Issues

Review Simulations

This will consist of three steps:

  1. Python Profiling
  2. Analyze Critic Parts
  3. Optimize

Also, parameters selection and sampling will be revised.

Experiment simulations plot saving

There is a problem when a plot of an experiment is saved

Saving attributes in ././datasets\0_specs.txt
0 0
Exception ignored in: <function Image.__del__ at 0x0000020DD6542200>
Traceback (most recent call last):
  File "D:\Users\Lucas\miniconda3\envs\dynamic_clustering\lib\tkinter\__init__.py", line 4056, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x0000020DD6476A70>
Traceback (most recent call last):
  File "D:\Users\Lucas\miniconda3\envs\dynamic_clustering\lib\tkinter\__init__.py", line 388, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x0000020DD6476A70>
Traceback (most recent call last):
  File "D:\Users\Lucas\miniconda3\envs\dynamic_clustering\lib\tkinter\__init__.py", line 388, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x0000020DD6476A70>
Traceback (most recent call last):
  File "D:\Users\Lucas\miniconda3\envs\dynamic_clustering\lib\tkinter\__init__.py", line 388, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x0000020DD6476A70>
Traceback (most recent call last):
  File "D:\Users\Lucas\miniconda3\envs\dynamic_clustering\lib\tkinter\__init__.py", line 388, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
RuntimeError: main thread is not in main loop
Tcl_AsyncDelete: async handler deleted by the wrong thread

Add Static Simulations

Once all dynamic clustering is developed, static simulations are going to be added. I strongly recommend to add a flag in the current simulations.

Minor Changes

Perhaps, some imports are unnecessary are refactors are possible. This issue will solve that kind of things.

GUI development

If we have time, a straightforward Graphical User Interface (GUI) is going to be developed with Napari.

sampling error during simulation

  File "/Users/lucasarielsaavedra/Desktop/dynamic_clustering/generate_datasets.py", line 69, in <module>
    an_experiment.move()
  File "Experiment.pyx", line 271, in Experiment.Experiment.move
    self.recharge_batteries()
  File "Experiment.pyx", line 293, in Experiment.Experiment.recharge_batteries
    particles_that_will_blink = np.random.choice(number_of_particles_not_currently_blinking, x, replace=False)
  File "mtrand.pyx", line 965, in numpy.random.mtrand.RandomState.choice
ValueError: Cannot take a larger sample than population when 'replace=False'

Error in recharge_batteries()

Traceback (most recent call last):
  File "d:\GitHub Repositories\dynamic_clustering\generate_datasets.py", line 34, in <module>
    an_experiment = Experiment(
  File "d:\GitHub Repositories\dynamic_clustering\Experiment.py", line 146, in __init__
    self.recharge_batteries()
  File "d:\GitHub Repositories\dynamic_clustering\Experiment.py", line 290, in recharge_batteries
    particles_that_will_blink = np.random.choice(all_particles, ceil(self.average_molecules_per_frame), replace=False)
  File "mtrand.pyx", line 915, in numpy.random.mtrand.RandomState.choice
ValueError: 'a' cannot be empty unless no samples are taken

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.