Code Monkey home page Code Monkey logo

optosim's People

Contributors

acolijn avatar cfuselli avatar marjoleinvannuland avatar

Stargazers

 avatar

Watchers

 avatar  avatar

optosim's Issues

data reading speed

So with the new data format we can read files quickly. I tried on a single file:

with h5py.File(file, 'r') as f:
group = f['events/']
data = {}
for name, dataset in group.items():
data[name] = np.array(dataset)

len(data)

Takes 0.6s to read 10k events. Time to modify event_reader.py

Simulation data structure

The way the simulations files are structured are extremely inefficient for loading large amount of data.
Restructuring is necessary to speed up.

Also need to change filenames extensions to hdf5 instead of hd5f.

Would also be better to have logs and models divided per run_id in the data folder.

Improve get training data

The way the training data is selected now can lead to unwillingly use the same data used for training, a better system would be nice also to allow training and testing the model with different data.

Make config parser

It would be nice to have a module or a function that reads in the config file and sets all the necessary defaults in the same place, deals with missing entries or wrong values etc..

Something like this, but better adapted to our needs.

import json

class Config:
    def __init__(self, config_file):
        self.defaults = {
            "detector": "XAMS",
            "nevents": 10000,
            "nphoton_per_event": [100, 100000],
            "photon_zgen": 0.5,
            "geometry": {
                "type": "cylinder",
                "radius": 3.2,
                "ztop": 1.2,
                "zliq": 0.0,
                "zbot": -6.7,
                "ptfe_zmin": -5.32,
                "ptfe_zmax": -0.25
            },
            "npmt_xy": 2,
            "pmt": {
                "type": "square",
                "size": 2.54,
                "ndivs": 10
            },
            "scatter": True,
            "experimental_scatter_model": True
        }
        
        with open(config_file, 'r') as f:
            self.config = json.load(f)
        
        self._validate_and_set_defaults()

    def _validate_and_set_defaults(self):
        for key, value in self.defaults.items():
            if key not in self.config:
                self.config[key] = value
            elif isinstance(value, dict):
                for subkey, subvalue in value.items():
                    if subkey not in self.config[key]:
                        self.config[key][subkey] = subvalue

    def get(self, key, default=None):
        return self.config.get(key, default)

    def __getitem__(self, key):
        return self.config[key]

    def __setitem__(self, key, value):
        self.config[key] = value

    def __contains__(self, key):
        return key in self.config

# Usage
config = Config('path_to_config_file.json')
print(config.get('detector'))
print(config['geometry']['type'])

Explain how to use package in VScode

  • what to do with the environment created

  • how to activate it for running code

  • how to activate it for notebooks

  • open the OptoSim folder as project folder to be able to source the created environment

  • change the default location for VS code server in nikhef login node to a place with some disk space like /data

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.