d-krupke / algbench Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 1.0 1.81 MB

Experiment execution and result management for empirical evaluations of algorithms in Python.

Home Page: https://algbench.readthedocs.io/

License: MIT License

Python 100.00%

algorithm-engineering python result-mangement

algbench's Introduction

PostDoc Researcher at the Algorithms Group (Prof. Fekete), TU Braunschweig, Germany.

I am an experienced algorithm engineer with a background in theoretical computer science. I have a knack for tackling complex, especially NP-hard, problems by fusing theoretical rigor with practical implementation. My work has been recognized with best paper (CIAC, ALENEX) and student awards.

Skills

Technical Skills 💡

Combinatorial Optimization: Proficient in solving NP-hard problems to optimality using techniques like Mixed Integer Programming, Constraint Programming, custom branch-and-bound algorithms, SAT-solvers, and Second Order Cone Programming.
Approximation & Meta-heuristics: Skilled in finding near-optimal solutions via approximation algorithms, meta-heuristics, and LNS-variants.
Algorithmic Foundations: Strong background in theoretical computer science, with a comprehensive understanding of algorithmic concepts and their practical applications, such as complexity, approximation, and graph theory.
Programming & Performance: Adept in writing maintainable Python and C++ code for complex algorithms, with expertise in performance tuning and modularization. Check out my repositories for examples.
Data Analysis & Visualization: Capable of managing, evaluating, and quality-checking complex data, as well as visualizing data sets for insights and decision-making. Refer to my dissertation for exemplary empirical evaluations.
Machine Learning: Familiar with machine learning techniques and have applied them successfully in research projects. Eager to explore their potential to augment classical algorithms, although this is neither a primary focus nor an area of expertise.

Research Skills 📚

Theory-Practice Bridge: Skillful in bridging the gap between theoretical computer science and practical implementation, highlighted by interdisciplinary collaborations and consultancy.
Interdisciplinary Collaboration: Extensive involvement in projects across diverse fields such as robotics, bioinformatics, automotive, and satellite management.
Creativity & Curiosity: Demonstrated curiosity and creativity in learning and combining new techniques, as evidenced by the diverse techniques used in my dissertation and the variety of projects undertaken.

Soft Skills 🤝

Teaching & Presentation: Proficient in teaching and presenting complex topics, as proven by the positive evaluation of my lecture on algorithm engineering and the popularity of my online material.
Project Management: Experienced in managing multiple projects and student teams concurrently, as evidenced by the number of successfully completed projects.

🔬 Research	📖 Publications	🎓 Dissertation
Explore my ongoing research projects.	Discover my published works.	My dissertation for a coherrent sample of my work (2017-2022).

🤝 Let's Connect!

I am open to research stays and collaborative opportunities to broaden my expertise and contribute to groundbreaking research. Let's work together to solve challenging problems and make a lasting impact.

algbench's People

Contributors

Stargazers

Watchers

Forkers

chekmanh

algbench's Issues

Create a simple function to check if the data is complete

A very serious problem we had recently was that a third party, that did not use AlgBench, executed their experiments without any error reporting. If during the execution of multiple weeks the network or the NFS had some short term problems, entries went missing without anyone noticing. Only after the painful data analysis, we noticed inconsistencies. Because the framework in the background was not changed (and simply was bad), also after two further iterations, we noticed inconsistencies due to incomplete data or wrongly copied data.

To make sure, the data generated with the AlgBench-Framework is complete, it would be good to have a verify-mode, that does not executed anything but just throws an error, if it would. This could be a ReadOnly-Mode will do nothing if the data already exists but throws an exception if it does not.

In this context, it may also be useful to have a finalize() function that sets a ReadOnly-Flag in the internal JSON, which will trigger this mode.

Add package information (like pip freeze) again, since pkg_resources has been deprecated.

Should have robust support for 3.7+.

Create an "archive"-function that makes a benchmark read-only

When writing a research paper, you may want to freeze a dataset and prevent accidental changes to it.
It would be easy to add a flag to a dataset that prevents algbench of manipulating it. Maybe, it should also create a hash for verification.

Create a fingerprint function that enable you to check if you are working on the same dataset

Create a simple function that creates a hash of the database. This fingerprint can be added to evaluations to verify that it has been run on the same data.

Allow deleting data to save space

The data saved by AlgBench is supposed to enable you to debug stuff and investigate observations. However, it can accumulate to many gigabytes.

You may want to clean up the results but still mark the parameters as already run and processed.
You may want to delete old stdout captures etc., without deleting the whole entry in order to have a more efficient database.

Write an efficient function that can reduce that data without deleting the "already run" property. There needs to be some thoughts about how to deal with the individual elements.

Create a "Changelog" reporting function

It is often interesting to see, in which timeframe and on which hardware the dataset has been created. It should be easy to automatically create a list with
date | hostname | git revision | package versions | number of entries

This should probably also have an export function, such that it can be provided in readable form for archived projects.

Make sure that AlgBench can also be used with local threading.

AlgBench has been designed to be used in parallel over multiple workstations or via different processes on the same workstation. However, it may not safe to use using multi-threading in the same process.

Error unzipping LookupError: unknown encoding: cp437

03_looking_at_results.pdf

Error when running code in jupyter notebook.

JSON-support for daytime and np.arrays

Suggestion from Michael: Extend the json compatibility to daytime objects and np.arrays. Might be easily realized by providing a decoder and functionality to the json converter.
Would this make for a useful addition?

AlgBench will not notice an interrupt if the interrrupt was handled by a native extension (like Gurobi)

There is a problem with bad entries when interrupting the benchmark via ctrl+c. Solvers like Gurobi and CP-SAT handle the signal on their own and stop the search but do not interrupt the Python-interpreter. Thus, they just terminate early and create bad entries (essentially, your shortened the time limit for this entry).

Check if there is some way to find out if ctrl+c was pressed during the creation of an entry.