Comments (3)
Hi @ezecastellano. Thank you for your suggestion. In the next commit, we will switch the budget from simulation time to total execution time. In this way, we will take into account also the time spent for test input generation.
from tool-competition-av.
@ezecastellano We plan something in the order of (few) hours. We do not have a precise number at the moment, as the final decision very much depends on how many participants we'll have. So I'll guess something in the range of 5-10hours. Less than 5 hours might not give the tools enough time to explore the space, while more than 10 hours becomes hard to manage on our side (each tool will be executed multiple times).
If you have a suggestion feel free to share it with us.
from tool-competition-av.
Hi Alessio, thank you very much for the quick response. Knowing that is already very useful to us.
Just to clarify, do you mean 5-10 hours of simulation time (--time-budget 56060) or 5-10 hours of total execution? We noticed that the --time-budget does not actually count the time spent in generating the tests.
from tool-competition-av.
Related Issues (20)
- Release the code_pipeline as pypi module
- Bump BeamNG.research and BeamNGpy versions up HOT 1
- Extend test and test execution evaluation using feature HOT 1
- Extend the pool of test subjects HOT 1
- Update links to point to SBST 2022 HOT 1
- Port the infrastructure to BeamNG.tech and latest BeamNGpy
- Descartes uses deprecated functions from shapely, consider bump it to the next version
- Test budget as time hinders reproducibility of results HOT 1
- Sparseness measures HOT 2
- New generation budget definition
- Improve the definition of test effectiveness and describe explicity how generators will be ranked
- Import git submodules using https not ssh
- The GUIDELINES page shows an outdate usage for the code_pipeline HOT 1
- Setting an OOB tolerance value of close to zero makes all the test fail
- Issues with --visualize-tests option
- Fix calculation of direction coverage
- Fix & speed up max curvature calculation
- What is the license HOT 1
- Double check frequency of monitoring for BeamNG Executor
- can simulator load a road that is more than 1km HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tool-competition-av.