Comments (6)
bradyz
commented on May 24, 2024
you should use the custom leaderboard repo that is included in this project - this custom leaderboard version disables these methods that allow for privileged information to be captured during training
additionally, i'm working on an updated version that should be public in the next month or so
from 2020_carla_challenge.
omerXfaruq
commented on May 24, 2024
Hello @bradyz , thanks for the response.
I used the original leaderboard repo in the $CARLA_ROOT directory (where I installed according to https://leaderboard.carla.org/get_started/#1-system-setup) and tried to run_agent.sh there.
It was missing
1-routes from 2020_carla_challenge/leaderboard/data , I copied the route directories.
2-team_agent autopilot, I copied leaderboard/team_code directory, from 2020_carla_challenge
3-carla_project files, carla_project from 2020_carla_challenge
However it still gave the privileged information capturing error error.
========= Preparing RouteScenario_19 (repetition 0) =========
> Setting up the agent
route_19_11_24_07_04_06
The sensor's configuration used is invalid:
> Illegal sensor used. sensor.camera.semantic_segmentation are not allowed!
Traceback (most recent call last):
File "leaderboard/leaderboard/leaderboard_evaluator.py", line 270, in _load_and_run_scenario
AgentWrapper.validate_sensor_configuration(self.sensors, track, args.track)
File "/home/lenovo/carlaPath_0.9.10/CARLA_0.9.10.1/leaderboard/leaderboard/autoagents/agent_wrapper.py", line 205, in validate_sensor_configuration
raise SensorConfigurationInvalid("Illegal sensor used. {} are not allowed!".format(sensor['type']))
leaderboard.envs.sensor_interface.SensorConfigurationInvalid: Illegal sensor used. sensor.camera.semantic_segmentation are not allowed!
My run_agent.sh:
#!/bin/bash
export PORT=2000 # change to port that CARLA is running on
export ROUTES=leaderboard/data/routes_training/route_19.xml # change to desired route
export TEAM_AGENT=auto_pilot.py # no need to change
export TEAM_CONFIG=saveDataPath # change path to checkpoint
export HAS_DISPLAY=1 # set to 0 if you don't want a debug window
export PYTHONPATH=$PYTHONPATH:$CARLA_ROOT/PythonAPI/carla
export PYTHONPATH=$PYTHONPATH:$CARLA_ROOT/PythonAPI/carla/dist/carla-0.9.10-py3.7-linux-x86_64.egg
export PYTHONPATH=$PYTHONPATH:leaderboard
export PYTHONPATH=$PYTHONPATH:leaderboard/team_code
export PYTHONPATH=$PYTHONPATH:scenario_runner
if [ -d "$TEAM_CONFIG" ]; then
CHECKPOINT_ENDPOINT="$TEAM_CONFIG/$(basename $ROUTES .xml).txt"
else
CHECKPOINT_ENDPOINT="$(dirname $TEAM_CONFIG)/$(basename $ROUTES .xml).txt"
fi
python3 leaderboard/leaderboard/leaderboard_evaluator.py \
--track=SENSORS \
--scenarios=leaderboard/data/all_towns_traffic_scenarios_public.json \
--agent=${TEAM_AGENT} \
--agent-config=${TEAM_CONFIG} \
--routes=${ROUTES} \
--checkpoint=${CHECKPOINT_ENDPOINT} \
--port=${PORT}
echo "Done. See $CHECKPOINT_ENDPOINT for detailed results."
from 2020_carla_challenge.
t27
commented on May 24, 2024
you should be using @bradyz 's version of the leaderboard repository.
This line should be present in your leaderboard_evaluator.py for it to work properly https://github.com/bradyz/leaderboard/blob/7104247204e0e591d43640d2c98490533bc5bbb8/leaderboard/leaderboard_evaluator.py#L42
from 2020_carla_challenge.
omerXfaruq
commented on May 24, 2024
@t27 thank you very much!
from 2020_carla_challenge.
omerXfaruq
commented on May 24, 2024
TL;DR
Finally I realize that the problem was path variables existing in ~/.bashrc like this: https://leaderboard.carla.org/get_started/#13-define-the-environment-variables
I deleted all path variables, and run the data_collection with leaderboard and scenario_runner from 2020_carla_challenge however i still got this error, it seems there is some small error in custom 2020_carla_challenge repo for data_collection
It stops just after running the run_agent and initializing vehicles on the world:
Current Error:
========= Preparing RouteScenario_19 (repetition 0) =========
> Setting up the agent
route_19_11_26_06_59_47
> Loading the world
> Running the route
Stopping the route, the agent has crashed:
> index 255 is out of bounds for axis 0 with size 26
Traceback (most recent call last):
File "/home/lenovo/2020_CARLA_challenge/leaderboard/leaderboard/scenarios/scenario_manager.py", line 152, in _tick_scenario
ego_action = self._agent()
File "/home/lenovo/2020_CARLA_challenge/leaderboard/leaderboard/autoagents/agent_wrapper.py", line 77, in __call__
return self._agent()
File "/home/lenovo/2020_CARLA_challenge/leaderboard/leaderboard/autoagents/autonomous_agent.py", line 115, in __call__
control = self.run_step(input_data, timestamp)
File "/home/lenovo/2020_CARLA_challenge/leaderboard/team_code/auto_pilot.py", line 170, in run_step
_topdown = Image.fromarray(COLOR[CONVERTER[topdown]])
IndexError: index 255 is out of bounds for axis 0 with size 26
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "leaderboard/leaderboard/leaderboard_evaluator.py", line 345, in _load_and_run_scenario
self.manager.run_scenario()
File "/home/lenovo/2020_CARLA_challenge/leaderboard/leaderboard/scenarios/scenario_manager.py", line 136, in run_scenario
self._tick_scenario(timestamp)
File "/home/lenovo/2020_CARLA_challenge/leaderboard/leaderboard/scenarios/scenario_manager.py", line 159, in _tick_scenario
raise AgentError(e)
leaderboard.autoagents.agent_wrapper.AgentError: index 255 is out of bounds for axis 0 with size 26
> Stopping the route
========= Results of RouteScenario_19 (repetition 0) ------ FAILURE =========
╒═════════════════════════════════╤═════════════════════╕
│ Start Time │ 2020-11-26 07:00:00 │
├─────────────────────────────────┼─────────────────────┤
│ End Time │ 2020-11-26 07:00:00 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time) │ 0.19s │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time) │ 0.05s │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.27 │
╘═════════════════════════════════╧═════════════════════╛
you should use the custom
leaderboardrepo that is included in this project - this custom leaderboard version disables these methods that allow for privileged information to be captured during trainingadditionally, i'm working on an updated version that should be public in the next month or so
Hello @bradyz , I misunterstood you there, sorry! After looking through again, I understood you correctly now.
I initially run the example in your 2020_carla_challenge repo, and was using custom leaderboard. After your reply I thought I need to use leaderboard inside $CARLA_ROOT and tried the things in my second comment:
Hello @bradyz , thanks for the response.
...
My Final run_agent.sh recieving error "index 255 is out of bounds for axis 0 with size 26":
#!/bin/bash
export CARLA_ROOT=/home/lenovo/carlaPath_0.9.10/CARLA_0.9.10.1
export PORT=2000 # change to port that CARLA is running on
export ROUTES=leaderboard/data/routes_training/route_19.xml # change to desired route
export TEAM_AGENT=auto_pilot.py # no need to change
export TEAM_CONFIG=sample_data # change path to save data
export PYTHONPATH=$PYTHONPATH:$CARLA_ROOT/PythonAPI/carla
export PYTHONPATH=$PYTHONPATH:$CARLA_ROOT/PythonAPI/carla/dist/carla-0.9.10-py3.7-linux-x86_64.egg
export PYTHONPATH=$PYTHONPATH:leaderboard
export PYTHONPATH=$PYTHONPATH:leaderboard/team_code
export PYTHONPATH=$PYTHONPATH:scenario_runner
if [ -d "$TEAM_CONFIG" ]; then
CHECKPOINT_ENDPOINT="$TEAM_CONFIG/$(basename $ROUTES .xml).txt"
else
CHECKPOINT_ENDPOINT="$(dirname $TEAM_CONFIG)/$(basename $ROUTES .xml).txt"
fi
python3 leaderboard/leaderboard/leaderboard_evaluator.py \
--track=SENSORS \
--scenarios=leaderboard/data/all_towns_traffic_scenarios_public.json \
--agent=${TEAM_AGENT} \
--agent-config=${TEAM_CONFIG} \
--routes=${ROUTES} \
--checkpoint=${CHECKPOINT_ENDPOINT} \
--port=${PORT}
echo "Done. See $CHECKPOINT_ENDPOINT for detailed results."
from 2020_carla_challenge.
bradyz
commented on May 24, 2024
it looks like you'll have to pull from master for the carla_project submodule
from 2020_carla_challenge.
Related Issues (20)
- Updated image model weights for CARLA 0.9.10 HOT 6
- Map not found Error while running a docker file in local machine
- Generation of Routes and Scenarios HOT 1
- Plotting image waypoints from the BEV perspective
- Is the data collected in map view? HOT 2
- Segmentation model weights? HOT 2
- route_19.xml, route_15, route_10 does not work HOT 7
- Not able to train STAGE_1 HOT 5
- The performance is not as good as LBC
- Does map model training use data augmentations?
- CoordConverter HOT 2
- Results of running a pretrained model is FAILURE HOT 2
- Where can I download the teacher model HOT 1
- Unknown error when submitting docker image to leaderboard benchmark HOT 2
- about sensormotor agent HOT 1
- the trained time took HOT 1
- Trying to update to work with CARLA 0.9.13 HOT 3
- cannot find image_agent.py HOT 2
- Pretrained Model Failed All with AgentBlocked
- About Converter world <--> cam conversion hack HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 2020_carla_challenge.