tasignotas / sdp Goto Github PK

Find the best way of fixing the distortion of the camera. Will look at other options and previous but at the moment using OpenCV to generate the calibration matrix.

I've gotten simpleCV to mostly work now after facing some problems:

1. Problem where /libs/lib/python2.6/site-packages/SimpleCV-1.3-py2.6.egg/SimpleCV/sampleimages/simplecv.png cannot be located.
   Solution: Simply make a sampleimages folder and find a simplecv.png image on google. Then paste it in.
2. NotImplementedError: font module not available
   (ImportError: libSDL_ttf-2.0.so.0: cannot open shared object file: No such file or directory)
   I've not solved this as libSDL_ttf-2.0.so.0 literally cannot be opened because it's shared(?)
   I ran the script install_simplecv.sh from the scripts folder. Must I run it in some other directory?

Thanks.
Mac

Find the direction the ball is moving in. Frame to frame analysis should help.

Kicker systems should be researched. Last year's examples may not help us much given how different their robots' dimensions are. Most had their kicker start from the vertical position, and just push out.

The issue should be revisited and discussed before any kickers are actually implemented, and certainly before the robot is modified.

The codec used to record the video files on the DICE machines seems to be unreadable by OpenCV Python (at least on Windows). They may need to be converted to another codec.

Given the video stream, detect the location of the ball.

Implement a PID Feedback System to calibrate displacement and rotation multipliers for the new robot design.

As mentioned in the group meeting today, we will most probably need several color masks for varying light scenarios. I propose we run some tests to find in which positions our current masking is performing poorly and add those to the system.

System: Run all color scenarios for each color at the same time using multiprocessing. The moment a location is found, return and stop the search on the other processes.

Performance: Tracking at the moment takes a small fraction of the computation time available to process 25 frames a second. Parallel processing should not increase the time too much and could result in much better results.

What do you think?

We should add some pieces to allow for the T plate to be placed on the robot. I am not entirely sure it is part of the first milestone but it should be easy enough.

Ideally, we'd have a build script that we can run on a dice machine and have it do all the hard work of changing the paths.

@RyanrDavies, @rowanborder, @victor-dumitrescu You have the most experience with it, could you write the instructions?

Have a look at what @victor-dumitrescu managed to implement and see if we could take it further and get reliable vision results.

Determine the speed a robot is moving in. Frame to frame analysis again.

Together with Mac and Maneshka we found a very fast way to detect the robots on the field based on masking and contour recognition. I presume it will work on the ball too.

A way to determine the front of the robot is required as the T is almost evenly sized and elliptical masking does not help. I will refactor the code later and show the issue. It would be nice if we could get other ideas how to determine the orientation with maximum precision.

One failed approach was to find maximum left, top, right and bottom values and use pixel distance to determine the orientation. Almost even sizing of the T makes this a failed attempt.

The question is, given a binary image with an evenly sized T shape, how do you determine which is the top. I can find majority of pixels inside the T, draw a bounding box or fit an ellipse over it but with limited precision and high volatility to camera distortion.

All ideas are welcome.

Make each Tracker run in a separate process.

Investigate the method of background subtraction and how it could be used to improve our vision accuracy.

Python is currently in use for both vision and movement for our robot. We already overcame an issue with NXTPython which Garry resolved.

We have the choice of using Java for vision, which would give us access to more vision code. Most SDP groups seem to use Java but plenty used Python for vision.

This issue is just to note that a decision must be made.

I found an alternative python wrapper for the NXT. Is it worth looking into?
https://pypi.python.org/pypi/jaraco.nxt

Investigate methods to remove camera distortion. Good starting point is here

Attempt to use the shape of the plate to our advantage to predict the orientation based on the black dot and parallel lines of the plate.

Find the direction the ball is moving in. Frame to frame analysis should help.

Research and test if Optical Flow could be used to improve object detection.

Design a PID system to correct for motor control errors i.e. over and under turning, overspeed and underspeed based on feedback from the vision system and onboard sensors.

What units and which relative vector is the angle to be computed? We talked about switching to radians. Can you please specify which units you are expecting?

Could anyone please talk to Garry and request the light to be fixed? Without it there's no chance to calibrate vision well.

Investigate if there are other color representations beside RGB (BGR) and HSV and how they could help us improve masking.

Build new robot designs for the attacker and defender with the arduino and test the robustness and usability of these designs.

Some code has been refactored already but we need to make it more general and find missing thresholds for the second pitch. Some of the tracking works just fine, the rest needs tweaking.

Mac mentioned a steering system akin to a car's, in which the front wheels may be on some kind of partially rotating axle. The back wheels would be used for drive.

If a concept design could be produced and reviewed by the team this could be a good.

It should be noted that other teams are also investigating this design. Their design features a horizontally spinning cog that rotates through 30 degrees, with the motor driving the front wheels by means of a vertical rod, which rotates to spin the wheels.

Rear-wheel-drive should make it easier to angle the robot quickly. Front-wheel-drive should allow better pinpoint steering control, but the turning circle would be horrible.

A boolean value representing whether a robot R has possession of the ball B. Possibly by using some proximity and orientation calculations with the vision?

Design a coordinate system for the football field from the stream.

Sections (attack/defense) need to be distinguished.

Grid overlay over the video would be helpful to be able to transmit these locations and allow to reason about it.

Find the direction a robot is facing. Frame to frame analysis again.

In order to avoid other teams connecting to our NXTs, we need to rename the Bricks. G7-A, G7-D for Attack and Defense?

Just stumbled on this. It's a technique to denoise images.

Might help solve our issues.

Optimize code and have a robot that can safely drive inside the field (attack and defence). No vision for now.

Before moving forward with the Arduino project, it would be best to test the communication latency to determine if it is worth.

The left and the right images are both the results of the same code but running on DICE machines which get the video from different amplifiers. (At least that's the only explanation I could come up with.) We need a way to deal with these variations.

Design the grabber for the attacker robot.

This means that it can only be used on the segments where the ball is not bouncing off the walls. It can accurately predict the position of the ball is we momentarily lose track of it, but it must be reset every time the ball stops to move in a continuous motion.
These are some examples of how the Kalman filter performs in these scenarios:

This means that using the filter depends on accurately measuring the direction and velocity of the ball.

Given a stream from the camera, find the location of our robot (for now).

Implementation of a dynamic GUI for real time calibration. Investigate some options - be it OpenCV, SimpleCV or Qt or anything else.

We need to know with accuracy the orientation of the robots on the pitch for catching, passing and shooting the ball. This could either be done using accurate computer vision or an on-board compass sensor. We should investigate the relative merits of both approaches.

We might need to run the code from the brick only. For that we might have to turn to LeJOS - Java implementation of the NXC language. It would be useful to have a look at it and try to build some code and test it, in case we do end up having to run it from the brick only. LeJOS

I wrote a little GUI app that lets the user create polygons for the whole field and each zone. Points are saved into json and can be retrieved by other apps. Some testing of how well it works and what functionality is expected and isn't availabale would be good.

Each selection should be terminated by pressing 'q'. Please follow instructions printed to the terminal. To run, do python configure.py

Install all libraries on a virtual machine.

Ensure we stay within the zone at all times in various scenarios

• Test timing again
• Timing for each of the defender zones might be different - verify

Design the defender robot. Preferable characteristic should be speed.

Research and test how communication with both Bricks from a single computer will work. Control of the robots might have to be turned into threaded processes.

Our programmers have produced a decent piece of code which works for milestone 1 on two wheels. It works well enough, and a video of the robot is available.

Ultimately the castor wheel at the rear may lead to alignment issues, so the robot design team's next task is to design a 4-wheeled, 4-wheel-drive robot.

Main planning system than links between the vision system, PID system and robot movement system.