Code Monkey home page Code Monkey logo

bytetrack_with_labels's Introduction

ByteTrack with (COCO) labels

output image

ByteTrack expanded with COCO or VOC labels.

License


Introduction.

ByteTrack is one of the best tracker in the field. However, it has two shortcomings.

Thread safety.

The C++ code uses function calls to static class routines.
It works well if you have one tracker running. With multiple instances of the trackers, you might face unpredictable situations.
Because every tracker, every thread, shares the same static variables, it can lead to rare conditions.
In this C++ implementation, we have removed all these potentially harmful calls.

Lack of labels.

ByteTrack only works with the bounding boxes of the objects and their probability. It takes no notion of the object class.
In most cases, it is no problem. Often, you have only one class in which you are interested, like pedestrians or football players.
However, on some occasions, the bounding boxes of different objects might coincide, as can be seen in the animation below.


A person is being marked as a car just because, at some point, their bounding boxes are (almost) identical.

We add the object class obj_id as an additional variable to the STrack class.

//---------------------------------------------------------------------------
class STrack
{
public:
	STrack(Ttlwh tlwh_, float score, int obj_id, BYTETracker* TrackInstance);
	~STrack();
......
public:
......
	int obj_id;

	Ttlwh _tlwh;
	Ttlwh tlwh;
	Ttlbr tlbr;
......
private:
    BYTETracker* myTrackInstance;               // Reference to BYTETracker instance
	byte_kalman::KalmanFilter kalman_filter;
};
//---------------------------------------------------------------------------

During Update(), this integer is filled with the object label.
It doesn't matter which kind of set you use. COCO, VOC or another.

//---------------------------------------------------------------------------
void BYTETracker::update(vector<bbox_t>& objects)
{

	////////////////// Step 1: Get detections //////////////////
	this->frame_id++;
......
	if (objects.size() > 0)
	{
		for(size_t i = 0; i < objects.size(); i++){
			Ttlwh a;
			a.t=objects[i].x;
			a.l=objects[i].y;
			a.w=objects[i].w;
			a.h=objects[i].h;

			float score = objects[i].prob;
			int obj_id  = objects[i].obj_id;

			STrack strack(a, score, obj_id, this);
			if (score >= track_thresh) detections.push_back(strack);
			else  			   detections_low.push_back(strack);
		}
	}

The distinction between object classes is done by giving a penalty in the IoU calculus when classes don't match.

for (size_t k = 0; k < btlbrs.size(); k++){
    vector<float> ious_tmp;
    float box_area = (btlbrs[k].b - btlbrs[k].t + 1)*(btlbrs[k].r - btlbrs[k].l + 1);
    for (size_t n = 0; n < atlbrs.size(); n++){
        float iw = min(atlbrs[n].b, btlbrs[k].b) - max(atlbrs[n].t, btlbrs[k].t) + 1;
        if (iw > 0){
            float ih = min(atlbrs[n].r, btlbrs[k].r) - max(atlbrs[n].l, btlbrs[k].l) + 1;
            if(ih > 0){
                float ua = (atlbrs[n].b - atlbrs[n].t + 1)*(atlbrs[n].r - atlbrs[n].l + 1) + box_area - iw * ih;
                ious[n][k] = iw * ih / ua;

                if(atlbrs[n].c!=btlbrs[k].c){    //not same object class? -> penalty
                    ious[n][k] *=0.5;
                }
            }
            else ious[n][k] = 0.0;
        }
        else ious[n][k] = 0.0;
    }
}


Final remarks.

The original code uses std::vector<float> to hold the bounding boxes.
We use a simple struct because of the fixed number of elements.
A std::vector requires more overhead. And the code becomes more readable.

We only adapted the C++ implementation. Python isn't supported. If someone likes to port the code to Python, be our guest.

BYTETrack
├── include
│   ├── BYTETracker.h
│   ├── dataType.h
│   ├── kalmanFilter.h
│   ├── lapjv.h
│   └── STrack.h
├── LICENSE
├── README.md
└── src
    ├── BYTETracker.cpp
    ├── kalmanFilter.cpp
    ├── lapjv.cpp
    ├── STrack.cpp
    └── utils.cpp

paypal

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.