MASTER OBSTACLE TRACKING: Robust Perception Skills for Computer Vision Engineers

299€ or 2 x 149€

MASTER OBSTACLE TRACKING: Robust Perception Skills for Computer Vision Enigneers

For Mid-Level Engineers looking to transform ordinary object detection skills into unique and values tracking skills.

Unlock the Research Secrets of the boldest Self-Driving Car companies to Ignite Your Computer Vision Skills and Build a Profile the Industry is Hunting for...

If, as a Computer Vision Engineer, you're interested in replacing common object detection know-how with brutally-advanced object tracking skills, then this page will show you how.

Here's the story:

In the early days of 2020, I was trying to figure out the topic of my first ever online course on self-driving cars.

At the time, the entire industry was only talking about two words: OBJECT DETECTION.

Every online course I was seeing had the exact same promise:
— "If you learn Object Detection, you'll become a Computer Vision and even Self-Driving Car Engineer! 👍"

That was the main thing, and entire waves of students were rushing into these online courses to learn how obstacle detection works (they still are today!).

But after spending over two years building autonomous shuttles and working with some of the smartest engineers in the country, I knew that this promise wasn't the truth...

It was a lie!

And I don't say it just from experience. I actually first learned about it from the great Amnon Shashua, CEO of Mobileye (Intel’s 15B$ self-driving car company), and one of the smartest minds in the self-driving car industry.

At the time, I was trying to become a Computer Vision Engineer, and was also buried in online courses on YOLO, Faster RCNN, SSD, and all of these object detection algorithms.

But one day, as I was listening to the CVPR 2016 conference, a sentence hit me:

"When you see an academic paper doing vehicle detection and putting a bounding box around the vehicle, that's not useful!"

Amnon Shashua, CEO of Mobileye

Wait, what was that?

Not useful? Almost instantly, I got obsessed about that sentence, not because it was inspiring or anything like that...

But because it was going at the exact opposite of what everybody else was saying.

Why was everyone telling me I should learn object detection (and only that) when the head of one of the best self-driving car company was telling me not to?

I've just spent months buried in online courses and fed to the idea that 2D Object Detection = Self-Driving Cars.

Intrigued, I listened to the rest of the conference to learn more.

And my suspicions were confirmed:

Object Detection is USELESS!

At least, doing 2D object detection for the sake of doing 2D object detection is.

For the first time, I felt like I got it. I never really tried to understand how 2D Bounding Boxes could be useful in a self-driving car! Detecting obstacles? Yes, but why? What do you do with 4 pixels and a class?

👉🏼 All of these hours spent learning object detection weren't useless, but they could be if I didn't know how to process the bounding boxes after they've been detected!

Put differently: object detection is useless without an evolution such as 3D Perception, Action Recognition, or, of course, Object Tracking.

The rest is almost history for me, I learned about tracking, got hired by a self-driving car company and applied object tracking to autonomous shuttles in France.

This is why several years after my discovery of object tracking, and after my experience as a Computer Vision Engineer working on object tracking in the self-driving car field...

I decided to pack my knowledge into an online course to teach engineers how to solve real-world problems like autonomous shuttles company have...

...But also build far more valuable skills in the market than mere "object detection".

For over two years, this course has been a best seller and received fantastic stories and feedback.

Here's for example an email from Hutch that was sent within the first hour of following the course:
—

—
I had lots of stories, from students who started with simple and basic 2D Object Detection skills and got hired as self-driving car interns just a few months later...
... to engineers who applied object tracking in their companies and built entire teams.

But my favourite was from an engineer named Ovidiu who reported to have doubled his salary using this course!

This course has been my first product during the pandemic, my first sale, and one of my best-sellers for two years... Until I decided to shut it down!

After two years of experience building dozens of online courses that helped students and engineers from everywhere on the planet double their salaries, get their dream job, and even build their own startups, I had to revamp this course and create a version that would not only include all the research and discoveries since it was first created, but also make it easier to understand, funnier to watch, considering the feedbacks I got for two years, and help engineers even better with their dream job search.

So please meet the new version: MASTER OBSTACLE TRACKING!

With better content, code, examples, projects and basically everything... here is the program:

Module 1 — Object Detection

We'll start with object detection, as it is still a pillar for obstacle tracking. In fact, most of the tracking solutions are based on object detection: we first detect objects, and then track them from frame to frame.

Here's what's included:

An exclusive look at YOLO, the undisputed champion of object detection for over 7 years. (including an easy-to-get explanation of Darknet, Anchor Boxes, and Bounding Box Regression).
An introduction to the video analysis field (including 3 techniques the entire self-driving car industry mix-and-matches to build robust motion prediction systems)
The honest, no "bull crap" truth about Computer Vision engineer salaries (and why skills such as object detection may actually draw your salary down)
Why a 17 year-old object detection technique still survives and gets chosen by publicly traded companies today over cutting-edge object detectors (hint: it has something to do with explainability).
A complete description of the two-stage Obstacle Detection algorithms including R-CNN, Feature Pyramid Networks, SPPNet, Faster RCNN, Mask RCNN, and more... (A strong belief: Any Engineer learning Computer Vision should go through these concepts at least three times, we'll make sure to double down on this part)
The subtle difference between Spatial Pyramid Pooling and ROI Pooling (we'll do a detailed explanation and understand why the later is perfect for object detection).
CENTERNET: Introduction to keypoint-based object detectors (I'll also show you how to extend these techniques to direct 3D Object Detection and Pose Estimation)
An exclusive look at YOLO, the undisputed champion of object detection for over 7 years. (including an easy-to-get explanation of Darknet, Anchor Boxes, and Bounding Box Regression).
Why you should NOT include object detection on your resume if you're just out of the beginner zone and trying to stand out (and what to do instead).
How to answer the question "What is the difference between YOLO and SSD?" (true story: I was asked to answer this question during a self-driving car interview and failed to answer it)
Research Review: A thorough exploration of YOLOv4 and its advanced Deep Learning Modules (PAN, SAM, FPN, DropBlocks, ...)
An look into RetinaNet and EfficientDet, and how to use Spatial Pyramid Networks and Bi-FPNs in object detection.
The complete list of the object detection algorithms you are REQUIRED to know before going to an interview.
✅ Coding 3 "Hermione Granger" techniques to grab and run YOLO algorithm (v3, v4, and v5) in just a few lines of code.

Let's take a break for a minute and describe more what I mean with this last bullet:

👉🏼 In this course and when developing your projects, you will want to spend as little time as possible on object detection, and spend as much time as possible on tracking.

And your code must express this. You can't focus most of your time on tracking if half of your project is about object detection.

This is why I will teach you 3 techniques I often use to "grab" a quick object detector in less than 10 lines of code without setting up anything.

Using these techniques, you'll be able to run a state-of-the-art object detection algorithm whenever you need it, and focus on the more important parts, such as tracking, sensor fusion, 3D, ...

Module 2 — Multiple Obstacle Tracking

Once we handle object detection, we'll dive into object tracking.

Here's what's planned:

A thorough understanding of the Hungarian Algorithm, an almost magic algorithm that is extensively used by almost every self-driving car company in the field and research paper.
The 4 Magic Steps that governed the field of Multi-Object Tracking for 6 years (and the algorithm that made these 4 steps completely irrelevant in a minute)
4 Techniques to track and follow bounding boxes from frame to frame.
Why Intersection Over Union is a terrible association metric and what you should use instead (while IOU might be a good idea at first, notice how it would also match a vertical pedestrian box with a horizontal car, I included two surprisingly good techniques to counter this problem)
Live Coding of the most popular obstacle tracking research paper from scratch.
How to "tweak" object tracking and repurpose it for Sensor Fusion (this algorithm was originally developed for data association problems such as tracking, but with a simple change, we can convert it into a sensor fusion algorithm).
Exactly when to use a Kalman Filter during tracking (and examples of an algorithm running without a Kalman Filter).
Why an entire parallel industry has been created to avoid using Deep Learning in object tracking.
A Re-Identification technique I accidentally discovered that helped improve tiny YOLO's accuracy without reducing its speed.
A simple but effective solution to filter ghosts in your detections (even the best obstacle detectors return ghosts and fail, we'll see how to solve this problem using tracking)
✅ OBSTACLE TRACKING PROJECT: Implement the SORT Algorithm to track bounding boxes from scratch.

Okay, let's take a closer look at the project. Here is the algorithm you'll build:

You'll notice is sometimes fails, especially when objects quickly change shape or gets occluded.

This is because the tracking algorithm we'll code is not relying on Deep Learning.

It's also a technique we call "tracking by detection".

The tracking field is split into two families:

Tracking by detection, or
Joint Detection and Tracking

Although the first technique is a go-to for most students, the state of the art techniques now build models that automatically detect and track objects using HydraNets architectures.

In this second module (and project) of the course, we heavily focus on tracking by detection (the most popular and historical techniques are there), but in the 3rd part, we are also going to focus on novel techniques...

Module 3 — Deep Obstacle Tracking

The third module is entirely new and not taught anywhere else. Here's the program:

Unlocking the Research Secrets of Strong SORT, the state-of-the-art tracking algorithm in 2022 (we'll look into NSA Kalman algorithms, EMA Features, Motion Aware Cost-Functions, and more...)
.
How Deep Matching works (+ an in-depth introduction to Deep SORT, one of the most popular deep matching algorithms).
An Introduction to Siamese Networks and how to train them (we'll also have a review on One-Shot Learning and Face Recognition).
The simple maths we use to calculate distances between feature maps.
Why you should never replace IOU with Deep Metrics (and how to build a cost function that makes sense).
Exactly where to include Deep Metrics in your tracking project (and the sanity checks you need to implement to make sure your siamese networks don't negatively impact your tracking algorithm).
A simple trick to 10x the performance of a Deep Tracking algorithm (warning: it will only work with "cascade matching algorithms" like DeepSORT, but is super effective nonetheless).
A quickie "idiot's guide" to Joint-Detection & Tracking algorithms.
The most important (and most little known to engineers) object tracking algorithms.
✅ DEEP TRACKING PROJECT: We'll take your existing tracking project and convert it into a Deep Tracking project, that uses Deep Learning to identify and track obstacles.
The actual tracking algorithm I implemented and deployed in an autonomous shuttle.

Okay, you might get a kick out of this. Here is an autonomous shuttle running and displaying two algorithms I created: object tracking & road segmentation.

Today, we have even better algorithms, let me show you:

The Multi-Task Object Detection and Tracking algorithm that got so good it was able to beat the best object detectors like YOLO and RetinaNet.
How to use Graph Convolutional Neural Networks in Motion Prediction & Tracking (we'll do a study case based on one of the most valuable self-driving car startup out there).
A look at the HydraNet architecture that implements object detection, tracking, and re-identification (it supposedly even works if the object gets out of the frame and comes back later).
4 Ways to use object tracking to "smuggle" in the computer vision and self-driving car industry (we'll see how to modify a basic object tracking project and turn it into a gold-medal portfolio project that is almost impossible to ignore.
And many more...

✚ Bonus: The Multi-Object Tracking Mindmap

A 5 min Overview of the Entire Stack of Object Tracking algorithms we've explored in the course.

They made the switch to Object Tracking Engineer

Mohammad Anas Khan, Object Tracking Edgeneer

"I recently completed the updated version of your obstacle tracking course. Really liked it."

I liked that you gave a quick walkthrough of all the major 2D OD networks. I also liked the part where we used additional metrics and not just IOU for our hungarian matching. And yes, seeing the classical sort perform better than deep sort gave insights into how problems can still be solved using good classical CV approach. Lastly, what better than seeing all of this being put into action on a live video?"

Pranjal Shingale, Obstacle Tracking Edgeneer

"The amount of information provided is indeed large!"

And this is much appreciated! However, a little more detail would be nice. I enjoyed the fact that we get to read the latest papers on this topic and that there is a connectivity provided between past and present results. Thank you very much for your efforts in reading and breaking down these latest papers AND for starting from the very basics.

Mark Moawad, Self-Driving Car Engineer

"I couldn't hold myself back..."

"Despite financially burdening myself, I couldn't hold myself back from buying the revamped Obstacle Tracking course. I believe in you and I am pretty sure that your course (along with Adrian Rosebrock's PyImageSearch University, which was recommended by you BTW) will empower me to be one of the Egdeneers!

I am sending you this email to thank you for the inspiration, for the enriching knowledge you share, and for teaching me a lot on the personal level as well as professionally. Sorry for bothering you with a big fat email, but I believe it is a must to say thanks."

Before going any further, here are some things to consider before enrolling:

Are there any prerequisites?

Although I reteach the concepts of Object Detection in this course, I recommend coming with existing background about object detection, Deep Learning, and Computer Vision in general.

There is a basic underlying reason why we can get to get super advanced and study research papers from 2022 in just a few hours... and it's because you already have some prerequisites.

Therefore, even if the topic is quite general, and can be useful to TONS of engineers way outside of the self-driving car space, I DON'T recommend taking the course if you:

❌ Don't have any Deep Learning skills
❌ Have never worked on object detection
❌ Can't code in Python, and don't know how Notebooks work
❌ Expect a course about YOLO (sorry, but in addition to being totally useless, this is a course about tracking rather than detection)
❌ Already know about the SORT and Deep SORT algorithms (most of the course and projects is about these)
❌ Have a phobia of math (we'll study lots of research papers and talk about maths quite often).

On the other hand, this course has been built for people who:

✅ Feel like they are stagnating in Computer Vision and Deep Learning and don't really have anything impressive to show.
✅ Have the prerequisites I listed above.
✅ Feel a need to stand out from the ordinary object detection engineers and build unique and in-demand skills.
✅ Are okay (and even find this somewhat cool) to read research papers (or die trying)
✅ Are motivated by the algorithms actually implemented in the industry and not just what's popular on LinkedIn.
✅ Are looking for a short, straight-to-the point explanation about object tracking (the course is about 3-5 hours and I made by best to reduce it as much as possible, if you are the type who judge a course by the number of hours and only enroll in 75+ hours courses, then this one might not be for you).
✅ Want a setup free course that you can run from anywhere in the world (we'll do everything on Google Colab).

If you validate these conditions, then this course might help you build valuable skills and get hunted by self-driving car companies.

I say "might", but I highly believe it can, because a lot of my students have been able to find their dream jobs after taking and applying the concepts inside.

Some of them have done it via the "Prediction" path.

The self-driving car industry is recruiting Prediction Engineers with skills similar to what's taught in this course.

Over the past few years, this topic has become so important that:

Waymo's 3 Pillars are now Perception, Prediction, Planning.
Aurora mentioned Forecasting as their main thing.
Nvidia has built an entire architecture around object tracking.
And almost every self-driving car (and even robotics) company is working extensively on tracking and prediction.

A lot of engineers have found their dream job simply because of a mindset shift...

With object detection skills, you can't stand out.

Everybody is already doing it, and today, it makes you look like a beginner.

I was once shocked to see my super-non-technical manager run a YOLO algorithm after only 5 minutes of work. If he could do it, anybody can. The value is in what you can do with the boxes, and tracking is an incredible step towards building a valuable cutting-edge profile.

This means that it isn't about having one particular job, but just being a more valuable Computer Vision Engineer than the others.

What about you?

299€ or 2 x 149€

MASTER OBSTACLE TRACKING: Robust Perception Skills For Computer Vision Engineers

Lifetime Access to:

Object Detection: From sliding windows to advanced detectors
Object Tracking: In-Depth Exploration
Cutting-Edge Deep Tracking Algorithms used in the field

Plus:

The Multi-Object Tracking Mindmap

Buy Now