Self-Driving Car Engineer Curriculum - From Beginner to Professional

Where should you start?

What is the content that you should absolutely do? What is the one you should skip?
Here is my curriculum for aspiring self-driving car professionals.

It is divided into 2 parts:
  • FOUNDATIONS - What are the coding skills and math level necessary to join the field?
  • SPECIALIZATIONS -  What are the skills that you will use in your day-to-day? What are the "project oriented" resources we should look into?

Step 1 - Common Knowledge

First and foremost, you need to validate some basic coding and maths skills. Here's what's needed.
  • C++ - The Main Language
  • Python - Useful to have
You shouldn't spend too much time on it, especially if you already have programming skills.
👉 Complete Python Bootcamp: Go from Zero to hero in Python 3
Python Programming Masterclass

  • Basic Linux Command Lines
👉 I'd say it's not necessary to take a course on this, learn while doing your projects and force yourself to use a Linux system.

These were languages, the other very important thing is maths.
  • Intermediate Probability
  • Intermediate Calculus
  • Intermediate Linear Algebra
👉 You can learn these on Khan Academy

Step 2 - Self-Driving Car Background

Once you know how to code, and once you know a bit of maths, you need a Self-Driving Car Background.

  • Autonomous Systems
👉 Follow my course SELF-DRIVING CAR: The guide to learn cutting-edge technologies that will change our lives for a detailed introduction that uses my field experience.
It is the only introduction you'll need.

  • ROS (Robotic OS) 
👉 ROS tutorials are a great help if you want to learn about the OS behind self-driving cars for free. You will work with command lines, and learn to build concrete projects.

  • Artificial Intelligence
👉 I highly recommend Artificial Intelligence for Robotics by Udacity for a practical free introduction to self-driving cars.

Specialization - Computer Vision

Computer Vision is a big part of self-driving cars, you'll need knowledge in AI, Machine Learning, Deep Learning, and image processing.

There are hundreds of Computer Vision courses out there; and a lot of them use the "self-driving car example". It doesn't mean it will be helpful on that topic.
Here's what you need to know and where to learn it.

  • Image Processing with OpenCV
  • Image Transformations with OpenCV
👉 PyImageSearch tutorials will help you a lot with OpenCV in general.

  • Camera Calibration with a chessboard
👉 Any tutorial like this one is enough as long as you understand the necessity to calibrate and manage to do it.

  • Traditional Computer Vision: Feature Detectors and Descriptors
  • Traditional Computer Vision: Feature Matching
  • Geometric Projections
  • Feature Tracking
👉 Sensor Fusion Nanodegree by Udacity, Module 2
👉 The course Visual Perception for Self-Driving Cars, by Coursera, covers it too; but I didn't try it and I don't know how much practical it is.

  • Machine Learning - the basics
  • Convolutional Neural Networks
  • Image Classification
  • Object Detection (2D, 3D)
👉 Deep Learning Specialization by Coursera (CNN module, or the whole course) is a great way to start.
👉 Computer Vision Nanodegree, Udacity

  • Object Tracking
  • Kalman Filters
👉 This notebook can help you!
👉 Kalman Filter Youtube Series
👉 Computer Vision Nanodegree, Udacity 

  • Image Segmentation

Specialization - Perception

Perception is the main field that includes Computer Vision. Here, I will describe all other tasks.

  • LiDAR Point Cloud Processing, Segmentation, Clustering
  • LiDAR Camera Fusion and 2D Projection
  • RADAR Processing, Detection, Speed Estimation

  • Kalman Filters (Linear, Extended, Unscented)
👉 This notebook can help you!
👉 Kalman Filter Youtube Series
👉 Sensor Fusion Nanodegree, Udacity
Sensor Fusion

LEARN KALMAN FILTERS: The Hidden Algorithm that silently powers the future
is my own Kalman Filter courses on obstacle tracking in Computer Vision!

Specialization - Localization

There are much less available resources on Localization than any other topic.
Here's what I found to be helpful.

  • Kalman Filters (Linear, Extended, Unscented)
👉 Kalman Filter YouTube Series by Michel Van Biezen

  • Monte Carlo Localization
  • Hidden Markov Models
  • Mapping
  • SLAM

Specialization - Motion Planning

Motion Planning means taking your vehicle from A to B. It mostly involved Artificial Intelligence foundations, and some reinforcement learning for research only.

  • Artificial Intelligence - Search and Path Planning
  • Reinforcement Learning

Specialization - Control Command

This could be much better, here's a starter kit.

  • PID, MPC, LQR Controllers

Make this Curriculum Better

There are hundreds of courses. I tried to included what I think are the best and most project oriented ones.

If you think you can find more content, please send me an email and we'll improve it!

Good Luck!