• 3 Reasons why you can't use Convolutions with Point Clouds, and the only type of Convolution that can be used to learn features directly on Point Clouds
  

• The Self-Driving Car  Engineer's most common ways to process point clouds with Deep Learning (at least one involves converting the point cloud into a voxel representation; we'll learn all about it)
  

• An in-depth overview of the 3D Deep Learning world, from meshes and video games to point clouds
  

• The difference between euclidean and non euclidean 3D Data, and why it changes the entire algorithm you need to use
  

• My Top 8 Libraries to work with 3D Data, including for tasks like 3D Classification, 3D Object Detection & Segmentation, but also 3D Reconstruction, Structure from Motion, and Augmented Reality.

• A quicky way to load and visualize any point cloud on your desktop (and what you need to do if your point cloud is in Binary format)
  

• A Deep Dive into the PointNet Algorithm, a pioneer in 3D Deep Learning, often considered most important algorithm in 3D Deep Learning
  

Now, when I say this, I mean it.

You are going to understand this algorithm top to bottom, because it's a foundation of the 3D Deep Learning world:

In this context, we'll also learn:

• The loss function of a point cloud classification network, and how to adapt it for another task
  

• The 3 Rules when Processing Point Clouds with Deep Learning (one of them involves invariance — a point cloud is random, and the order of points can be changed, unlike pixels)
  

• The real reason PointNet isn’t good enough for large point clouds data, and how PointNet++ solves it
  

• An Intro to Spatial Transformer Networks — A key algorithm in Point Cloud processing and canonical learning
  

• How to tweak a 3D Network for 3D Semantic Segmentation with Deep Learning (and why most voxel based approaches can’t do it)
  

• (BONUS) The PyTorch Cookbook — A 16 Page PDF showing you how to handle PyTorch in 5 minutes
  

• Spatial Encoding: How to design a 3D Deep Learning algorithm to learn a specific shape; and how to tell the difference between two similar shapes
  

• How self-driving cars manage to find and classify objects, even though they only see a few part of it (often, the point clouds are one part of a car, never the entire 3D model)
  

• PROJECT 🔥: How to implement, train, and run a point cloud classification algorithm from scratch with PyTorch

Ok let’s pause a minute. This first part is all about using point based approaches, and PointNet is the best example of it. If you understand PointNet, you’re halfway through understanding most point based 3D object detection and segmentation techniques.

In our case...

•  An Introduction to 3D Voxelization, and 2 techniques to convert a point cloud into a voxel grid
  
  

• The difference between Occupancy Networks and Voxel Networks, and why most Occupancy Networks are overhyped
  
  

• The one reason people choose to convert point clouds into Voxels, and the biggest drawback of voxelization networks
  
  

• A « real life » insight into how we use Voxel based approaches in self-driving cars
  
  

• The one mistake even LiDAR Professionals make when working with voxels (
  
  

• Exactly when to use a 3D CNN, and how many layers are good enough to build a performant algorithm
  
  

• A very important (but almost unknown) « trick » to do in your convolutions to avoid running convolutions on empty voxels (90% of the space is empty, but because of voxelization, a standard convolution must multiply their 3D kernels with 0 most of the time, we’ll see how to avoid it using Sparse computing)
  
  

• How many voxels you should have when working with a self-driving car dataset, and how many points should they contain
  
  

• An introduction to Sparse Computing, and the Step by Step implementation of Sparse Convolutions
  
  

• The subtle difference between sparse and submanifold convolutions, and why both are needed in most networks
  
  

• PROJECT 🔥: Voxelize an environment by building 3D Voxels on top of 3D Point Clouds

Something I want to talk about is how we won't just do something classical and boring here, but we'll really think about how experts do.

For example, we won't just look at 3D Convolutions, but we'll have an entire module on Sparse 3D Convolutions, which are used in Point Clouds, and even Submanifold 3D Convolutions. The understanding of this difference is usually not taught, but if we want to "get" 3D Deep Learning, we'll need to really dive deep into these concepts.

Similarly, in the second project, you’ll build your voxels manually, which will help you develop a better understanding of how voxels work. 

But there’s more: 

• An exclusive look into the 3D Convolutions, and the Feature maps learned from them
  
  

• Occupancy Networks: How to build 3D Occupancy Grids using Sparse Voxels (the algorithm behind it is also used by Tesla’s team in their own Occupancy Networks)
  
  

• The difference between 2D and 3D CNNs, and an animated view of how 3D CNNs process voxels
  
  

• The implementation of the MeanVFE function, the feature encoding layer used in the State Of The Art algorithms in 3D Object Detection
  
  

• How to achieve true mastery of 3D Deep Learning
  
  

• How to tell if a company is using Voxels or Point based approaches in their Neural Nets
  
  

• PROJECT 🔥: How to build and train a 3D CNN that loads a point cloud, converts it to a voxel, encode and extract features, before running classification

This last project will help you build not only a strong understanding of voxelization, but the very algorithm you’ll build is also the backbone used by PV-RCNN, VoxelNet, and other state of the art 3D Object Detection techniques.

When you go through it, you'll have learned very strong skills in 3D Deep Learning, whether in the self-driving car space or not!

Now here is where it gets interesting:

After I launched this course, I got fantastic reviews (this course has been the most popular — even though it's closed most of the year)...

But I also got asked several times "Can you add lessons on 3D Object Detection??? Something that would go really in-depth?? So we understand how to build the bounding boxes in 3D? Not just use networks, but actually code them..."

This felt impossible. 3D Object Detection is a HUUUGGGEEE topic... but on the other hand, it made sense to learn it after you've build the skills in Deep Point Clouds...

If you want to become THE Apex Predator on LiDAR Point Clouds, you will also need to know the advanced 3D Object Detection approaches.

This is why I spent a lot of time on the topic, and then released what is NOT a lesson, but an ENTIRE COURSE JUST ON 3D OBJECT DETECTION!