What is image segmentation?
What are the different types of segmentation?
How can you use segmentation in your everyday life?
Let's take this group of sheep.
When you first learn about Deep Learning and Computer Vision, your first task is to predict what's in an image using Convolutional Neural Networks (CNNs).
If you need to classify an image between a cat and a dog, this is multi-class classification. If your image can contain both a cat and a dog, this is multi-label classification. A classification network is a set of convolution operations and a fully-connected layer for classification. Resnet or VGG are examples.
When you dive deeper in Computer Vision, you learn object detection.
Instead of predicting a label per image, we predict bounding boxes for each object. Hence, we can detect and localize all 3 sheep. A lot of detection algorithms now use regression to predict the bounding box coordinates.An object detection network can be one of the two families: Region proposal or Single Shot Detectors. Algorithms such as YOLO or FASTER RCNN are well-known.
Finally, we have semantic segmentation and instance segmentation.
The problem is different. We want to classify every single pixel in the image. As you can see, the output is also an image. An image where each pixel has a color and represents a class.
Instance segmentation is different because every sheep is in a unique color.
Semantic Segmentation networks are a set of encoder-decoders that uses specific operations, different than convolutions, to recreate an image. UNet, DeepLab, or FCN are popular algorithms. Instance Segmentation is again something else. It combines object detection and semantic segmentation. Mask-RCNN is an example.As we get more advanced in the analysis of our image, semantic and instance segmentation are one of the most final things we can get. Every pixel is classified. The scene is completely understood.
What's the point? Semantic analysis has a lot of advantages, especially in the autonomous world.
The most known use case is to use segmentation to predict the driveable area.
For a self-driving car, it can help detect sidewalks, lane lines, parked cars, and understand where it can drive. Most self-driving car companies use it.
With the right dataset, we can even train the network to understand traffic regulations and mark an adjacent lane as "forbidden".
It is useful for self-driving cars, but even more for autonomous robots.
Indoor robots like Vacuum cleaners can use this to avoid obstacles, detect crumbs, and navigate the environment.
I had an amazing experience with Aerial Images.
A couple years ago, I worked for a consulting company involved in a big competition for the autonomous plane of the French Army in the future. The goal was straightforward: develop a plane that can be fully autonomous. Although I cannot reveal more, one of the features the company was looking for was aerial image recognition.
Another application of aerial analysis is farming.
Satellites and drones today can segment precisely agricultural regions and act accordingly.
Segmentation is becoming the most popular algorithm in medical imaging.
The U-Net algorithm even got released with the name "Convolutional Networks for Biomedical Image Segmentation."
Thanks to this use of technology, we can detect lung cancers and their sizes faster than any radiologist.This use can even be extended to dental X-Ray images.
Radiology with Deep Learning is using Semantic Segmentation.It doesn't stop there, it works in 3D!Semantic Segmentation is very popular for RGB images, but try to imagine what we can do using this with 3D point clouds?The conversion problem is directly solved.
Instead of wondering: "How are we going to use these pixels in the real-world?"
We directly have the pixels in 3D.
The RANSAC algorithm is very popular for this type of task, but today neural networks can do the task!
I created a training course on semantic segmentation.
It is unlike any course. I followed a lot and I always got out with more questions than answers.
Let's not do this!
In this course, you will get out with no questions, and a complete understanding of cutting-edge algorithms used today. Every convolution used, every weird name you couldn't understand, everything.
And as it is the case with One Step Ahead, you will leave the course with a project.