Think Autonomous

SEMANTIC VISION: Modern Segmentation Approaches for Self-Driving Cars

Learn segmentation the way modern architectures are actually built: as reusable blocks. Code UNet, PSPNet and DeepLab from scratch in PyTorch, then learn what to do with the mask once you have it.

Ready to order →
Semantic Vision course cover

Learn Image Segmentation through the blocks every modern architecture is made of

If every new Deep Learning architecture feels like starting over, and you want segmentation skills you can reuse in many fields, read on.

Image segmentation classifies every pixel of an image. It powers a long list of applications:

It's also one of the best ways to learn Deep Learning itself.

Computer Vision Engineers tend to plateau at one of 3 levels: Deep Learning, 3D, and Time. Most stay stuck at the Deep Learning level. They understand it, yet every new architecture feels like a new subject to learn from zero.

Segmentation is a good way out, because segmentation models use most of the blocks you'll meet in modern Deep Learning: encoders, decoders, skip connections, multi-scale context, attention. And many segmentation architectures are reused for other Computer Vision tasks.

New papers rarely start from zero

You can watch videos, follow tutorials, or spend hours on a paper until it clicks. It works, but you pay that cost again with every release.

Most research papers combine existing blocks in a better way. Take DeepLabv3+, one of the architectures we build in the course. It's an encoder-decoder that adds Atrous Convolutions and a Spatial Pyramid Pooling module.

Pyramid Pooling + Auto-Encoder = DeepLab

Put segmentation models side by side, from the classics to today's foundation models, and the same blocks keep coming back:

Pick a block to see where it's used

You build these in the course

UNet2015 · Semantic Vision
Encoder-DecoderSkip connections
PSPNet2017 · Semantic Vision
Atrous ConvolutionPyramid Pooling
DeepLabv3+2018 · Semantic Vision
Encoder-DecoderSkip connectionsAtrous ConvolutionPyramid Pooling
SegFormer2021 · Segformers DLC
Encoder-DecoderSkip connectionsSelf-Attention

Recent models you can then read

Mask2Former2022
Encoder-DecoderSkip connectionsSelf-AttentionCross-AttentionMask queries
SAM2023
Encoder-DecoderSelf-AttentionCross-AttentionPrompt encoder
SAM 22024 · images and video
Encoder-DecoderSkip connectionsSelf-AttentionCross-AttentionMemory attention

dashed the part that's new in the paper, the only part left to study

So rather than learning architectures one by one, learn to engineer the blocks

You already did it with activation functions.
You learned how the most used ones work, and now you understand any paper that mentions "ReLU" or "Sigmoid".

You did it with Convolutions too.
You learned about strides, pooling, maybe residual blocks, and now a paper that stacks convolutions with a stride of 2 reads easily.

This course does the same at a larger scale, with blocks. Once you know how they combine, a new architecture becomes a known set of blocks plus the part that's new, and that's the part you study.

The same skills apply to perception for autonomous robots, medical imaging, aerial mapping, or your own background removal software.

New architectures keep coming out, including Transformer-based ones. They are still assembled from blocks.

Let's see the program.

INTRODUCING...

SEMANTIC VISION: Modern Deep Learning Techniques for Computer Vision Experts

Learn Image Segmentation by developing an extensive understanding of Deep Learning blocks.

Semantic Vision course cover

Module 1

Intro to Image Segmentation

In the first module, we'll begin by understanding how to use image segmentation in self-driving cars, and we'll then learn about Auto-Encoders, the core architecture used in image segmentation.

Image segmentation on a driving scene

You'll learn

Wait a minute.
Let's pause on this one.

Most segmentation courses use self-driving cars as their example.

Very few explain what to do with the segmentation mask afterwards.
This question is much harder to answer, because it demands an understanding of the main system you're using image segmentation for. This is why we'll see how to use image segmentation masks in the context of self-driving cars.

We'll see how to use it in the Sensor Fusion module, but we'll also study novel architectures such as Bird's-Eye-View Segmentation.

With that, the first module also has

Module 2

Modern Deep Learning

The second module is purely "architecture". Once we know how to build autoencoders that can recreate images, classify every single pixel, we'll implement our first AutoEncoder.

UNet architecture

What you'll learn

Let's take a break.

Although this module explores lots of architecture, the first implementation you'll do is on the UNet module.

You probably heard of UNet before. It's the U-shaped network introduced in 2015 for biomedical segmentation, and it's still one of the most reused architectures today. If you understand the building blocks of this network, you have a first layer to then tackle the rest, and this is what we'll do.

You'll actually build your own UNet Architecture, and train it for a drivable area segmentation project, just like this one:

Drivable area segmentation with UNet
The PyTorch 101 Cookbook

Module 2 Included Bonus:

The PyTorch Cookbook!

Module 2 is about implementing a UNet with PyTorch. But you may not be used to PyTorch. And yes, it can be a blocking point!

If you'd like to understand how to learn the fundamentals of PyTorch...
... you'll get access to this bonus 15-page PDF I created just for the occasion to share the 3 most important PyTorch concepts to get started:

  • How to get rid of the confusion from Datasets & DataLoaders
  • The PyTorch way to define a model (with the __init__() and forward() functions)
  • What to know before training and running a model with PyTorch.

During the entire book, you'll get access to "minimal codes" of the things you must include at every step.

After module 2 is done, comes the cutting-edge...

Module 3

Cutting-Edge Segmentation

This third module is all about "blocks".
Once you know how segmentation works, and after you've built your first auto-encoder, you'll implement more advanced modules!

Segmentation outputs across network stages

You'll learn

The power of this module is that you won't build 1, or 2, but 3 complete architectures in total! And by coding 3 different algorithms, you'll have the chance to see some modules being repeated, and develop a good understanding of how to copy and paste blocks from one project to another to build just about any architecture.

And this isn't over yet!
Right now, let's answer some questions about these 3 modules, and right after, you'll see the Transformer extension of the course...

Yudhisteer in Mauritius: autonomous golf carts, autonomous drone tracking, image segmentation

AN EDGENEER'S TESTIMONIAL

"I was able to apply this to my own work, building autonomous golf cars in Mauritius"

"I am Yudhisteer from Mauritius and I have a background in Industrial engineering. I wanted to know more about Self-Driving cars and was searching for courses. That's when I discovered Jeremy's courses. Someone posted a video of an Image Segmentation algo on roads in Paris and I was completely baffled by it. It was based on a paper dated only a few years back and it was impossible to find a tutorial online to explain how to apply it. I enrolled on that day itself!

The good news is that I am not only taking Jeremy's courses for knowledge but I am also able to apply it in my work. Working in an industry with a lot of golf-cars around, I thought why not automate them. With Jeremy's guidance, I successfully taught my golf-car its driveable lane and the adjacent one. Truth be told, I planned on building the first autonomous car in Mauritius and perhaps in South Africa. Years ago this idea would have seemed far-fetched but now, I believe it is just some clicks away."

Yudhisteer Chintaram
Yudhisteer ChintaramImage Segmentation Edgeneer

Frequently asked questions

What are the prerequisites to follow the course?

I want to think of this course as the most accessible Computer Vision course on my platform. So if you understand backpropagation, if you're good with Python, and if you've already done other projects such as image classification, you're good to go!

How long is the course?

All my courses are developed to be as short as possible. So is this one! I don't want you hanging there 80 hours getting lost in blocks. So we'll cut down to the essential.

  • If you already know how image segmentation works, but need to learn more about Deep Learning blocks and advanced implementations, it should take you 3-5 hours
  • If you have never done any image segmentation before and have no clue what an autoencoder is, expect 8-10 hours before you're confident with it.
What type of job can I get with this?

In the course, we spend some time studying a job offer for Deep Learning Researcher in a healthcare startup in Paris. In this job, you'll build novel architectures to find fractures in X-Rays using Deep Learning. Cool job, right?

There are many other fields that use image segmentation, but you can consider that most Computer Vision jobs will be interested with this skill. And as listed above, image segmentation is something you can take to healthcare, construction, defense, or consulting for large groups.

Who is this course for and not for?

Well, first, it's definitely not for you if you're not already a bit used to Deep Learning.

❌ If you have never trained an image classification network, or even object detection, you should do that first.

❌ Then, if you absolutely want to learn one architecture, this course won't be for you. It's all about combining multiple blocks to build an infinite number of architectures.

✅ Put differently, this course is for those who want to learn about the game of Deep Learning, rather than one single algorithm.

✅ It's for those who are motivated to build cutting-edge segmentation skills, that can be applied in multiple industries.

✅ And it's for those who want to understand how to use segmentation in the self-driving car space. For those who wonder "what's after the segmentation output?"

If you're in either one of the 3 cases, then it is for you.

How far does the course get, technically?

I'm glad you asked.
This course you've just seen is the "core" version of Image Segmentation. But I wanted those who really want to master the latest research papers to have a block named Transformers.

Transformers are too big a topic for one module. They need a course of their own.

This is why I built another course, and I'm plugging it as an exploration (DLC) right after this segmentation course.

It has been built to be followed right after the image segmentation course, let's take a look.

DLC

THE SEGFORMERS EXPLORATION

A deep dive into Attention, Transformer Networks, and Vision Transformers applied to Image Segmentation.

The Attention Mechanism, taught for Computer Vision Engineers

The Transformer started in NLP and is now a standard block in Computer Vision, from Vision Transformers to segmentation models like SegFormer. This DLC takes it apart block by block.

What's included

  • An introduction to the Attention Mechanism and Spatial Transformers
  • What is an Attention Mask, and the main difference with Convolutions Feature Maps
  • How Transformers work; and the 3 main differences between Transformers in NLP and Transformers in Computer Vision
  • A step by step walkthrough of Query, Keys, and Values + A Workshop Example to visualize the query, keys, and values!
  • 5 Common Questions about Attention and Transformers visually answered (this will explain self-attention, cross-attention, masked-attention, and 2 other questions)
  • How to code a Multi-Head Attention Network and Visualize what each head is focusing on (you got it, we'll be very, very visual in this course, so you will get to see all Attention heads at every stage of the network)

Let's see what this is about:

Attention heads visualized at every stage of the network

But it doesn't stop there, we'll also learn:

  • What is a Segformer Network, and a breakdown of the Encoder and Decoder parts (with a research paper breakdown)
  • A detailed explanation of the main blocks used in Segformers; including Efficient Self-Attention, Overlap Patch Embeddings, Mix FeedForward Networks, and more...
  • ✨ Research Paper Breakdown: We'll go over 3 Research Papers: Attention is All You Need (Transformers), An Image is Worth 16x16 Words (Vision Transformers), and SegFormers.
  • How to stack several Transformer Blocks together (and the difference between blocks and stages)
  • A "from scratch" workshop on Segformers to solve the full-frame image segmentation problem.
  • Drivable Area Segmentation with SegFormers: the SegFormer workshop has been adapted to the Drivable Area Detection workshop of the Image Segmentation course, and you'll get access to the code to compare the 2 problems.
  • and many more...

FROM THE EDGENEERS

Testimonials

"The price could have stopped me from buying the course, but it didn’t! I got in and found a good content that was well designed."

I like how jeremy explain some concepts. The courses explain well the content and it is not easy to find in another webpages. I also like the pipeline courses, and that Jeremy creates courses with top industry concepts.

Yes, the price can be high, but if you have the money the course is well explained and it have good content (taking in to account that I am from south america)

Something to add: I like how jeremy create the stories and connect with the concepts.

Jeremy just creates top content during all the year!

Walter Mayor
Walter MayorImage Segmentation Edgeneer

"Better than Udacity, Coursera, and edX."

"I currently engaged in image segmentation and SegTransformer course. What can I say about your course? It’s just an awesome coursen, better than udacity, Coursera and edX.

Before taking the course, I had some doubts about whether I could make the career transition from automation to computer vision.

However, after the knowledge from your course, I have gained great confidence in the field of computer vision."

KdahanImage Segmentation Edgeneer

"I followed the Segmentation course. It's well structured, and we have every resource necessary to get results and even adapt it to other problems."

LinkedIn post: driveable area segmentation results on a Paris road
Staberlin Valentain
Staberlin ValentainImage Segmentation Edgeneer

"He completed the course and got an internship at Bosch company."

"After graduating from college with a data science degree, he [my grandson] couldn't find a job. So I bought your courses for him to learn to further improve his ability. He completed "IMAGE SEGMENTATION" course and then he got an internship at Bosch company. Thanks to the " IMAGE SEGMENTATION" course, he is doing well on a project about segmenting photos of company equipment."

Ton Quang Toaigrandfather of an Image Segmentation Edgeneer

PICK YOUR ADVENTURE

What will your Image Segmentation Journey look like?

*For all courses, 2-time payment plans are available at no cost

Semantic Vision course

CNN BLOCKS

SEMANTIC VISION

  • SEMANTIC VISION (299€)

299€

Enroll Now
Segformers Edition: Semantic Vision + Segformers DLC

CNNs + TRANSFORMERS

SEGFORMERS EDITION

  • SEMANTIC VISION (299€)
  • SEGFORMERS DLC (199€)

399€

Enroll Now