3 Insights from Autoware's Transition to End-To-End Learning with Samet Kütük

Did you ever wonder... why are self-driving cars taking so long to come? I had that question too when starting, and my first answer came from Sebastian Thrun, godfather of self-driving cars, who talked about reaching 90% of use cases easily, but then finding huge difficulty in going from 90% to 100%. Recently, Andrej Karpathy, former Lead of Tesla Autopilot described something similar as "the march of 9s":

"When you get a demo and something works 90% of the time, that's just the first 9 and then you need the second 9 and third 9, fourth 9, fifth 9..."

This is what's taking long, but instead of focusing on this, tons of companies lose time focusing on the first 0-90%. Back in 2017 or so, we were all trying to get to 90%, and for this, we were re-developping all the software, algorithms, and so on... At some point, probably 30 startups were all spending millions developing the exact same algorithms.

This is when Autoware comes in the play. Started by Tier IV, Autoware is an open-source self-driving car software that allows you to achieve the first 9 in just a few weeks. Rather than re-developping yet another version of the same code, you jumpstart from the existing state of the art, and finetune it for your needs.

This month, our membership The Edgeneer's Land is welcoming Samet Kütük from The Autoware Foundation.

And let me start with a small snippet about how he defines Autoware:

Together, we recorded a new Fragment of The Edgeneer's Land, my community membership experience, in which he takes us through the building and management of Autoware. How does it work? How do you build a self-driving car with a full remote team? This is everything Samet teaches in our new fragment...

In this post, I'd like to give you a small sample of that interview, highlighting a very interesting moment where Samet talked about End-To-End learning...

3 insights from Autoware's End-To-End Learning Transition

Since its creation, Autoware has been implementing a "robotic" architecture, meaning implementing the traditinal "4 pillars": Perception → Localization → Planning → Control.

But recently, Autoware announced a new plan to evolve to an End-To-End architecture, a single neural network that takes in the input sensor data, and automatically outputs the steering angle and acceleration value. I have a complete article explaining the differences with detailed examples here.

4 Pillars vs End To End: How to pick an autonomous vehicle architecture

How to design an autonomous vehicle architecture? Should you implement an End-To-End solution, or a more traditional one? Let’s see…

Read from the most advanced autonomous tech blogJeremy Cohen

So here is the sample I'd like to share:

As you can see, there is a lot to uncover from just one minute. Let me share 3 highlights from that:

1. Level 5 is NOT easy to reach, may not even be possible, which is why Autoware focuses on Level 4+, in which humans, while not asked to takeover the car, could be asked to drive under regions or conditions that aren't appropriate.

2. The Transition will NOT be achieved immediately, but rather then result of several steps:
   1. Current — Starting from a traditional Robotic stack
   2. Step 1 — Learned Planning
   3. Step 2 — Deep Perception & Learned Planning
   4. Step 3 — Monolythic End-to-End (single network)
   5. Step 4 — Hybrid End-To-End using a "guardian" for redundancy

This can feel similar to how Tesla did their own transition to End-To-End (which I cover in this article:

Breakdown: How Tesla will transition from Modular to End-To-End Deep Learning

It’s no secret, Tesla is going to use End-To-End Deep Learning. But how? What will it look like? Will the Occupancy Network and HydraNet stay? Here’s a full breakdown…

Read from the most advanced autonomous tech blogJeremy Cohen

The difference between Modular and Monolythic End-To-End has been explained in their GitHub repository talking about the new planning algorithm:

Alright, let's continue with a third and final idea:

3. The algorithms for End-To-End have already been built.

We are not talking about a distant future, according to Autoware, it's possible to achieve End-To-End with today's algorithms, including (but not limited to) CenterPoint as the 3D Deep Learning algorithm for LiDAR Detection, AutoSeg as the Foundation Model in Perception, AutoSteer and Diffusion Planner for the Learned Planning approaches.

See? They are already there, and even though 2, 3, or 5 years from now, they may evolve and get replaced, the logic of Modular End-To-End (Step 2) has been implemented.

For example, the AutoSeg algorithm is a "HydraNet" that has a single backbone that split into several heads for lane lines, ego path, free space, segmentation, objects, and 3D. The outputs of these heads are then passed to the deep planner.

Summary & Next Steps

• Autoware is an open source self-driving car organization that builds a self-driving car software used over the world by thousands of engineers and teams
• Autoware is the solution I recommend to get started in self-driving cars; rather than building a software from scratch, get Autoware working quickly, and then finetune and customize it for your applications.
• Autoware is transitioning from a robotic architecture to an End-To-End Learning architecture, and there are 3 highlights from it:
  • It won't reach Level 5, but a Level 4+ that can drive almost anywhere
  • The transition will happen in 4 steps, adding planning, perception, then turning into a monolythic architecture, and finally hybrid.
  • The algorithms and modular logic have already been implemented and are working, such as CenterPoint, AutoSeg, or Diffusion Planner.

Next steps

Interesting in getting access to our Autoware Fragment? It's going to be very cool, and feature several things, such as:

• The Full-Length interview with Samet on Autoware
• An even deeper dive on Autoware's End-To-End Transition (this was just a 1 minute video - we do it for the full section on End-To-End).
• A complete breakdown on many algorithms used by Autoware, and a near plug & play solution to start running Autoware's software on your computer by tonight

Intersted? Apply to the Edgeneer's Land at the button below before October 25, 2025 (use the direct link if you're already a pre-approved client).

Fragment #13: Open Source AV Secrets

Exclusive Interview with Autoware - Autoware E2E Transition Full Breakdown - Autoware Platform Run & Algorithm Dive

Learn More about Fragment 13