Foxglove: Why the future of Physical AI is collaborative

Ever heard of Spider-Man's "Guy in the Chair"?. He's called Ned, and he's Peter Parker's best friend, but he's also the one who helps Spider-Man and whispers to him what to do when he's on a mission against the Vulture and the other bad guys. Batman has Alfred. Iron Man has Jarvis. All superheroes have someone or something that supports them while they're busy fighting the villains. Don't they?

In Robotics, we may be fighting the Perception & Planning villains, but what is our support system helping us achieve that?

Often, it's tools.

Every time I go visit a company, for example for a seminar, they show me some of the internal tools they've built. Most of the time, it's things they use every day and have invested a lot in. Many times, I've noticed companies were putting their entire robotic infrastructure on a particular tool provider: Foxglove.

Foxglove is your guy in the chair. It's a platform you use for visualization, testing, data management, and robotic workshops. Companies like Wayve, Shield AI, Gatik, and NVIDIA use Foxglove every day, especially because of a specific feature I will call "Multi-Player Mode".

Before we get to this, I'd like to give you a quick intro to the platform, so let's see:

There are a few things that I learned from Roman that I'd like to share with you:

• Can Foxglove replace RViz?
• Why Multi-Player Robotics is the future of Physical AI
• What being a CTO really is about

I. Can Foxglove replace RViz?

Traditionally, companies have used ROS (Robot Operating System) to build their self-driving cars, along with its free visualizer, RViz. As great as RViz can be, it has many drawbacks that a free and open product can have:

• Very simple viewer (only one 3D view, no side-by-side viewer with other plots etc.)
• Only on ROS with Linux (can't share a problem with managers who don't have ROS on their computer and are comfortable with it)
• Doesn't evolve (AI, robots, software)
• Single player mode (no collaboration, insight sharing, comments)
• No new or advanced features (data retrieval, querying, marking)

Looking at it this way, RViz feels very 2017-like; but these days. Wouldn't it feel normal to simply "ask" your data for some events and be able to retrieve them?

When working with free tools like RViz, you get a free visualizer, but you never get access to much. Foxglove, on the other hand, has a team funded and working every day on making these tools better.

The industry has a lot of discussions on "Saas vs Vibecoded"; I think when you experience Foxglove, there is no comparison to be made. A tool like RViz, or a custom visualizer, can work for one task, but the depth and scope of what you can do (and what you will be able to do in the future) is unmatched.

A note on MCAP Files

Foxglove has built something called MCAP (em-cap). It's a file format that replaces the previous multimodal log formats used by ROS. They benchmarked it against what ROS 2 was using at the time (SQLite-based bag files) and took the results directly to the ROS technical governance groups.

Results were so good that MCAP became the default recording format in ROS 2. Run ros2 bag record today, and you're using a Foxglove-designed format. This means when you go with a platform like Foxglove, you aren't following something; you're part of the leadership.

This brings me to where they lead too: Multi-Player Robotics.

II. Why Multi-Player Robotics is the future of Physical AI

Sebastian Thrun, known as the godfather of self-driving cars, once shared a story about a major problem his team faced during the DARPA Grand Challenge in 2005. Their self-driving car was able to drive through the desert for hours, perfectly following the destination. But during the long drives, the car sometimes detected a gigantic obstacle getting closer and closer and slammed the brakes.

Unexplainable.

Yet, it happened a certain number of times. So they ran the car again and again for hours to see the bug of the giant obstacle appearing in the middle of the desert. At some point, someone noticed this problem always happened after exactly a specific number of minutes, which led them to the solution (something about a timestamp refresh, from memory).

This would never happen with a tool like Foxglove, because we wouldn't need to re-run the car physically for hours and hours. What would happen is:

• We'd record the events to a file
• We'd replay the recording in the browser
• We'd share the problem with other engineers, some watching from other countries
• We'd simultaneously plot certain information, mark interesting comments until we debug the problem

Put differently, we wouldn't be in "Single-Player" mode, but in Multi-Player Robotics Mode.

Foxglove's real strength is collaboration. They enable Multi-Player Mode for you. You can share a recording with someone else who works on a Windows computer and has no idea or desire to install your exact version of ROS and Ubuntu. They can collaborate with you, tag important moments, and do a range of tasks so you jointly visualize and solve the problem.

Roman talked the most about events. While reviewing data, an engineer drops a timestamped annotation right on the timeline. "The foot was out of alignment" to use my own example from the demo.

Teams that never move past a single laptop and local files don't hit this wall at all. It shows up the moment there's more than one person who needs to look at the same robot's brain.

III. What being a CTO is really about

As I run monthly interviews with self-driving car and physical AI companies, I sometimes get to speak with executives, CEOs, CTOs, and Directors of Engineering. You can learn a lot from them.

One question I asked Roman was "What is different in a CTO rather than an engineer?"

He gave me a lot of interesting answers, but one quote in particular stuck with me.

He then went on to explain how he's spending A LOT of his time in Slack/Discord groups of Foxglove users, just seeing what people struggle with, need help with, and ask questions about... to shape the product!

This is something engineers rarely do. A typical engineer, especially when young and inexperienced, would see tasks like support and validation as "beneath them". When you're in startup mode, you quickly learn that nothing is beneath you. I have changed tires, fixed windshields, and done all sorts of silly tasks an engineer isn't supposed to do because I was in a startup.

As a CTO, you need to care extremely about the market and can't work in a silo. You need people to get value out of it.

On that last "value" point, there is a complete video in the Behind-The-Scenes of our interview that I absolutely recommend for figuring out where the value is in a product.

Summary & Next Steps

• Physical AI isn't just algorithms; it's also tools. Countless companies develop tools internally for tasks like visualization or data management. Foxglove developed them for you (and keeps on improving them so you can focus on the algorithms)
• RViz is a great visualization tool, but it has the drawbacks of a free visualizer. It's never improved, hasn't been enhanced with new features, and is stuck in a 2017-like world. On the other hand, a platform like Foxglove is continuously improved to stay current.
• Foxglove pioneered the MCAP format, which is now the default file format in ROS 2. When you record a scene, everything is stored in an MCAP file, which Foxglove can visualize directly.
• The future of Physical AI and Robotics happens in Multi-Player Mode. When using a tool like Foxglove, MPM lets you share recordings with others in their browsers, mark events, and collaborate on a problem rather than fixing it alone.
• A CTO tends to care about customers in ways other roles may not. They spend a lot of time reading where users struggle to anticipate the next features. They are constantly in touch with their market; it's about the market first and the features second.

Interested in the full workshop?

Here's what you get in this training:

• You will understand how Foxglove works (you can add that to your resume) and have the skill of turning everything into a Foxglove file (so any project, portfolio demo, dataset, anything… thanks to our bonus workshop).
• You'll be introduced to MCAP files, the now-default format used in the industry, and have been through the SDK to build your own
• You will understand how CTOs think, and the 2/3 critical mind-shifts engineers lack that CTOs have
• You'll get valuable career and hiring insight from a working CTO (what it actually looks like to lead technically while staying hands-on, what he screens for when hiring engineers at an early-stage startup, and how he thinks about fundraising a $40M Series B)
• Plus a lot of other bonuses included…

Get access to this, PLUS my 2 past fragments here:

https://www.thinkautonomous.ai/the-edgeneers-land

(P.S: This offer may not be available by the time you read this, make sure to join the membership while it's live)