A lot of people recently reached out to me wondering if self-driving cars can work in rainy or snowy conditions.
You might see a lot of videos of self-driving cars riding in California or in sunny environments
These environments usually also have very few pedestrians and cars.
The ideal condition for self-driving cars
Maybe you have wondered the same question
What if it rains or snow?
No one seems to talk about this
Companies will communicate on their upcoming improvements
But is it technically feasible?
The answers seem to converge to a future yes
But if it’s possible, why has no one succeeded yet?
Sure we see some videos on YouTube showing it can work
But again, we don’t know what the result will be like in real-life
Our opinions seem to lead to a no.
In general, self-driving cars are made in 4 main parts. Localization Planning Control
Instantly, what works well?
We can assume localization can work assuming the GPS reception is good.
In the case of a storm or heavy clouds, it might not be the case.
For a GPS with RTK (Real-Time Kinematic) correction, clouds often lead to a float number instead of a fixed number
What does it mean? That we can’t really trust it.
localization is 50/50
We can also assume planning/navigation works well
If localization is good, there should be no problem guiding the car to a specific route. AI algorithms are still the same.
A* (A-star) algorithm is often used to plan a trajectory.Planning works , a finite-state machine is used to handle specific situations like overtaking a vehicleControl can work , but we should be careful of slippery roads
What about Perception?
Perception is another story
It is what causes problems to the functioning of self-driving cars in rainy or snowy conditions.
What is perception about again? Sensors
What sensors are used?
Depending on the strategy or the company, it will use cameras, LiDARs, and RADARs. Radars work under rain or snow, but the sensor is not really trustworthy alone.
It is generally noisy by essence but can see through obstacles, and through fog or other bad conditions.
LiDARS use laser and light to sense obstacles.If it rains or snow, impact points will appear everywhere around the car
Because the laser might touch every drop of rain and snow it crosses.
The last sensor we can use with the RADAR is the camera.
Technically, a camera simply retransmits what we see with the eye.
And humans are able to drive on the road in heavy conditions.
I wanted to see if it would really work in a condition where no line is visible and no clear road is defined
I ran a custom freespace algorithm I trained to detect drivable areas.
Here is the result.
It is quite impressive
It can detect my lane even if the camera is filled with snowflakes…
… and the wipes are constantly on…
The next step was to see if obstacle detection would work
And the result didn’t disappoint again
What does it mean?
Self-driving cars rarely rely solely on the camera
So it’s unlikely that we would assume self-driving cars can work under rain or snow based on these two pictures
The camera has a lot of potential
Companies like Tesla are betting big on this sensor
It is the closest we have to human eyes
Some scenarios like Fog will still be a big issue
Making the driving impossible
But rain might be managed thanks to this sensor.