The most high-stakes lawsuit in Silicon Valley, a nasty battle for the first commercial self-driving car, centers on a 54-year-old technology called Lidar.
Alphabet (Google’s parent company) claims that Uber stole Lidar-related intellectual property from its self-driving car company, Waymo, by hiring one of Waymo’s leading engineers, Anthony Levandowski. On Monday, Waymo got a court win when the judge ordered Uber to return any allegedly stolen documents and barred Levandowski from working on part of Uber's self-driving program.
The future of an industry, the safety of all its passengers, and a whole lot of money (not even counting the lawsuit) rest on Lidar, which allows self-driving cars to see and navigate the world around them. Waymo, Uber, and the world's biggest car companies are all racing to make Lidar systems that are safer than human drivers and only cost a few hundred dollars per car.
“Whoever cracks the nut, to make Lidar work with a safe self-driving car, will own the market,” roboticist Edwin Olson of the University of Michigan told BuzzFeed News.
For something suddenly so important to the future of the auto industry, and at the heart of a stunning titan vs. titan lawsuit, Lidar is quite old in terms of technology. Pioneered soon after the 1958 invention of lasers, Lidar (or LiDAR, LIDAR, or LADAR, which we’ll get to) works by bouncing light off far-away things to reckon their distance and shape. Since light travels at an unvarying speed of 671 million miles per hour, the time it takes for projected light to bounce off remote objects and return tells you the distance to whatever is around the Lidar system, typically down to a centimeter.
Unlike oncoming headlights, automotive Lidar lasers typically operate in the near-infrared spectrum, invisible to the human eye, which means they don’t blind anyone.
“That’s pretty useful for a self-driving car,” roboticist Chistoph Mertz of Carnegie Mellon University in Pittsburgh told BuzzFeed News. (Carnegie Mellon has been a hotbed for self-driving car research: In 2015, Uber poached an entire CM robotics lab to try and beat Waymo, one of the backstories of the Levandowski lawsuit.)
On top of distances, changes in the reflectivity of returning Lidar pulses can provide information about whatever the laser hit, whether it’s hard or soft, light or dark (beer foam is 88% reflective, for example, while pavement is only 17%), and which way it’s facing, with a precision of around one-tenth of a degree.
Some Lidar systems take millions of measurements every second. By building up a mosaic of these measurements in 360 degrees, Lidar can paint a three-dimensional picture of the world around it. Voila: a car that “sees,” at least out to a football field’s distance.
At least six of the main commercial self-driving car efforts use Lidar.
At least six of the main commercial self-driving car efforts — Ford, Toyota, Volvo, and Honda, plus Uber and Waymo — use Lidar. Although the basic technology behind Lidar is old, engineers at each of these companies are still trying to figure out how to cheaply take millions of Lidar readings and mold them into a car’s map of the world. It’s hard.
Each Lidar system includes lasers, sensors, lenses, a clock, and a lot of computer circuitry and programming to work out all the calculations. Waymo buys some components from the best-known Lidar company, Velodyne. But Waymo’s lasers and sensors are controlled by its own electronics, ones at the center of the lawsuit, according to court documents. For now, Uber's self-driving cars use Lidar units developed by Velodyne.
Lidar systems are expensive: Velodyne’s 32 laser and sensor system costs $8,000 — far too high to add on to the price of a car meant for typical buyers. Lidar systems also tend to be bulky, spinning like an oversized police siren atop cars to get an all-around view. The laser pulses might come every 5 nanoseconds and require high-speed clocks to accurately measure their return time. The laser light has to be tightly focused by a carefully engineered lens (and how this is done is one point of dispute in the lawsuit). Expensive sensors are required to characterize the returning light.
And it takes a lot of computer power to process those millions of measurements into something that computer algorithms can use to hit the brakes when a deer jumps onto a road.
“The challenge is making something work better than the human eyeball, which is hard,” Olson said. “We aren’t really there yet.”
So far, only one notable self-driving car doesn’t use Lidar. Elon Musk’s Tesla instead relies on radar and a camera system, which can see to greater distances and is much cheaper, around $200. But cameras don’t work so well at night, and they can’t calculate distances off flat surfaces (like say, the side of a truck) very well. Lidar actually works better at night, because there isn’t any other light interfering with the laser reflections. (Black cars, which don’t reflect light well, can be a problem, though.)
Lidar manufacturers, broadly speaking, have two avenues for improving their performance — amping up the power of the laser and the sensors, which is costly, or improving their capabilities so that they use tighter laser beams and less power. For the latter (also costly) approach, solid- state lidar systems are coming, which park the lasers and sensor on a silicon chip. If and when solid-state systems arrive, they’d replace the mechanical spinning gumball machines now seen on most experimental self-driving cars.
“That’s why we built a mega-factory,” Mike Jellen of Velodyne told BuzzFeed News, after a $150 million investment from Ford and Baidu, a Chinese automaker. (Velodyne’s competitors have also promised solid-state systems). Velodyne has promised a $50 price tag for the solid-state Lidar computer chip within the next three years, despite some skepticism.
Given advances, getting Lidar systems down to the hundreds of dollars in cost, rather than thousands of dollars, looks achievable, Mertz said. And this is exactly what all companies in the self-driving sector are racing to do.
Today’s self-driving cars most commonly fuse together signals not only from Lidar, but from cameras, radars, and other sensors as they interpret the world. They also rely on GPS signals and internal maps, and sometimes identifying signals transmitted from car to car.
“Every kind of sensor is making progress, which is a good thing,” Mertz said. “We are going to need robust, redundant systems,” he said, for self-driving cars, with Lidar as a keystone technology. He gives it a few years, while Olson is more pessimistic, giving it a decade.
Many Lidar systems don’t perform exactly as well as advertised, Olson said, and academic self-driving car labs spend a lot of time checking out limitations of new systems and sharing the results with each other. Some Lidar systems rely on clever ideas, optimized to pick out car tail lights to save power, for example, but with limited range for detecting anything else. Bad weather reduces Lidar’s range as well, and it doesn’t always perform well on wet surfaces.
“We’re going to have to add something to the paint on cars, maybe, to help Lidar,” Olson said. Anyone wanting to see more chrome on cars, he added, might be disappointed as well: “It can acts like a mirror and sends the [light] away instead of reflecting it back.”
There’s enough uncertainty about Lidar that at an April 12 tutorial on the technology for the Waymo v. Uber lawsuit, Judge William Alsup asked for a clarification on whether to call it LiDAR, an acronym for Light Detection And Ranging, or LADAR, the acronym for LAser Detection And Ranging.
The word "LIDAR" originated not as an acronym, but a mash-up of the word “light” and “radar."
“I notice that sometimes it is spelled with a small ‘I’, and sometimes with everything else capitalized, and sometimes it is just the ‘L’ capitalized,” the judge complained.
It’s all the same thing, he was told.
(For word nerds, a three-paragraph Wikipedia history of the etymology of Lidar concludes that the word originated not as an acronym at all, but a mash-up of the word “light” and “radar,” first published in a 1963 astronomy report in New Scientist magazine.)
While most major innovations rely on a series of small improvements, Lidar “may be somewhat different,” economist David Mowery of the University of California, Berkeley, told BuzzFeed News. Whichever firm develops that cheap, reliable, Lidar for self-driving cars might effectively establish it as the dominant technology, regardless of its inherent flaws or strengths, as the rest of the auto industry immediately builds its cars around the technology it has waited so long for. In other words: The winner of this lawsuit might be the next Henry Ford.
Ultimate success in the self-driving car industry awaits more than just a cheap Lidar system but the computer brain to run it flawlessly, cautioned Olson, who is also a founder of self-driving car startup, May Mobility. Despite that, he jokes that he is the “most pessimistic person in the autonomous car startup industry.” Beyond Lidar that beats the human eye, he said, the industry still needs a computer that drives better than the human brain.
“What keeps me up at night are the totally unpredictable situations — the cop directing traffic at a broken stoplight,” he said, or the deer in the road. “Our visual cortex is really good at understanding these situations right away.” Computers aren’t.
Despite the 30,000 US traffic deaths every year, people are surprisingly good drivers overall, he said. “Matching human performance is really hard.”
Priya Anand contributed reporting to this story.