We’re here to make safe perception possible by enabling cars to pass their vision test. It’s a bold statement, we know. How do we make this a reality? To answer this question, our CEO Daniel Langkilde took the opportunity at AutoAI in Berlin to explain our data-driven approach and iterative way of working.
We know that not everyone could attend the event, and we need to spread the word! Continue reading to discover what we’re all about - and why you need us.
At Kognic, we provide software to help develop perception systems. Everyday, we ask ourselves: how can we develop cars that pass their vision test? We're trying to make progress when it comes to improving our KPIs and performance so that we reach use cases that can be valuable to our customers. Now, we’re approaching a point where we need to be able to argue the performance and safety of our products before customers, regulators and many more. And while the interest in progressing quickly is great, we also have the back pressure of what we can argue safety about.
Where do we put our focus on? Three words (or 3 P’s): perception, prediction and planning. The perception system is what turns the chaos around the car into a structured representation or view of the world. Prediction understands the intent of each object on and around the road. And planning plots a path for the vehicle based on perception and prediction. This is to a large degree a very complex process, as you can see in the following image.
What becomes a challenge here is to define safe behavior for millions of edge cases. Getting global coverage is important, too. As an example, it’s great that a product can work in Phoenix, Arizona, but you need to be able to operate safely in other places in the world. And global coverage requires massive amounts of data. Let’s also add the fact that we live in a context that is constantly changing. Bearing in mind these factors, is there anything that can make companies succeed in the race towards autonomous mobility? At Kognic, we believe so! The key is to be able to progress quickly and iterate, but in a way that enables constant monitoring and measurement.
How can you quantify, slice and dice and understand the performance of your perception system? We’re not talking necessarily about maximizing performance, but about balancing safety and performance.
First of all, when it comes to safety, there are regulatory, ethical and other requirements that make safety a hard constraint. Then, it’s the turn of performance, which according to some OEMs is about setting high performance goals and, when you know how much you can achieve, nailing down the product while maintaining safety constraints. That needs to be balanced with value. And here comes the big question. What is a valuable product that has sufficient performance and is also safe? We don’t have the answer to this question yet. But we know there’s one solution: high engineering velocity.
It’s not about planning better, but about testing faster and learning faster. And the major challenge here is that you, as a customer, want to have that speed but still be responsible, and accurate when you launch. That requires sophisticated tooling which makes the work of engineering teams smooth, so they can deliver in the fastest and best way possible. And we at Kognic have built that tooling. Because we think that’s what you actually need. You need tooling to move faster, to iterate quicker, to learn more, and do that while being able to measure what your progress is.
The way we see it, you should start measuring, and then take the decision of what the Operational Design Domain is going to be as you approach a point where you have no choice, rather than beginning with specifying what a car is going to be able to do at some point in the future. We know that this requires constant monitoring of KPIs, and a lot of ODD understanding. Detailed statistics are needed here if you want to understand your ODD, and check that you actually have sufficient coverage of it.
How do you know the performance KPIs you’re measuring work in the Operational Design Domain that you have? Tracking both simultaneously: every time you expand your ODD you need to see how your coverage improves, how that impacts your perception performance KPIs and whether you’re still viable to launch. Because the performance KPIs and the ODD are only valid if there’s enough statistical significance to your observations in that ODD.
Finally, you need to work in a highly iterative way. Realistically, you probably have the best way to accelerate your data mining or refinement process: the machine learning algorithms in your perception teams will always be better than anyone else’s for your sensor, and your specific use case. However, you need to take into account the reliability of all of it. In other words, and as we often say at Kognic, calling it ground truth doesn't actually make it true. It is still a sampling out of a probability distribution and it's gonna have variants. But if you have a performance goal, your ground truth and your model, you will never supersede your ground truth in your model and the reliability of your KPIs is gonna be whatever variants you have in the sampling of humans on which you've trained. The lesson here is that you need to minimize that, and maximize the closeness to your performance goal.
Measuring that is hard. And it starts with localizing your performance expectations. You don't care about two centimeter accuracy of detecting a car at 200 m. It would make no sense. You care if it's close to you and if you're in a safety relevant scenario. Again, it’s essential to measure performance and understand your ODD. And in these cases, at Kognic we think that, instead of planning, you should iterate, and learn, and try, and define what you think is a viable product based on the progress you've made, and then learn more, and launch, while still maintaining safety. This is likely the most differentiating factor to succeed in the industry.
To conclude, we think the overarching problem here is: what data do you need? You might think that you want to plan first, and then specify later. At Kognic, we think you should explore, learn, measure, label, and look for what is necessary for the product you’re trying to launch. And then iterate! Fast!
To succeed, you’ll need great tooling. With our powerful Perception analytics and Ground Truth Platform, many engineering teams are already developing their safe perception systems. And we're glad to be helping the most ambitious companies from across the automotive industry in achieving this with our market-leading offering.
We know the future is autonomous. We invite you to take a step closer to safe automated driving and develop your perception system too, starting from today - with vision you can trust.