Accident rates of self-driving cars: A critique of the Sivak/Schoettle study

To what degree are self-driving cars likely to reduce accidents and traffic deaths? This is a very important but very hard question which has implications for testing, insurance, regulations and governments considering to accelerate or delay the introduction of autonomous cars. Now two researchers, Michael Sivak and Brandon Schoettle, of the Transportation Research Institute at the University of Michigan have examined this problem in a short study titled “Road safety with self-driving vehicles: General limitations and road sharing with conventional vehicles and arrived at four conclusions which – when read carefully – provide little insight into the problem but when read casually seem to raise doubts about the expectation that self-driving cars will be significantly safer than human drivers.

As an example the abstract summarizes their second conclusion as follows: “It is not a foregone conclusion that a self-driving vehicle would ever perform more safely than an experienced, middle-aged driver”.

Who could argue against this statement? Of course, this is not a foregone conclusion. This is a hard problem and a substantial question. Neither would it be a a foregone conclusion that a self-driving vehicle would ever perform more safely than an experienced, young driver (or even an unexperienced young driver). But many readers will interpret this conclusion that the authors – after having analyzed the issue – have found substantial problems that raise doubts as to whether autonomous cars could ever perform better than experienced, middle-aged drivers. But the full text of the report contains just one sentence which further examines this problem:

“To the extent that not all predictive knowledge gained through experience could exhaustively be programmed into a computer (or even quantified), it is not clear a priory (italics by the original authors) whether computational speed, constant vigilance, and lack of distractability of self-driving vehicles would trump the predictive experience of middle-aged drivers”. (Page 4)

Nobody can argue with this statement. It would be a good introduction to a chapter that looks at this problem in more detail, provides some framework, examines the different aspects etc. etc. But this does not materialize.

If we read the study carefully, then we find a pattern that valid questions are being raised, a small number of the aspects relating to these questions are outlined, and then the questions are rephrased into conclusions which themselves are questions. This is unfortunate because the topic is extremely important. More than a million people die in traffic accidents every year. If – twenty years from now – we might look back from a situation where traffic accidents have fallen by more than a factor of five, then we will be able to state with certainty how many lives could have been saved if self-driving cars would have been introduced a few years earlier. We might find that tens of thousands of people have lost their lives because governments and regulators did not realize the risk of delaying a highly beneficial technology and business and innovators were reluctant to advance the technology because of a climate of mistrust and skepticism with respect to the technology. Of course, from the perspective of today this is not a foregone conclusion but we need to make an effort to understand the risks and likely accident patterns of autonomous vehicles much better.

There are lives at stake both if we are too optimistic and too pessimistic over the potential of this technology. But the problem is not symmetric: If we are too pessimistic with respect to the potential of this technology, then we can easily find ourselves in a situation in the future where we find in hindsight that thousands of lives have been lost because of this pessimism and the resulting delay of the introduction. On the other hand, if we are overly optimistic with regard to the technology, and accelerate innovation in this area, it is unlikely that thousands of lives will be lost because the cars do not perform as safely as expected. We can be confident that certification bodies will do their work and uncover problems before they can cause thousands of deaths and regulators will most surely step in immediately when these cars do not perform as expected. At the current stage therefore, pessimism about the technology’s potential may be much more deadly than optimism (which should not be confounded with being blind about the risks).

We should work together urgently to formulate a theory of human traffic accidents and self-driving car accidents which can help us shed light on the issue and understand and organize the many different aspects of this problem. This is hard but it can be done. Please contact me at info.2011 ( at ) inventivio ( dot ) com if you are already working on this topic, if you know of a suitable approach for covering this problem or if you are interested in working together on this topic. I will post one approach on how this could be achieved next week.

Changes:
2015-01-23: Added link to the full text of the study.