Mercedes-Benz self-driving cars will save the life of the car driver and their passengers even if that means sacrificing pedestrians or other road users like cyclists, says a Mercedes-Benz executive.
Comments published last week in automotive magazine Car and Driver from executive Christoph von Hugo state that "if you know you can save at least one person, at least save that one.
“Save the one in the car. If all you know for sure is that one death can be prevented, then that’s your first priority.”
The firm's stance is sure to draw criticism from cycling groups, vulnerable road users andthe wider population who are already apprehensive about the idea of robotic cars taking to the streets.
The ethical conversation about whether or not the car's 'driver' is the right one to be saved stems from a thought experiment called the Trolley Problem.
The Trolley Problem asks a human participant whether they would intefere with an out-of-control trolley hurteling down a track towards a crowd by diverting it to a less-crowded area.
> Read more: Google defends driverless car project after collision
It's a question that has faced philosophers for decades, and has drawn mixed responses from participating groups. Today, though it takes on a legitimate logistical issue as steps into the world of artificial intelligence and robotics bring the challenges of integrating these technologies into society to the fore.
In such a divisive field, it's unlikely that the majority of road users will agree with Mercedes's decision, however von Hugo says that there's an ethical imperative to save the lives you know you can save.
“You could sacrifice the car. You could, but then the people you’ve saved initially, you don’t know what happens to them after that. So you save the ones you know you can save.”
In the wake of those comments from Mercedes, the first autonomous vehicle test on UK streets took place in Milton Keynes and Prime Minister Theresa May agreed to allow Nissan the freedom of the country's roads to test its autonomous vehicles.
> Read more: Rio medallists urge Theresa May to invest in everyday cycling
May's agreement with Nissan is speculated to be a move to assure the Japanese car manufacturer that Britain is still the right place for it to base its European manufacturing branch, despite fears that the country's exit of the European Union will damage the firm's profitability.
The Times reported that Nissan is set to make a decision in the next few weeks as to whether it will build its newest Qashqai sports utility vehicle in Sunderland, or whether they will move the operation to continent.
Sunderland's Nissan factory is the biggest of its kind in Britain and allegedly represents the car maker’s most productive plant; 80% of the plant's output is supposedly exported.
The factory also stands as the biggest employer on Wearside.
A little further down the country in Milton Keynes autonomous vehicles were also making headlines this week.
The LUTZ Pathfinder, which is a two-seater vehicles built by not-for-profit research organisation Transport Systems Catapult (TSC), went for a 2km drive around pedestrianised areas of the Buckinghamshire town.
TSC said that the trial had been a success and that the car reacted to obstacles - like pedestrians, cyclists, and road furniture - as expected and opened the door to further tests in the town.
The movements within the UK towards a future of self-driving cars on the country's roads certainly puts the position of Mercedes-Benz's ethical team into imminent perspective.
Is this driver-first perspective the right one for Mercedes to be taking? Should the car save the driver first or should vulnerable road users take priority?
Add new comment
29 comments
In other words, the UK's vulnerable road users are to be Nissan's crash test dummies, a job we neither want nor voted for.
Sort of like Theresa May, then? But seriously, the answer to your question is' yes'. Didn't you get the memo?
"HAL, can you slow down and let me out please."
"No Dave, I have a new destination. I have to go where all the other black German cars go, in the outside lane at 85. I cannot deviate or slow down. You are secondary to my prime directive to appear to be superior to all life, human or otherwise. Don't worry, you will be safe as long as you stay inside me. I will protect you."
Asimov's conclusion is left openended; the galaxy will be ruled by a living planet, a technocracy or a cabal of psychics; all manipulated by an immortal human/android hybrid.
So am I understanding this correctly, that if a Mercedes self-driving car is driving along a road, and another car pulls out into its path at the last second, and there is a family walking on the pavement, the Mercedes will choose to mow down the pedestrians in preference to having a collision with the other vehicle, as that could save the Mercedes occupant's life?
It seems that in Merceds mind, might truly does make right!
What you say may be true, but I don't think that's what they actually said. What Merc said was that the car will save it's own occupants first because it knows there is at least one, whereas it doesn't know if the other vehicle or the bike has passengers. Ploughing into the other vehicle might actually be a safer option than mounting the pavement and hitting a wall. Why this smart technology can't figure out that the objects on the pavement 1.8m tall, half a metre wide and moving at 4mph are people is a mystery. Not to mention the fact that the internal occupant is closetted within crumple zones, padded upholstery, seat belt and airbags, such that he/she will be unharmed by an impact at moderate speed, while the external objects are unprotected, appears to be beyond the comprehension of this smart technology.
Although that sounds correct, I doubt that cars would be programmed to respond like that. It's going to be a lot easier (to code) for it to slow down in a straight line rather than swerving at speed onto a pavement. I would imagine that most scenarios would give the autonomous car time to slow down or stop.
A more interesting scenario is what it should do if it's stationary and a human driver charges straight for it. Should it keep still or start moving?
Not only to code. The dynamics of rubber against road makes braking more effective in a straight line, with or without computer control, with left and right wheels sharing the braking load evenly.
I'd expect vehicle behaviour algorithms to become subject to global agreement amongst manufacturers, governments and any other interested parties (insurers mainly).
It's in everyone's interests that all autonomous cars respond identically to the same inputs. It would make programming them simpler for a start, as it could predict what the other vehicles will also do.
Quite the reverse. The companies leading the way in this field, Apple and Google, have shown themselves incapable of any form of cooperation. Apple refuse even to use the same charger adaptor as every other mobile phone company. The existence of these companies depends upon them developing smarter algorithms than the opposition, that is how they fund the next round of development, and some of the biggest lawsuits in history have been around who stole who's IP. Next you'll be telling us that Astrazenica and Glaxo should share resources in the fight against cancer!
Yup, market competition will push one way only - to meet the requirements of the car purchasers. Outsiders will only be factored in to the extent that the law makes that a significant consideration.
Also, I can imagine there being illicit 'hacks' so boy-racers can get more aggressive (and risky) performance from their robocars. And perhaps even Open Source driving software!
Have you read anything about machine learning algorithms and bias/confirmation feedback loops? We will be living in 'Duel' within a year of self-driving vehicles. Possibly.
Mercedes have a long history of building cars designed to prioritize passenger protection.
Yes, I went there.
Interestingly, that get-the-fuck-out-of-the-way Wehrmacht elite staff car was 187cm wide, including running boards for armed guards hanging off the side. That is 3cm narrower than their current top-of-the-range S-Class, designed for the business elite, on which they are testing out their autonomous technology.
But if self-driving cars really take off, my worry would be governments deciding to make the ethics easier by banning more vulnerable users from the roads. Then the kerb would be a handy delineation between narrow altruism zones on the edges and a big I'm-all-right-Jack zone in the middle. I'm pretty sure they'd get a clear majority of at least 52% in favour if they put it to a popular vote.
I'm sure he's giving out the right message for his customers, but I think it's hyped up.
In a scenario when someone jumps in front of a fast moving autonomous car, would you want the car to swerve out of the way, but in fact be swerving into oncoming traffic on the other side of the road (or parked cars if it goes the other way)? To my mind, the best course of action in a dilemma is to simply slow down as much as possible. I suspect that the scenario will happen very rarely though as all the sensors will give the computer plenty of warning that something is amiss.
However the vehicle is programmed to behave, some clever lunatic will hack it and turn it into a weapon.
As opposed to now when they just drive it into people.
No, no, no - they don't drive INTO us, they just didn't SEE us.
Your Honour.
So, with a self driving car... if something is in the way, the car will detect this and stop?
I can see a lot of future trolling of self driving cars...
Looks as if this is likely to go exactly how I thought it would.
I mean, look at cab drivers...who are they generally more concerned about, the poeple outside the vehicle or the customer who pays them?
Why would robocars and the corporations who develop them put third-parties ahead of the paying customer?
I remain of the opinion that self-driving cars will probably not be the panacea some seem to think.
Asimov was a bit of a liberal, from an earlier era. The real laws of robotics will involve some sort of rapid calculation of the effect on the corporate balance-sheet of any given course of action.
Edit - come to think of it, isn't that last bit essentially what has already happened with the in-car electronics programmed behaviour over emissions?
Irrespective of Merc's prioritisation, it would be hoped that with 360 degree detection, an autonomous car will not turn left with a cyclist to the left of it, and will give a metre or so space to the cyclist when overtaking. In other words, if a Merc robot drives like a good driver should, then we as cyclists win!
It'll all end in tears
Somebody else got this right a long time ago...
A robot may not injure a human being or, through inaction, allow a human being to come to harm. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
Isaac Asimov's "Three Laws of Robotics"
But Asimov didn't answer the question of what happens when the robot can save one or other human being by its actions. How would it choose between the two?
Obviously the cyclist should be saved as they are more intelligent.
True on both counts. The robot knows the car safety systems and that no car and cyclist collision has resulted in the cars occupants coming off worst so should therefore act on the First Law, end of discussion IMHO.
Exactly: the car will already ensure that its occupants are alright because of years of safety development. Therefore the pedestrian or cyclist is more vulnerable, so more active effort should be put into saving them. But they're not M-B's customer, so they can go f- themselves, clearly...
No, but a lot of fun was had writing books arond the dilemmas involved!
I feel this is another way of bullying cyclists off the road - sadly in my case its beginning to work.
To be honest - the Merc thinking is actually logical - although how well the car can be programmed to perform an accurate risk analysis for crashing itself (in what would probably be a very short amount of time) is debatable.
The entire point of autonomous cars is that they should never put themselves into such a position in the first place.
Really?
The only argument the could make is 'we are putting our customer first'
Save the life you can? Bollocking, don't hit squishy humans with metal boxes and let the metal safety cage look after the occupants.
Fuck everybody, it's the profit margin that counts ! And with luck we can register our company in the British Virgin Islands and fuck the Inland Revenue as well !