[personal profile] mjg59
Traditionally, devices that were tied to logins tended to indicate that in some way - turn on someone's xbox and it'll show you their account name, run Netflix and it'll ask which profile you want to use. The increasing prevalence of smart devices in the home changes that, in ways that may not be immediately obvious to the majority of people. You can configure a Philips Hue with wall-mounted dimmers, meaning that someone unfamiliar with the system may not recognise that it's a smart lighting system at all. Without any actively malicious intent, you end up with a situation where the account holder is able to infer whether someone is home without that person necessarily having any idea that that's possible. A visitor who uses an Amazon Echo is not necessarily going to know that it's tied to somebody's Amazon account, and even if they do they may not know that the log (and recorded audio!) of all interactions is available to the account holder. And someone grabbing an egg out of your fridge is almost certainly not going to think that your smart egg tray will trigger an immediate notification on the account owner's phone that they need to buy new eggs.

Things get even more complicated when there's multiple account support. Google Home supports multiple users on a single device, using voice recognition to determine which queries should be associated with which account. But the account that was used to initially configure the device remains as the fallback, with unrecognised voices ended up being logged to it. If a voice is misidentified, the query may end up being logged to an unexpected account.

There's some interesting questions about consent and expectations of privacy here. If someone sets up a smart device in their home then at some point they'll agree to the manufacturer's privacy policy. But if someone else makes use of the system (by pressing a lightswitch, making a spoken query or, uh, picking up an egg), have they consented? Who has the social obligation to explain to them that the information they're producing may be stored elsewhere and visible to someone else? If I use an Echo in a hotel room, who has access to the Amazon account it's associated with? How do you explain to a teenager that there's a chance that when they asked their Home for contact details for an abortion clinic, it ended up in their parent's activity log? Who's going to be the first person divorced for claiming that they were vegan but having been the only person home when an egg was taken out of the fridge?

To be clear, I'm not arguing against the design choices involved in the implementation of these devices. In many cases it's hard to see how the desired functionality could be implemented without this sort of issue arising. But we're gradually shifting to a place where the data we generate is not only available to corporations who probably don't care about us as individuals, it's also becoming available to people who own the more private spaces we inhabit. We have social norms against bugging our houseguests, but we have no social norms that require us to explain to them that there'll be a record of every light that they turn on or off. This feels like it's going to end badly.

(Thanks to Nikki Everett for conversations that inspired this post)

(Disclaimer: while I work for Google, I am not involved in any of the products or teams described in this post and my opinions are my own rather than those of my employer's)

Date: 2018-01-18 01:19 am (UTC)
From: (Anonymous)
I wish there were open source libraries / frameworks that could be applied.

For example I work on an iOT type device, and decided to allow local control from a mobile device (technically anything that speaks HTTP) without traffic having to go out to the internet and back again. There is a pairing step where you have to enter a random code shown on the device which then provides a cookie if correct. That at least protects should someone put the device directly on the Internet.

But what I can't do is secure the connection (ie HTTPS instead of HTTP) because SSL needs certificates which you can't do for local only names. Self signed certs and similar won't work in regular browsers (try explaining the administration and installation of them to regular folk!)

I also send information to our servers. I'd be delighted to send them to some other service (local or remote) but I am not going to implement one of those, and nothing appears to exist in the open source world.

The law

Date: 2018-01-18 09:30 am (UTC)
From: (Anonymous)
I think laws need to keep up.

In the end, what kind of place do we wish to live? Do we really wish snooping on people to be the default from now on and the next thousands of years? I think more systems engineering and resulting lost opportunities in the data that can't be collected is a the right long term sacrifice. But I don't see it happening without the law stepping in - these issues are so subtle that you can't leave it to consumers.

I think the EU is working on it, slowly.


German telecom law as example

Date: 2018-01-18 11:48 am (UTC)
From: (Anonymous)
There is a parallel thing in German law:

Let's say I have a landline phone. The phone company let me choose between different call records: Either only the prefix (which is important for the costs) or full numbers. If I select full numbers, the company lets me sign a paper, that all persons in my household (potential users of the landline phone) know about the full number call records resp. I have the obligation of making them aware of the fact, that I can later see all called numbers!

That does mean for the fridge with egg detection: The company delivering such a device should have the duty of letting their customers sign a paper, that all potential egg eaters in the household know about the automatic egg ordering mechanism. And the customers responsibility is to explain the fact to the members of the household, guests etc.


(Note, that I'm lacto-vegetarian and therefore did not eat the missing egg.)

better check first

Date: 2018-01-19 02:31 pm (UTC)
From: (Anonymous)
https://xkcd.com/1807/ applies

Date: 2018-01-20 06:11 pm (UTC)
From: (Anonymous)
Why do these devices *need* to log every query, or voice, or action? Why do they not follow the principle of logging as little as possible?

relevant literature

Date: 2018-01-22 04:57 pm (UTC)
fche: (Default)
From: [personal profile] fche
Do you have an opinion on The Circle (book)? It seems to present a steelman case for trading privacy for anything an omniscient corporation can give you.
From: (Anonymous)
It shouldn't be necessary to send sound recordings to Amazon or Google to be recognized there. Speech recognition can take place locally, and data that are sent out of the house can be reduced to the minimum necessary to execute the requested function. Certainly a command to turn on the lights doesn't have to leave the premises.

I suppose Amazon and Google have no interest in a redesign that would respect privacy more, but perhaps a competitor will enter the market.

Cf. Benjamin Mako Hill

Date: 2018-02-01 02:47 am (UTC)
From: (Anonymous)
Another good post. IMO, you are making a similar point, in the IoT domain, to the one Benjamin Mako Hill made a few years ago in the email domain: https://mako.cc/copyrighteous/google-has-most-of-my-email-because-it-has-all-of-yours .

The point is: even if some people behave in strongly privacy-conscious and privacy-respecting ways, other people's failure to act similarly responsibly will almost certainly compromise their efforts substantially.

- sampablokuper


Matthew Garrett

About Matthew

Power management, mobile and firmware developer on Linux. Security developer at Google. Ex-biologist. @mjg59 on Twitter. Content here should not be interpreted as the opinion of my employer.

Page Summary

Expand Cut Tags

No cut tags