Five reasons NOT to use Facebook posts for insurance premiums…

The announcement by Admiral that it was going to use an algorithmic analysis of Facebook usage to ‘ analyse the personalities of car owners and set the price of their insurance’, as reported in the Guardian, should not be a surprise. In many ways it was inevitable that this kind of approach would be taken – the idea that ‘useful’ data can be derived from social media behaviour is fundamental to the huge success of Facebook – but it should not be greeted with much pleasure.

Rather, it should be met with a serious level of scepticism and a good deal of alarm. Facebook themselves seem to have understood this, and have revoked Admiral’s right to use the system – possibly because they understand the problems with the system, and possibly because they can foresee some of the tactics that could be developed in response to it (more of which below). This, however, does not mean that Admiral and other insurers and indeed those in other sectors will not continue with this approach – which is something about which we should be very careful. There are distinct dangers here, and reasons to be concerned that it marks the beginning of a very slippery slope.

1     Inappropriateness

The first reason is perhaps too obvious for many to notice is that the whole idea can be seen as inappropriate. This is not what social media was designed for – and neither is it what users of social media are likely to expect it to be used for. Comparisons between the ‘offline’ and ‘online’ worlds need a lot of care – but here they are apt. Would we expect insurers to go through our address books and calendars, record and listen to our conversations in cafés and pubs, check our taste in music and what films we watch at the cinema? Most people would find that kind of thing distinctly creepy – and yet this is what using an algorithm to analyse our social media activity amounts to. Even if it is consensual, the chances of the consent being truly informed – that the person consenting to the insurer analysing their social media activity actually understand the essence of what they’re allowing – are very small indeed.

2     Discrimination

The chances that this sort of a system would be discriminatory in a wide range of ways are also very significant – indeed, it would be surprising if they are not discriminatory. Algorithmic analysis, despite the best intentions of those creating the algorithms, are not neutral, but embed the biases and prejudices of those creating and using them. A very graphic example of this was unearthed recently, when the first international beauty contest judged by algorithms managed to produce remarkably prejudiced results – almost all of the winners were white, despite there being no conscious mention of skin colour in the algorithms.

This kind of discrimination is likely to be made even worse by the kind of linguistic analysis that seems to be hinted at by Admiral. As reported by the Guardian,

“evidence that the Facebook user might be overconfident – such as the use of exclamation marks and the frequent use of “always” or “never” rather than “maybe” – will count against them”

In practice, this kind of analysis is very likely to favour what might be seen as ‘educated’ language – and make any kind of regional, ethnic or otherwise ‘non-standard’ use of language put its user at a disadvantage. The biases concerned could be racial, ethnic, cultural, regional, sexual, sexual-orientation, class-based – but they will be present, and they will almost certainly be unfair.

3     Digital divides

The problem of digital divides – where people who have good access to or experience with digital technology have an advantage over those who have less – is likely to be exacerbated by this kind of a system. A ‘savvy’ user will be able to game the system – from taking more care of the language use (and avoiding exclamation marks!!!) – while those with less experience and more naïveté will lose out.

Those who don’t use Facebook are also likely to lose out – making the ‘normalisation’ of Facebook use even stronger. This is, perhaps, one of the slipperiest of the slopes involved here: the more we make Facebook ‘part of the infrastructure’, the more we hand over our rights, our freedoms, to a commercial entity that really does not have our best interests at heart.

4     The perpetuation of illusions about algorithms and surveillance

The illusion that algorithms are somehow ‘neutral’ or ‘objective’ is one that seems to be growing, despite the increasing understanding amongst those who study ‘big data’ and social media that they are anything but neutral. This illusion combines with the idea that surveillance – including algorithmic analysis of ‘public’ data – is harmless, unless you have ‘something to hide’. Admiral’s new idea is described only in terms of providing a ‘service’ – and the suggestion is made that it can only ‘reduce’ premiums, not increase them. As well as showing a deep misunderstanding of how insurance works (and even how mathematics works) that idea emphasises how the surveillance itself (and this is surveillance) is not harmful, unless you’re trying to hide some dark secret as a risk taker.

These twin illusions combine into the facade of neutral, objective algorithms undertaking analysis that can only harm you if you’ve got something to hide. This is a doubly pernicious illusion.

5     The chilling effect

The idea that surveillance has a chilling effect – that being watched makes you alter your behaviour, reducing freedom of speech and of action – is one that has been understood for a long time. The concept of the Panopticon, theorised by Bentham and taken much further by Foucault, was initially about a prison where the prisoners knew that at any time they might be watched, and hence they would behave better. It is based, however, on the presumption that we want to control people’s behaviour – and for people whose behaviour is already known to be aberrant and in need of ‘correction’. For a prison, this  might be appropriate – for citizens of a democracy, it is another matter.

It might seem a little extreme to mention this in the context of car insurance – but the point is clear. If we think that it is generally ‘OK’ to monitor people’s behaviour for something like this, then we are pushing open the doors (which are in many ways already at least ajar) to things that take this a lot further. Monitoring social media usage these days means monitoring almost every aspect of someone’s life.

Survival tactics

What can be done about this? The first thing is to develop survival tactics. Keep the more creative sides of your social life off Facebook – and in particular, never mention things like extreme sports, motor racing, admiring Lewis Hamilton or anything even slightly like that on Facebook. Don’t mention or organise parties on social media – or find ways to describe them that the initial algorithms won’t be able to decode immediately.

This should already be becoming the norm for young people – and there is evidence to suggest that it is. Insurers are just another in the long line of potential snoops to be wary of, from parents and potential employers onwards. Thinking about your reputation can have a direct impact on your finances as well as other prospects, and young people need to understand this. Some even maintain several profiles – a ‘safe’ one that the parents can ‘friend’, that employers can scrutinise and insurers can analyse, and a ‘real’ one where real social activity goes on. I would advise all young people to consider using these kinds of tactics – this idea from Admiral is just one example of the kind of thing that is likely to become the norm.

It is, perhaps, an understanding of this that lies behind Facebook’s revocation of Admiral’s permission to use their data. Facebook needs young people (and the rest of us) to post more things, more accurate, more ‘sexy’, and if surveillance like this means that they post less – a trend which has already been observed and about which Facebook is already wary – then Facebook will care. That does not mean they will stop this kind of thing, but more that they are likely to make it less obvious, less overt, and less clearly risky.

The future: algorithmic accountability and regulation 

Ultimately, though, all of this is just scratching the surface – and may even produce a degree of overconfidence. If you think you can ‘fool’ the system, you’re quite likely to end up being the fool yourself. What is needed is for this to be taken seriously by the authorities. ‘Algorithmic accountability’ is one of the new phrases going around – and it is matters. Angela Merkel recently spoke out against the way that Google are ‘distorting our perception’ through the way their algorithm controls what we see when we search. What she was asking for was algorithmic accountability – not for Google to reveal its trade secrets, but for Google to be responsible, and for the way that algorithms control so much to be taken seriously. Algorithms need to be monitored and tested, their impact assessed, and those who create and use them to be held accountable for that impact.

Insurance is just one example – but it is a pertinent one, where the impact is obvious. We need to be very careful here, and not walk blindly into something that has distinct problems.

Brexit and consequences…

Yesterday morning I tweeted about Brexit (as I’ve done a fair number of times), and it went just a little bit viral. Here’s the tweet:


It was an off-the-cuff Tweet, and I had no idea that people would RT it so much, nor that it would provoke quite as many reactions as it has. I’ve replied to a few, but, frankly, it’s not possible to reply to all. The responses, however, have been quite revealing in many ways. As usual, people read Tweets in different ways, and of course this particular Tweet is far from unambiguous. I was asked many times what is the ‘this’ that I’m saying is the fault of the ‘Brexit people’. And who I meant by ‘Brexit people’. I was told I was wrong to lump all Brexit people together. And that we should be looking for unity, not stoking the fires of division.

Some thought I was specifically talking about the dramatic fall of the pound. I wasn’t, but I might have been. Others thought I was blaming Brexit voters for ‘anything and everything’. I wasn’t. Actually, what I was doing was getting angry with those people who voted for Brexit but are now saying ‘we didn’t vote for this’ when they see Theresa May’s increasing nasty and xenophobic government do things like threaten to use EU citizens in the UK as ‘bargaining chips’, sending foreign doctors home as soon as we’ve trained enough ‘home grown’ doctors, and ‘naming and shaming’ companies that employ foreigners.

The thing is, if you voted Brexit you may not have wanted that to happen, but that’s the effect of your vote. And you were warned, many times, that by voting for Brexit you were helping the far right. By voting for Brexit you were ‘sending a message’ that immigrants weren’t welcome. By voting for Brexit you were likely to give more power to the worst kind of Tory. This is what I said on my blog in February, when the campaign was just beginning:

“What’s far more likely with Brexit is that an even more right-wing Tory government will come in, and with even fewer restrictions on their actions will destroy even more of what is left of our welfare state, our NHS, all those things about Britain that those on the left like. It shouldn’t be a surprise that Iain Duncan Smith and Chris Grayling are amongst the most enthusiastic Brexiters. Win the vote and you’re giving them what they want.”

That’s what happened – and I was far from alone in predicting it, and warning people that if they voted for Brexit they’d get more nastiness and a more right-wing government. Now we’ve got it, and if you voted for Brexit, that’s the result.

I’m not, as I’ve also been accused, ‘lumping all Brexit voters together’, suggesting that they’re all racists and xenophobes. Of course they’re not. They have all, however, helped the racists and xenophobes. That’s what the vote did. That’s cause and effect. Some people I know and respect have strong and detailed analytical economic reasons behind their vote – and some expounded them in response to my tweet – but, frankly, that’s by-the-by. Even if their economic arguments  are sound (and I remain unconvinced), they still unleashed the xenophobia.

Others try to suggest that what’s happened is all for the good. We should be making lists of foreigners, we should be replacing foreign doctors with Brits and so forth. That’s also all well and good – but in that case, why be angry with my Tweet? You should be proud of the consequences, if you like them.

I am, of course, one of the out-of-touch metropolitan elite, and I know it. I don’t expect to be listened to. I don’t expect to have any result – but I still have the right to be angry. And I am. I only wish I’d been angrier earlier.

Warning signs – and surveillance…

There are many things being said at the Conservative Party Conference that should be worrying people – from the idea that we should be sending foreign doctors home and ‘naming and shaming’ companies that have the temerity and lack of patriotism to dare to employ foreigners onwards. Military in schools just sends one extra shiver down the spine – these things, when looked at together, do not paint a pretty picture at all. The direction our government is headed is one that is ringing alarm bells for many. Even if you don’t believe the current government is ‘extreme’, the idea that it could become extreme should be taken very seriously indeed.

That, in turn, should raise even louder alarm bells at the current plans for surveillance. The powers that are being granted to the authorities under the Investigatory Powers Bill that is currently making its final steps through parliament are extremely potent and worrying even in the hands of a trustworthy, ‘moderate’ government – but in the hands of an extreme government they become something far, far worse. Tools such as Internet Connection Records, though very poorly suited to the purpose for which they are being put forward, are very good at the kind of profile-based politically-motivated population control that totalitarian regimes thrive upon. The same for many of the ‘bulk powers’ built into the Investigatory Powers Bill. It is bad enough – dangerous enough – to give these kinds of powers to a government that can be trusted, but by putting them into law and building the ‘necessary’ systems to implement them, we are giving them to subsequent governments, governments that may be far less trustworthy, and far more worrying. Governments like those that we have seen more than glimpses of at the Conservative Party conference over the last few days.

When the recent revelation that Yahoo! secretly scanned all of its customers incoming emails on behalf of the US intelligence agencies is added to the equation – with the added twist that Yahoo! had been subject to a massive hack – the picture gets still worse. As I point out in my new academic piece on surveillance, it is a mistake to think of commercial and governmental surveillance as separate and entirely different: they are intimately connected and inextricably linked. If we accept, unthinking, corporate surveillance as harmless, innovative and just about a bit more annoying advertising, we miss the bigger picture. By accepting that, we accept government use of the same techniques, government ‘forcing’ corporations to work with and for them and so on – and not just our current, relatively benign (!) governments but future, more extreme, more alarming, more dangerous governments. If Amber Rudd wants to know whether a company is employing too many foreigners, why not scan all that company’s emails, monitor all the web-browsing from that company’s computers and use profiling to work out which of the employees are probably ‘foreign’, then target them accordingly. Naming and shaming. Labelling. Deporting.

As Bruce Schneier put it:

“It’s bad civic hygiene to build an infrastructure that can be used to facilitate a police state.”

The combination of the level of corporate surveillance, the interaction between corporates and governments, and the disturbing political developments all over the world – from the Conservative Party conference to Donald Trump (and Hillary Clinton is no saint in surveillance terms!) to extremism in Hungary and Poland and more – is making his warning too important to ignore.

It is not too late to change direction – at least we had better hope it is – and we should do everything we can to do so. In the UK, all the opposition parties should fight much harder to limit and amend the Investigatory Powers Bill, for example – as should those within the Conservative Party who have any sense of the traditions of liberty that they purport to hold as important. Whether they will is another matter. This Conservative Party conference should be a warning sign for all.

A better debate on surveillance?

screen-shot-2016-09-21-at-18-57-00Back in 2015, Andrew Parker, the head of MI5, called for a ‘mature debate’ on surveillance – in advance of the Investigatory Powers Bill, the surveillance law which has now almost finished making its way through parliament, and will almost certainly become law in a few months time. Though there has been, at least in some ways, a better debate over this bill than over previous attempts to update the UK’s surveillance law, it still seems as though the debate in both politics and the media remains distinctly superficial and indeed often deeply misleading.

It is in this context that I have a new academic paper out: “Data gathering, surveillance and human rights: recasting the debate”, in a new journal, the Journal of Cyber Policy. It is an academic piece, and access, sadly, is relatively restricted, so I wanted to say a little about the piece here, in a blog which is freely accessible to all – at least in places where censorship of the internet has not yet taken full hold.

The essence of the argument in the paper is relatively straightforward. The debate over surveillance is simplified and miscast in a number of ways, and those ways in general tend to make surveillance seem more positive and effective that it is, and with less broad and significant an impact on ordinary people than it might have. The rights that it impinges are underplayed, and the side-effects of the surveillance are barely mentioned, making surveillance seem much more attractive than should be – and hence decisions are made that might not have been made if the debate had been better informed. If the debate is improved, then the decisions will be improved – and we might have both better law and better surveillance practices.

Perhaps the most important way in which the debate needs to be improved is to understand that surveillance does not just impact upon what is portrayed as a kind of selfish, individual privacy – privacy that it is implied does not matter for those who ‘have nothing to hide’ – but upon a wide range of what are generally described as ‘civil liberties’. It has a big impact on freedom of speech – an impact that been empirically evidenced in the last year – and upon freedom of association and assembly, both online and in the ‘real’ world. One of the main reasons for this – a reason largely missed by those who advocate for more surveillance – is that we use the internet for so many more things than we ever used telephones and letters, or even email. We work, play, romance and research our health. We organise our social lives, find entertainment, shop, discuss politics, do our finances and much, much more. There is pretty much no element of our lives that does not have a very significant online element – and that means that surveillance touches all aspects of our lives, and any chilling effect doesn’t just chill speech or invade selfish privacy, but almost everything.

This, and much more, is discussed in my paper – which I hope will contribute to the debate, and indeed stimulate debate. Some of it is contentious – the role of commercial surveillance the interaction between it and state surveillance – but that too is intentional. Contentious issues need to be discussed.

There is one particular point that often gets missed – the question of when surveillance occurs. Is it when data is gathered, when it is algorithmically analysed, or when human eyes finally look at it. In the end, this may be a semantic point – what technically counts as ‘surveillance’ is less important than what actually has an impact on people, which begins at the data gathering stage. In my conclusion, I bring out that point by quoting our new Prime Minister, from her time as Home Secretary and chief instigator of our current manifestation of surveillance law. This is how I put it in the paper:

“Statements such as Theresa May’s that ‘the UK does not engage in mass surveillance’ though semantically arguable, are in effect deeply unhelpful. A more accurate statement would be that:

‘the UK engages in bulk data gathering that interferes not only with privacy but with freedom of expression, association and assembly, the right to a free trial and the prohibition of discrimination, and which puts people at a wide variety of unacknowledged and unquantified risks.’”

It is only when we can have clearer debate, acknowledging the real risks, that we can come to appropriate conclusions. We are probably too late for that to happen in relation to the Investigatory Powers Bill, but given that the bill includes measures such as the contentious Internet Connection Records that seem likely to fail, in expensive and probably farcical ways, the debate will be returned to again and again. Next time, perhaps it might be a better debate.

More on Corbyn’s Digital Manifesto…

Yesterday a piece I wrote about Corbyn’s Digital Manifesto was published on The Conversation – you can find it here:

The natural constraints of a short piece, and the requirements of The Conversation meant that I didn’t cover all the areas, and my own tendency to, well, be a bit strident in my opinions at times means that it may not have been quite as clear as it could have been. I would like to add a few things to what I said, clarify a few more, and open up the opportunity for anyone to comment on it.

The first thing to make absolutely clear is that though I was distinctly underwhelmed by the Digital Democracy Manifesto, it is far better than anything produced by Labour to date, and vastly better than anything I have seen by the Tories. My criticism of it was not in any way supporting what the Tories are currently doing, nor what they are likely to do. I used the word ‘meh’ in my piece because I wanted (and still want) Labour to be bolder, clearer, and more forward-looking precisely so that they can provide a better opposition to the Tories – and to the generally lamentable status quo on internet policy. As I tried (but perhaps failed) to make clear, I am delighted that Corbyn has taken this initiative, and hope it sparks more discussion. There are many of us who would be delighted to contribute to the discussion and indeed to the development of policy.

The second thing to make clear is that my piece was not an exhaustive analysis of the manifesto – indeed, it largely missed some really good parts. The support of Open Source, for example – which was criticised aggressively in the Sun – is to be thoroughly applauded. You can, as usual, trust The Sun to get things completely wrong.

I would of course like to say much more about privacy – sadly the manifesto (in some ways subconsciously) repeats the all-too-common idea that privacy is a purely personal, individual right, when it actually underpins the functioning of communities. I’ve written about this many times before – one piece is here, for example – but that is for another time. Labour, for me, should change its tack on privacy completely – but I know that I am somewhat unusual in that belief. I’ll continue to plug away on that particular issue, but not here and not now.

What I would hope is that the manifesto starts an open discussion – and starts to move us to a better understanding of these issues. If we don’t understand them better, we’ll continue to be driven down very unhelpful paths. Whether you’re one of Corbyn’s supporters or his bitterest opponents, that’s something to be avoided.

Dear Labour MPs and Members

Dear Labour MPs

I’m sorry that our party is in such a mess. I’m also sorry that it seems so hard to find a way forward – and I’m afraid that right now, you’re not really helping.

The thing is, Labour needs its members – so it really isn’t a viable option for you, as a parliamentary party, to either ignore what members want or to suggest that many members are somehow not really in tune with the party – suggesting that they’re all entryists, Trotskyists, or similar. There are, of course, some who are like that – but most really aren’t, and unless you understand that and pay a bit more respect to the members, the party is really in trouble.

That’s the thing – you really need to understand why so many members voted for Corbyn last year, and why, particularly, they didn’t vote for the three candidates arrayed against him. Until you understand that, and in particular that Labour members aren’t just stupid for doing so, but tap into that energy, that feeling of hope that Corbyn gave to people, then there’s little chance of your regaining the trust of the members. You need to understand why things like the abstention over welfare – even if it can be technically justified – alienated so many people, and why a principled stand is sometimes crucial even if it doesn’t make perfect parliamentary logic.

I hope that you can find a way. We really need to bring the party back together – which means members and MPs need to find a way to come back together.

With hope

Paul Bernal

Dear Labour Members

I’m sorry that our party is in such a mess. I’m also sorry that it seems so hard to find a way forward – and I’m afraid that right now, you’re not really helping.

The thing is, Labour needs its MPs – so it really isn’t a viable option for you, as a party membership, to either ignore what MPs want or to suggest that many MPs are somehow not really in tune with the party – suggesting that they’re all Blarites, Red Tories, or similar. There are, of course, some who are like that – but most really aren’t, and unless you understand that and pay a bit more respect to the MPs, the party is really in trouble.

That’s the thing – you really need to understand why so many MPs supported the vote of no confidence in Corbyn, and why, despite the clear support of the members, they still can’t really work with him. Until you understand that, and in particular that Labour MPs aren’t just stupid for doing this, but recognise why what MPs in parliament do that matters, and that MPs do work hard and are committed to the Labour Party, there’s little chance of Labour being an effective party or winning an election. You need to understand why what happens in parliament matters – even if it isn’t always clear.

I hope that you can find a way. We really need to bring the party back together – which means members and MPs need to find a way to come back together.

With hope

Paul Bernal