It's been an interesting few weeks in the world of state surveillance, with Laura Poitras's film on Edward Snowden, Citizenfour, competing for airtime with tales of Russian submarines stalking Sweden. The latter has a reassuringly old-fashioned air to it, like Daniel Craig's retro-Bond or a modern reimagining of The Riddle of the Sands. Just as the Swedes are being urged to spend more on their navy, so Theresa May insists that haystacks must be gathered if we are to find needles. John Naughton traces this insistence to a particular moment in time: "If they [NSA, GCHQ] are collecting the whole goddam haystack (and they are), then it’s because of the pressure placed on them by their political masters — the 'war on terror', the political pressure to 'join the dots' and the injunction (e.g. from Vice President Cheney after 9/11) to ensure that 'this must never happen again'. In that sense, the NSA, GCHQ etc. are just rational actors trying to meet impossible political demands".
While 9/11 certainly accelerated the process, I think it is wrong to see the historic incursion of state surveillance into the online realm as a response to a particular threat. This would have happened anyway because of the opportunities opened up by the technology and the symbiotic relationship of the state and business. Proof of this is provided by the coincidental revelation of the extent of MI5's surveillance of Eric Hobsbawm and Christopher Hill in the era of snailmail. The agency's thoroughness was prompted not just by bureaucratic mania but by the desire "to establish the identities of his [Hobsbawm's] contacts and to unearth overt or covert intellectual Communists who may be unknown to us", and to establish "the identity of his [Hill's] contacts at the University and in the cultural field generally, and to obtain the names of intellectuals sympathetic to the party who may not already be known to us". In other words, network analysis.
It is in this light that Theresa May's claims are not merely specious but downright dishonest: "It’s hugely important, this ability to have a large amount of communications data. We cannot emphasise enough that the collection of bulk data is not mass surveillance. Precisely because what happens is this targeted process, which means this is not about just some sort of mass look at everybody’s data. Most of the data will not be looked at all, will not be touched." This perpetuates the myth that surveillance concerns itself with what you know or believe, and thus risks intrusion on private property (the liberty of conscience), rather than who you know. In practice, it has always been chiefly concerned with association, from the religious and civil wars and witch-crazes of the 16th and 17th centuries to the Combination Acts of the 18th and 19th.
The implicit claim is that who you consort with (from fellow workers to the Devil) is legitimate grist for the surveillance mill, but that what you say remains private and what you believe remains sacrosanct, at least until your associations promote you to a target. In the pre-Internet age an equivalent claim would be that we only steamed your post open in order to establish who was corresponding with you. Not only is the structural boundary between the realms of "what" and "who" fuzzy to the point of non-existent, but the epistemological distinction is nonsense: the "who" can only be understood in the context of the "what". May's essential proposition is that freedom of conscience can be preserved if we accept the curtailment of the freedom of association. This shows that the state's attitude has evolved little since the days of the Tolpuddle Martyrs.
What the Home Secretary's ingenuous position also ignores is the structural bias inherent to any agency, i.e. the extent to which her "targeted process" can be abused for political or sectional ends, which was ultimately the tipping-point for Snowden. This is not just about actual conspiracies, such as the cabal around Peter Wright that targeted Harold Wilson, but "canteen culture" and the associated presumptions of guilt. For example, one of the MI5 reports on Hobsbawm describes the historian's uncle Harry as a "sneering, half Jew in appearance, having a long nose". That MI5 was institutionally antisemitic is hardly news, any more than the revelation that the Metropolitan Police considered anyone to the left of Tony Blair suspicious and leftwing activists a positive menace.
Zoe Williams, considering the history of police spies fathering children with surveillance targets, puts her finger on another flaw in the policy of mass-data-gathering, which is institutional inertia: "One day, using average human judgement, of a woman he knew inside out, Lambert must have known that Jacqui was not a terrorist but rather a person of radical views. The thing we will never know is how long after that penny had dropped he continued to spy on her. One year? Three? Five?" The continuum with MI5's earlier efforts is clear: "When, for that matter, did MI5 realise that Eric Hobsbawm had no intention of defecting to Russia, and was simply agitating for radical left possibilities within UK politics? When did it realise that Christopher Hill was not intending to restart the English civil war, with a mind to recreating a Leveller revolution three centuries later?" She is being ironic. In most cases the state does not start surveillance because it has a firm suspicion, so the moment at which the target is "cleared" can never arrive. The spying stops when new priorities relegate you as a target.
The state will always seek to gather haystacks, not just because it can but because it has an insatiable appetite for data. If you cannot know in advance which data are valuable, you will gather everything. There is no effective limit, other than the inherent restrictions of the technology, and no effective constraint on the use of the data, other than the conscience of the agency, so it is ridiculous to claim a categorical distinction between the haystack and the needle. Part of our attraction to the theatre of spy-subs and old-fashioned cloak and dagger operations is nostalgia for clear boundaries, for a time when surveillance was state-on-state and concerned obvious targets such as military secrets and the uncovering of moles. The reality is that modern surveillance has been turned largely towards our own populations, in Russia as much as the UK, because that is where the technological opportunity has led us.
Marxist historians and leftwing campaign groups weren't threats to British national security in the second half of the 20th century, but they were convenient targets because they were so readily identifiable and easy to infiltrate. The disbanding of the Met's Special Demonstration Squad in 2008 owed more to the changing technological base and new priorities after 2005 than the unit's excesses. Similarly, the growth of the surveillance state since 2001 reflects the growth of the data industry and the convergence of the Internet and mobile communications. Without 9/11 and 7/7 we'd have ended up in much the same place today: the land of giant haystacks.