Disproportionate use of commercially and publicly available data: Europe’s next frontier for intelligence reform?
Study
Executive Summary
Intelligence services across Europe are increasingly processing commercially available data as well as a broad range of information they deem ‘publicly available’. To gain access to such data, they
- purchase data(sets), either ad hoc – when specific information is needed – or on a rolling basis by means of subscription from various data brokers
- purchase data on the darknet (which may emanate from leaks or stolen customer data)
- buy finished intelligence on the market, without any access to the underlying data (thus outsourcing time and resources for the analysis to private actors)
- purchase from various providers the tools needed for automated analysis of commercially and publicly available data
- obtain large (bulk) datasets through voluntary submissions of private sector entities, courtesy requests, or gifts
- purchase or otherwise acquire large datasets through the use of authorised undercover agents or covert human intelligence sources (CHIS).
What these types of access have in common is that they are non-compelled; that is, the entity which provides the intelligence service with access to such data is not obliged by law to do so. This distinguishes these practices from signals intelligence (SIGINT) and computer network exploitation (CNE, commonly known as government hacking), where data held by the private sector can be obtained through compulsion or penetration.
Notably, whereas compelled and direct access have been subject to increasingly dense regulation and oversight in established democracies,[1] governments’ purchases of commercially available data or their acquisition and processing of publicly available data still face far fewer legal restrictions and less robust (if any) authorisation and oversight procedures. This deficiency erodes public trust in government and is at odds with the promotion of the rule of law and democracy in Europe. Vague or missing legal restrictions and insufficient oversight may also increase the risk of disproportionate access to personal data without sufficient accountability. In turn, this may increase risks that various rights will be infringed, notably those to privacy, informational self-determination, and freedom of expression.
While the quantity and easy availability of commercially and publicly available data is profoundly transforming the practice and governance of contemporary intelligence, European lawmakers remain rather oblivious to the gradual paradigm shift and risks involved. To date, regional and European legal frameworks for privacy and data protection are either not applicable or insufficient to rein in these ill- governed practices of national intelligence services. National legal frameworks also lack precision, clarity, and substance: Hardly any European intelligence law currently provides robust legal safeguards, let alone ex ante authorisation and ex post oversight for the various types of data purchases and automated open-source analyses.
Having identified a wide range of governance deficits at both the European and the domestic level, this report shows that the golden era of surveillance is far from over. Indeed, the current labyrinth of public–private co-productions of intelligence and, in particular, non-compelled government access to commercially and publicly available data ought to attract far more legislative attention as well as oversight practice. It should be the next frontier of intelligence reform, and this report aims to provide actionable recommendations, food for thought, and pointers for progress to the pioneers of future intelligence reform.
[1] For a comparative overview of good legislative provisions and oversight practice on bulk collection, see intelligence-oversight.org.
Thorsten Wetzling & Charlotte Dietrich