The NSA's Shadow Database

Last week saw the release of yet another independent report condemning the NSA's bulk collection of Americans' phone records. In its report, the Privacy and Civil Liberties Oversight Board (PCLOB) called for an end to the program, concluding that it is unlawful, threatens the privacy of Americans, and does little to keep the country secure. Its recommendations reflect a deepening public consensus that President Obama and Congress must do more to halt the NSA's efforts to vacuum up as much of our personal data as possible.

One element of the privacy board's report, however, hasn't received the attention it deserves. That's the push to limit the NSA's unfettered access to a pool of Americans' phone records called the "corporate store" — a problem that I first wrote about in August. The privacy board wants to stop the NSA from searching or datamining this pool of sensitive data whenever it pleases. That's a good idea. In particular, included among the report's four immediate recommendations is this one:

This should set off alarm bells about how much private data is likely sitting in the corporate store and how few restrictions currently apply to the NSA's use of this data. Indeed, elsewhere in its report, the privacy board estimates that the corporate store holds roughly 120 million phone numbers and the complete calling records of 1.5 million people — from 2012 alone.

Here's how the corporate store works. It's basically a database that warehouses the results of the NSA's queries of the raw telephone data it collects daily. When the NSA runs a query on that raw data — called the "collection store" — the results are dumped into the corporate store, where they are combined with the results of previous queries. And, crucially, the corporate store is not subject to the restrictions that intelligence officials have been so eager to talk about.

Intelligence officials often claim that the NSA's access to our phone records is extremely limited. They emphasize that, under orders from the FISA court, the NSA queries its call-records database only when there is "reasonable suspicion, based on specific and articulated facts" that a phone number is linked to specific foreign terrorist organizations. For empirical support, they cite the fact that the NSA conducted less than 300 queries of the raw phone data in 2012.

But these claims disguise much larger numbers and a wide open backdoor to much of the same phone data. Because NSA analysts were, for years, permitted to examine the call records of all individuals within three "hops" of a specific target (now two hops, since the president's January 17 speech), a single query could yield millions of phone numbers. And with each query, all of those numbers are added to the corporate store.

This should give you a sense of just how many numbers "three hops" can mean:

With that in mind, here's the key point: The NSA is not limited in how it can further search, analyze, or datamine the phone records amassed in the corporate store. Instead, as set out in the court order governing the program, the NSA "may apply the full range of SIGINT analytic tradecraft" to this data, even by combining it with information collected under other surveillance programs. In short, the corporate store is very big, and the rules restricting its use are very weak.

So, for instance, if you happen to order take-out from the same restaurant as someone targeted by the NSA, your complete calling records can be examined by the NSA—and saved in the NSA's corporate store indefinitely for future searching without any type of suspicion or other restriction.

The privacy board deserves credit for focusing on this gaping loophole in the current rules. While many, like the President's Review Group, have called for dramatically reining in the NSA's phone-records program, the privacy board is the first government entity to recognize that these reforms must include restrictions on the NSA's ability to amass an unregulated shadow database that it can search at will.

Many of the NSA's surveillance programs depend on what might be termed "the illusion of restraint." That is, they are designed to appear far more narrow and restricted than they are, with built-in backdoors, incredibly loose targeting rules, elastic definitions, and disguised ways of scooping up Americans' data in vast quantity. The subject of the privacy board's next report — the NSA's dragnet surveillance of Americans' international phone calls and emails under the FISA Amendments Act — is another case in point. Let's hope that report, due in the next few months, is just as vigilant about identifying all the ways the NSA collects and searches our private data as it pleases.

Learn more about government surveillance and other civil liberties issues: Sign up for breaking news alerts, follow us on Twitter, and like us on Facebook.