Ed Felten on the 30% Collection Claim and Technical Debt
Ed Felton has his own take on last week’s claims that the NSA was only collecting 30% of phone data.
He suggests my observation–which he calls an argument–that the dragnet combines data from multiple sources is unlikely because it would pose a great risk to NSA’s credibility.
Theory A: Not under this program: One theory is that the NSA is actually getting a lot of domestic phone call data from another source, so this is another one of the “not under this program” evasions. This would mean the NSA is getting domestic phone call data via some method other than a Section 215 court order. For example, Marcy Wheeler argues that the data is coming from a foreign partner agency.
The argument against this theory is that it assumes the NSA is still willing to deceive the public and policymakers with the “not under this program” maneuver. The price to the agency’s credibility of getting caught in such a trick at this late date would seem to be fairly high.
Of course, on the specific issue of geolocation (which the reports claim is part of the problem) the Administration has always engaged in this game (and was doing so as recently as October), assuring us they don’t collect geolocation under this program.
More importantly, I think Felten misrepresents who might be misinformed. The issue, I believe, is not exclusively about misinformation (though there’s some of that); it’s about classification.
My observation is that the NSA collects a great deal of cell data under EO 12333 authorities — an observation backed by (among other sources) Snowden-released documents.
The question, then, is how much the NSA and ODNI are willing to talk about EO 12333 activities. And the answer to that has consistently been “unwilling.” As recently as October, James Clapper outright refused to answer an Amy Klobuchar question pertaining to EO 12333 authorities. When I asked former senior DNI official Jill Rhodes about EO 12333 collection last Friday — referring exclusively to information ODNI had declassified — she would not address that question either. We should assume that Intel Community sources will not discuss issues pertaining to EO 12333 — publicly at least– all the more so when they involve GCHQ involvement. I believe the Intelligence Committees have more information, but even there, Dianne Feinstein is quite clear that they have less oversight on EO 12333 activities than they do on FISA ones.
In addition, it’s worth noting that the only way Administration figures can have told the truth in all statements — both in their explicit claims to the Courts and Congress that they need the entire haystack and in their anonymous claims they only get 30% of phone data under Section 215 is if the haystack incorporates data from other sources as well. Which the public record shows to be the case.
All that said, I do think Felten’s explanation is part of what’s going on. He suggests the NSA may just have never properly solved some of the underlying problems they claim to be facing today.
Why might straightforward technical issues be holding up the program? One reason is that the program might be mired in technical debt.
For those not familiar with the concept, technical debt is a concept from software engineering. If your project has an engineering problem to address, the “right” response is to understand the underlying cause and address it in a careful (yet cost-aware) fashion. Alternatively, you can slap on a quick and dirty “band-aid” solution that makes the problem go away in the short run but leaves the system more fragile and bug-prone. If you opt for the band-aid approach, you are taking on technical debt. Until you pay back the principal by addressing the underlying engineering problem, you will have to keep paying interest on the debt by devoting engineering effort to coping with extra crashes and bugs.
Although prudent managers take on technical debt at times, there is also a trap—as with financial debt—in which the burden of interest payments makes it more difficult to dig yourself out of debt, and your engineering staff spends all their time “putting out fires” rather than improving the product. Worst case, you can’t keep up with interest payments and can only pay the bills (i.e. keep the system alive) by taking on further debt. Then you slide into technical insolvency, where the system never really works right.
Government systems seem to be at higher risk of technical debt or insolvency, for reasons that would require another post to unpack.
This is why I said that some of the absurd claims peddled to the journalists have some grain of truth, such as the claim that crises in 2009 and 2013 prevented the NSA from fixing this problem. The claim is absurd if you believe the issue was seen as important in 2001 when NSA set up the dragnet or between 2006 and 2008 when NSA operated happily under FISC oversight or in 2011 to 2012 when the NSA was, in fact, working on precisely the issues the leaked reports say underlie the difficulties.
But it’s not absurd if the issue has been a problem primarily during those crisis periods when NSA didn’t manage the issue.
And given that we know Verizon was having problems in 2009 pertaining to the mix of foreign and domestic records, I think it’s safe to say that NSA kluged together solutions during the last crisis.
All that said, i suspect it is a technical debt created by legal debt, in part. While I think the issue here arises from legal arbitrage (the interest in doing what ever is most flexible under the law), I do think that may create technical issues (that should be a cinch to solve).
deleted
well, it’s a theory but that’s all it is, and a nice explanation of technical debt. We’ve all been there. The interest payments aren’t always as draining as he claims though. Sometimes it doesn’t cost you a ton of interest and take up all the capacity of your developers. Sometimes it’s not a high maintenance bandaid at all but it paints you into corners. You might end up not being able to do certain things without major reengineering or changes might take longer, etc. But it doesn’t necessarily cost you a lot in your day to day maintenance.
And as far as technical debt goes, given the amount of money flying around in the intelligence community and the ability to just outsource big projects to big contractor teams (which is what Binney claims happened when they rejected his Thin Thread system and just outsourced another huge system), I’m less inclined to believe that resources were an issue.
Then there is his argument that the NSA would not risk their credibility by saying something that was so obviously untrue and might be exposed. Well they seem to do that every day when they ringfence their answers and statements around 215 program or the FISA programs. And when they use their custom-made redefinitions of words.
Thanks for your thoughtful response to my post. I agree that one of the most important questions related to domestic call phone data is whether NSA has been acquiring masses of domestic phone call data under authorities other than Section 215.
The narrative NSA has been offering about domestic phone call data is that they started collecting it per a directive from President Bush, then brought it within the 215 authority around 2006. It would be hard to square this narrative with the facts if it turns out that there has been massive-scale collection of domestic phone call data based on non-215 authority for years.
I think it’s quite likely that there’s “technical debt” from technology problems.
Consider how fast these systems have grown. Consider the pressures the technology managers have been under. Consider how much money they have to throw around.
You build a system to collect something. A new data source comes along. Software engineers wants to modify the first system to accomodate the second source. With all the money in the world and an extreme disincentive to tamper with the first system (“If you break it, people might die!”), you just build a second system. Then a third, a fourth, etc. With each of these, your technical debt builds geometrically.
Of course, we don’t know. But I’ve seen no reason to believe that, in the way it manages technology, that the NSA is any different than your basic dysfunctional bureaucracy.
@Ed Felten: I think that you should also consider the legal and procedural debt as well. Simply put the NSA was happy to operate under a variety of hazy rules and has, over time, jumped through some hoops, run under others, and generally ignored the legal mire that they have waltzed into. As Marcy has amply illustrated their use of overlapping orders and hand waiving has led to increased legal debt which they are currently fighting by fighting any attempt to examine the actual processes they use. In some respects their unwillingness to inform also stems from this same problem in a legal realm.
“The argument against this theory is that it assumes the NSA is still willing to deceive the public and policymakers with the “not under this program” maneuver. The price to the agency’s credibility of getting caught in such a trick at this late date would seem to be fairly high.”
ROFL. You think they give a tinker’s damn about any “price to their credibility?” THEY FLAT-OUT LIE TO CONGRESS. Furthermore, what exactly would that “price to the agency’s credibility” be paid with? Funding cuts? Oh no, that would mean we’re “soft on terrorism.” Can’t have that.
I find it entirely plausible that the NSA is willing to deceive the public and policymakers in any capacity whatsoever that results in their being left unfettered to pursue their ostensible goal of “protecting” us from all the bogeymen hiding under our beds and in our closets. The “argument against this theory” is in my opinion extraordinarily naive. How many times do you need to be burned, Felten, before admitting that the stove is pretty hot?
@Ed Felten: Actually, if you look at the NSA IG report, it’s clear the dragnet lumped in 12333 data with PSP data from the start (though there must have been a transition between 2006 and 2008 until they were re-lumped after the 215 authorizations) — and current reports show they date the 12333 dragnet application dates to 1998. That doesn’t necessarily mean the 12333 data includes a ton of USP data except as a counterparty to a foreign interlocutor. I’ll have a follow-up on what the 2009 crisis shows; it’s clear the mingling of 12333 data with 215 data was a key problem, which I think led to kluged solutions.
@Saul Tannenbaum: Right. But that’s why I raised legal debt.
We know they collect cell phones in bulk. The question is not why the can’t do it (or can only do it with AT&T and Sprint). It’s why they didn’t choose to do so with Verizon and T-Mobile from the start. And I think that’s likely to be a legal question as much as a technical question.
@emptywheel: I misread your closing paragraph, dropping the “in part” in the first sentence.
That said, I fall back on Bruce Schneier’s observation that NSA data collection is robust: using multiple legal and technical means to collect the same data. It’s an academically interesting question whether the obfuscation that comes with this robustness was designed to make this stuff next to impossible to understand if leaked or whether the confusing nature happened organically.
@Saul Tannenbaum:
Great points.
Gosh, it is getting old that we the people keep on having to ask the, “…bug or a feature?” question.