Call Detail Ruckus
Matthew Yglesias points out why the basic data-mining explanation for the collection of call detail records (and whatever else) from the major telcos makes no mathematical sense.
Mathematician John David Farley suggests some other things the NSA might “constructively” do with the data. James Joyner complains that Farley seems not to consider that the NSA may actually be doing these things.
David Ignatius guesses that the idea is to make it easier to track suspects as they switch cell phones by tracing back the patterns of called numbers from the old phone to the new one.
The telcos swear up and down that they didn’t give no information to no National Security Agency, no way.
Various folks dig up executive memoranda that appear to give companies the privilege to lie to the public and conceal their activities if the Director of National Intelligence says it’s okay, calling into question those denials.
What do I think is the government’s reason for collecting all this call data from the telcos? They need a reason? Look, the government, institutionally, tends to want to know ever more about its citizens. In that sense, the vaunted post-9/11 mentality just serves to help gratify a preexisting desire.
Less snarkily, why does there only have to be one reason? This is actually the problem: once the government has a capability, it sets about finding more and more uses for it. Here’s just one: Do-it-yourself pen registers. Pen registers officially require a court order. There’s not much worthy of the name “review,” but the feds have to, per the law, at least go before a judge and say, “Hey, this TN/MDN/IP address is relevant to an investigation, now sign.” But if we know anything about the Bush Administration, it’s that they just hate getting the approval of courts for anything investigative, even tame ones like the FISA Court.
But if you’re adding realtime traffic data to historical records for most major carriers, you don’t need to go to the telcos and say, Put a pen register on this here number. You’ve got all the tools the telco itself would use to comply. So you just do it. Sweet!
Like I said, I would never argue that do-it-yourself pen registry is “the reason” the government decided to collect CDRs (and whatever else). There’s an embarrassment of reasons. And the data itself breeds more of them.

Comment by Leonard —
May 17, 2006 @ 9:45 pm
Yglesias is right that any sort of pattern-finding applied to the data as a whole, with no outside input, is likely not to work. However, as one of his commenters points out, outside input is easy enough. As soon as you ID any ”terrorist” or number, you can quickly construct the calling network that contains it, then mathematically analyze that for patterns. I think this could be useful information by itself, but of course, why stop with just one form of outside input? Instead, once you’ve got a potential cell you go to FISA an wiretap it to listen. (or screw FISA and do it anyway! Who’ll stop you?) Or, you link up all those numbers with addresses and names. That is, you do all the things that they’ve assured us are not being done. Of course I read their denials here as ”we’re not doing anything to the whole dataset” other than linking numbers. With specific suspect numbers, I expect they’re doing the full monty.
Now, say I’m a terrorist and I want to hide from this sort of thing. That’s reasonably easy to do, but it does require a bit of extra coordination. You have to sync up ditching cell phones for new ones while getting the new numbers out in some ”out of band” fashion. This would be easy to do using the Net, but maybe these guys aren’t so smart, or just didn’t know. Now, they should. Any terrorist worth anything will be taking measures. Of course, this pattern in itself, if they can detect it, would be highly suspicious. But they’d need to mine the entire corpus including linking it with outside info to do that.
Comment by Nell —
May 17, 2006 @ 10:35 pm
Now, say you’re a government worker, or a Congressional staffer, who wants to talk with a journalist about something. You apparently need to start thinking and acting clandestinely (rather than just discreetly) — like a criminal, terrorist, or spy.
Comment by anon —
May 17, 2006 @ 11:15 pm
Matt’s post is not wrong, as far as it goes, but it rests on a false assumption — that a trustworthy confidence interval exists for any technique being used by the NSA.
If you want to use this data to actively search for terrorist activity, you need a profile of terrorist phone activity as distinct from other types of phone activity. (Note that this is a truly blind model; if you have some particular numbers of interest, then you can use those as a starting point for a search.) I have no idea if such a profile exists, and neither does the NSA at this point, I’d wager. So how will they train the system? By attempting to build a profile from what they know about the phone activity of previous terrorists. That’s going to be a small, atypical data set, and I find it hard to believe it’s very good at this point — or that it will ever be that good.
Now as a passive database, one that ”comes into play” once a suspected terrorist’s phone is identified — I can see how that might be useful. But I don’t know why you couldn’t just obtain those records when needed, rather than create a massive database in advance.
But really, to put this all in proper perspective, I would point you towards Chris Bray’s cab driver.
Anon
Comment by Leonard —
May 18, 2006 @ 12:45 am
Anon, the reason why you create the DB is to have it there immediately when you ID a ”terrorist” number. Then you can instantly ID other associated numbers. If you can only start gathering data after IDing a suspect number, (a) it takes time, and (b) you’re likely to miss numbers, unless you wait a really long time. And the problem iterates. Consider a number two hops away from the initial number. To get there, you must first ID the intermediate number, then get a warrant for it. Then watch some more, etc.
The danger here is not that the thing doesn’t work.
Comment by jlw —
May 18, 2006 @ 9:06 am
But are phones really how a terrorist communicates? Seems extraordinarily risky, even without the publicization of the NSA snooping. If I recall correctly, the 9/11 squads used image swapping sites to pass along information. If anything happens in the next year or so, I’d bet the key messages will be sent via Flickr or YouTube rather than NexTel.
Comment by Leonard —
May 18, 2006 @ 9:33 am
Jlw, I don’t know how terrorists communicate. Certainly the phone network is there, and powerful, so I’d guess that they do make use of it some. Perhaps not for most sensitive stuff, but I’d guess it’s pretty hard to avoid entirely.
There’s useful info to be gained even if the bad guys get off the phone network with each other. There’s still things that anybody does with phones which the Feds would want to know about. I.e., you ID some guy as a terrorist; now you look into his phone history and see a bunch of calls to flight schools.
Of course, for any given thing in life, it is probably at least possible to take it off the phone network. I.e. you could drive around to flight schools to get prices. But this, in itself, exposes you to a new set of risks, especially if you look and sound like an Arab. Also, it slows you down.
Another way in which you could use this info even against pretty hardcore guys would be to fish into their ”normal” acquaintance network which they do use phones to communicate with. I.e., say I’m a terrorist trying to do an operation in Baltimore. And say I’ve completely kept my group off the phone net, by using the Internet. But I still use a cell phone, to call people I know in my everyday existence. Friends, family, business acquaintances, etc. Now you get suspicious of me for some outside reason and link me with my phone number. It may be that you can get some useful info just by talking to all my ordinary acquaintances to see if they’ve noticed anything funny about me. ”Yes, he’s out of town for a month. Business, he said.”
Comment by jlw —
May 18, 2006 @ 12:28 pm
I donno, Leonard. Seems like a lotta nail conversion going on there–we’ve got this hammer, what can we do with it? (I mean, that’s terr-ist related.)
I think it’s safer to assume that al-Qaeda agents will operate much as the Cold War Communist agents did: coded messages hidden in plain sight, drop points, deep covers, and whatnot. The fact that the September 11 teams had critical contact with the U.S. economy (i.e. flight schools) is a bit of a fluke. The packages that are likely to be delivered in the future–explosives, biological agents, small arms fire–can be handled by agents who won’t need sophisticated training. They will serve the group best by having zero contact with it, at least until a single coded message arrives activating the scheme.
No, sweeping up phone communication contacts is terrorist-related only if you consider Dana Priest or Joe Trippi terrorists.
Comment by No Nym —
May 18, 2006 @ 1:14 pm
I think people are missing out on the real story here. Yes, mining is dumb and will implicate innocents. Yes, using the data only after you have a suspect number is a good way to go. There’s even a weak MI-5 style argument that getting warrants, ala criminal prosecutors, is too slow for counterterrorist work.
What people should be upset about is not merely that they’re assembling these data, or that they will be abused. What should have us scratching our heads, in a post-Katrina world, is that if they perceive the need for this kind of thing, they obviously a)suspect there are a good number of terrorists, sympathizers, supporters, or funding agents still inside the US; and b) have absolutely no clue who they are, where they are, or even how to go about finding them.
Remember, Lindh infiltrated AQ very early and showed how low their bar was (Moussoui is a similar case). If the CIA, FBI, et al feel they need this kind of thing, it provides good evidence that, 5 years after 9/11, our intelligence and law enforcement apparatus is ineffectual at counterterrorism.
I believe that one could also sell this argument to Fox news, and get the rightosphere upset about it, as well.
Comment by Leonard —
May 18, 2006 @ 2:24 pm
No Nym – I’m not really upset by data mining phone calls; it’s too bloodless for that. (It’s upsetting to see pictures of children screaming with pain, whose legs were reduced to hamburger using my taxes.)
Rather I consider state-security maximization a logical progression of an entire policy direction that we shouldn’t be taking. We shouldn’t need this sort of intelligence against ”terrorists”, not because, given the existence of terrorists who hate us, it wouldn’t help. But because, we shouldn’t ”give” the statists those terrorists. They – the state – took the actions that are the terrorists’ animating grievances against us, and those actions were not necessary. A policy of isolationism would have kept us out of the Middle East, and thus, no 9/11, no Iraq Attaq.
”Our” ends – ”regime change” the world to stop terrorism – are so desperately important to us as to require us to accept warfare as our means. That includes, as a permissible (though regrettable) side-effect inflicting pain on innocent children. That’s grave, even evil, stuff we’re doing. Given that, it’s pretty hard to argue that means as relatively innocuous as data gathering and wiretapping should not be allowed. I don’t see any way to reject the more innocuous means without also rejecting the more offensive ones. And that means giving up the war.
Even if we return to isolation and peace tomorrow, we’ll still have to maintain intrusive security measures for a long time while we wait for the old generation of terrorists to age and die, and for the world to accept that we really have defanged ourselves.
Comment by Barry —
May 18, 2006 @ 3:17 pm
”There’s even a weak MI-5 style argument that getting warrants, ala criminal prosecutors, is too slow for counterterrorist work.”
It’s worth repeating ad nauseum that the FISA procedure are not too slow. Period. There’s no argument, let alone even a weak one.
Comment by Neel Krishnaswami —
May 18, 2006 @ 6:05 pm
Full agreement, Barry — I remember when FISA creeped me out because of the ludicrously low barrier it presented to federal snooping. Who’d have thought that the feds would decide that even that token obeisance to the rule of law was too much work?
Comment by Hob —
May 19, 2006 @ 4:42 pm
I just figured out why that David Ignatius article seemed so fishy to me: it only makes sense if you think that in a terrorist network of 5 or 10 or 15 people, only one guy changes his cell phone number at a time, and then keeps calling all the same numbers to give the feds a fighting chance at doing their pattern-matching. That would be stupid even if there weren’t any huge NSA project – if just one of those other numbers were traced, the new cell phone would be blown anyway. The article might as well have been headlined ”Columnist in Need of an Excuse.”