Heckuva Job, Mikey

by digby

NSA killed system that sifted phone data legally

Sources say project was shelved in part because of bureaucratic infighting

By Siobhan Gorman
Sun Reporter

May 17, 2006, 10:27 PM EDT

The National Security Agency developed a pilot program in the late 1990s that would have enabled it to gather and analyze massive amounts of communications data without running afoul of privacy laws. But after the Sept. 11 attacks, it shelved the project — not because it failed to work — but because of bureaucratic infighting and a sudden White House expansion of the agency’s surveillance powers, according to several intelligence officials.

The agency opted instead to adopt only one component of the program, which produced a far less capable and rigorous program. It remains the backbone of the NSA’s warrantless surveillance efforts, tracking domestic and overseas communications from a vast databank of information, and monitoring selected calls.

[…]

n what intelligence experts describe as rigorous testing of ThinThread in 1998, the project succeeded at each task with high marks. For example, its ability to sort through massive amounts of data to find threat-related communications far surpassed the existing system, sources said. It also was able to rapidly separate and encrypt U.S.-related communications to ensure privacy.

But the NSA, then headed by Air Force Gen. Michael V. Hayden, opted against both of those tools, as well as the feature that monitored potential abuse of the records. Only the data analysis facet of the program survived and became the basis for the warrantless surveillance program.

The decision, which one official attributed to “turf protection and empire building,” has undermined the agency’s ability to zero in on potential threats, sources say. In the wake of revelations about the agency’s wide gathering of U.S. phone records, they add, ThinThread could have provided a simple solution to privacy concerns.

A number of independent studies, including a classified 2004 report from the Pentagon’s inspector-general, in addition to the successful pilot tests, found that the program provided “superior processing, filtering and protection of U.S. citizens, and discovery of important and previously unknown targets,” said an intelligence official familiar with the program who described the reports to The Sun. The Pentagon report concluded that ThinThread’s ability to sort through data in 2001 was far superior to that of another NSA system in place in 2004, and that the program should be launched and enhanced.

Is it possible that these people are actually working for Al Qaeda? It’s almost impossible for anyone to fuck things up this consistently without consciously trying.

WTF????

BTW: I don’t know if everyone has noticed, but this reporter Siobhan Gorman has been doing excellent work on this NSA story for the Baltimore Sun. Some kudos accolades are in order.

.