In 1986, when the world first heard of the events now known as the Iran-Contra affair, John Poindexter, then national security adviser, purged more than 5,000 incriminating emails. Unfortunately for Mr Poindexter, backup files existed.
He won’t be making that mistake again, and neither should anyone else after the formal launch this week of a Pentagon research project headed by Mr Poindexter that is devoted to sifting every electronic trail generated in America to hunt downterrorists.
“It takes what had been in the realm of paranoid conspiracy theorists and puts it in the realm of a potential reality – right here and now,” said Jody Patilla, a consultant for the digital security company @Stake, and a former data analyst at the national security agency.
“In the 50s and 60s, there were considerable abuses in surveillance of US citizens by the federal government. It was made very clear to us over and over again under what circumstances you could collect information about people. This is a very big change.”
The scale of the project, called Total Information Awareness (TIA), is dazzling: computers will be developed to trawl through the vast quantities of data generated by US civilians in their daily lives.
Academic transcripts, ATM receipts, prescription drugs, telephone calls, driving licences, airline tickets, parking permits, mortgage payments, banking records, emails, website visits and credit card slips can all be monitored.
Also dazzling is the intrusiveness that it would impose: a level of surveillance and monitoring of ordinary Americans that is unprecedented in peacetime, and was impossible before the electronic age.
The move has even alarmed supporters of the George Bush administration.
Thirty civil rights groups have written to the White House to oppose the project as well as Mr Poindexter’s return to government, and commentators have accused the administration of creating an Orwellian America.
Although years away from being operational, TIA is seen as a product of the post-September 11 age and the Bush administration’s use of issues of national security to enact sweeping legislation such as the Homeland Security Act, which provides the legal underpinning for TIA.
The project has an initial budget of only $10m (£6.3m), but contracts have been signed with computing firms amounting to tens of millions.
TIA requires the development of software so that information from, for instance, decades-old life insurance records can be accessed as easily as a picture stored on a current driving licence.
It would draw on film from CCTV cameras on motorway toll booths and police speed traps, along with new techniques such as facial recognition, to track potential suspects.
The ultimate purpose of TIA would be to predict potential terrorists by tracking a lifetime of seemingly innocuous movements through electronic paper trails.
Much of the queasiness about the project is generated by memories of Mr Poindexter. He came up with the idea and has a “passion for the project”, Pentagon officials said this week.
But most Americans associate him with Iran-Contra, in which funds from illegal arms sales to Tehran were secretly funnelled to Nicaraguan rebels. In 1990, he was found guilty of conspiracy and obstruction of justice, but the verdict was overturned on appeal.
That unease is unlikely to be alleviated by the logo for Mr Poindexter’s Information Awareness Office, a vaguely Masonic all-seeing eye on top of a pyramid, or the motto: “scientia est potentia”.
At its heart, TIA overturns the notion of a presumption of innocence, civil libertarians say, because it exercises a technique that relies on the widest possible use of information: data mining. They argue that all of the stored data is already available after production of a warrant or during the course of an investigation. The idea behind the project is that terrorists exhibit certain patterns of behaviour that can be winnowed out by mining gigabytes and gigabytes of seemingly mundane activities.
But it is highly controversial even as a scientific endeavour.
Steven Aftergood, of the Federation of American Scientists, said: “It may be that there is no such thing as an identifiable pattern of behaviour associated with terrorists. It may be untrue that terrorists preferentially fly out of certain airports using certain airlines using tickets paid for by certain credit cards, or that they all buy certain books from Amazon dotcom, or that they all originate from particular countries.
“All of those assumptions may be unfounded, in which case there is no meaningful pattern for which to search through this universal database.”
The problem is, however, that once the algorithms are out there looking for a particular type of person, it may be difficult to reset them.
There are also mountains of corrupted data, with wrong names and addresses. That deepens the danger for abuse, and the prospect of innocent individuals becoming entangled in the government’s terrorist search engines.
The rumblings against Big Brother have grown so great that the Pentagon was compelled to intervene to ease anxieties. The undersecretary of defence for technology, Pete Aldridge, told reporters this week that test runs of TIA will use fabricated data.
“There is some real data that we use, but it’s normal data that’s available legally,” he said. “Most of the data is synthetic. It’s generated just to exercise the analysis.”
He also said that Mr Poindexter would not be deciding whom to spy on.
ZNetwork is funded solely through the generosity of its readers.Donate