The Pentagon. Image: David B Gleason/Flickr (CC BY-SA 2.0)
Anyone with a free AWS account could have viewed information stored in the cloud by the US Department of Defense, according to security researcher Chris Vickery.
The UpGuard cyber risk team recently revealed that three publicly downloadable cloud storage servers on Amazon Web Services (AWS) contained a huge quantity of data collected for apparent US Department of Defense intelligence purposes.
The repositories seemed to contain billions of public internet posts and news commentary from a broad range of countries, including the US, and this data was held by United States Central Command (Centcom) as well as the United States Pacific Command.
Both of these organisations are Pentagon unified combatant commands heavily involved in US military operations across Asia, the South Pacific and the Middle East.
Eight years of user data left exposed
UpGuard explained: “The data exposed in one of the three buckets is estimated to contain at least 1.8bn posts of scraped internet content over the past eight years, including content captured from news sites, comment sections, web forums and social media sites like Facebook, featuring multiple languages and originating from countries around the world.
“Among those are many apparently benign public internet and social media posts by Americans, collected in an apparent Pentagon intelligence-gathering operation, raising serious questions of privacy and civil liberties.”
UpGuard said that following a cursory examination of some of the data, it can see some loose correlations to regional US security concerns – for example, posts about Pakistani or Iraqi politics. Posts were in many languages, with an emphasis on Arabic, Farsi, and other Central and South Asian dialects spoken in Afghanistan and Pakistan.
While it notes that much of the vast amount of data captured is “benign”, it is still not clear why such large swathes of data were accumulated. It is the belief of UpGuard that the majority of the posts are from law-abiding citizens, which is alarming.
Third-party vendor
An apparently defunct third-party private sector government contractor called VendorX was employed to create the software that formed the data stores, which shows once again that third-party vendor risk is a growing issue.
Researcher Chris Vickery found the original buckets in September 2017. The three AWS buckets were configured to allow any AWS global authenticated user to look through and download the content.
Although on the surface it looked like a disparate collection of data, it apparently “appears to constitute an ingestion engine for the bulk collection of internet posts” – in other words, a conveniently searchable database.
Poor security protocols
In a damning indictment, UpGuard concluded that a simple change in permissions settings would have kept this massive repository of data secured.
Major Josh Jacques, a spokesperson for Centcom, said to CNNMoney that the data is used for measurement and engagement activities of online programmes on public sites, and denied it was being processed for intelligence purposes.
Although the information was publicly available, the failure to fully secure it and the mass retention of the data raises serious concerns.
The Pentagon. Image: David B Gleason/Flickr (CC BY-SA 2.0)
