Imperva's Breach Post-Mortem: API Key Left ExposedImperva Says Key Was Stolen and Used to Take Critical Customer Database
Cybersecurity vendor Imperva's breach post-mortem should serve as a warning to all those using cloud services: One mistake can turn into a calamity.
See Also: The Global State of Online Digital Trust
In August, Imperva warned that a customer database for its web application firewall product, formerly known as Incapsula, had suffered a breach (see Imperva Alerts Customers About 'Security Incident').
Breaches by security vendors are usually cause for extra worry due to the level of access they inherently have as part of protecting organizations.
Imperva's leaked data included email addresses, salted and hashed passwords, and for some customers, API and TLS keys. Imperva's WAF is popular with banks, and according to its website, its WAF customers also include GE, Siemens and PayPal's Xoom.
On Thursday, Imperva CEO Chris Hylen offered a more detailed post-mortem on what went wrong and writes that the company "profoundly" regrets the incident.
Hylen addresses why it took a little over six weeks to come forward with more information, writing that there's "a natural tension between the desire to share newly discovered information with customers and the need of an investigation to progress in a forensic and regimented manner."
He adds: "Our approach to balance this tension is to focus on being fact-driven in our communications to employees, customers, partners and the community, which continues to mean that we must confirm findings and assessments (and take actions to protect all of our customers) in order to responsibly share additional details."
Stolen AWS API Key
Imperva learned of the breach through "a third party requesting a bug bounty." It's unclear if the security company paid a bounty. Imperva is listed on bug bounty management company HackerOne's directory, although the site notes the entry is a community-created listing and hasn't been verified for accuracy.
The customer data that was taken in October 2018 was a snapshot of customer data that was current as of Sept. 17, 2017, Hylen writes.
The database was created as Imperva was seeking to improve performance and add to its ability to scale as its customer base was growing, Hylen writes. To scale the user database, Imperva migrated to Amazon Web Services' Relational Database Service.
"Thus far, we have not found any malicious behavior targeting our customers (logins, rule changes, etc.) and have implemented procedures to continue monitoring for such activity."
—Chris Hylen, Imperva
It created a snapshot of the database for testing. But it made a critical mistake: leaving an internal compute instance containing the AWS API key accessible from the internet.
The key was stolen, Hylen writes, and "was used to access the snapshot."
The disclosure meant Imperva's customers had to quickly take action. Its customers changed more than 13,000 passwords. Also, 13,500 TLS certificates were replaced, and 1,400 API key were regenerated, Hylen writes.
"Thus far, we have not found any malicious behavior targeting our customers (logins, rule changes, etc.) and have implemented procedures to continue monitoring for such activity," Hylen writes. "We remains vigilant, however, and will continue to monitor for malicious behavior."
Although it's clear that the API key should have never been left exposed to the internet, Hylen writes that company has undertaken a host of measures that would help it detect mishaps.
For example, the instance that contained the key "plus other unused and experimental instances discovered as part of the investigation were archived to preserve logs, and subsequently decommissioned." It is also doing daily scans and audits of its S3 buckets.
Log events, including VPC NetFlow logs, - which come from AWS CloudTrail and Guarduty - are now forwarded into its user and entity behavioral analytics software, Hylen writes.
"We have also developed SOC dashboards to monitor and alert on malicious activity at the customer account level (API and management console)," he writes. "These leverage our product's built-in audit logs."
Within its WAF, Hylen writes, the company has added a new feature that allows customers to download full audit reports, which include logins, password changes, rule changes and "dozens of other event types. These reports can be generated via the management console and API."