John Reed Stark

The Capital One data hack has attracted a great deal of attention, not least because of the size and extent of the breach, but also because the hacker apparently managed to steal data from The Cloud. In the following guest post, John Reed Stark, President of John Reed Stark Consulting and former Chief of the SEC’s Office of Internet Enforcement, takes a closer look at this aspect of the Capital One data breach and asked whether Amazon, the cloud service provider, can be held liable for the hack? Stark takes a close look at the technology involved and analyzes the potential liability issues between Capital One, on the one hand, and Amazon, on the other. A version of this article originally appeared on Securities Docket. My thanks to John for allowing me to publish his article as a guest post on this site. I welcome guest post submissions from responsible authors on topics of interest to this blog’s readers. Please contact me directly if you would like to submit a guest post. Here is John’s article.

 

***********************

 

“The lady doth protest too much, methinks.”

Amazon Web Service’s (AWS’s) announcement after the Capital One data breach was immediate, unqualified, unapologetic and absolute. AWS emphatically denied any culpability for the hack, which involved the theft of over 100 million Capital One credit applications residing on the AWS cloud service, declaring:

“AWS was not compromised in any way and functioned as designed . . . The perpetrator gained access through a misconfiguration of the web application and not the underlying cloud-based infrastructure. As Capital One explained clearly in its disclosure, this type of vulnerability is not specific to the cloud.”

In the complex world of data breaches, the AWS announcement seems surprisingly bold. How could AWS be so sure so soon of its blamelessness? 

First off, the FBI shared only minor details about the Capital One hacker’s modus operandi, which is typical of the FBI, not only because the Capital One investigation is confidential and nonpublic, but also because the investigation is ongoing. The FBI could still discover, or might already have discovered, a range of inculpatory facts relating to AWS but opted not to share the information with the public.

Second, AWS made no mention of any extensive investigation conducted by an objective and independent third party. Engaging an independent law firm and digital forensics firm to confirm internal findings, and recommend future remedial actions is a fundamental element of digital incident response – and a process that is typically trumpeted out immediately. Yet, AWS makes no mention of undertaking any sort of independent, transparent analysis, which would not only shed sunlight into AWS’s practices, but would also contribute to a far more thoughtful, conscientious and meritorious defense. Right now, AWS’s position seems self-serving and lacks credibility.

Finally, given that cyber-attackers will often traverse across a company’s network and into the networks of its vendors or vice versa, cyber-attacks can often result in disputes as to the culpability for an attack. Indeed, just about every corporate data breach that involves a third party vendor results in some level of finger-pointing between the two, especially when tens (or even hundreds) of millions of dollars are at stake, which is likely the case with the Capital One data breach.

AWS’s quick and categorical denial allows for no wiggle room, hedging or vacillation, and seems more befitting a glowing promotional advertisement than a cautious liability pronouncement. Lance Armstrong was more equivocal when he made his first doping denials (and we all know how that turned out).

AWS may very well be correct about their liability (or lack thereof) – they can make a range of powerfully compelling arguments. Moreover, security within cloud applications can be incredibly challenging. However, understanding the relative liability of parties involved in any data beach is complicated and the juxtaposition of law and technology pervading the Capital One/AWS situation is worthy of thorough analysis and discussion.

This article: 1) Provides some background (including some legally relevant technical details) about the Capital One data breach; 2) Discusses the oddly co-dependent relationship between Capital One and AWS; and 3) Analyzes the potential liability issues arising between the two.

The Capital One Data Breach

On July 29, 2019, FBI agents arrested Paige A. Thompson on suspicion of downloading nearly 30 GB of 100 million Capital One Financial Corp credit applications from a rented cloud data server. The FBI says Capital One learned about the theft from a July 17, 2019, email stating that some of its leaked data was being stored for public view on the software development platform Github. That Github account was for a user named “Netcrave,” which includes the resume and name of Paige A. Thompson. According to the FBI, Thompson also used a public Meetup group under the alias “erratic,” where she invited others to join a Slack channel named “Netcrave Communications.” 

KrebsOnSecurity actually entered the open Netcrave Slack channel on July 30, 2019, and reviewed a June 27, 2019 commentary by Thompson, which listed various databases she found by hacking into improperly secured AWS cloud accounts, suggesting that Thompson may also have exfiltrated tens of gigabytes of data belonging to other major corporations.

Along these lines, on August 14, 2019, the FBI alleged in a “Memorandum in Support of Motion for Detention,” that Thompson has not only stolen data from Capital One, but has also targeted “more than 30 other companies, educational institutions, and other entities,” hijacking multiple terabytes of data. Much of the data hacked from the other entities, which the government has yet to identify, does not appear to contain personal identifying information, though the review of the data is not yet complete.

According to the Wall Street Journal, Thompson suggested in certain online postings that she had accessed data at several other entities, including Ford Motor Company and UniCredit SpA, Italy’s largest bank, and Michigan State University. Ford said it was not affected. UniCredit and Michigan State University have said they were investigating the incident.

An SSRF Attack?

Understanding the technical aspects of the Capital One data breach unfortunately requires tolerating the jargon of an alphabet soup of acronyms and bewildering network nomenclature. However, although somewhat confusing, a brief discussion of at least one technical theory of Thompson’s hack could impact liability, and is worthy of discussion, namely the cyber-attack referred to as Server Side Request Forgery or “SSRF.”

During an SSRF attack, Capital One’s firewall (referred to as a web application firewall or “WAF”) is tricked into running commands that it should never have been permitted to run, including those that allow it to gain a foothold and communicate with AWS. (To learn more about SSRF, read this article by Hackerone).

According to the criminal complaint, Thompson allegedly obtained credentials for an account known as “******-WAF-Role” — WAF refers to web application firewall — which then allowed her to pull up a list of more than 700 AWS S3 buckets and folders and extract data from them. The criminal complaint does not specify how she obtained the AWS credentials, though Capital One and AWS have publicly cited a “firewall misconfiguration” as the cause of the breach.

According to KrebsonSecurity, the problem stemmed in part from a misconfigured open-source WAF that Capital One was using as part of its AWS operations. KrebsonSecurity explains the apparent SSRF Capital One attack as follows:

“The misconfiguration of [Capital One’s] WAF allowed the intruder to trick the firewall into relaying requests to a key back-end resource on the AWS platform. This resource, known as the “metadata” service, is responsible for handing out temporary information to a cloud server, including current credentials sent from a security service to access any resource in the cloud to which that server has access. In AWS, exactly what those credentials can be used for hinges on the permissions assigned to the resource that is requesting them. In Capital One’s case, the misconfigured WAF for whatever reason was assigned too many permissions, i.e. it was allowed to list all of the files in any buckets of data, and to read the contents of each of those files.”

In a blog postEvan Johnson, manager of the product security team at Cloudflare, explains SSRF further and in plain English:

“Every indication is that the attacker exploited a type of vulnerability known as SSRF in order to perform the attack. SSRF has become the most serious vulnerability facing organizations that use public clouds. SSRF is not an unknown vulnerability, but it doesn’t receive enough attention and was absent from the OWASP Top 10 [a list of The Ten Most Critical Web Application Security Risks]. SSRF is a bug hunters dream because it is an easy to perform attack and regularly yields critical findings, like this bug bounty report to Shopify.”

AWS’s Position

AWS remains steadfast in its position of 100% innocence. In a statement provided to KrebsOnSecurity, AWS said it is inaccurate to argue that the Capital One breach was in any way caused by AWS’s identity access management, AWS’s instance metadata service, or AWS’s firewall. Per AWS:

“The intrusion was caused by a misconfiguration of a web application firewall and not the underlying infrastructure or the location of the infrastructure . . . AWS is constantly delivering services and functionality to anticipate new threats at scale, offering more security capabilities and layers than customers can find anywhere else including within their own datacenters, and when broadly used, properly configured and monitored, offer unmatched security—and the track record for customers over 13+ years in securely using AWS provides unambiguous proof that these layers work.”

Capital One appears to agree with AWS’s assessment. According to the Capital One news release announcing the incident, the firewall configuration vulnerability that Thompson exploited is “a specific configuration vulnerability in our infrastructure . . . not specific to the cloud.” In other words, Capital One’s firewall runs on AWS’s servers but may have been targeted no matter where it was located. Capital One even touts its cloud infrastructure as helping with its incident response, stating:

“The speed with which we were able to diagnose and fix this vulnerability, and determine its impact, was enabled by our cloud operating model.”

The AWS/Capital One Love Affair

The lack of accusations and unwavering amicability between Capital One and AWS is not at all surprising. The mutually adoring relationship between the two companies has always been more akin to husband/wife than company/vendor.

Throughout AWS’s website, AWS has posted a myriad of glowing video and case study tributes to Capital One, almost as if Capital One were owned by AWS. Upon close examination of all of this awkward Capital One spin, there seems to exist an almost eerie entanglement and symbiosis between the two mega-corporations. Here are links to just a few of the many case studies scattered throughout the AWS website, together with some of the AWS-celebratory Capital One quotes contained in each:

  • A case study entitled, “How Capital One Reduced its Data-Center Footprint, Expanded its Use of Microservices, and Reimagined Banking Using AWS.” (“In everything we do at Capital One, we always start from what our customers need and work back from there to figure out how to give it to them. The most important benefit of working with AWS is that we don’t have to worry about building and operating the infrastructure necessary to do that and can instead focus our time, money, and energy on creating great experiences for our customers.” George Brady, Executive Vice President and Chief Technology Officer, Capital One);
  • A case study entitled, “On-Demand Infrastructure on AWS Helps Capital One DevOps Teams Move Faster Than Ever” (“By using AWS, we’ve cut the time needed to build new application infrastructure by more than 99 percent. With the virtually instantaneous infrastructure available on AWS, our DevOps teams have the building blocks they need to start developing any new product as soon as they understand the intent behind it.” John Andrukonis, Chief Architect, Capital One); and
  • A case study entitled, “Capital One Contact Centers Innovate Faster Using Amazon Connect” (“The flexibility of Amazon Connect lets us add new features in weeks instead of the three to six months that our last solution required.” Rajiv Sondhi, Vice President for Software Engineering and Digital Technologies, Capital One).

AWS also showcases the interconnectivity of its Capital One relationship on the AWS website, hailing the AWS/Capital One alliance as the cornerstone of its technological prowess, stating:

Capital One is using AWS as a central part of its technology strategy. As a result, the bank plans to reduce its data center footprint from eight to three by 2018. Capital One is one of the nation’s largest banks and offers credit cards, checking and savings accounts, auto loans, rewards, and online banking services for consumers and businesses. It is using or experimenting with nearly every AWS service to develop, test, build, and run its most critical workloads, including its new flagship mobile-banking application. Capital One selected AWS for its security model and for the ability to provision infrastructure on the fly, the elasticity to handle purchasing demands at peak times, its high availability, and its pace of innovation.” 

AWS even goes so far as to promote Capital One by featuring articles written by outside publications about Capital One. The level of praise and attention that AWS bestows upon Capital One knows no boundaries and has obviously drawn substantial investment of time, effort and resources by both companies.

AWS clearly considers Capital One to be more than a treasured customer or partner. AWS treats Capital One as if it were almost an outside source of revenue – and the feeling is mutual. In fact, at a 2015 AWS conference, Capital One’s CIO Rob Alexander gushed ad nauseum over AWS, touting how Capital One uses the breadth and depth of services, high availability, speed, and elasticity of the AWS cloud for its mission-critical applications and will reduce its data-center footprint from 8 to 3 by 2018.

The AWS/Capital One Contract

When determining cybersecurity-related vendor liability such as AWS, what matters most is the contractual arrangement between AWS and Capital One, which arguably tilts any liability issues in favor of AWS. For instance, per AWS’s standard customer agreement(available on AWS’s website), provisions 4.14.2 and 4.3, relating to data security responsibilities state:

4.1 Your Accounts. Except to the extent caused by our breach of this Agreement, (a) you are responsible for all activities that occur under your account, regardless of whether the activities are authorized by you or undertaken by you, your employees or a third party (including your contractors, agents or End Users), and (b) we and our affiliates are not responsible for unauthorized access to your account.

4.2 Your Content. You will ensure that Your Content and your and End Users’ use of Your Content or the Service Offerings will not violate any of the Policies or any applicable law. You are solely responsible for the development, content, operation, maintenance, and use of Your Content.

4.3 Your Security and Backup. You are responsible for properly configuring and using the Service Offerings and otherwise taking appropriate action to secure, protect and backup your accounts and Your Content in a manner that will provide appropriate security and protection, which might include use of encryption to protect Your Content from unauthorized access and routinely archiving Your Content.

The above provisions are not ambiguous, and clearly define data security responsibilities to belong to the AWS customer. In other words, AWS mandates that once Capital One opted to rent one of AWS’s digital storage warehouses, it became Capital One’s responsibility to make sure the online warehouse entry door was always locked.

AWS Configuration Defaults

The AWS Customer Agreement does, however, include a provision about AWS’s responsibilities, which could arguably require that AWS maintain appropriate firewall default configurations that would thwart a SSRF attack. Section 3.1 of the AWS customer agreement states:

3.1 AWS Security. Without limiting Section 10 or your obligations under Section 4.2, we will implement reasonable and appropriate measures designed to help you secure Your Content against accidental or unlawful loss, access or disclosure.

According to a blog post by Evan Johnson, manager of the product security team at Cloudflare:

“The [SSRF vulnerability] is common and well-known, but hard to prevent and does not have any mitigations built in to the AWS platform . . . The impact of SSRF is being worsened by the offering of public clouds, and the major players like AWS are not doing anything to fix it . . . In my opinion, it’s clear that AWS’ product offering is not complete since this is a major and recurring problem amongst their biggest customers. AWS should do something about this because IAM [identity access management] is the root of all security within AWS.”

Along these lines, some experts agree that: SSRF threats on AWS remain a known weakness discussed among security experts since as early as 2014; AWS has been repeatedly asked to improve measures for SSRF defense; and AWS could fix the problem with stronger communication and updated WAF default configuration settings.

The Capital One AWS Contract

The Capital One/AWS contract is not a public document and whether the Capital One contract is actually consistent with AWS’s advertised customer agreement is unknown.

The AWS contract terms for Capital One might very well vary, especially given the lengthy history of, and co-dependent relationship between, AWS and Capital One. Amazon first created AWS, more than a decade ago after building out similar capacity for its own retail site, which had unusually extreme scaling requirements, such as during the Winter holiday season, when demand for server capacity would skyrocket. AWS then began selling the service to outside companies, including Netflix, which uses AWS for streaming.

As noted above, companies like Netflix and Capital One are more than just AWS partners, they are AWS cheerleaders, marketers and promotors. Thus, it seems likely that Capital One receives some level of compensation or other form of consideration (e.g. a discount or other customer VIP status) — and that their contractual relationship is bespoke.

AWS is also used by a number of government agencies and members of the intelligence community, including the CIA, on what’s called the “AWS Secret Region” — a massive collection of data centers available for storing and analyzing unclassified as well as top-secret data. Similarly, the government probably maintains its own unique AWS contractual requirements, unique to federal procurement and federal rules and regulations.

A Brief Side Note About Contract Law

During the 1967 film, “Cool Hand Luke,” when tedium in a prison chain gang seems to overtake the inmates, the incarcerated Luke, played by Paul Newman, quietly (and heroically) pronounces to his fellow inmates, “I can eat 50 eggs. ”

Amid the excitement surrounding Luke’s provocative claim, a gambling frenzy arises and the wager is on. Terms are negotiated among the various believers and non-believers; a rules committee is formed; and Luke agrees to a one-hour time limit in which to eat all 50 eggs.

Dragline, another prisoner, played by the late George Kennedy becomes the head of the syndicate backing Luke’s claim while the guards, equally engaged, direct the prison cook to prepare 50 hard-boiled eggs for the contest. The countdown for the challenge begins and Dragline immediately begins peeling the eggs and arranging them in front of Luke. A prisoner jailed for counterfeiting nicknamed “Society,” who is leading the group betting against Luke, angrily accuses Dragline of cheating, “He peels his own eggs – that is understood!” yells the outraged “Society.” Dragline responds, anointing himself as Luke’s “official egg peeler,” and tells “Society,” in one of the best lines of the film:

“You just may be great at hanging paper around big cities. But country boys like me are not entirely brainless. When it comes to the law, nothing is understood.”

Counsel for AWS would be wise to heed Dragline’s admonition and be mindful not to rely too heavily upon the AWS customer agreement, which may not be 100% enforceable.

Amazon pioneered the multibillion-dollar business of providing Web-based, on-demand computing resources to companies, it dominates the cloud infrastructure business, with roughly 48 percent of the market, according to estimates from Gartner. This confers upon AWS tremendous bargaining power during contractual negotiations – but such one-sided leverage can become a double-edged sword.

Like the “limitation of liability” agreements pasted to the backside of parking garage claim tickets, courts carefully scrutinize such one-sided and click-through boilerplate user agreements, citing them more akin to adhesion contracts and sometimes void certain provisions because of the possibility of unequal bargaining power, unfairness, and unconscionability.

The Capital One Liability Wildcard

Despite the rigorous contractual provisions of the AWS customer agreement and the exculpatory public statements issued by both Capital One and AWS, there is one ominous wildcard thrown into the liability calculus that could become a problem for AWS: Thompson is a former AWS employee who worked in the company’s S3 cloud storage technology group, and is suspected of exfiltrating data from other possible AWS customers. Capital One uses S3, which has led to speculation she may have had an inside track on Capital One firewall weaknesses.

For instance, the criminal complaint against Thompson hints at other AWS-related victims of Thompson’s hacking. The complaint states:

“A search of a bedroom believed to belong to Paige A. Thompson resulted in the seizure of numerous digital devices. During the initial search of some of these devices, agents observed files and items that reference to Capital One and the cloud computing company [AWS], other entities that may have been the target of attempted or actual network intrusions, and “erratic, “the alias associated with Paige A. Thompson.”

With respect to the 30 companies the FBI mentions in its August 14, 2019, “Memorandum in Support of Motion for Detention, there is no specific mention of any sort of AWS culpability or involvement, though the investigation is clearly continuing.

Moreover, at present, no one has determined:

  • Whether Thompson’s knowledge from her time at AWS gave her specific insight into how to breach the firewall of Capital One;
  • Whether AWS had any knowledge of any previous Thompson (or other attacker) orchestrated unlawful network intrusion using SSRF or a similar hacking modus operandi; or
  • Whether Thompson used her time at AWS to develop her cyber-attack against Capital One or any other company.

As more information is stored in the cloud, staff system engineers like Thompson, trained to become experts using these cloud systems, could become a threat to other companies. If the FBI uncovers evidence of multiple AWS-related Thompson data exfiltrations, or if it’s established that Thompson somehow unlawfully used proprietary AWS information in order to carry out her hack into Capital One, then that could trigger further investigation. And if further investigation uncovers that AWS should have done more to alert Capital One about server configuration SSRF vulnerabilities or errors, the findings could lead to possible AWS culpability.

AWS Need For Independence and Transparency

Whether it is British Petroleum struggling to handle the aftermath of an oil refinery explosion killing 15 Alaskan workers; Wells Fargo adjusting its operations after a massive company fraud committed by 5,300 employees against over two million customer accounts; or any company experiencing a threat to its customers, the same lesson always rings true. Confront the issue head-on with independence, transparency and integrity.

Strong leaders seek answers from independent and neutral sources of information when responding to data security incidents. Despite its immediate and loud proclamation of innocence, AWS should nonetheless:

  • Engage a former law enforcement agent or prosecutor from an independent and neutral law firm or consulting firm (preferably never engaged before) to conduct an investigation and report its findings to the board;
  • Direct the law firm to engage an independent digital forensics firm to examine possible AWS culpability for the Capital One data breach;
  • Report the investigation’s progress to shareholders, regulators, customers and other interested constituencies and fiduciaries every step of the way; and
  • Disclose the details of the findings to those users impacted.

In stark contrast to trying to characterize an incident, effective leaders should begin with the above critical steps, which evidence strong corporate ethics; fierce customer dedication; and steadfast corporate governance.

C-Suite executives and boards of directors are not politicians and do not have the luxury of conducting potentially self-serving investigations and offering sanitized reports and findings; they have fiduciary obligations to shareholders, customers and others to seek the truth and they should do so with independency, neutrality, transparency and candor. Otherwise, any formal findings can lack credibility and integrity, and no one, including Congress, regulators and law enforcement, will take the investigative effort seriously.

Looking Ahead

AWS’s categorial denial while definitive and brazen, will do little to ward of public scrutiny and skepticism of its culpability. Top-ranking Republicans on the House Oversight and Reform Committee have already sent a letter to Capital One CEO Richard Fairbank and Amazon founder Jeff Bezos, demanding by August 15, 2019, briefings detailing how the breach occurred. Specifically, U.S. Rep. Jim Jordan, a dogged and intense Ohio Republican, wrote to Amazon CEO Jeff Bezos:

“Because Amazon Web services will provide the trusted Internet connection and cloud support for the 2020 Census and could potentially run the Department of Defense’s Joint Enterprise Defense Infrastructure cloud computing system, the committee may carefully examine the consequences of this breach.”

Along the same lines, Oregon Senator Ron Wyden sent an August 6, 2019, letter to Amazon CEO Jeff Bezos asking about the nature of the hack and whether vulnerabilities in the company’s cloud services had anything to do with it. According to the Wall Street Journal, Amazon has responded to the Wyden letter, and is now considering additional changes that it can make to its cloud subsystems that will better protect its customers, including running checks and alerting customers if they have the kind of firewall misconfiguration that Thompson allegedly exploited.

We have also yet to hear from other AWS customers about the Capital One data breach and whether any have suffered similar attacks. Recent comparable AWS cloud S3 bucket misconfiguration incidents include: a June 2017 attack involving 200 million voter records; a November 2017 attack involving dozens of terabytes of a U.S. military social media spying archive; and an April 2019 attack involving 540 million Facebook profiles).

The reality is that the AWS SSRF exploit is nothing new. A Swiss security engineer described such an attack two years ago in a blog post specifically titled, Abusing the AWS metadata service using SSRF vulnerabilities. Similarly, Netflix, one of AWS’s biggest customers, became so concerned that about a year ago they published some strong research on credential compromise relating to AWS cloud configuration.

Not surprisingly, according to a now-deleted July 31, 2019 tweet, Netflix senior security software engineer Michael Wardrop reacted to the Capital One breach by reporting that Netflix had asked AWS to address its exposure to SSRF, tweeting:

“Netflix asked AWS to follow GCP [Google Cloud Platform] and require a special header for metadata service requests [to defend against SSRF attacks]. Unfortunately we didn’t get a satisfactory response.” 

If AWS in fact received a request from Netflix to add critical SSRF security protection and was negligent in ignoring it or in the least, passing along concerns to its other customers such as Capital One, liability could shift form Capital One to AWS – and class actions against AWS would quickly follow.

In fact, one class action lawsuit was already filed on August 5, 2019, against AWS for the Capital One breach, arguing, inter alia, that AWS knew about the SSRF vulnerability and “have done nothing to fix it.” Per the class action complaint, “The single-line command that exposes AWS credentials on any EC2 system is known by AWS and is in fact included in their online documentation . . and also well known among hackers.”

Under any circumstance, whoever seeks the truth about the Capital One breach will undoubtedly need to examine the economic realities of the extraordinary cozy relationship between Capital One and AWS, and the mammoth historical alliance between the two. The relationship between AWS and Capital One seems so snug that executives from either company almost warrant recusal with respect to any investigation of the other. Like lifetime NBA teammates, neither company will be comfortable leveling accusations or even publicly criticizing the other.

In my view, the Capital One board of directors needs to step-in and step-up. The board is best suited to disregard the possible conflicts inherent within the AWS/Capital One relationship, to rise above the intimate personal relationships that likely exist between the companies and to find out what really happened. To me, it will also take a board effort to initiate the bold step of accusing AWS of any wrongdoing.

Sometime after the Capital One data breach fallout, we will hopefully receive a definitive post mortem, and learn whether Thompson’s hack was due to:

For now, liability remains somewhat foggy, and assigning blame seems premature. Indeed, we only know what is true in the early phases of every data breach investigation: that nothing is certain except uncertainty.

_____________________________-

John Reed Stark is president of John Reed Stark Consulting LLC, a data breach response and digital compliance firm. Formerly, Mr. Stark served for almost 20 years in the Enforcement Division of the U.S. Securities and Exchange Commission, the last 11 of which as Chief of its Office of Internet Enforcement. He currently teaches a cyber-law course as a Senior Lecturing Fellow at Duke Law School. Mr. Stark also worked for 15 years as an Adjunct Professor of Law at the Georgetown University Law Center, where he taught several courses on the juxtaposition of law, technology and crime, and for five years as managing director of global data breach response firm, Stroz Friedberg, including three years heading its Washington, D.C. office. Mr. Stark is the author of “The Cybersecurity Due Diligence Handbook.”