The "Circle of Blame"

I prefer the "Chain of Blame" because of the better rhyme scheme... all kidding aside, Andrew Conry-Murray has done some good reporting on this story.

One money quote:

While PCI provides more concrete guidelines than, say, Sarbanes-Oxley, merchants are quick to complain that it's both too specific and too vague. For instance, the standard requires use of stateful packet inspection firewalls. "What if I choose to use another technology that I believe is equivalent?" says Michael Barrett, chief information security officer of PayPal, a Level 1 merchant. "You have a whole big fight with your auditors or you hold your nose and do it."

Level 1 merchants also clash with QSAs over issues such as "compensating controls"--technologies or processes used in place of specific requirements on the PCI checklist. "We believe our controls are adequate, but they are different from how the standard is written," Barrett says. "So you argue with auditors. Those kinds of things make you want to tear your hair out."

There's also a level of subjectivity in PCI that many find disturbing. The training for QSAs provides few guidelines for resolving this subjectivity. One PCI expert, who requested anonymity, says of the training: "When you ask if X or Y would be acceptable, or how to apply X in situation Y, they always answer 'Use your best judgment.'" He says that when others in the class pointed out how wildly their opinions could differ in a given situation, the instructor "had no answer other than to say 'do your best.'"

"It's a question of interpretation of the auditor, and the sophistication and skill set of the auditor," says Jay White, global information protection architect at Chevron, also a Level 1 merchant. "PCI was more painful than it had to be, but we've learned we have to help the auditors understand how we meet their objectives, even if they don't at first see it."

This lack of guidance can lead to significantly different approaches to compliance, even among auditors at the same Qualified Security Assessor. In one case, a company brought in a PCI expert to monitor a QSA's recommendations. The expert says the QSA had insisted the company deploy a million-dollar technical control when a simple change in operational procedure would have addressed the issue. "The assessment company then sent out someone completely different," the expert says, "and he disagreed with the recommendations of the prior QSA from his own company!"

This inconsistency can have significant repercussions for Level 1 merchants. If a merchant exposes card data, Visa dispatches a team of forensics security consultants to determine if the merchant was compliant with PCI at the time of the breach. "If a 'compliant' merchant gets compromised, I can guarantee you I can find at least one thing in the compliance report I could argue about," says the PCI expert. "This provides just enough wiggle room for the brands to point at the merchant or QSA and argue the standard was interpreted wrong."

Being judged noncompliant can result in substantial fines for the merchant and its acquiring bank, including higher per-transaction card processing fees. A judgment of noncompliance would also be useful to law firms contemplating action against the merchant.

More interesting points:

One major clothing retailer we spoke with said auditors examined four out of 1,000 stores, a sample size of just 0.4%. The retailer says all its stores share the same configuration and are centrally managed, but it's all too easy for security problems to go undiscovered with such small samples. "I could hide a multitude of sins from a QSA," says the PCI expert.

And while some retailers complain that auditors are too strict, the current system lets retailers seek out QSAs who may apply the standard less rigorously than others. "I've read several compliance reports that have been provided to us after the fact, and I wouldn't consider them appropriate," says the PCI expert. "They passed, but I don't know how." When asked if merchants are shopping for QSAs that provide an easy assessment, he says: "I can guarantee you that. Why wouldn't they?" Even the PCI Security Standards Council, which trains and certifies QSAs, admits that quality levels may not be consistent among the more than 100 active QSAs.

"It's a competitive game," says Bob Russo, general manager of the council. "One QSA might do an on-site assessment for X number of dollars, and another QSA will do the exact same assessment for less. A merchant thinks, 'If this guy is charging me $50K and this guy charges me $10K, there's a question there.'"

In response, the council is introducing a quality assurance program, due later this quarter, to ensure that all QSAs are performing assessments with the same rigor. "The goal is to make sure it's a level playing field so we don't have accusations from QSAs or merchants that some people are rubber-stamping," Russo says.

The question of rubber-stamping ties to the issue of liability. If a compliant merchant is breached, does the QSA bear any responsibility? It's a question that makes QSAs uncomfortable.

"Who's to say a retailer doesn't take what we say and toss it into the garbage?" says Barbara Mitchell, manager of security product marketing at Verizon. Along with Internet Security Systems and TrustWave, Verizon wins much of the assessment business for Level 1 merchants. "We should have some skin in the game, but if a retailer decides to not listen to our recommendations, it's a murky area," Mitchell says. "If we assume liability, we want to review all the stores, all the servers. That shoots the cost up to a prohibitive degree."

Retailers we spoke with were unclear about the liability question. "I think it would depend on whether our controls were deficient and on the audit process," says the network architect at the major clothing retailer. "I think there would be some level of liability, but we've not dug into that. There may be language in the contract I'm unaware of, but my focus has been on controls to prevent a breach rather than where we will point a finger." Unfortunately, finger-pointing is inevitable if credit card data gets stolen. "When a breach happens, if they see something out of whack, they will go back to the auditor, like Enron and Arthur Andersen," says Teri Quinn-Andry, product marketing manager for Cisco Security Solutions.

Then there's the problem of depending on what is, essentially, an honor system for Level 2, 3, and 4 merchants. There is no outside validation of a company's responses to the self-assessment questionnaire. "The reality is, you don't have to be compliant, if your business wants to take that risk," says the IT director of a Level 2 cruise ship operator.

"A lot with PCI is left to your interpretation," agrees Alan Stukalsky, CIO of Church's Chicken restaurant chain, also a Level 2 merchant.

So what does it all mean. I think it means a very volatile system with a lot of liability risk and uncertainty. I think it means that taking shortcuts could get both merchants that self-assess and QSAs into hot water (including hot water of the "going out of business" type for smaller merchants and QSAs). I think it means probably more comprehensive and expensive assessments when QSAs start getting hit with lawsuits.

So what can be done to smooth out the risk? More on that later from me... any thoughts from others?