Bob Lord: What I preached as Yahoo’s CISO (Q&A)
SAN FRANCISCO—It’s rare to pinpoint a probable purveyor of a major cybercrime. It’s even more rare to witness your alleged Internet attacker in court. Bob Lord, who stepped down as Yahoo’s chief information security officer on September 12, recently did both.
“As a victim of criminal activity, it was very good to see the wheels of justice move,” Lord said during a recent phone call of the indictment of hacker Karim Baratov. In a San Francisco courtroom on August 23, he watched as Baratov, a 22-year-old Canadian citizen born in Kazakhstan, pleaded not guilty to 47 charges of computer intrusion, conspiracy, and economic espionage for his alleged role in a 2014 breach that compromised more than 500 million Yahoo accounts. A status conference involving the parties is scheduled for October 10.
Baratov, said Lord, who joined me on stage for a conversation at the tech conference Structure Security here Tuesday, was “not emotional” as he entered his plea. “As a victim, I felt a measure of gratification,” he added. “I wonder what his life would’ve been like, had somebody else grabbed him and given him a real tech job.”
READ MORE ON YAHOO AND DATA BREACHES
‘Yahoo is not safe to use,’ former company engineer says
How to dump your Yahoo, Flickr, and Tumblr accounts
Opinion: Why Yahoo should have fought FISA like Apple fought the FBI
Opinion: Massive Yahoo breach highlights why to hit ‘delete’
What to do when you’re caught in a data breach
Special report: How data brokers slice up your private life
Parallax Primer: How to protect your payment apps
As the top security expert at one of the best-known tech companies in the world during the revelation of some of the worst data breaches ever to make headlines, Lord, who joined Yahoo in November 2015, faced enormous pressure and scrutiny.
On stage at Structure Security, Lord explained that less than a year after the headlines started running, he left Yahoo in part to rethink how security experts at the top of the corporate ladder should be influencing their bosses and their subordinates. I asked him what he learned from his time at Yahoo, and what other CISOs could learn from his experiences. What follows is an edited transcript of our conversation.
Q: Why did you leave Yahoo?
You may have heard that Verizon purchased Yahoo, and it had previously purchased AOL. Those two companies are being put together under the Oath moniker. One of the things that I would like to have done more in my career is notice inflection points and ask myself whether the horse that got me to this well was the right one to get me to the next one.
I’ve had some interesting experiences, and I think that there are interesting opportunities. I might be able to help either boards or CEOs or other people really understand the way I think about security, and the way that they can be a little more structured in their thinking about improving the security posture of a company. I’m just exploring things right now, and if nothing else works, I’ll go back to being a CISO.
Did the breaches that were revealed last winter have any impact on your decision to leave the company?
No. I actually want to compliment the way that the company organized its activities. We came across the breach, and we understood what it was. I basically rang the bell, and my company jumped into action. So I couldn’t really be more proud of everyone, from CEO Marissa Mayer on down. Everyone acted very quickly.
Unfortunately, that meant a lot of weekends and late nights and early mornings. But really, the overall response was fantastic, so no complaints there.
What did you learn as Yahoo’s CISO that you wish you had known before you got to Yahoo?
I can answer that better from the perspective of Twitter. There are things that I eventually learned the hard way, through my experiences at Twitter, and you can read about the breach there. Maybe I sent some of you [in the audience] an email about that.
One thing that evolved in my thinking—and this is really hats-off to the security team at Twitter and other places—was really trying to understand what the attacker life cycle was: being responsive to what the attackers do and how they do it. And really trying to think about the economics, not just for the defenders but for the attackers. I came upon this philosophy of really reorganizing my thinking. From the very beginning at Yahoo, I set out to really preach that.
“But the real issue with security is not a technology problem. If it were, the hundreds of vendors at the RSA Conference every year would have fixed the problem.”
It was not entirely clear to me that this was going to be a well-received philosophy at Yahoo. During our first board presentations, we had a slide saying we are up against dedicated human adversaries who organized their work in campaigns.
I would spend a few minutes talking about that. I talked about what dedicated means: They show up in uniforms or flip-flops. And I talked about what human means: Every time we make an improvement in our security posture, our adversaries get to decide what to do about it, if anything. And the campaign part means that we not only are going to see things over a long period of time, but an attack against any one company may be part of a larger effort to do something.
And, oh boy, am I glad I did that from the very beginning.
What was the reaction at Yahoo? Not just from people who reported to you, but also from people who were above you and looking to you to guide the company’s cybersecurity posture?
People were very interested in this approach. When I first talk about these things, people nod and say, “Of course, dedicated; of course, human; of course, campaigns.”
But when you really start to peel back the onion, most security teams don’t actually act that way. And most boards don’t work with their CISOs in ways that are truly informed by that philosophy. So people were polite. They listened. But over time, we were able to start thinking much more completely. And I tried to teach other people how to think about security.
You’re talking about a cultural change.
Yes. We spend a lot of time talking about technology. But the real issue with security is not a technology problem. If it were, the hundreds of vendors at the RSA Conference every year would have fixed the problem. We don’t have any CISOs raising their hand and saying, “I got this, I’m bulletproof, no problem.”
“You shouldn’t grade your own homework.”
No one is saying that. It’s not a technology problem. It’s the people in process problem. And behind that is the cultural element. It’s figuring out how to teach people so that they’re going to do the right thing without having the CISO and security team tell them what to do.
Now that you’ve left Yahoo, would you say that the culture there has changed sufficiently enough to make the company and its products safer, more secure?
I think it was improving even under [Yahoo’s prior CISO], Alex Stamos. I inherited a lot of the things that he did, and I really was standing on his shoulders.
The hiring of a full-time red team, for example, and the work with the bug bounty program—these are examples of not doing your own homework. You shouldn’t grade your own homework. And so these are examples of us looking for people to tell us what else is wrong. And that’s one of the things I think boards need to do more of.
They shouldn’t be asking, “Are we safe?” They should be saying, “What else is wrong?” And when the CISO runs out of things that are still not right, hire a consultant to find more.
What are some of the things that Yahoo still needs to work on?
You may be aware that the company is the subject of numerous constraints and legal things I can’t really go into. So I’m not going to go into that. But I would say, with the industry as a whole, really embracing these concepts is going to be a major thing.
I constantly teach about the attacker life cycle or the kill chain, and different companies have their own interpretations and philosophies that describe the major milestones that the attackers must go through to accomplish their goals.
We came up with one [at Yahoo] that we thought was good. I worked with the red team on that, and then it came into play, in terms of specific diagrams or simplified network diagrams. And every time we talk about something that we wanted to do—often things that would inconvenience developers or employees—we’d go back to this picture and say, “This is how lateral movement works. This is why we’re trying to degrade lateral movement, and in order for us to do that, we have to do these things.”
“Let’s just say it for what it is: Everyone’s been pwned, and so it’s incredibly important for people to act this way.”
We tried to really teach people the basis for that. And I think the mark of success will tend to be when people come back, and they argue with the security team in the context of that attack life cycle, in the context of these human beings that are out to get you. And that’s because I’ve learned that they really are out to get you, and they’re quite persistent.
Companies, broadly speaking, need to continue to improve on really understanding the attacker life cycle, making sure that the security programs are responsive to that. If you’re a big company, you’re going to have to do ISO 27000 or the NIST cybersecurity framework, and you’re going to have to have teams manage that. You just have to do that.
But that’s clearly not sufficient. Many companies that have been breached have had rigorous programs in place for years. Really understanding what the attackers do is going to be another major component.
How should modern CISOs prepare for the inevitable breach?
I’d like to say I had grand plans, and I had the checklist of things I wanted to do. But I did a few things opportunistically, and a few things extremely well and with purpose.
One thing I did was develop our existing relationship with the FBI. If you have a breach, you should know who your FBI agents are. Thankfully, I refreshed my contacts with the agency when I joined Yahoo. Who expects there to be a breach in the first few months, the first year of your tenure?
But isn’t that something the modern CISO should expect at some point? The company is either about to be breached—or it has already been breached, and it just doesn’t know it yet?
Already been breached. Let’s just say it for what it is: Everyone’s been pwned, and so it’s incredibly important for people to act this way. We can talk about the basic story in a second, because I don’t think that people actually have been walked through some of the facts.
“If you think that the security team is going to keep you safe…we tried that for years. It doesn’t work.”
No one really expects the Spanish Inquisition. You don’t really expect a foreign government to be going after you, and to spend an extended period of time trying to hack you. You don’t really expect a foreign-intelligence service to hire criminal hackers to break in.
CISOs know it’s possible. But how many of them act like that? Moving the intellectual knowledge into your heart and your gut—that’s the culture transformation that you mentioned. Believing that’s actually the case, and then changing your behavior accordingly. I think that’s really where we need to go.
What is it like trying to change the culture at a company as large and established as Yahoo? I can’t imagine that it’s easy to get people who have been there a very long time to think (and act) differently.
Yes and no. I had a lot of advantages. The culture was already good. The Paranoids, which is the information security team within the company, had a very good reputation. In fact, I didn’t even want a CISO job. I had a really great gig at Rapid7—fantastic people, and I was learning a lot. I took the interviews [for Yahoo’s CISO position] because I was curious, and I loved the brand, and I loved the Paranoids.
So when I showed up, there really wasn’t pushback. We tried to communicate better, streamline things, and improve on their ability to reach out and teach people how these attacks work. It was effective.
I have to say I had it relatively easy. After the breach, we turned things up a few notches. We spoke at the company’s weekly all-hands meetings a lot more, as you might imagine. I did fireside chats with people in front of the entire company and walked them through what the red team and various other teams were doing around security.
A lot of that was already there. But continuing to push on that is really the key. And taking personality and personas out of it was something I desperately wanted to do. I wanted to teach them how to fish.
How do you change the message, or change the tone of the message, so that employees above and below you are more receptive?
Again, maybe I just lucked out. People were receptive. The question was, What do I tell them to do? They’re willing to do things.
I pushed hard on showing people actual metrics. At every company I’ve worked at, every time that I’ve gotten good metrics—and especially trend analysis—people were receptive.
“The reality is that as CISO, I’m your personal trainer. If you have a lower-back problem, I’m going to find a way to make sure you do the stomach crunches. You’re going to get them in, and they’re not going to aggravate your lower-back problem. That’s the sort of thing I can do. You have to do the pushups, you have to do the crunches, you have to push away the extra piece of cake, and you have to stop drinking so much. And if you don’t do it, you should have the heart attack, not me.”
Trend analysis requires much more work. But it’s much better than showing how many attacks, how many bombs. Everyone can intuit that an upward trend, if it’s a trend of risk, is bad. And if it’s a downward trend, that’s probably the right direction.
So I worked hard to really show people what was happening through metrics and trend analysis. And then the message just kind of sold itself. I have to be aware that I could go into a room and, if I have the CISO hat on, or if I’ve got the shaved head and earrings, I’ll bring a certain mystique. But that wears thin. And I don’t want to rely on that.
Would you say that Yahoo is safer now? At some point, we’re going to find out about another Yahoo breach. Not necessarily because it’s Yahoo, but because we know that it happens to all companies. How can you gauge your success as a CISO?
Internally, we had really strong metrics. I think a lot of the metrics that we had were trending in the right direction. And when we exposed those to people, they got even better.
What should those metrics be? What should other CISOs be looking at?
We looked at a number of things, and CISOs in general will do this. But we took a really hard look at some of the best practices. I wish I could say that there’s a magic silver bullet, but the fundamentals are the fundamentals.
We looked at the patching strategy. Obviously, it’s not always sufficient. And we looked at the training—and not just employee awareness training—to make sure that it’s very specific to the job function. I think that we’re all aware of security. But general awareness really hasn’t helped much.
How does the idea of pursuing the perfect over the good apply to the role of a CISO?
We’re somewhat cursed in the security world because we’ve seen things like encryption technologies. You’ll sometimes hear somebody say, “With this many bits, you will be safe until past the heat death of the universe.” We think that we can get that kind of assurance elsewhere. Well, we can’t.
That’s why I go back to that idea of being up against humans. As soon as you internalize that, you’re going to be a little upset. Because you realize that many of the things, if not all the things, you’ve done are going to be insufficient to actually keep them out, and you have to really change the way that you think.
That has to start at the board and CEO level. If you think that the security team is going to keep you safe…we tried that for years. It doesn’t work.
The reality is that as CISO, I’m your personal trainer. If you have a lower-back problem, I’m going to find a way to make sure you do the stomach crunches. You’re going to get them in, and they’re not going to aggravate your lower-back problem. That’s the sort of thing I can do. You have to do the pushups, you have to do the crunches, you have to push away the extra piece of cake, and you have to stop drinking so much. And if you don’t do it, you should have the heart attack, not me.
What are some mistakes CISOs make that would prompt you to have them do extra pushups?
The No. 1 thing is the focus on tools. Companies get into transactional relationships. CISOs get into them with boards and their teams: “Oh, I just need to go buy this tool; that’s going to solve it,” or, “This thing has machine learning—that’s going to be great.”
That’s going to get you into big trouble. It’s going to cause you to really think about things from the tools perspective. But as I said before, if that approach would’ve worked, it would’ve worked over the last 20 years. It has not.
Last question: Tell us about your experience with Russia’s FSB [the successor to the KGB] and the breach.
So this is all public information, and there are two things I advise everyone to do, whether you’re an executive or CISO or on a board, or anyone else. Read two documents. One is the 10K report from Yahoo. Read it, wait two weeks, read it again, then ask yourself: Would you really have done better? Because I think that’s the fullest public description that we’re going to see for a while.
The other is, read the indictment of the criminals who actually perpetrated this 2014 hack. Four people were indicted in March of this year. Two of them are FSB officers. They, in turn, hired two criminal hackers, who then did a number of things. One was responsible for the actual penetration into Yahoo. And so this blending of an intelligence service with criminal hackers is remarkable.
It’s also remarkable that one of the hackers, Alexsey Belan, had been indicted also in 2012 and 2013, and he had an Interpol red notice. He should’ve gotten picked up anytime he traveled outside of Russia. He did travel outside Russia. He did get picked up. And somehow, he escaped. I’d love to know exactly what happened there.
But instead of having him go through the Interpol red-notice process, he got turned, and he started to hack on behalf of the FSB. This is a real-life spy story. I can’t go through the whole thing. It is remarkable.
And in the back of your mind, as you read it, think about the attacker life cycle and the kill chain. That’s in the indictments. It’s a little bit long. But it’s all there.