A Leanpub Frontmatter Podcast Interview with Tony Robinson, Author of Building Virtual Machine…

A Leanpub Frontmatter Podcast Interview with Tony Robinson, Author of Building Virtual Machine Labs: A Hands-On Guide

Tony Robinson is the author of the Leanpub book Building Virtual Machine Labs: A Hands-On Guide. In this interview, Leanpub co-founder Len Epp talks with Tony about his background in IT and security, the nature his work as a security analyst, his book, why he uses Kirby as his avatar online, and at the end, they talk a little bit about his experience as a self-published author.

This interview was recorded on November 21, 2017.

The full audio for the interview is here. You can subscribe to the Frontmatter podcast in iTunes or add the podcast URL directly.

This interview has been edited for conciseness and clarity.

A Note About the Leanpub Frontmatter Podcast

This summer we split the old Leanpub podcast into two distinct podcasts:

Frontmatter, which is a general interest podcast where you can listen to Leanpub authors talk with Leanpub co-founder Len Epp about their books and their areas of expertise, from data science to molecular biology, to the history of labor and management. And for those interested in the nitty-gritty of what it takes to be a successful self-published author, at the end of each episode Len asks the author about how they made their book and how they are spreading the word, and other publishing shop talk.

Backmatter, a new podcast focused specifically on the publishing industry and its latest trends. In each episode Len interviews a professional from the publishing world about their background and their insider’s perspective on what’s happening in the huge and evolving world of book publishing.

Len: Hi, I’m Len Epp from Leanpub, and in this Frontmatter Podcast, I’ll be interviewing Tony Robinson.

Based in Michigan, Tony is a Senior Security Analyst at Hurricane Labs, a Cleveland-based security services provider. In addition to his duties as an analyst, focusing on network security monitoring — which I’m going to be asking him about a bit shortly — he also plays a role recommending new technologies and providing threat intelligence data to clients and a number of other things.

Tony is the author of the book Building Virtual Machine Labs: A Hands-On Guide. His book teaches you everything there is to know about building and maintaining a virtual lab environment using today’s most popular hypervisors.

You can read Tony’s blog posts at hurricanelabs.com/blog/author/tony-robinson. I should also say that under normal circumstances, you can follow Tony on Twitter @da_667, though he’s currently taking a break from Twitter until Thanksgiving — which sounds like a good idea to me.

Tony: I’ve just got family that’s out of town, and don’t get to see them very often. Not only that, we moved into this place in June of this year, and we’re hosting families. So I just want to make sure that I don’t set anything on fire, make a mess for Thanksgiving.

Len: That sounds like a really good idea. Thanks very much Tony, for being on the Frontmatter Podcast.

Tony: A pleasure.

Len: I always look to start these interviews by asking people for their origin story. So I was wondering if you could talk a little bit about how you first became interested in — let’s say — computers and software.

Tony: My story goes back to high school. A lot of people that I know in computers and information security, they date their stuff back to when they were just going to grade school, they had computers, they grew up with them.

My story starts in high school. We had access to a computer. My dad was a Ford factory worker, so our first computer was given out through the Ford program that happened at about the turn of the millennium, where they were giving out heavily discounted PCs to different Ford families.

We had a shared PC and I just got really curious about the internet. I wanted to know how computers work. I wanted to know how computers communicate over the internet. And that just sparked my passion from there.

I decided to go through the IT route. I wasn’t a terribly great programmer, so I opted to go through computer networking. One of the schools I was at had a Cisco networking program, and that springboarded me towards systems administration.

I did that for a few years here in the State of Michigan, and then simply by a twist of fate one day, I go to a book store, and I find a magazine called 2600: The Hacker Quarterly. and that gets me towards information security. I figured out that information security is actually a passion for a lot of people, in that it’s a job function that’s available out there, and there are people that get paid to break into systems or defend systems on a day-to-day basis. I was having a great time as a systems administrator, but I was really curious about security.

So I took a leap and ended up in Maryland, working for Sourcefire, pre-Cisco merger. Then from there, I worked for the NSA for a period of time. I did a little bit of network security work. I’m not going to go too deep into that, because don’t need black helicopters over here or anything, but I worked for them for a little while.

I worked for a large power company, securing the power grid, securing enterprise users, for, probably the second largest power company in the country.

I went back to Cisco for a little bit. They said, “We have seen what you have been doing.” I had honeypot projects, I had talked at security conferences, I had written numerous blog posts. And they liked what I had going on, and they brought me back for a little while. And then we decided we wanted to come back to Michigan, me and my wife, and Hurricane Labs was nice enough to bring me on. They’ve given me a ton of freedom, and a ton of ability, which is part of the reason why that book got written. Is because they gave me the freedom to do so. And that just kinda brings me here to today.

Len: I’ve got a few questions — actually one that I hadn’t been intending to ask about, although it had kind of crossed my mind was — so you grew up in Michigan?

Tony: Yes.

Len: So your dad worked for Ford. What was that like, being part of the Detroit, Michigan auto -

Tony: Actually we did grow up in Detroit, so -

Len: One of the things obviously a lot of people know about Detroit’s story is how there was this period of decline for a while in the auto industry, and things like that. I was wondering if that affected you or your friends or your family in any particular way?

Tony: There’s a misconception, especially in the government and in other organizations, that if you have a family who’s working for Ford Motor or the auto industry, that you have it pretty good. And things were pretty tight for us. That’s part of the reason why I wasn’t really introduced to computing until I was a teenager, is that money was a little bit tight. But we had family, we stuck together, and we helped each other out.

My parents are the main reason that I managed to get through college, because I lived at home with them, they let me do that. I worked my tailfeathers off, and they supported me, they helped me get through it. So money might have been a little bit tighter than we would’ve liked, but we managed to make it work. And he retired a few years back. He has about 27 years of experience at Ford.

Len: Thanks for that, and for clearing up that misconception that — yeah, I think some people sometimes cynically or even sarcastically adopt towards the industry.

Speaking of studying in university, one of the themes of this podcast is, if you were to start over now, and things do change very quickly in IT — would you study computer science formally in university again? Or if someone were starting out now, would you recommend it?

Tony: It’s funny that you mention that. I get this a lot on social media and Twitter.

I have a lot of people — every so often, I’ll say, “If you want advice, or if you want recommendations” — I do like a hands-off career guidance kinda thing, where people say, “This is what I know,” and I ask them, “Where do you want to go?” And I say, “This is the path I think you should take.”

So in terms of general IT background, the best thing I can say for people who want to get involved, is that there are a ton of resources out there that weren’t available before. There are ways to get cheap hardware through companies that are trying to get rid of their old stuff. Like ServerMonkey or SaveMyServer or North American Technology Exchange, is one of them. There’s plenty of cheap hardware available out on the internet.

There are free training resources. Cybrary.it has a ton of resources out there if you’re a dedicated self-learner and you just don’t have the money to work with. There are people who are recording their content and putting it out there for free. And some of it might be based off of certifications that you may or may not be interested in. But the thing I would take away from that is that, the certifications — you don’t have to have them, they just give you a foundational knowledge to build up your capabilities.

One of the reasons why I wrote the virtualization [book] is another aspect of that — getting the hands on experience. It’s really hard to get your foot in the door in IT and information security in general, because people want experience, and if you say, “I’ve built labs, I’ve done this, I’ve worked with this technology.” Or, “I’ve done computer security challenges and I networked at these conferences, both IT and security,” it’s really easy for you to build a network of peers that know you and know what your capabilities are and are willing to vouch for you.

If there’s anything I would do differently, I would find out about conferences in the areas near me, or any local security or IT gatherings, and just make yourself known. Just make friends and — sometimes that’s easier said than done, because a lot of us computer types are introverts. Sometimes it just takes a little bit of breaking out of your shell. But it’s very much worth doing.

Len: One thing I’ve heard people say sometimes, is that even if getting a formal degree isn’t absolutely necessary for a career — and it obviously isn’t — if you do want to work for some of the big companies, it can often be something of a requirement, like for Google or Facebook or something like that. Is that true of government work as well?

Tony: For government work, it’s kind of a 50/50 split. First off, if you have a background in education, you’re currently going to school, if you’re fortunate enough, there are scholarship programs for service. Especially for the Department of Defense and various other orgs, whereby you pledge to submit to five years working for the federal government, and they will pay back your student loans.

So if you’re going to school, and you’re thinking about it, it’s a good way to get experience. It’s a good way to get your foot in the door. And also to have your student loan debt kind of managed.

Len: Pretty important these days to get that managed.

Tony: Exactly. And the other side of that is if you have enough years of experience, and going to work for the government is a passion, it’s something that you want to do. Like you want to do the mission, or you want to work for your country, is [work for] the government and better their security standards or IT standards — then if you have enough years of experience and can back it up on that piece of paper, your resume — then more often than not, they’ll be happy to have you.

Len: Speaking of experience actually, I have one question — and I might have a few more. But I have one particular question about it, which is — what’s your day like, working for Hurricane Labs as an analyst.

Tony: A lot of the time, I have a lot of freedom to do practically whatever I want. If that’s playing around with virtual machines, if that’s developing training material — like right now, I’m working with an individual by the name of Chris Sanders. He develops security training material, and he’s not associated with Hurricane, but so long as I’m not working a critical case or something that isn’t waiting for my attention, they’re letting me record training based off the book. It’s not a conflict of interest to them, they don’t mind that I’m doing it.

So my days could be helping out the other security analyst, and looking at an event. It could be communicating with customers and utilising soft skills to explain what’s going on with them, or hand hold them as they need it. I could be demoing new technology, or trying to figure out better ways to integrate some of the data or threat intelligence we have, and better give it to our customers or have it work out in their favor.

So it’s kind of a weird role. If you’re familiar with Game Of Thrones, it’s like a Tyrion Lannister role, in that I drink and I know things.

And I kind of have like an interesting advisory role, and it’s a lot of fun, becauseit just has a huge degree of trust, and I’m very thankful to have it.

Len: And you said a couple of words — “critical” and “event.” I guess I’m just curious about — some people, like — myself to some extent, might have a romantic notion about what it’s like to be working in security, every once in a while — I mean, do things happen like, your own version of a red phone goes off, and it’s like, “Tony, we need you right now, someone’s being attacked and we need to repel it?”

Tony: Typically that kind of thing happens whenever there’s a new oh-day, or zero-day vulnerability, a new vulnerability that is out there, and actively being exploited. It’s: what do we need to tell our customers? How is this going to affect them? What things do we need to look out for? What are our recommendations to make sure that you don’t become a victim of this, or this particular vulnerability or this particular piece of malware that just came out.

So it’s usually whenever something new comes out, or some researcher drops some new piece of malware or vulnerability, that things get thrown into high gear, and I’m writing recommendations, or I might be working with customers and explaining what needs to be done.

Len: One thing I’m really curious about is the relationship that security analysts might have with enterprises that they engage with. What I mean by that is, what’s the attitude like from the enterprise side, generally? Obviously without necessarily referencing any specific client, or even experience you’ve had, what’s your general sense of how — let’s say people in the executive, the C-suites — relate to security? Is it something that they treat like an annoyance? Is it something that they’re not well-versed in, but they do take seriously?

Tony: That’s a heck of a question because I’ve seen all ends of the spectrum there. There have been C-suites that — you have compliance requirements in security. If any of these sound familiar — PCI, HIPAA, Sarbanes-Oxley — these are compliance requirements that you have to have certain security measures in place, in order to do business. And some C-suites, some management see it — either we can accept the risk, and we can accept the fines, and it’s business as normal, or it’s, “We’ve been burned by this before, and we need to take it a bit more seriously.”

I’ve seen both ends there. Sometimes it’s a bit frustrating when all they want to do is just accept the risk, and just move forward. And then other times it’s really refreshing when they realise — this is customer data, this is private information that they’re liable for, and they realise the gravity of the situation. This is people’s lives that they have in their hands.

Len: That reminds me of one of the things I’ve come across, ot so much in direct personal experience, but just reading around — which is that often you can find in some companies, a culture where — in the C-suite, it’s actively promoted that one should not have any domain-specific expertise. And it’s one of the things I ask people about when it comes to basically IT-related matters.

Maybe if you’re just shipping toothpaste, as opposed to shoelaces, you can adopt a kind of abstract relationship towards what you’re doing, and just apply — to put it in a sort of derogatory way, business school models — is that something that’s actually not appropriate for a world that has been eaten by software?

Tony: In some aspects, and in some ways, it is and it isn’t. I know that this is a lot of “it depends” kind of answers, and I apologise for that. But since there’s a grey scale in our world today, it all runs down to threat profiles.

What I mean by a threat profile is — how likely, or how valuable is the data you have? How valuable are the resources that you’re company produces? How much money does your company make? Do you stand to lose?

It really depends on how seriously some companies are willing to take security. I mean, in some cases — if all you’re doing is producing toothpaste, it might not seem like such a serious thing until hackers manage to get control of your industrial processors, or the industrial controllers, and can shoot toothpaste halfway across the production line, and you have no idea why your toothpaste tubes are coming out empty, and things are malfunctioning left, right, front and centre. At the same time, that might be a minor hiccup or an annoyance for a single facility.

Whereas, somebody else who is producing let’s say for instance — let’s play around with Stuxnet, and say that you’re at a nuclear facility, and you’re producing yellow cake uranium. That is a very sensitive process, and something where if hackers get control of your industrial control systems there, they can throw off the entire process — could cause millions of dollars of damage, infinite push back. All sorts of issues where it’s really hard to recover from those kinds of things. And the impact can be significant.

So it’s all dependent on your threat profile, and how valuable the data and processes and things that you produce actually are.

Len: For those listening who perhaps haven’t heard of Stuxnet, or haven’t read about it in detail — it’s a fascinating story.

Basically, as I understand, government actors managed to really get into the details around how certain devices worked in the production of — I guess — enriched uranium, in Iranian nuclear facilities. And it was such an obscure hack that it took researchers a long time to figure it out. It was [based on the] nitty-gritty of the way the machines worked.

Tony: It was really fascinating, because, like you say, they acquired an in-depth knowledge of how those machines worked. And not only that, it actively hit itself. It did a bunch of fascinating things, and there was a ton of fallout, because this malware that was targeted from one particular place just ended up spreading outside of that, because of unforeseen circumstances.

Len: Speaking of potentially foreseen circumstances. You mentioned that you worked for an electric utility. When I was preparing for this interview, I was reminded of how, just in the recent past, the well-known broadcast journalist from the US, Ted Koppel, wrote a book called Lights Out. I don’t know if you heard about this.

He was sort of making the rounds on the late night talk shows and other things, talking about the vulnerability of basic infrastructure, like electric grids, and the incredible cost there could be to society if something like, say the electricity for all of New York State or something like that were just turned off by hackers. So he was deliberately fear-mongering. I was wondering if that particular problem keeps you up at night, and if it doesn’t, is there something else that does?

Tony: It’s funny that you mention that. There was an organisation whose name escapes me. I’m going to have to find the name of the place. But it was an organisation that is known for insuring obscure things. Like they would insure if a supermodel had a really gorgeous pair of legs, they would insure that. If a singer had a wonderful voice, they would insure that. And they had research to back up why particular things were as expensive as they were, or to the market to say, “We think it’s worth this, therefore the insurance value is this.”

And this organisation — whose name still escapes me, I’m going to remember it later, and it’s going to drive me nuts — it was asked to them, “If you had to insure the North American power grid, how could you put a value on that?” And there was a ton of research that was put into it, as to how much money would be lost if this portion of the country hypothetically lost power, if it was a major uban area, if there was major fallout?

And thinking about it — thinking about all the damages and all the costs and all the potential lose for life, is really rather crazy. All we have to do is look at what happened in Puerto Rico recently, to see that a catastrophic loss of power and infrastructure could result in a massive loss of life. There are aspects of a disaster, where if you lose power you typically don’t think about it — like life support systems in hospitals, and things of that nature — where they’re relying on power in order to stay alive.

So it’s really interesting to think about how cobweb-delicate things like the power grid could truly be. If somebody had access to those systems, how easy it would be to plunge portions of the country or the entire country into chaos, depending on what kind of access they’ve got?

I know that that sounds like fear mongering as well, from my side, but researchers from an organisation, Dragos Security, they are really good at sussing out issues, from security issues in the power grid, and determining whether or not an issue in a power plant or a water treatment plant that happened some place in the country — whether or not it was just an errant piece of malware that attacked a system, or if it was actually orchestrated damage that happened.

And nine times out of ten, at least right now, it’s just malware happens to get on a system, and there’s no major impact. But that is something that keeps me up, and keeps me wondering — if someone actually had a level of access to do those kinds of things, how bad could it possibly get?

Len: Actually for some reason, that reminds me of one question I was really looking forward to asking you is: so, using the term AI loosely, as a lot of people do these days, machine learning is often what they’re referring to — but what concerns are there in the security community around that? So for example, could a system — and perhaps this is something that already exists — can you build something that can just run infinite attacks against itself, and innovate that way? And then suddenly you’ve got an operator out there, that there are no humans behind, that can actually iterate attacks on its own?

Tony: It’s kind of funny, because there was a DARPA challenge at DEF CON. It was a year ago, where they had systems that were designed to both attack and defend one another. They had a set of AI and machine learning systems that they threw onto an enclosed network, air-gapped network on its own. And they brought them to Las Vegas, and it was a simulation on how AI would find vulnerabilities and attack vulnerabilities. It was just absolutely insane, how fast these things work.

A lot of people are worried about artificial intelligence and machine learning and what it can mean in terms of automating attacks and automating security and things of that nature. A lot of these systems, I’m not going to say that they’re signature-based, because they’re not, but machine learning and AI depend on having a set of data to learn from, and a constant environment, to where they’re able to say, “This is normal and this is not.”

And because computer networks and data systems change so frequently, it’s really hard to be able to train these AI and machine learning systems to be able to adapt to changes in the environment. So if you rename a file, or if you change a directory from where it’s running, and you try running that same executable again — something that’s machine learning or AI-based, might think that it’s fine to leave it alone because, I renamed “powershell” to “derpshell,” and moved it to one file, from one part of folder, to another. And suddenly everything is okay. I’ve never seen that before, and I’m going to assume that it’s alright, because it’s not in my data sets and I don’t know what to do with it.

Len: So we don’t need to worry about security analysts being beaten the same way AlphaGo has cracked Go.

Tony: I wouldn’t say necessarily, just because there’s — at least for the foreseeable future, there’s going to be a human aspect, somebody that has to help train the machine, somebody — I use this term lightly, because it’s information security has a lot of acronyms in terms from the military, but AI and machine learning is a “force multiplier,” where you have somebody that is manning it, and making sure that it’s running properly.

It can help you do more things. It can help you look at more data. But it needs somebody there to validate it and give a thumbs up, and say, “Yeah, this is accurate.” Or, “This is what I’m expecting,” say. “Hold the phones, why are you doing it that way?”

Len: One question I have is about the concept of red team and blue team. I’m sure everyone listening who’s familiar with the security space knows all about what that is, but I was wondering if you could talk a little bit about that, and about the competitive aspect of [security]?

Tony: It is rather competitive, I will definitely say. And for the most part, my career in both IT and security has primarily been blue team. And sussing out the bad guy.

Len: I just wanted to ask if you could explain what red team is and what blue team is?

Tony: So to put it basically, blue team are the protectors. They are the ones that are identifying the vulnerabilities, patching the vulnerabilities, putting up the security systems or the layers of that security onion for an enterprise, in order to protect it against attackers.

Whereas the red team — they are the ones that, depending on what type of test you want done, they might have a limited scope or a limited range that they can attack.

Their goal is to see how far they can get, and determine what access they can get within that scope. Or you could have a contract with a red team that could be complete adversary emulation. Meaning, they’re going to use any tactics that a nation state actor or an actual hacker or a bad guy on the internet could use. So there’s no holds barred, there’s no — that’s not out of scope, and you can’t touch that. There’s, “Did I get in? Did I achieve my objectives, and how did I get there?” So the red team is attack. I equate them to the sword. And the blue team is the fence, equating them to the shield.

Len: And before I interrupted you, you were saying that you’ve found yourself being blue team for most of your career?

Tony: Yes, I’ve found myself blue team most of my career. Because it’s kind of like a natural transition, coming from an IT background, in that, if you’ve done systems administration or network or computer networking or network administration, a lot of the skill — you know how these systems are built, and you learn how to defend them. And you learn where the weak points are, and what needs to be shored up.

That [isn’t???] to say, the same can’t also be true for a red team. If you’ve done systems administrating or network administration work, you know what the systems administrator mindset is. You know where they might be lazy. They might reuse the same credentials for a bunch of their systems on the network. Or they might put all their credentials in one place on a network share or something, and if you know where to look, then you have the keys to the kingdom.

Len: Is it conventional in these exercises for the blue team’s job to be complete when it’s repelled an attack? Or does blue team go after red team?

Tony: Sometimes it’s funny in that, if the blue team’s good enough, they’ll spot the red team, and they’ll just mess with them. They’ll just say, “You’ve got a shell on the system.” Like, “Nope, I’m going to quarantine that.” By a shell, I mean like a remote session on a computer. They’re just going to kill that connection, or — to kind of answer the question, blue team isn’t considered a failure if they weren’t able to spot the red team. The goal is for both sides to learn from one another.

So if the red team couldn’t get in, they ask, “What were you guys doing to block me?” And we’d say, “We were using this technology. We’re using these things. We saw you trying to do this, so we just assumed that this was coming next.” The blue team might say, “We never got an alert, we never saw anything. So how did you all get in?” And that’s when they produce the report, and they say, “Here’s all the things we did, here’s how we got from one system to another. And produce the evidence that they got there. So both sides learn from one another. There isn’t really a loser in these exercises.

Len: And in real life, if a company — say — is being attacked, is it ethical for it’s analysts to go after the attackers? I assume it is, but is there some limitation on that? I guess I’m just curious about the real, in the trenches. Do you want to perhaps not antagonize them and just kind of bat them away, and become a boring target? And do you expose yourself by attacking them in return?

Tony: It’s interesting that you mention the idea of attacking the real bad guys on the internet. Ghat’s something that a lot of companies want to do. However, like everything else, it’s not really a black-and-white problem. Especially when the bad guys can make it look like they’re coming from another system or another network.

Let’s say you’re the bad guys, you’re party A, and you want to attack party B. The bad guys could attack a separate party C, and make it look like their attacks are coming from them. So their actual target will say, “It’s coming from those guys, it’s not coming from the attackers.” And they could try and attack them back. And then you have a giant mess where they’re attacking one another, and nobody really knows who started it.

So a lot of the times when it comes to cyber attacks or compromises — if the attackers were successful, your job is to — it’s almost like looking at an investigation or a crime scene. There are digital forensics experts, incident response experts who go in there and they collect all of the logs, all of the data off the systems; they collect as much as they can, and try and determine, how did this happen? How long have they been here? What data did they take? And all you can really do at that point is make recommendations to harden those systems or harden their defensive posture to make it harder — make themselves less of an easy target, or less low-hanging fruit for the bad guys to want to go towards.

Len: And at what point does law enforcement get involved in a case of say, an attack that’s successful? Or even just ongoing?

Tony: It really depends. God — I keep saying that a lot.

Len: It’s okay, it’s a complex matter.

Tony: It could depend on a number of factors. Is the target something that contains a lot of personal identifying information like credit card numbers, like social security numbers, date of birth, medical records — those things that are governed by regulatory compliance? If a target that is holding credit card numbers or credit information gets breached, then the government’s going to be all over that saying, “What happened? Why didn’t your security controls work? Why didn’t any of this compliance that you’re required to do fix anything for you?”

I know that Equifax is everybody’s favourite punching bag these days, but that’s a perfect example, is that Equifax had hundreds of thousands of records. And they’re governed by a lot of compliance requirements, because they have credit card information. Because they have — practically, they know everything about you. And then they had such a massive breach, a massive failure in security posture that all of this data left — and they didn’t know, they didn’t know for quite a while.

It was somewhere around — I want to say it was 60 to 90 days before they realised something was up. And in terms of attacker timelines, that’s an eternity.

So if it involves regulatory compliance, or anything that is regulated, the government’s all over it. If it’s a mum and pop shop that might not see a lot of business, or might not be critical to national infrastructure or anything of that nature — they might be a little bit more lax about that.

Len: I was curious about the spectrum of activities, and where they might get touched by different law enforcement agencies and regulations and things like that.

Tony: I can just tell you, from my past — I’m not getting into too many details, but there was an incident at one of the places I worked at. Where we thought we had control malware. And it turns out, it really wasn’t as bad as we thought it was. But then, the DHS got involved. They wanted copies of the data. They wanted to send things off and, when you send data to the government, it’s kind of like a — it’s a one way street. You’re sending data to them. And if they’re feeling fortunate, they might give you something and they might let you know what happens. But most of the time, it’s kind of just like a black hole. And if anything comes of it, they might let you know what happened down the line.

Len: Is it a stressful job?

Tony: Depending on what jobs you, or what job you’re doing in security. If you’re a security operations analyst and you get a little bit of anxiety having to deal with angry customers, it could be very frustrating. I know people who are incident responders, or they might otherwise be contractors, or they might be professionals that get sent out to customer locations or fly in by plane to new locations every single day of the week.

And some of them thrive by that life, and love being able to see new things and go see new places. But after a while, it could be kind of stressful. A lot of the time with my job — I feel like my job’s a little bit unique in security, in that I’m given a lot of freedom and it’s not too terribly stressful most days. But for your average analyst, until you develop a little bit of experience and have a little bit of experience under your belt, it can be a little bit stressful at first, if you don’t know exactly what you’re doing or what the next steps are. So it’s very important to stay cool under pressure.

Len: You just triggered something there, of not knowing. As I understand it, there’s a kind of unresolvable tension in security between transparency and secrecy. For example, how much information should you share with other people? Does it make the community stronger? I guess the blue side — for everybody to share as much information as they can about their own defences, or should you keep something in your back pocket? I had a martial arts trainer once who said every Sifu keeps a few tricks up his sleeve that he doesn’t teach. Is there something equivalent to that in the security world generally?

Tony: There’s two sides to that coin. Sometimes the red teamer’s, or penetration testers as they’re sometimes called as well, they might have their own custom pieces of software, custom implants or methods that they use for getting into systems that they keep a secret. They might share them with clients, and tell them how they used it to get onto a system. They write up their reports. But amongst friends, it’s a secret and I’m going to keep it safe — that’s my secret sauce, that’s how I do it.

Whereas the other side of that is, in spite of what I said a moment ago about the government being a one-way street, they facilitate something called [ISECS???], so there’s different verticals, like financial, industrial control systems — different verticals in our country. They have these threat-sharing programs where it’s kind of like a forum, where you can say, “We’ve see this threat,” or the government will say, “We’ve seen this threat out in the wild, you might want to check your logs for it.” And maybe one organization in that vertical will say, “We’ve seen this. Here’s some file hashes. Just FYI, if you see this, be aware that this is probably something bad.”

So threat-sharing initiatives are a great thing — information security and IT conferences where the red team will debut new tools that they just developed. And they’ll say, “Here’s cool new things that I’m using to get around these security problems.” And then the blue team might respond and say, “Here’s how I’m finding your new security tools, and I’m going to tell everybody how to do it.” It kind of turns it into a game of cat and mouse, when you’re sharing all of this knowledge. But at the same time, it’s better to share this information — so that both sides can improve themselves, rather than just let it remain a secret and keep it stagnant.

Len: Speaking of games, I saw a reference in something you wrote once to capture the flag. I wanted to ask you, what’s that? I mean, I know the playground game.

Tony: Okay so, a lot of security conferences around the country, they have CTF’s. And these CTFs, they’re kind of challenges to test yourself. They might be programming challenges, they might be, “I want you to get onto the system, and I want you to find this file called ‘flag.txt,’ and I want you to tell me what the hash is in it.” You collect the hashes, or you collect the flags, and you get points, and you might get prizes.

There are a variety of different CTFs I’ve seen before, where they could be, “Here’s a forensic image, and I want you to find the flag in it.” Or, “Here’s a file that I need to -” or, “Here’s an executable, I need you to reverse engineer it and get it to spit out the flag.” It might be a program that the flag’s a hidden function that doesn’t normally execute, but you debug it or you mess with it enough, and it’ll print the flag out. So they’re kind of these interesting little challenges that you can use to better yourself.

One of the more common ones that I recommend is a set of CTFs called, “Over the wire.” You can easily Google for it, but it’s a kind of introductory set of challenges, where they’ll say, “We need you to find this file that’s exactly this size and it has this many lines in it.” And you would go and say, “Well, what program can I use to sort through this list of files to find files to meet this criteria?”

And there’s introductory challenges like that, and then there’s the crazy complex stuff where — like the FireEye flare challenges that they do yearly, the malware company, where they’ll say, “Here’s an executable, we want you to reverse engineer this and get the flag for the next challenge. And you can’t access the next challenge until you get the flag from this challenge.” And so on and so forth. Their exercise is kind of like that hands-on training, like building up a lab, like I recommended earlier. They will grind you out, they will make you go crazy, they’ll make your head pound — and you won’t want to stop until you solve it.

But there are things that will teach you a ton of information about different types of systems. Even if you never solved the challenge, and you have to read somebody else’s write-up, they’ll teach you a lot about security and computing in general.

Len: Speaking of labs, I guess that’s a good opportunity to move on to the subject of your book. Could you talk a little bit about virtualization and what a lab is, and why it’s important to understand this?

Tony: So virtualization is a relatively common technology these days. Virtualization is, to put it in layman’s terms, essentially taking your computer and dedicating a set of resources off of that computer to run another computer, or another system inside of it. For example, I run Windows at home. I run Windows 10. And it has a hypervisor or virtualization software on it, called Client Hyper V.

So, I will start that up, I’ll say, “I want to create a new virtual machine.” And then it’ll ask me, “How much RAM, how many CPU cores, how much drive space do you want to give to this virtual machine?” And I say, “Oh give it a core, give it a gigabyte of RAM, and give it like 10 gigs of disk space.” And then it’ll say, “What operating system do you want to run on it?” And you give it an install CD, or point it to a file that has the installation information on it, nd you tell it to go, and it acts like it’s it’s own computer, but it’s its own little virtual computer that’s running inside of yours.

I use virtualization in lab environments to help me recreate customer issues with Hurricane Labs, or to help me test out new pieces of software that my boss might want to use. He’ll say, “I need you to set up an environment 10, run these two pieces of software and see if they can be integrated.” And I’ll spin up two VMs. I’ll install the software, and I’ll try and get them talking to one another. It’s very important for learning new pieces of software, learning how to network things together. Virtualization is very, very convenient for trying to learn those hands-on skills these days.

Len: And what’s a hypervisor?

Tony: A hypervisor’s just a fancy word for virtualization software. There are two types of hypervisors, in that there’s bare-metal hypervisors that are an OS operating system, that are just strictly dedicated to running VMs. And then there are hosted hypervisors — this virtualization software that runs on top of an operating system that’s already installed. A popular bare-metal one would be VMWare’s ESXi, whereas a popular hosted hypervisor might be [VirtualBox])https://www.virtualbox.org/_, running on top of Windows, or running on top of Linux, or something like that.

Len: You spoke about air-gapped systems earlier, and that reminded me of a question I wanted to ask about virtualization, and doing it in the context of security.

Forgive me if this is just a misguided question, but if you’re, say, working with a live virus, or some form of attack for which there has been no defence developed yet, would you use that in a virtualized environment on a normal machine? Or would you use virtualization for that at all, I guess is my first question? And the second one is — would you always air gap in those circumstances, where you’re building an environment to test solutions?

Tony: So it’s interesting. Because the funny thing about the book that I wrote is that a lot of the things that I do are really — a lot of the things that I tell readers to do are more focused towards building a malware analysis lab. Kind of what you’re talking about, where the ways to jump out of that virtual environment, or to jump out and attack other systems on the same network — I make it a point to say, to disable those things. Like, VirtualBox and VMWare might have shared folders, or the ability to drag and drop to and from a virtual machine to your physical computer’s desktop.

I tell users explicitly, “Disable all of these things. Because if you’re running malware, the malware could escape the lab. Or if you’re trying to simulate a penetration test or attack a host on your virtual network, that way those attacks won’t affect hosts outside of that.” The idea of isolating or air-gapping the network is pretty important. It’s something that I recommend if people are going to run virtual labs.

Even if it’s just for something as simple as testing out new software, if you keep that network isolated, or if you keep it well-controlled, then there’s little risk to your other systems if you’re just experimenting or running malware or, doing just about anything.

Len: And taking all your knowledge, you decided to write a book about it. It’s a huge, huge book. Very detailed. At the end, you talk about how many words there are, and how many screenshots there are and things like that. It’s massive and thorough.

You mention in your introduction that there was a fair amount of shouting at the screen, I think, or at the process that happened while you were writing it. I was wondering if you could talk a little bit about what form that took for you? Because there are a lot of — I mean I’m a shouting-at-the-screen kind of person. But writing books can be… it’s a journey.

Tony: It definitely is. And before moving on, I’m going to say that, before I decided to write this book, I had tried writing a book two other times, and for various reasons here or there, either I couldn’t progress it enough, or I didn’t have a clearly-defined idea. This whole book writing thing is very new to me, and I had no idea how I was going to do this.

And one day my boss at work asks, “I’d like us to have some training material for our level one technicians to work with, and do you guys feel like writing something?” That was all that it took for me to go off on this completely crazy direction in building a virtual lab. So a lot of my frustrations with the book is that a lot of people who are IT or infosec think that they’re masters. And then you actually try to teach something and document how you do it, and the programs will throw curve balls at you that you didn’t expect.

Or they operate in ways, like, “Ha, that’s interesting it doesn’t normally do that for me.” And you have to figure out why it’s doing that weird thing. Before I had started writing this book, I had never used Microsoft’s Client Hyper-V for virtualization at all. And it had a couple of unique problems that I ran into for getting networks to talk to one another, or getting them to communicate — the VMs on different networks to communicate.

I had to do a lot of research, and contact Microsoft’s TechNet forum for recommendations on how to resolve my problems. It’s just little things like that, things not operating exactly how you expect them to, where I was just like, “I’ve torn down my lab, I’ve fiddled all of these switches. I’ve set all of these configuration options, and you’re just not working for me.” And then your only recourse is to say, “Okay, you guys are my only hope, please help me.”

Len: I think you rallied some troops around you — you had a fair amount of assistance along the way, which is actually a pretty common feature of successful books.

Tony: It’s fascinating. When I was making the book, I made it free — I made it available for free while I was writing it. And I mean, I haven’t seen that done too terribly often. I’m not going to say I pioneered it. But while I was writing it, I would make another portion of the draft available, and I would put it up on my website and say, “Take a look at this, maybe it’ll help you. If you have any suggestions, let me know.” And people say, “This isn’t accurate.” Or, “This page has a bunch of errors.” Or somebody would say, “This isn’t this type of virtualization software, you’re a little bit confused.”

I learned something very quickly, and that is, it’s very easy to get people to give you recommendations on the internet. So I’m going to kind of segue a little bit here, and say the hacker mindset is to use things in a non-conventional manner, or to go off the rails a little bit. And on the internet, it’s very easy for people to give you feedback or tell you why something that you wrote isn’t necessarily accurate, or isn’t necessarily correct.

The way that I use that to my advantage, is I put these drafts out there, and I would say, “Here’s some free information, let me know what you think.” For every issue or negative bit of feedback, I would say, “I need to account for that.” Or, “I need to improve this section.” Or, “Maybe I should move these parts around?” So over the course of time, I just kept snowballing the material.

Primarily, the reason why the book is 600 pages, is because I cover five different hypervisors in the book. A lot of people say, “Qhy didn’t you cover this one? Why didn’t you cover that one?” It’s like, “The book’s 600 pages now man, I can’t make it any bigger — because literally it won’t fit in the binding if I print it.”

Len: That’s I think a relatively rare story in the conventional self-publishing world. Obviously Leanpub itself was built around the idea of publishing a book while it’s in progress, and getting feedback and iterating while you go along. It’s interesting though, it is rather rare for people to make it free while it’s in-progress. What they will do sometimes is make the price very low at the beginning, and then increase it as the book increases in length, and the certainty that it will be delivered increases as well.

Tony: That was something interesting. Like I had said before, I had never done book publishing, being an author and writing some kind of technical book was something that I always wanted to do, and kick it off my bucket list. I had no idea where to get started. Originally I was going to publish with another company, and they didn’t accept the book. And then one of my other friends who published for a separate company said, “You should try and do self-publishing.”

At the time I was like, “I don’t know if I can handle that. I don’t know if I have the resources to do that.” I am pretty sure there are grammar errors all over this thing, because in spite of English being my first language, I’m terrible at it. So while I was saying, “Who should I go through for self-publishing?” one of my followers on social media said, “You should try out Leanpub.” And another one said, “You should try out CreateSpace.” And I looked at them, and there’s no non-compete agreements between either of them, I’m aware. One site doesn’t say, “If you publish with us, you can’t go with the other.” So it’s just like, “Well, why not do both of them?” [Note: Sometimes Amazon does impose restrictions, for example, you may not sell your book elsewhere for a lower price — eds.]

There was another individual, by the name of Ray, who recommended Leanpub to me because of the model that you guys have, where it’s just constant improvement over time. If I had done things differently, I might have considered doing it that way. So I learned a lot in writing this book, and I’d actually be really interested, at a security conference, saying, “This is what I learnt about doing this.”

Len: If anyone out there is thinking of writing a book, listen to some stories first, is something I would definitely say. And if there are people who — what Tony’s suggesting — are generous enough to talk about that process, it will save you an incredible amount of time and shouting, to just understand that the experience you’re having is probably not the first time in human history that someone’s experienced that, and not the first time in the world of writing and self-publishing that people have experienced that too.

Especially if your book is going to be very useful to people, it probably will necessarily involve, at some point, grappling with something difficult, and solving a real problem. That’s sort of what makes it useful to other people.

My last question for you is a selfish one. I’ve seen that you use the character, Kirby, as your avatar around the internet. Kirby is one of my favourite Smash Bros characters to play, and I was just wondering if there’s a story behind why you use Kirby?

Tony: I’m going to tell that. And just for giggles, I’m going to tell the story about why my book was originally named “The Avatar Project,” just for funsies too.

The reason I chose Kirby as my avatar, and that I like him so much, is because Kirby represents the ability to dynamically change to your environment. Kirby saps up people’s powers and uses it against them. I like that, because it demonstrates the ability to adapt to changes, and adapt to difficulties.

Kirby doesn’t just sit down and take it, he learns learns from his mistakes. He learns from adversity and he overcomes it.

I know that’s a lot to take from a little pink puff ball, but that little pink puff ball has cracked a planet in half in some of his games. He can wield a light saber, can do kung fu — all sorts of crazy things. I think it’s just an interesting character, because it represents the ability to adapt to your situations.

Now, the reason I chose “Avatar” as my working book title, or, “The Avatar Project,” is based off of one of my favourite video games, XCOM 2. Long story short — spoilers, but the games been out for a few years now — is that your soldiers are trying to prevent an alien force from enacting a project that they call “The Avatar Project.” And “The Avatar Project” is this alien being that is like the perfect combination of human genetics and alien technology.

And I thought “The Avatar Project” sounds like an ideal book name, because it’s building a lab environment in your own image. I’m giving you the baseline to do it. And then it’s up to you to recreate it in your own image using the knowledge that you’ve picked up since, from us working on the book together, from my guiding you, and starting you off at that baseline. It’s up to you to determine how your avatar, or how your network is going to be built.

Len: Thanks very much for that answer, and for that poetic explanation of Kirby. That was fascinating.

And well, thanks very much Tony for being on the Leanpub Frontmatter Podcast, and for using Leanpub to sell the PDF of your book.

Tony: I appreciate it very much, and I am happy to be here. I just hope that the book helps somebody else out there. I love to teach, I love to teach new things. So if you got something from it — then my job is done.

Len: Thanks.

Originally published at https://leanpub.com/podcasts/frontmatter/tony-robinson-11-12-17.