The Leanpub Blog: On Writing, Publishing, Self-Publishing and Ebooks
Leanpub Podcast Interview #37: Ben Frain
published Sep 27, 2016
Ben Frain is the author of the Leanpub book Enduring CSS: Architect and maintain large-scale CSS codebases. In this interview, Leanpub co-founder Len Epp talks with Ben about his interesting career path, his book, and his experience self-publishing on Leanpub.
This interview was recorded on July 13, 2016.
The full audio for the interview is here. You can subscribe to this podcast in iTunes or add the following podcast URL directly: http://leanpub.com/podcast.xml.
This interview has been edited for conciseness and clarity.
Ben Frain
Len: Hi, I’m Len Epp from Leanpub, and in this Leanpub Podcast, I’ll be interviewing Ben Frain. Ben is based in Cheshire, England and is a senior front end developer at bet365, one of the world’s leading online gambling sites, with over 19 million customers. Ben has previously published two books with Packt Publishing: Responsive Web Design with HTML 5 and CSS 3 and Sass and Compass for Designers.
He has spoken about web development a number of events, and on various industry podcasts. In a previous life, Ben was a TV actor and technology journalist, and he has an ongoing interest in writing screenplays. He blogs about CSS web development and other tech subjects at benfrain.com, and you can follow him at @benfrain on Twitter.
Ben is the author of the Leanpub book, Enduring CSS: Architect and maintain large-scale CSS codebases, and we’ll be talking about his book a little bit later. In this interview we’re also going to talk about Ben’s professional interests, and at the very end, his experience using Leanpub, and why he chose it to self-publish his book. So, thank you Ben for being on the Leanpub Podcast.
Ben: Thank you. Thanks for having me.
Len: As I think you know, I usually like to start these interviews by asking people for their origin story. And I know from an interview I read about you that in your case, the path to where you are now is perhaps a little more circuitous than it is for most people, and I was hoping you could talk a little bit about how you came to where you are in your career?
Ben: Yeah, okay. Well obviously I’m primarily a software developer now. And the closest I got to software development in my younger years was, when we had a BBC Model B computer when I was a young boy, and we used to get computer magazines here in the UK. I don’t know if you had them in the States? They contain a program, which you could basically spend four or five days of your life typing out. And then, hit go and they never worked. So that was sort of my initial foray into programming.
I basically started my career - I always wanted to be a film director. So, from sort of young teens, that was where my head was. I did film directing at university, and tried desperately to get into that career. Where I live in the UK isn’t exactly a hotbed for the film industry, but off the back of doing that at university, I used to do a lot of acting as well. An agent took me on for the acting, and so I followed that path for a short time, and did about five years of TV acting here in the UK - none of which set the silver screen alight, but sort of soaps that are well known over here, like Coronation Street and the like. I was in that a couple of times.
The frustration with that sort of career was that I was imagining myself getting alongside Bobby DeNiro - and I was actually getting 10 lines in a not-so-great soap opera. So my idea of where I wanted to be in reality weren’t measuring up. I’d got to a point where I’d started to write my own stuff. I’d written a screenplay, and I wanted to try to get that off the ground. I’d done short films before; the same time I was doing the short films, was when I started to be sort of tallied in with the beginnings of web design. This was about sort of ‘98 - something like that, or ‘99. A lot of people do websites for their band; my first website was one for a short film that I’d made.
And so these two lives started to develop - one where I was furthering my web development skills, and the other one was where I was trying to get this film career going. And the reality was that on a long enough timeline, what I wanted to do didn’t pan out, and what I never planned on doing, did. So, that’s kind of where I ended up.
I tried to get films off the ground, myself, tried to get them financed. It was very tough and I basically failed. I was into writing screenplays, and that made no money, but at the same time I was writing for technology magazines here in the UK and abroad. So that was where I was making my money at the same time. So yeah, I was making my money making websites for other people and writing for magazines, and then actually trying to get these screenplays off the ground, that never saw the light of day. So eventually, it just came to the point where I was so much better at the web development, that was taking off, and that’s kind of the path that I followed.
That kind of brings us up to about 2011, something like that. I’ve been doing web development freelance for quite some time, and then decided to try my hand at a book.
I did a couple of books with Packt Publishing. I’ve always enjoyed the writing process, because I feel like I probably learn as much by writing something for other people, as I do just trying to learn for myself. There were good things about writing for a publisher, and there were lots of bad things. And we’ll get to the bad things later on, maybe? That kind of gives you the versions of my torrid career path, I guess?
Len: I have a question specifically about the process of getting a screenplay funded. Maybe like some other listeners, I’ve written a couple myself, totally amateurishly, and entered them in screenplay contests or passed them around to friends, but I was wondering, when you say you tried to get them off the ground, what does that mean? Does that mean approaching producers that you’ve been introduced to? Or trying to get introductions to people with money who are interested in producing a film?
Ben: It was a bit of both. I’d done a bit of screenwriting, and I managed to get a literary agent as well off the back of that. So much like my acting ideals didn’t match the reality of what agents wanted me to do, it was the same case with the screenwriting. They wanted me to be writing episodes of soap operas, and I wanted to be writing blockbuster films.
But throughout the course of the different people that I knew in the industry from doing what I did for some time - it was basically any producers that would let you bend their ear. There were production companies, as well, that - you probably had a very similar experience. You get a lot of responses from people that sound quite promising, but ultimately end up in nothing. And so when you’re in the beginning of that journey, you always feel like you’re on the cusp of something happening. And for me, at least, it never actually did. And so I got better at recognizing those situations.
But the one that I was going to try and make myself, what I planned to do was find 1,000 people to each put in 1,000 pounds. And so everyone would have an equal share in this million pound, low budget film. But I just didn’t anticipate what a skill producing is, and how difficult it is to actually go out and get money out of people. It was basically something that I never considered that would be the difficult bit. So I spent all the time on the script, and the director, and the cinematography, and storyboarding - and all that sort of stuff. And I just sort of thought, “Well, of course people would want to put in 1,000 pounds to be part of a film.” I just thought that would be a given. And nobody did. So it didn’t pan out that well.
Len: That’s really interesting, it sounds like having a screenplay and trying to get it produced is a little bit like being a startup and approaching venture capitalists. Almost none of whom will ever give you an outright “no”.
Ben: Yeah, yeah.
Len: And certainly, unless you’re a total basket case, they’ll want to maintain positive relationships with you - and with everyone they meet, in case something pans out. And that’s interesting, the sort of crowdfunding model that you’re describing. Have you ever thought about using Kickstarter or something like that?
Ben: Well there was a thing a while back, when I was last seriously dabbling in it, called Trigger Street - I don’t know if it’s still going. I think it was something that Kevin Spacey had set up. That was quite interesting, because you put your screenplay in, and it was voted on by other members of the site, and the highest ones each month got put before a panel. And so, things like that are potentially - I think the truth of the matter is, Len - at the point I’ve moved past it mentally. I’ve got two young sons now, and I think that the one remaining thing that I’d like to do for them is to try and write a novel for them, in few years’ time. My belief in that I could get one of these screenplays sold is sort of fading every day, sadly.
Len: It sounds like quite an adventure though, a definitely I think we should all live our lives aiming for what we most desire. It was interesting - speaking of moving on - you’ve written about the change, or at least there’s an interview I read online about changing from being a lone wolf - working for yourself - to being an employee for a company, and I was wondering if you wanted to talk a little bit about how that affected you, and what you’ve lost and what you’ve gained?
Ben: Yeah, certainly. Like I say, for about ten years I worked just for myself, freelancing - essentially like a sole worker. And that was all absolutely fine. I think the rub of that is that what you actually like doing is just one portion of your day. Today obviously, you’ve got to chase jobs coming in. You’ve got to chase payment, and all that sort of stuff people in the freelance trade don’t like, particularly. And I certainly didn’t either.
The good parts about freelancing is obviously, once you organize yourself, and you know how to put structure into your day, there’s a leeway that if at four o’clock one day you just know that your brain’s not working very well, you can go out and get a cup of coffee or whatever. So that feeling of being like you imagine an adult would be, where you’re in charge of your own destiny day to day - that’s what’s really nice about it.
But the flipside - I was approached to join bet365 by, it was like a recruitment agency in the UK, and when you’re a freelancer doing public-ish work on the web, recruiters are contacting you quite a lot of the time. And so that was no surprise, but the thing that they described to me was quite interesting. And it was fairly close to where I live. And they didn’t say specifically who it was at the time, but they said it was a large company. And there’s only two really big companies near to where I live - one of which is bet365, and the other one is JCB, who’s - you know like you’ve got Caterpillars there in the US? The big monster trucks or cars? Are you Canada or US?
Len: I’m based in Canada. The way I describe it, I live on a Canadian island in the Pacific Ocean.
Ben: That sounds pretty good.
Len: Called Vancouver Island, yeah.
Ben: Oh yeah, Vancouver. Yeah, I’m not a complete hillbilly. I do know where Vancouver is. So yeah, so it wasn’t JCB, because I’d already been in and spoken to them. So it turned out to be bet365. Anyway, so I joined them fully believing that I would be there maybe a couple of weeks, and somebody telling me what to do every day would very quickly wear thin.
But it turns out, it was - there’s a quote that I’d heard from somebody and they said, “People don’t leave jobs, they leave bosses”. And I always think that’s quite interesting. Because the guy who took me on, is still my sort of my boss. He’s just a really good guy, and I get on with him very well, I enjoy working with him. And I think that sort of thing is the part working for a company that I’ve missed but didn’t realize I missed.
So I’m working with like two or three people pretty closely every day, and I enjoy that. I suppose it must be like people who are in a band, being able to sort of bounce ideas off people, which is what you miss when you’re working for yourself, and what you really gain when you’re sat face to face with somebody. And that’s not to say you have to be doing that all the time, but it’s nice if you’re doing anything creative to be able to stop and sit around a table and put some sketches down and go, “This is what we’re thinking of doing”.
Being able to bash that stuff back and forth is the stuff that I actually enjoy doing the most. I suppose it’s probably the closest to the creative side of things, that you’ll have done in terms of screenwriting and that sort of stuff. The real buzz of it is when the ideas bounce back and forth, and you come up with something that no one person would have come up all by themselves.
Len: Speaking about buzz, I’m not sure if everyone listening to the podcast knows this but in England it’s more or less legal to bet on anything. I think I’m correct, right?
Ben: Yeah, that’s right, yeah.
Len: I actually lived over there for 9 years, and I’m familiar with the sight of the oddsmakers and things like that. I was wondering if there is any special buzz around working for a betting company? Is there an extra kind of pressure or constraint on you?
Ben: No, I mean I think it’s more - the funny thing is, I was never somebody who ever bet on anything. I was never brought up with betting in our family at all. That’s not to say that people in my family don’t. But I can remember distinctly thinking, “I don’t know if I’m comfortable with a betting company”. It felt slightly weird to me, almost like a tobacco company or something like that.
But the reality is, yeah it’s very much a cultural thing. It’s just not a big deal here. It’s not considered weird in any way. It’s sort of normal day to day, I suppose in much the same respect as people would go out and have a drink in a bar, and 99% of people are absolutely fine. You occasionally get people that don’t respond well - when you get alcoholism and all that sort of stuff. And by the same token you get the odd people - in terms of gambling, that go the same way. They get addicted to it. So it’s a really difficult subject to really feel out the edges of.
But in terms of what I do day to day, which is build the interfaces for these things, there are very few applications - if you want to call them that - on the web, which are, by necessity, as complicated as a betting application needs to be. So that does present some unique challenges, which are pretty exciting for what I do, to try and solve, to try and figure out - and architect, and all the rest of it. And so from my point of view, it’s a good thing, because other than that, I’d be doing the same sort of brochure sites, and magazine sites, and all that sort of stuff that are much more common.
Len: Yeah, it sounds exciting to me. I met a couple of people who worked doing the odds, and they could be literally anything, right? Like when will Charles and Camilla get divorced?
Ben: Oh yeah.
Len: And the idea of just putting odds on that - I just imagine that the user’s interaction with something like that must be kind of particularly - tense.
Ben: Yeah, the thing that I’m most involved in it’s, called, “The Sports Book.” Which is - it’s betting on live sporting events, or sporting events around the world. It was unbelievable to me when I got there and saw the scale of what they do. I mean, as an example you might go onto the site and there’s maybe 18 live soccer games happening right now, and we let you choose any one of those games, and we’ll either have a live video feed of that event happening, or we’ll give you an animated, interactive kind of like computer version of it - showing you what’s happening right now as well. So at all these events around the world, there’s guys and girls feeding back exactly what’s happening at this game at this point in time. And based upon that, the odds are updating, which lets users choose which opportunities they want to bet on second to second. Like I say, that’s the level and scale of interactivity that you just really can’t find in many other areas.
Len: Yeah, I’m looking at it right now, actually. It looks really good. I see there are e-sports as well - are you a part of that?
Ben: Yeah, all that sort of stuff. Like if you go to mobile - primarily, if you go to mobile.bet365.com on your phone - what you see there is the sort of stuff that our team prototypes, builds up. And then we have another team of devs as well, that take it and mesh it in with all the live data and all the rest of it. And we’re just constantly putting new things in there. Just when you think we thought of everything, there’s another whole set of features we want to write in. So it certainly moves really quickly, it’s a competitive market to be in. And so yeah, it gets me out of bed every day, which is good. If it keeps your brain challenged and keeps you solving problems, that generally makes me quite happy.
Len: It’s a really fascinating industry.
I was wondering - I kind of have to ask the obligatory Brexit question. But I guess it’s kind of reverse Brexit question - which is, as far as you know did the EU ever try to put limitations of the British cultural practice of betting?
Ben: Well, funnily enough, it’s across the whole of Europe. There really isn’t anywhere that doesn’t allow betting. I think if anything, America and Canada are kind of unique. China tries to say it doesn’t allow betting, but China’s obviously always been massively into gambling and betting and all that sort of stuff. And so, there’s actually quite few places around the world that don’t do betting as standard. There’s occasionally bits of legislation that are particular to different areas. Like, Italy has a certain set of specific requirements for in-play bets and things like that. But by and large, they’re all at it.
Len: I guess that just speaks to my own limitations. I mean having spent so much time in Britain, and seeing oddsmakers everywhere - but when I would travel in Europe, I guess I just probably didn’t notice that that was a practice elsewhere as well.
One more question, about the switch from working on your own, to working for a company.
I made this switch about a year ago from working remotely, to working in an office. And you’ve written about how much you enjoy being able to set up your own physical environment, and how sometimes there are things that you can’t do in an office, that you might be able to get away with doing at home. And I was wondering if there’s one or two things, sort of pro tips you can give for setting up a physical environment - things that you’ve thought about and found solutions for.
Ben: I guess the main thing is to try and get people to understand your whims. Particularly if you’ve worked for yourself for a long period of time, you know how to have things the way that you want them. And so - I mean, I’ve got to be honest - there aren’t many things that I’ve wanted at my current place of work, that they haven’t accommodated in some way.
I suppose the key difference for me is, if I was working for a company and it had a crummy, horrible monitor I had to sit in front of, and they weren’t prepared to buy a new, better monitor - I would quite happily buy my own monitor, and bring it in if they would allow me to. Because I’d rather do that - have a nice working environment - than I would use crummy tools. Because you sit in front of these things every day, and that’s how you ply your craft.
Obviously whether they should or they shouldn’t buy something better for you is a different question. But I’d rather just buy the things that I want to use to do my job, and accept that as a part of my professional outgoings, if you like. When I think about people in other trades, and what they have to pay for the tools that they need…. The sort of stuff - like the absolute crème de la crème of kit - you can get a top of the line MacBook Pro for a few thousand dollars, massive cinema displays for same sort of money. In the scheme of things, compared to other professions, it’s not a great deal of money.
Len: That’s very true. It also sounds like in an environment where one is getting recruitment calls all the time - I imagine there’s a bit of an incentive on the part of companies to accommodate a programmer’s needs. One of the things at Leanpub that we really like to provide is actually - this just reminds me - is really nice noise cancelling headphones, so the programmer can just go into their zone and stay in their world. One of the interesting things about it is that putting on the headphones is code for -
Ben: Leave me alone.
Len: “Leave me alone. I’m in the zone. I’ve built my stack in my head, and let’s be productive now.”
Just switching topics, your book, “Enduring CSS” is about a method, I suppose we could call it, that you’ve developed for organizing, and doing CSS architecture on big projects. And I was wondering if you could explain a little bit about how you came to develop the approach, and what the approach is?
Ben: Yeah, okay. So obviously, CSS and HTML has been the core of what I’ve done over the years. I’ve dabbled in other technologies, the basics of the three tiers of the front end, that being JavaScript, CSS, and HTML. I’ve specialized in the CSS particularly and the HTML.
And what happens when you move to something of size, and something with a lot of people that are all working on that code base? I guess that historically, you’re always told to lean on the Cascade in CSS. So you make some styles, and you inherit from those styles as much as you can. You try and write as little CSS as you can, and therefore when you want to, you make a pattern of things for your site, and when you want to change something across the whole site - the thought being that you go back to that one thing that everything inherits from, and you make those changes.
Which is all well and good, but what I found once I started on these sizable projects with many, many different visual components - it wasn’t just a brochure site - was that often these things that started life looking quite similar, often needed to transgress as time went on, and develop their own life and identity. And it was actually a major inconvenience to have to - you would make a change to one thing, and it would end up inadvertently changing something somewhere else. Which, on something of scale, is really, really problematic, because you need to be quite confident when you’re committed into a large code base. That the changes you make are going to make just those changes, they’re not going to start visually changing things in other areas.
The core principle, and there’s been a few sort of methodologies over the years, like BEM is one, which is Block Element Modifier, and then there’s been things like OOCSS. The two main ways of approaching CSS at scale, are you either go for a kind of atomic approach, which is categorized by atomic CSS, where you make a little class for every single permution of CSS. And so, at some point, your CSS can’t get any bigger - but the downside of that is that you have to write 10 or 20 classes on every single note in the DOM, to get it to appear the way that you want. So that’s one approach - the benefit being that your CSS stays quite small.
The flip side of that is you try and contain styles. And so by doing that, you put one particular class on each note in the DOM. And then every single thing that’s needed to isolate and contain, and render that thing perfectly, is put on that class. So you’re going to get a little bit more repetition - maybe a lot more repetition because you’re going to be declaring things like the display type, the colors, background colors. Lots and lots, but things like gzip, when it’s all smashed together - nullify a great deal of that. And so the ECSS methodology is based around the idea of isolation and containment. So the principle is that by isolating these things with a namespace, and various other characteristics, there’s no danger of styles leaking out from one component into another.
Len: You also talk about stylesheet entropy. I was wondering, is that something in what you just described? Or is that something else? It just sounds like a really interesting idea.
Ben: I’m not sure the exact context of where I’ve said that. It sounds like the sort of thing I would say, certainly. The place where I came to look at the styles on a big site where for months and years people have been working on a CSS code base. It grows, because nobody has the confidence. It’s back to that thing I was saying before, of, people are too afraid to make changes to existing CSS, because they don’t know what it’s going to affect.
Nobody seems to know what changing a class 200 lines above is actually going to do. And so, they don’t do that. And they use a very specific site, and they write some more CSS. And so you end up in this situation where the CSS only ever grows, because nobody has the confidence to fully understand what the existing code is doing. And that’s kind of what I’m talking about. And so, once you develop a very clear set of guidelines and principles for how you’re going to architect this stuff - it becomes very simple, manageable to deal with those styles or update them, remove them, knowing full well you’re not going to destroy everything.
Len: I imagine the target audience for you book is people who are either on large scale projects that have suffered from this kind of entropy, or people who are scaling up, and who might want to adopt a best practice before they proceed?
Ben: Absolutely. And I think, with all these things, there’s trade-offs. There’s no right way to do anything. There’s just a set of problems and the solution that fits those problems. And so, I’m quite clear and quite vocal about saying, “Before you do anything, you need to understand the problems you actually have”. Don’t go and pick a methodology - whether it’s mine or anybody else’s - and just think, “Well, that sounds pretty good”, and slap that into your situation, because you need to know what your situation is. You need to know exactly what the problems are that you’re trying to solve before you try and fit something to them. And so for me - obviously - it’s a methodology that works perfectly for what I do. But people need to understand the problems they’re solving before they try and apply stuff - that’s my take, I guess.
Len: You’ve got a section where you talk about - and forgive me if I pronounce this wrongly - “Pin Cing Do” - which you say translates roughly as “the way of pragmatic coding”. And it means, more or less, solving the problems you actually have. But you immediately then go on to say, “But the problems you have might not be the problems I have”. And as pragmatic as one individual might be, there’s still the problem before them, there’s this sort of combined problem of collaborating.
Ben: Yeah. I think, as well, it’s a bit of a bugbear of mine. We tend to be a bit like lemmings in web development, and we just kind of follow what somebody decides is the next hot thing, without actually stopping and deciding if it’s the right thing for us. And so that’s what the whole “Pin Cing Do” thing is about, is about having the confidence, and developing the confidence to know what suits your particular circumstances - and knowing what things to concentrate on. Some of that takes time, but some if it also requires conscious effort to do that, I think, and not just get carried away on the latest hot technology.
Len: On the subject of writing and creativity and publishing, I wanted to ask you - so you’ve written previously two books for what we might call a conventional publisher, and you decided to switch to self-publishing. Before I ask you specifically about Leanpub, I was wondering if you would like to talk a little bit about why you decided to make that switch - at least for this book?
Ben: The things that are good about a traditional publisher, is obviously that they’ve got the scope and the avenues to get your book out in all the right places. Which, if you’ve never written books before, there’s some advantage to doing that.
The disadvantages are - they take a massive chunk of your royalty, you have to use horrible tools generally to write the books, you’re working in Word templates and the like. And you’re also stuck to a contract, which means you’ve got to do stuff to a certain timeline - to certain conventions, and all that sort of stuff. That can be a good and a bad thing. In fairness, I don’t think I would’ve written my first book if I hadn’t signed a contract, because that motivated me to write it.
Sometimes not having the things that you’ve got to hit, means that you never actually end up doing anything. And so having something in place is good, but for me, it was perfect to move to Leanpub, because having done that a couple of times, I sort of understood how to structure stuff and get stuff done, whilst at the same time I could move more quickly into my own pace, because I didn’t have a contract to worry about.
That’s the other big thing that’s a real annoyance with typically published books: no matter how fastidious you are with the detail, once that thing is published and you find those little four or five things which are wrong or need fixing or are slightly out - but you’re done. They’re going to stay there and irritate you and everybody else forever. That’s really frustrating. And what’s lovely about Leanpub is, a reader can email me, and within 10 minutes, I can have the revised version up and live for people to re-download. And that feels like how it should be in this day and age.
Len: I couldn’t agree more. I have the same sort of attitude towards seeing a mistake and wanting to fix it, especially if you believe that it ought to be technically possible.
You made a comment about Word, and so I imagine that you wrote your book in plain text using our Leanpub-Flavored Markdown syntax?
Ben: Yeah I used Sublime text data - that’s where I spend like 95% of my working day, is writing code, so it’s very easy for me to just flick open a new tab, put it in Markdown syntax, and away I go. And so that’s generally how Istart writing anything - it starts out as Markdown, whether it’s a blog post or a book or anything really, a bit of documentation that I’m writing for work. And so, it just suited me perfectly, because that’s the way my mind works.
And the only time I got - well, not even frustrated, because it was kind of a think above and beyond - but I wanted to have an online version of the book as well - ECSS.io. And what I wanted to try and do is use that same Markdown text to generate the HTML, so that I only ever had to update in one place. And there’s just a couple of things which I wanted to be able to do, in terms of like putting classes on things, which I couldn’t do in the Markdown once it had run through the converter. These are sort of minor things. Maybe I’ll bug you about another time.
Len: The last question I always ask people is - what can we do for you that we haven’t done, or what can we fix that was broken? So we’ll get to that in a few minutes I think.
Ben: But by and large, it’s just been incredible, because I’ve been able to put the book up on Leanpub. And obviously, I think I started off saying it was like 70% complete or something like that. So I kept chipping away at it, till I got it to a point that I was happy with it. And then once it had run on Leanpub in a sort of finished state for a while, it’s easy to then go and stick it on Amazon, or stick it on iBookstore. Like literally within an hour, you can have generated the right files and the right stuff that they need to get it out there. And it’s just incredibly liberating, when you’re used to dealing with the massive machinery of a big publisher.
Len: That was actually going to be my next question. Did you publish it in-progress? And that sort of related to switching from working for a traditional publisher to self-publishing - is the issue of motivation and timing and stuff like that. A lot of self-published authors talk about how they like the freedom, but sometimes you miss the pressure, because it helps you get motivated to finish.
One thing we’ve tried to do is sort of replace that with the concept of in-progress publishing. So you publishing something that’s 70% finished, and then you maybe start getting readers before you’re done, and then - the fact that you have readers, and that your book is out there - provides you with some motivation.
Ben: Definitely. It definitely did for me, Len. Because I’ve got a sort of weird thing where - it only had to be one person buying that book at the 70% stage, and I felt a debt of honour, if you like, to finish that thing for that person. So the more people that buy it along the way, obviously just adds the weight. And they become like the contract, if you like - or that’s certainly the way it felt to me.
Len: I know your book is on Amazon, as well as available in the Leanpub bookstore. And for all kinds of reasons, I can see that the pricing is different. One is that Leanpub has variable pricing, wo you can set a minimum and a suggested price, and people can choose to pay anything above the minimum, including above the suggested price. But on Amazon, your book is set at $9.99. I imagine that’s partly because Amazon starts to punish you…
Ben: Exactly.
Len: …if you make your book worth more than $9.99. I was wondering if you could maybe explain a little bit about what that’s like for you as an author - to have your book available in two different places, for two different prices?
Ben: Yeah I mean, the reality is, it probably won’t be on Amazon for much longer, because it only went on about 10 days ago, and I didn’t really realize at the time that that would be the case. It’s incredibly frustrating, because obviously they take a massive chunk of the money anyway, that they kind of strong arm you into setting it at a certain price.
Len: I was just going to say - as I understand it - I mean there are all sorts of complications to Amazon’s royalty structure. But one of the main divisions is that you can earn up to - I think - a 70% royalty on a sale, if your book is worth up to $9.99. But after $9.99, it goes down to 30%. [Editor’s note: it’s 35% above $9.99, which isn’t much better than 30%, but is a bit higher.] And so, if you price a book at $11.00, you can make less money than you would if you priced it at $9.99.
Ben: Exactly, yeah.
Len: And this is a particular issue for people who are writing technical books, or books that have a higher inherent value, perhaps that you might say - to the reader, than other types of books. Because if it’s for your work, for your career - to solve a problem that can make an entire organisation more efficient, then that book might be worth $1000, not $9.99. And Amazon is inherently structured to not be friendly to books of that kind.
Ben: Yeah absolutely. And like I say, it’s something that - I wanted to get it on there for an experiment to see how it’d do, to see whether it would sell substantially more units that way. But like I say, it’s been on there about 10 days and it’s not sold any more than I have on Leanpub, and obviously it’s a much more frustrating process. In time, once there’s reviews and stuff like that, whether that makes a difference, I’d be keen to know. But that’s sort of something I’m saving up to talk to you about.
Len: I’ve just got one more question before we jump to that, which is, you’ve mentioned making corrections, and you’ve published in-progress. I was wondering, is engaging with readers something that’s been important to you throughout this process?
Ben: It is, and I think definitely, Len, with the subject of this book particularly, because it’s not like a simple, “Do these things, and there’s the answer and it can only be one answer”. It is the sort of book where, if you’re a CSS developer, you’re starting down this road. You’re going to have a lot of questions, even after writing that book. And so, being able to have back and forth with people and, yeah, that sort of thing.
You basically have to meet people on whatever medium they want to do. Whether you ask for it or not, some people are going to email you. Some people are going to reach out to you on Twitter. Some people are going to put a comment on the site’s page. So you just try, and have to be, as vigilant as you can to meet people wherever they are, I think.
Len: That’s an approach that a lot of people take. It’s one of the reasons that we produce output in PDF, EPUB, and MOBI - so that people can put their books up for sale on other places, and try that out - if that’s something that they want to do.
Ben: And I think the other thing, honestly is - well, I’ve found incredible - is the variable pricing. I remember seeing that and thinking, “Oh that’s ridiculous, nobody’s ever going to pay more”. And all the time people do, and bless them for doing it, because it’s just lovely. Because literally I’d say every one in three, is probably somebody who puts a little bit more than you’ve even asked for. And from an author’s point of view, that’s obviously fantastic. I never would’ve imagined that in a million years.
Len: Yeah, we also get people who have bought a book and then read it, and then approach us saying, “Hey - can I now pay more?” which is essentially a kind of tip afterwards. Because since people can choose - I have a strong opinion about this - it creates a very different commercial relationship between the customer and the thing they’ve bought and the person or people who produced the thing, because it introduces choice on their part, about what to do. And a choice on their part about value. And that can change. That can be different from what yours is as an author, and it can actually change after they’ve had an experience reading the book.
Ben: Yeah, it’s fantastic. I mean - I don’t care however famous people are, it’s always nice to get an email from somebody saying like, “Thanks for that, good job”. Or, “Really enjoyed it”. Or whatever. It makes your day. And so, anything that makes that easier to do, is fantastic - to open that dialogue with people.
Len: That gives me an interesting idea about a feature we obviously could add to our tip jar, when we do eventually develop it - which will be sending a happy message to the author. It’s something we probably would’ve done anyway, but it - you just made me think about what a good feature that will be.
Moving on to the final part of the podcast. What is it that we can do for you that we haven’t done, that you think would be a good addition to the process?
Ben: I think just reviews are the main thing for me. I know so many people that, when they look at something, particularly books, without any kind of testimonials and reviews of the book - it just makes it more likely to go away and have a think about it. And I think that’s the only thing. I’m sure you must have thought about it 100 times.
Len: Yeah we have. We have thought about it. Amazon made online reviews famous, quite some time ago now. The only reason they don’t exist yet on Leanpub, is because we’re a small team and we’re bootstrapped, and there’s just always an endless number of things to do. But definitely reviews. And more broadly reader-author and reader-reader interaction is a big part of Leanpub’s future, and doing more work around that.
Currently the substitutes for reviews are the reader testimonials that authors can add, which are obviously all going to be positive. But it puts a face in the name to a claim about the book, which is a positive thing. And we also make it possible for people to make samples available through download, so that someone can download a sample of the book, and have a look. It’s sort of the equivalent of being in an old time-y paper book store, and taking something off the shelf -
Ben: Yeah, I’m going to flip through.
Len: And having a flip through. And the last thing is that, we have our 100% Happiness Guarantee, which means that with a couple of clicks, people can get a refund within 45 days of purchase. Not everyone knows about that when they come to Leanpub, and when they buy a book. But people who do know, do understand that buying the book is actually risk-free. And so they might be more likely to just buy a book, without a recommendation, because they know that it’s risk-free. But reviews are definitely something that we need to add.
Ben: Yeah I mean other than that, the only thing that I can really think of is maybe more possibilities for extending the Markdown output. But that’s really niche - really - because I don’t think most people will be making an online version of the book. So that’s certainly not something I worry too much about, because like I say, I think I’d put that at the bottom of the stack, if I was on the other end.
Len: It sort of is in a way for us as well. And there’s a bit of a theoretical reason around that, around the sort of concept of Lean Publishing - which is that 99% of one’s work, as a writer, ought to be going into writing. And especially before a book is finished. Formatting is kind of - it’s sort of like - I guess, literally ought to be an afterthought, because if you’re doing too much formatting while you’re writing, what you’re probably doing is procrastinating. But that being said, of course we want to have an awesome online reading experience. And we want our books to look really good. So it is something we - when authors come to us with a suggestion for something to add, it’s something we think seriously about.
Ben: But by and large, I’ve got to say, not just because I’m on the podcast, but it’s just been incredible. Like I would certainly - anybody listening who’s thinking about - doing Leanpub, I would encourage them to do so. There’s very, very, very little negative I could say about the whole experience.
Len: Well thanks, thanks very much for that. We really appreciate it. Actually I have one last, last question. Which is - you have an awesome book cover. How did you get that?
Ben: Funnily enough, the boss I mentioned earlier - he designed it for me. He’s just one of these irritating people that can do everything really well. And so,when I said I was going to do this self-published book, and all the rest of it, and I didn’t know what to do for a cover, we had a couple of iterations of it. The first one he did had like isobars. Like you would look at a map and see isobars, sort of hinting towards that idea of sort of terrain and topology and stuff like that.
But ultimately, we sort of felt that was maybe a bit of a vague notion. And people won’t understand it, and think it was just squiggly lines. And so, the new one - with the landscape - hopefully attests to the notion of - writing your CSS code, it’s going to last as long as the hills. Which obviously - not quite. But hopefully you get the notion.
Len: It’s a great idea for a cover, and it just looks awesome. So thanks, thanks to your boss as well from me I suppose. Just to wrap up, Ben - thanks very much for being on the Leanpub Podcast, and for being a Leanpub author.
Ben: Not at all, thanks for having me, and thanks for making such a great product.
Len: Thanks.
Leanpub Podcast Interview #36: Daniel Dib and Russ White
published Sep 20, 2016
Daniel Dib and Russ White are the authors of the Leanpub book Unintended Features: Thoughts on thinking and life as a network engineer. In this interview, Leanpub co-founder Len Epp talks with Daniel and Russ about their careers, their book, and their experience self-publishing on Leanpub.
This interview was recorded on July 12, 2016.
The full audio for the interview is here. You can subscribe to this podcast in iTunes or add the following podcast URL directly: http://leanpub.com/podcast.xml.
This interview has been edited for conciseness and clarity.
Daniel Dib and Russ White
Len: Hi, I’m Len Epp from Leanpub, and in this Leanpub podcast I’ll be interviewing Daniel Dib and Russ White. Russ is based in North Carolina and has been working with large-scale networks for more than 20 years and written a number of books, co-authored over 40 software patents, and spoken at many different venues around the world. His blog is called Network and you can find it at ntwrk.guru. He’s currently on the architecture team at LinkedIn, where he works on next-generation data-centric designs, complexity, security, and privacy.
Daniel is based in Sweden and is currently Senior Network Architect at Conscia Netsafe. Daniel has worked on some of the most demanding networks in Sweden, and is committed to mentoring new and promising engineers. He is a Cisco Learning Network VIP, and a Cisco champion. You can find his blog called Lost In Transit at lostintransit.se.
Russ and Daniel are the authors of the Leanpub book, *Unintended Features: Thoughts on thinking and life as a network engineer. Their book is focused on helping people who are building careers in networking technology, and helping them think about their knowledge-based skills and experience in an informed way. As such, it’s not a book so much about technology, as it is a really valuable and generous contribution from experienced professionals who want to help other people improve their careers in either network engineering, or information technology more generally.
In this interview we’re going to talk about Russ and Daniel’s professional interests, their books, their experience self-publishing using Leanpub, and at the very end maybe one or two ways we could help improve Leanpub for them and for other authors. I said Russ is based in North Carolina and Daniel was based in Sweden, but currently they’re both coming to us from the Cisco Live event in Las Vegas.
Thanks guys for being together in the world to do this, and for being on the Leanpub podcast.
Russ: Thanks. I mean, I think you pretty much covered the whole thing, so we can just quit now.
Len: Thanks Russ. Russ, why don’t we start with you? I usually like to start these interviews by asking people for their origin story, so could you tell us how you first became interested in being a network engineer, and how your career got you to the point where you are now at LinkedIn?
Russ: I actually kind of fell into it, accidentally, it wasn’t something I was actually planning on. I started my life doing graphic design and doing electronic engineering, and ended up in electronic engineering in the Air Force, working on airfield systems, radar systems, and BOR’s … and stuff, and somehow or another ended up in this small computer support center, probably because I’d learned how to code, and they needed a coder to do some odds and ends. So I ended up there building systems and fell into the project of helping replace the core network - the optical fiber ring at McGuire Air Force Base - and replace the telco switch, which is an old 1960s telco switch, for which our primary troubleshooting pool was WD-40 in the physical rotary switches in the frame.
Len: Wow.
Russ: It’s pretty old isn’t it?
So from there I went into networking intentionally. I went to a small company, then went to BASF, and worked there for a while. Then I moved to North Carolina on a lark with no job, and ended up at Cisco Systems in the technical assistance center. I was at Cisco for 16 years, then I went to Verisign Labs after that, because I thought it was time to move on and do something different. And then I was at Ericsson for two years working in the engineering area.
And by the way, when I ended Cisco I was actually working in engineering, I was actually working on the coding side of things rather than the network design side of things. Specifically, I was kind of in a mixed role in the engineering department, doing coding and network design and stuff. And then from Ericsson, I got to know some folks at LinkedIn, and they pulled me over on the architecture team to do next-gen network design, next-gen data center, hyper scale design, which has been really interesting and fun so far.
Len: Many of our listeners are themselves software engineers, and I was wondering if you could tell me a little bit about what - Daniel actually, I’ll ask you a similar question about from your career later - but what’s a day in life like for someone who’s doing network design at LinkedIn?
Russ: So, LinkedIn is a little bit different because we’re a hyperscaler, so we don’t do normal network design. It’s kind of hard to explain. But hyperscalers, we’re all about simplicity - and just getting a lot of boxes into a single data center, and figuring out how to make the network as much as possible. So it’s kind of a little bit odd and different from doing enterprise design, and service provider design, or transient service provider design - which I’ve done before as well - in that we’re much more concerned about how our control plane interacts with the apps guys, and the application side, and the microsegmentation, and security, and stuff like that.
So I have a little bit more interaction with application and the house security guys, and etc., than I would possibly at an enterprise shop. But it’s also very interesting in that I’m also more involved in how the sheet metal is bent, and what type of chipsets we’re using, and things like that, which you don’t normally get into necessarily in the enterprise side.
Then life for me is mostly throwing interesting ideas on the white board, and see if we can make them stick. Talking to partners, talking to vendors, figuring out how we can get where we want to be from a five-year plan perspective, helping the ops guys understand what we’re trying to do, doing training. And for me I do a lot of ITF work, which is in LinkedIn’s best interests, and B2B security, and talking to people in that world on a regular basis, trying to figure out how LinkedIn participates in the larger community, rather than just being our own little closed-in network that nobody really pays attention to.
Len: Thanks for that. That’s a really, really great answer. Daniel, now it’s your turn. What’s your origin story? And where are you in your career right now?
Daniel: My background came from growing up with a Nintendo and everything. Then I started to get an interest in computers. So I was using the Commodore 64, and then the Amiga, and then I moved on to the PC world. And then I progressed through school technical education and then I moved on to the university, and I found an interesting program there, and the university was Cisco Academy.
So that’s how I kind of got involved into the networking world. And I caught on quite fast, I figured out that this was really interesting and that it was something that I wanted to work on in the future. So I completed the program in the university for three years, and then I started my career as a network engineer, in the networking industry, and working for a service provider at first. And then I moved on to another role, where I also did a bit of design and working on different customer projects. And about a year ago I ended up working for Conscia Netsafe.
So I’m working as a network architect, meeting with customers and discussing designs, doing a lot of writing and diagramming, and explaining to people, and also acting as a subject matter expert in different technologies, helping my customers on different projects. So I guess that’s where I’m at right now.
Len: Thanks very much. One question I like to ask people who’ve formally studied IT, is, do you think, if you were to go back, would you do it again? And I’m asking because I find that actually about half of the people who I speak to did have a formal education in IT at university, and half didn’t. And about half of those who did have a formal education in IT, say they wouldn’t do it if they went back. And I was wondering if you would- if you would do it again?
Daniel: Yeah, from my perspective I would definitely do it again. That’s mainly because of two reasons. The first reason would be that it’s a lot about the soft skills you learn at universities. You learn how to consume information, digest it, and do proper writing, formatting, and those are skills that are really useful if you want to move into an architectural role in the future. And the second reason is that I really like the way they had the program set up, with a lot of hands-on and stuff. So we actually got to do a lot of work on real devices, and learn from the basics - how to cable stuff, how to configure stuff, and so, I think it’s a good way to learn the basics.
Len: I guess things might be a little bit different in a world where you’re dealing with very complex - where you’re going to be given the responsibility of dealing with very complex networks, with very large and established organizations, as opposed to, perhaps, being a developer for a start-up.
Daniel: Yeah, sure.
Len: Russ, on the subject of education, I saw from one of your bio’s that I think you’re working on a PhD right now?
Russ: Yeah, I’m working on a PhD in Philosophy. I was at dinner Monday night or Sunday night, and somebody asked me, “So how does a philosophy degree make you a better engineer?” My answer was, “Why do I always need to be a better engineer? Maybe I just need to be a better person.” But my answer for that is pretty much the same as Daniel’s. Doing a PhD has helped me understand a lot more about ethics, and about how the technology is applied. And it’s also helped me a lot with my research skills and writing skills, and just having the tenacity to go on and do what needs to be done, and to be more of a whole person.
And I’m asked all the time, “Should I get my certification, or should I get my degree?” I always say, “Both”. And then people say, “Well, should I get my third CCIE, or should I get my CCDE, or should I get my degree?” And I always say, “If you have a CCIE and a CCDE, go on and get a degree. Don’t continue piling up certifications. Spread out and do other things, because those are important things to do in your life”.
Len: I’ve got to say I’m a personally very much on the same side as both of you on this question. I got a doctorate in English Literature, before becoming an investment banker. and to me the value of - in addition of the things that both of you have said, the value of taking a few years to engage in a highly focused way, with a large subject is in itself just inherently rewarding - and informs everything else one does in one’s life. And there’s no substitute for that, I would say.
Russ: I would call that developing intellectual virtue. To go to an Aristotelian term - since I’m doing philosophy, you always have to find a way to work your Aristotle and Plato into this - but this whole concept of intellectual virtue, the ability to learn how to focus, and to learn how to learn - I mean to learn how to consume information very quickly. I think that’s part of what Daniel and I are trying to address in this book, is that, we engineers, we tend to go, “Oh, I know how to work a CLI, I’m kind of done now”. Well no, you’re not really done now.
Len: On that subject, you’ve got a section on culture. And there’s a story about General Howe’s Dog, and I was wondering if one of you wouldn’t mind telling that story, and explain why it’s so important.
Russ: Yeah, it’s this little bitty book called, General Howe’s Dog that you can find that explains the whole thing. Essentially at one point General Howe and General Washington - in the American Revolution for anybody who’s not up on their history - were facing each other over a battlefield, and General Howe’s dog, which was some sort of a small terrier, actually slipped across the battle lines, and was captured as an enemy combatant, and taken to - I guess he was biting the soldier’s heels or something? Anyway, so he was taken to General Washington. And General Washington actually sent a soldier back under flag of truce with a very, very nice letter - you can read the whole thing, it’s available online - explaining that they had found this dog, and someone had recognized it as General Howe’s dog - and returning it to him.
And I think the significance of that in our world is, we get really combative about what we believe in and what we do. And if you can actually find two people who are fighting each other - literally to the death, and sending men to their death - but yet they had the humanity to figure out how to return one another’s dogs - with a nice note, under sign of truce, I think maybe we can all learn how to get along better, and not treat each other so nasty in the middle of big arguments over technology and stuff.
Len: And you had a specific experience you related that to, I think it was you Russ, in your career in the Air Force. I was wondering if you wouldn’t mind talking a little bit about that.
Russ: Yeah. I think that was the Banyan VINES fight at the McGuire Air Force Base. We were deeply involved in an absolutely huge, knock-down, drag-out fight between Banyan VINES and Novell Netware. And I was on the Banyan VINES side - believe it or not - even though I was at C&E at the time. And I was studying for my CBE, but I thought VINES was a better technology - and so did pretty much everybody else on McGuire Air Force Base.
But we were basically overruled by the headquarters. And they said, “No, you, thou shalt use Network Netware.” And it actually ended up ending several people’s careers over it, because the reaction on it was so, so bad. And I wonder sometimes, I think part of that was just political scapegoating. But sometimes I think that’s just because we get emotionally invested in technologies in ways that are really, really unhealthy - and technical solutions in ways that are really, really unhealthy. And it helps really a lot if you can disengage from that, and just realize it’s a tool. It’s just a tool. It’s nothing else. You don’t fall in love with your hammer, and shouldn’t fall in love with your router.
Len: And what does it mean to have your career ended in the Air Force? I’m actually curious about that. When I read that in the book, I’m not exactly sure what that means?
Russ: Well, people were given low performance reviews for their participation in the whole project on the VINES side. Which essentially stops you from ever getting a promotion again. So the next time you’re going to re-sign up, or re-contract with the Air Force, re-enlist, you might as well not bother, because your career is stopped, you’re dead.
Len: And why do you think people fall in love with their router?
Russ: Because it’s such a beautiful shade of blue. I don’t know? I think that this is an ownership issue, and I think it’s healthy in some ways. There’s a pride in building a really nice building. If you’re a carpenter, you go out and you build a really nice building. Or if you’re an artist, you draw a really, really beautiful picture. And there’s pride of ownership in that - that’s really important, that drives you to excel, and that’s really good.
So in a technology situation it’s really good to fall in love with the technology enough that you really, really focus on, “How am I going to do an excellent job, and do the best I can to make this technology fit in my network, and do the best job that I can?” But the problem is that you need to be emotionally detachable in a way that allows you to say, “Okay, in the end I wasn’t right. That wasn’t the best solution”. Or even if it was the best solution, “I was overruled by somebody who thinks they know better, so I’m just going to do my best with what I have”. So I think it’s more a matter of, if you build the house, you own it. And because you own it, and you’ve built it, you have an emotional attachment to the work that you’ve done in that space, and I think that’s the way we treat technology.
Len: Daniel, there’s a section in the book on personal integrity, and I was wondering if you wouldn’t mind talking a little bit about what the both of you are saying in that section, and how it relates to one’s career as a network engineer.
Daniel: I think personal integrity is a lot about having values that you follow. So you should do learning in a proper way, not try to take short cuts, because if you do that, you’ll end up being punished later. Every time you try to understand a new technology itself, and based a lot on previous technology, so if you take your time in the start - and learn the basics, and learn the protocols - you’ll do a lot better further into your career.
Len: I actually had a question that’s specific about Cisco. I interviewed Nick Russo, who I believe both of you know, very recently. He’s got a new Leanpub book as well. And so, Daniel for example - you work for Conscia Netsafe in Sweden. But here you are in Las Vegas at a Cisco conference. And I was just curious about how that relationship works?
Daniel: Yeah, so we’re a systems integrator and a Cisco Gold partner. So obviously we try to keep up with everything Cisco is pushing out. So we usually have, like, four or five guys every year that goes to the US conference to pick up what’s new and what’s trending. Then we try to bring that knowledge back home and train our other staff, so that they get up to speed on those technologies as well. So yeah, always trying to stay ahead of the curve, and seeing what’s new.
Len: When we were speaking just before we started this interview, Russ, I think that you mentioned that there were 25,000 people there at the US conference?
Russ: Well, I actually don’t know what the number is this year. I know last year it was 25,000, and I think it’s probably that size or larger this year. I haven’t heard an official number, but I know that they’ve taken up the Mandalay Bay Convention Center, and it is - they’ve taken up every stick to the floor space in the entire Convention Center, and it’s huge.
Len: If someone listening was considering going into a career in Network Engineering, I mean it sounds like obviously it’s a massive industry, would you recommend it? Is it something that’s growing?
Daniel: Yeah, I think it’s really, really a nice industry to be in, because there’s a lot of things happening right now. In the last couple of years, it’s really starting to pick up pace with all the new technologies - and trying to move towards more automation and software-defined networks and everything. So it’s kind of interesting place to be. A bit scary at the same time, but also really rewarding if you take the time to learn these new technologies.
Len: Why is it scary?
Daniel: Because if you have a current set of skills and then there’s new job role sets asking for a new kind of skills, and if you don’t have them yet it can be, it can feel a bit scary - if you can learn those new skills or not….
Russ: Yeah, in the networking industry there’s a couple of things we always say. One is nobody has a job more than two years. I mean you just don’t know what it’s going to be in two years. So you just might as well plot your own career path, and not be too freaked out about, or too concentrated on working for one company for the rest of your life. Because it might happen, and that would be nice. But it might not happen, so emotionally prepare yourself for that not to happen.
And the second thing is we always have this thing about the half-life of any skill being about two and a half years, so anything you learn today is generally useful for about two and a half years, then it starts to taper off in its usefulness. In about five to seven and a half years, it’s generally old enough that no one cares anymore, and this actually speaks to the whole concept of what Daniel was talking about before.
When you learn a protocol, or when you learn something from new - you need to learn the theory, and try to understand it. And understand it at a level that helps you relate it to other things, so that you’re actually building a level of knowledge that’s not just surface knowledge. I don’t just know how to configure something. I actually know how it works, so then a new technology comes along that’s similar, I can take that knowledge that I have, and I can actually push it into the new concept or the new technology - without having to relearn a whole lot of information.
Len: Going way back, Russ, can you think of an example of something that was extremely important at the beginning of your career that has disappeared? Maybe because of that two and a half year half-life?
[Laughter]
Russ: How many do you want? X25, Token Ring, old SONET. Although SONET’s still around today, but it’s really more DWDM. And let’s see, ATM. Oh my goodness, ATM is like, in horrors. When I first learned networking, we were actually using Token Bus and Tap Ethernet ThickNet, which doesn’t exist anymore. And we were using Tommy Conrad ARCNET stuff, which is so far gone, that nobody even knows what I’m talking about when I say, “Thomas Conrad ARCNET” any longer.
So yeah, I mean there’s a ton of these things. But to make that example full - if you think about it - Tommy Conrad ARCNET stuff, or ARCNET, actually worked a certain way on the wire. It used a certain signaling pattern - and carried things in a particular way, and stuff like that. Well, those things really haven’t changed, even with DWDM and things like that. The principles are still the same. The bits are different, the techniques are little bit different - but the basic principles are the same. So in learning ARCNET in depth, and Ethernet ThickNet in depth, I’ve actually been able to apply that general concept of knowledge to everything I’ve learned since then, as far as physical media goes.
Len: You guys will have to forgive me if maybe one or two of the next questions I ask you are actually totally irreverent, because I’m not an expert in your field. One of you mentioned something about scariness. I was reminded of a recent pub table conversation that I had, where the concept of a giant solar flare destroying all the networks came up. I’m sure you’ve probably heard that one. It’s out there in the popular culture that something can happen that can fry all of our networks. Now I’ve got twp experts - and so I want to ask for all the pub table conspirators out there, is that true?
Daniel: Well–
Russ: Yes.
Len: Oh no. And why is that? Why is that true?
Russ: Well, every piece of wire acts as an antenna, and if you get enough power on the wire, you can fry the receiver that’s attached to that wire. So if you get a large enough solar flare, or what’s called an EMP - whether it’s natural or man-made, or whatever the case might be - you can actually fry all the receivers on all the wires, in the entire world, or in a particular region, pretty quickly - which would take all the networks down.
Now, you try to do things like using shield cables and stuff like that. There’s lots of ways you can try to work around it. But for the most part, these things are so fast and so powerful that it’s very, very difficult to counter them. There’s actually an incident in US history where this kind of thing happened in the 1800’s, and it took out all the telegraph receivers.
Len: In the country?
Russ: In the country. It took down the entire telegraph network in the country. Just this way.
Len: And so is this a once in a century, or a once in a millennia - or a once in a - a kind of epoch kind of event?
Russ: That’s a very good question, and I don’t think anybody actually knows the answer to it.
Len: Okay, okay. Well now we can all be a little bit scared.
Daniel: I know that Russ has worked with radars as well, and I’ve seen some incidents in that space, and I think it’s like somewhere around every six to eight years they have some kind of incidents due to like the solar energy effecting the radars. So it can happen.
Len: And would it affect undersea internet cables?
Daniel: Not sure about that. I doubt it, but–
Len: Well thank you very much for answering that question. I was wondering if either one - or both of you - has an opinion about how the world will change for the general consumer, if and when we get to a point where we’ve all got gigabit per second connections?
Russ: Well, tough question. So we’ll all be uploading videos, and there’ll be so many videos, none of us can watch them.
Len: Live streaming all the cats in the world at once, or something like that?
Russ: Yes, that’s exactly right. We’ll all put little cameras on our cats, so we can all watch each other’s cats all the time. And no one will be watching anyway, except the other cats. I don’t know? You almost wonder how much more information we can tolerate. We’re already sipping from a fire hose, and I just wonder how much more we can tolerate. I mean, you think - well - there’ll be machine-to-machine, sure. But then that’s almost scary too right? Because now I have machines looking at things I’m not looking at, and making decisions about things that I’m not sure I want them making decisions about.
Len: So are you on the Elon Musk, Stephen Hawking side of worry about AI then, Russ?
Russ: Yeah, actually I am.
Len: It’s a really hard question, and I think it is something that people should be concerned about.
Russ: Well, go back to philosophy, right? That’s one of the things - we all know our networks, and we know our technologies. But we don’t think about the ethical impacts of those things, and what it means philosophically and culturally. And, so we tend to just run off and go do what we can do, and we don’t think about what we ought to be doing. So that’s one of the reasons I find philosophy so fascinating.
Len: On that topic, what is the subject of your thesis, if you’ve decided one yet?
Russ: Oh wow. Thesis, dissertation or degree? The degree is in Apologetics and Culture. The dissertation is in the unintended side effects of the technological revolution. As a matter of fact, that’s actually what I’m writing on.
Len: And what are some of those unintended facts that you’ll be focusing on?
Russ: I’m looking at things like the Stanford Prison Experiment and the Panopticon, and how people change under observation, and how that impacts our social interactions on a personal level. My professor - my major professor - actually will not buy things from online retailer, because he values the personal interaction of actually walking into a store, even if he doesn’t know the person on the other side of the cash register at a personal level. He just values that human interaction so much, that he refuses to do a lot of things online as far as buying goes.
So, I’m trying to look at those types of things, and look at how communities react, and whether or not it fragments communities, or, in what ways does it solidify communities, in what ways does it fragment communities? And then going beyond that - that’s the problem side of it, but maybe eventually start thinking about, well, how do you try to ameliorate some of this? How do you actually try to counter this in a way that’s intelligent? Or can it be countered? I’m not really even sure it can be, but–
Len: You mentioned the concept of the Panopticon. And for anyone listening who’s unfamiliar with that idea - it was developed by Jeremy Bentham, I think, in the late 18th century? Maybe the early 19th century? If you’ve ever seen an image of a prison that’s in the shape of a circle, and there’s a guard in the center - who can just, kind of, turn around - and then see all the cells arranged around the periphery of the circle, that’s a Panopticon. Or one version of a Panopticon. And the idea is that, you can make it so that everyone can be simultaneously observed. I guess this is one way of explaining it.
Russ: Without the guard being observed. That’s the interesting thing, is the original one was set-up with all these mirrors and stuff. Well, it was never really built, but it was designed with mirrors and stuff, so that the prisoners could never tell when they were being observed, but the guard could. One guard could observe every prisoner.
Len: That’s right. I forgot that very important part of it. The sort of brilliant and terrifying efficiency of the idea was based on the idea that the guard couldn’t be seen by the prisoners themselves. So they could actually be in a situation where they’re not being watched, but they don’t know. So they have to behave as though they’re always being watched.
Russ: Right.
Len: This is a fascinating metaphor, and I think extremely relevant in our era, where people are walking around - not so much with Google Glass, although that might come back now that Pokémon Go is a thing - but people are walking around with their phones everywhere, and those phones have cameras on them. And in a sort of analogous way, anything you do on social media can immediately make you a global villain or celebrity. And because everyone can suddenly see it, there is a sense, in which, in our world nowadays - I guess unless we’ve plugged out in some way, or we’re avoiding people - we sort of all have to act like we’re being watched, and I guess there might be a connection between that, and what you guys are writing about in your book - in your way. Which is it’s sort of a set of guidelines for how to behave in a professional context, where one does need to act like everything one does is going to be assessed, and is being watched.
Russ: Yeah.
Len: I guess there’s not a question in there, more of an observation. But yeah, on that note - what led the both of you to blog about these subjects, and then to get together and write a book about it? I guess Daniel, if you wouldn’t mind answering that question?
Daniel: We’ve both been blogging for several years, and I kind of started out blogging when I started studying for my CCIE. And as I progressed in my career, I got more interested in network design and architecture, and more about some of the soft skills. And I noticed that there wasn’t a lot of people writing about these things, and I thought that some of the things were missing for people, to be able to get the information on how to develop their careers and their thinking.
And then I started to get to know Russ, and I had noticed that he was like the main guy writing about these things. Like the General Howe story, and these kind of interesting blogs about how to think. And I got in contact with him, and asked if he wanted to do some writing - and that’s how we got started. So we started to take all of the stuff we had already written. And then we did some new writing, and starting to edit it, so it could flow a bit better in a book form, and hopefully people find it useful. We’ve already received some feedback from people that it’s kind of unique content, and that they’re enjoying it. So hopefully it’s a decent-size book that’s easy to consume, and that will help you progress in your career and thinking, and that’s the goal of the book.
Len: And if you had some advice to give to any budding engineers listening to this podcast - if you could just get one thing into their heads at the beginning of their career, what might that be?
Daniel: I think that goes back to what we mentioned earlier. To take your learning seriously, and learn the protocols - so that you can apply new knowledge later. Try to think about why things work in a certain way, and not just learn the bits and bytes about things. Because then that will keep you from progressing your career as rapidly as it could be if you were thinking about these things.
Len: Russ, I was wondering, you’ve written a few books, and also you’ve co-authored more than 40 software patents - I was wondering if you wouldn’t mind telling us a little story about maybe one of the bigger challenges that you’ve had in your career, where you overcame something?
Russ: A challenge. That is a hard question. So let me think. In the soft skill side, one of the hardest things for me to overcome is I’m actually a pretty big introvert, and so I’ve had to school myself particularly in public speaking. I started doing public speaking maybe 20 years ago almost now, and it’s still nerve wracking, many times when you’re up there dealing with things.
So I think one challenge I’ve had to overcome in my life is that - this whole thing of just not liking crowds, and not liking being in front of people and speaking - which I have come to peace with, I’m okay with it. And the biggest way I did it was just - if you’re afraid of heights - you go to the Empire State Building, and you walk out on that glass thing. And you look down, and you go, “Hey, I did that and nothing happened. I didn’t fall, and I’m okay - so maybe heights aren’t necessarily so bad”.
I wouldn’t say it’s that easy to overcome. But what I have found, is when I face challenges like that, I just need to practice and be intentional about it - to be intentional about what I’m learning. So I think that’s one thing; when I first started, I was really afraid of being in front of people, and being around, and being in big crowds. And I went to the ITF, and I pushed myself to go to the microphone and make comments. And I was called stupid, and I’m still called stupid today - so that’s okay, I don’t really care anymore though.
And so, I think that’s one of those things that you have to overcome - the emotional connection to your work. The first time I wrote a book - you almost feel like crying the first time it comes back from an editor and it’s all completely red. So you go through that a couple of times, and you just have to learn how to emotionally detach yourself from those situations, enough that you can work effectively, and be good. So I think those are the types of things that I think I’ve had to overcome in my career, that have taught me more about being consistent, and being intentional when I’m learning, and how I’m learning.
There’s technical challenges. I mean there’s always technical challenges that you run into, and it’s that same skill set. You’ve just got to pursue, and just keep going, and keep trying to learn things - to get to the point where you feel like you’re skillful enough to actually do the job.
By the way, we all feel like imposters. I don’t know if anybody has ever brought that up on your podcast, but every engineer in the world feels like an imposter. So if you have a listener out there who’s listening to this podcast, and says, “Yeah, but I’m not as smart as you” - yeah. Well, I’m not as smart as you think I am either, so it’s fine. We’re all imposters.
Len: That’s really interesting. I don’t know if anyone has brought that up before. On that note, I was going to ask - I mean, one of the things about people who come to Leanpub, and who we then interview on the podcast, is that they’re like the both of you. They’re writing, and whether it’s easy or not, putting themselves in public, and being called up to speak at conferences and things like that. I was wondering, is that conventional for people developing careers in your field, to do what the both of you are doing? Like blogging and making books, and speaking at conferences, and things like that - or does that make you exceptions? And you can brag if you want.
Daniel: I think blogging is probably a lot more common than writing a book. Writing a book takes a lot more commitment, and it can be challenging - like Russ mentioned, to go through all of the editing and stuff. And normally it’s not very rewarding financially either, unless you’re writing something for the masses. So people don’t really see a big return on investment from a purely financial perspective. But on the other side, you go through a learning process. And also it can be a good way to meet with people, and to learn to connect with other people. And also it could be nice to put on your resume, that you published something.
Len: And did you both go self-impose an editing process on yourselves for this book, or did you get the opportunity to forego some of that this time around?
Russ: Yeah. I think we kind of cross-edited each other’s work in a way that was helpful. Finding things that needed to be fixed, and making it flow better - that’s a skill in and of itself. So we actually did not hire an editor, but we did cross-edit, and we did pay attention to what we were trying to do a little more carefully than we might have perhaps with a blog post, or something like that.
Len: Thanks very much. That kind of practical knowledge is really important to people who are setting out to write books, and to know that it can be achieved that way, and that you can get quality work in the end.
Russ: Yeah. I think it’s important if you’re doing co-authors to have each person cross–read the other person’s material, and edit it. Because you want to get as close as you can to one voice in that situation, which is pretty difficult to achieve in a lot of cases.
Len: Yes, definitely. I can only image how the interaction must have to work in order to write a book with someone else that way. I have one question about - sort of a selfish question, to ask both of you - about Leanpub. Now that I’ve got you here, if you could wave a magic wand, and have us build a feature for you - or fix a problem for you, what would you ask for?
Daniel: Hmmm.
Len: The answer can be nothing. I mean that would be kind of fine from our perspective.
Russ: I think from the publishing end it works really well. I mean, you could guarantee each author 10,000 sales, but I don’t see how you’re going to do that, so–
Len: We’d love to be able to do that.
Daniel: I think it’s been easy to work on the platform, and it’s nice when you’re co-authoring something, because some of the other platforms won’t let you do that easily, and split the income and stuff, so yeah.
Len: Oh, thanks for that. Yeah. That’s something we don’t think about all that much. It’s just something we built when it was pulled out of us by people asking. And we’re like, “Well of course, people should be able to coauthor books easily”.
I guess I just have one last - and I’m switching subjects pretty dramatically - but one last question about your work, is related to energy sustainability - it’s something people read about in the media when it comes to network architecture. Is that something that both, or either of you, takes into account in your work?
Daniel: Do you mean like green IT, or something like that?
Len: Yeah. I mean that in a very straightforward, from-a-place-of-no-knowledge perspective. I mean, is that something that day-to-day one has to build into one’s designs or concerns?
Daniel: Well, I’ve seen some cases where we could like, propose to our customer that they buy new switches, and the power savings alone would - they would have return on investment for, like, five years just by the power alone. So that’s kind of a good way to both be a bit more environmentally friendly, and save some money as well, so there’s definitely some use cases for it.
Russ: Yeah. In the hyperscale environment, of course, the data centers are built all around energy usage - it’s a huge, huge deal for us, how much energy we use per server, per amount of compute, process type of things. We’re very tightly controlled in our data centers on how much energy we use to do a particular task.
Len: Okay, well, thanks very much guys, for taking time away from the exciting conference to talk to me. I really appreciated it. I certainly learned a lot. And it was great to have such a wide ranging discussion about so many things. So, thanks very much for taking part in the interview, and for being Leanpub authors.
Daniel: Thank you, Len.
Russ: Thank you, Len. This was great.
Len: Thanks.
Leanpub Podcast Interview #35: Janelle Klein
published Aug 06, 2016
Janelle Klein is the Austin-based author of Idea Flow: How to Measure the PAIN in Software Development. She is the CTO of the recruitment consultancy New Iron and founder of Open Mastery, a network for peer mentorship based on data-driven mastery of software development.
In this interview, Leanpub co-founder Len Epp talks with Janelle about her career, the ideas and experiences that are the inspiration for “Idea Flow”, the concept of “Idea Flow” itself and how it can transform the relationship between developers and management to profoundly improve productivity and quality, and at the very end, Janelle also talks about her experience self-publishing on Leanpub.
This interview was recorded on May 27, 2016.
The full audio for the interview is here. You can subscribe to this podcast in iTunes or add the following podcast URL directly: http://leanpub.com/podcast.xml.
This interview has been edited for conciseness and clarity.
Janelle Klein
Len: Hi, I’m Len Epp from Leanpub, and in this Leanpub Podcast, I’ll be interviewing Janelle Klein. Janelle is an Austin, Texas-based NFJS tour speaker and technical mentor. She’s the founder of the Software Mastery Circle [which is now branded as “Open Mastery” - eds.], which is dedicated to data driven software mastery and aligning the interests of business and software engineering. Janelle is the CTO of the recruitment consultancy, New Iron, that specializes in building software teams.
Janelle is also the author of the Leanpub book, Idea Flow: How to Measure the PAIN in Software Development. The book is about the hugely important topic of technical risk management and presents, in Janelle’s words, “A modern strategy for systemically optimizing software productivity with a data driven feedback loop by measuring the pain, or friction, in developer experience. People can identify the biggest problems, understand the causes, and run experiments to systematically learn what works.”
In this interview, we’re going to talk about Janelle’s professional interests, her book, her experiences using Leanpub at the end of the interview - and ways we can improve Leanpub for her and other authors. So thank you, Janelle, for being on a Leanpub Podcast.
Janelle: Thank you for having me.
Len: I usually like to start these interviews by asking people for their origin story, and I was wondering if you could tell us a little bit about how you first became interested in Agile and software development.
Janelle: Sure. I’ll start from college - well, slightly before college. When I graduated high school, I ran off and married my high school sweetheart. Not the best decision I ever made. But that got me running off to California to be a professional song writer and following my dreams.
And then I got into college, and I started to realize what a career in music might actually be like. And it just sort of sucked all the passion out of me, thinking about having to write to make money. And I knew absolutely nothing about software development. We had a computer growing up that I played lots of games on. So I’ve always been a huge, avid gamer.
And my ex-husband at the time - he was in the military doing network management, computer-y stuff, kind of a hardware geek - he’s like, “Let’s take this assembly programming class. That’ll be so cool. Assembly’s like awesome. It’s all this low level cool shit, right?” And so I was like, “Okay. That sounds like fun.” So we went and took this assembly programming class, and I was like, “Programming my calculator in math class, I can figure this out.”
I got basic instructions, and I started writing reams and reams of assembly code. I wrote the game, Breakout, with the paddles, and all that, completely in Assembly, with the full 256 colors and little music and beeps when the thing hit the wall - just completely in Assembly. And my teacher was like, “Well, at this point, just show me what you’re working on. You can work on whatever you want, and you get an A.” And I’m like, “That’s cool. I should take more classes like this.”
But that moment really changed things for me, because it was this moment of unlimited exploration - where I can go and create anything I could dream of. It was this ultimate kind of artistic medium, and that’s when I fell in love with software development. I’ve had one experience after another, that’s been this giant open-ended problem of following whatever dreams that you might have, and turning ideas into awesome tools. As I’ve learned how to do that with other people, it’s been one of the coolest experiences imaginable. And really, software development is very much a love of my life.
Len: That’s a fantastic story. When I’m interviewing Leanpub authors, many of them are in software development, and it’s curious - I think at this point, it’s now less than half of the people I’ve interviewed who end up in software, took something like computer science in university. People come into software development from so many different directions. I was wondering how you ended up at New Iron doing consulting work?
Janelle: So I ended at this kind of strange company, New Iron, doing - I mentioned it’s a software niche recruiting company, or you mentioned that in my intro - I started working for New Iron just as a contractor on this project at a semiconductor company. I got really involved in semiconductor, and lean manufacturing and supply team optimization and process control. And doing back end, high volume data automation stuff, and problem-space wise, it was really cool.
But as I started to learn more about lean - because the company was all about everything lean - I started to get involved in their lean consulting program, and teaching lean practices around the company, but in the context of software development. After figuring out how to solve the problems on my own project, and just getting better at software development, and how to figure out what your problems are, and turn around a failing ship, I got really good at helping other people to solve those problems.
And so it was never really that complicated in consulting. The main thing I discovered is that the job of a consultant is essentially to come in and identify the elephants and point to them. Usually, if you just listen to what people are saying, they’re already talking about what all their problems are. A lot of the time, I’ll just echo the same things that the team is already saying, and just explain it to management - but with a nice Keynote and explaining things in management-speak.
And suddenly all these communication problems where engineers couldn’t talk to management became my niche. It was figuring out how to communicate all this pain with metaphors and stuff, so that I can bridge that gap. A lot of the failures of projects are caused by these problems, with this inability to communicate. And so, with consulting, it was kind of a natural niche for helping out with that.
And then with New Iron, since we specialize in technical assessment and developer mentorship, I’ve got to focus on teaching people, doing a bunch of stuff with community, mastering the art of technical assessment, and teaching people how to build more effective teams. I mean, it all kind of went together with consulting.
In school I almost got my PhD, but I decided not to, because I was really eager to get into industry, and I really enjoyed like all the HCI kind of stuff. I always had this love of science and research. And I thought about going back to get my PhD, and then realized, “You know, I don’t need a need a degree to be a scientist, right? If I want to go and do research and learn about problems, I can just set up my own little research lab and learn about problems.”
And so I started treating developers kind of like guinea pigs, and set up a little lab in my work environment, and then started codifying the things I was learning about how to teach the art of software development into patterns and principles using these tools. A lot of that learning created the basis for Idea Flow. So I didn’t scare everybody away. I decided to write my book as kind of a first person narrative story of my own experiences, so it didn’t read like a dissertation.
Len: I’m really looking forward to actually talking to you about your book. It’s really interesting. I just have one question I’d like to ask you before we get to that though, which is about - there’s a line, I think, on New Iron’s website that says, “Being a good developer doesn’t make you a good interviewer.” This is something I’ve been reading more and more about in the tech press lately. It’s become a little bit of a meme, I guess, especially in relationship to people with various kinds of disability. I was wondering how you would recommend people hiring software developers to approach giving interviews?
Janelle: I think the main piece of advice I would give developers, is to focus on testing decision-making abilities - rather than current, active skills. The profession of software development is all about being a professional learner. It’s knowing how to find and look up information - how to break down problems, and how to learn your way out of a broken situation.
And so what I eventually came to is, that we need a method of technical assessment that focuses on assessing a person’s capability to reason about problems and navigate trade-off decisions. As opposed to, “Do you know what TDD is?” or “Can you write a unit test?” Because at the end of the day, it’s not about what you can do right now. It’s about what you can learn when you’re on the job. Because you’re never going to be faced with the same problem and the same challenges.
I mean, this industry changes so fast. We’re professional learners. We’re professional problem solvers. And that’s what makes people good, is their craftsmanship skill, that comes from years of intuition based around problem-solving, is really what skill is. And that’s what we need to be assessing for. So the main thing I do is put real problems in front of people. And then I ask them lots of questions to poke into their reasoning process, as opposed to, “Can you get from A to B?” It’s about, “What kind of options are you exploring?”
Len: Do you recommend doing things like asking riddles to see how people respond to unfamiliar questions?
Janelle: I’m not good with riddles, so it’s hard for me. Yeah, probably not answering riddles. I would hate to be given a riddle in a - I guess the other thing I’d say, is that when people get nervous, their ability to make intuitive leaps to solutions generally shuts down a bit. But their recognition skills don’t. If you put something in front of somebody, and they don’t recognize that there’s a code smell of sorts, or some problem with things, then that’s usually a sign that something is actually wrong or missing.
Whereas, just because somebody can’t come up with a right answer - it’s just the world is not that black and white. And I know technical assessment is really hard. For example, with respect to certifications, we’ve been working on a way to systematize technical assessment, so then I could teach other people how to do my interview technique.
It’s taken us a decade to figure out how to do that. But a lot of people punt on the whole certification problem, because, “It can’t be done, it’s too hard.” There’s so many problems with commercialized certification in our industry, that everybody is like, “Oh well, we shouldn’t even try, because it’s just going to cause so many problems.”
I think we need to move toward more of a model, where it’s open certification. Where we’re testing decision-making skills, and it’s an industry public debate about working on getting better at this, as opposed to just giving up on the problem, because it can’t be done. And just make technical assessment a free standard, almost, that we work on together as a community. Because we all benefit from having a clear definition of what good is. I mean, we need that.
Len: Yeah, definitely. In that context, I’m curious what you think about nervousness in interviews and things like that? Because the interview is a situation that the person will never find themselves in again - more or less, if you hire them. They’re going to then be working. And so people can be nervous in interviews, but not nervous when they’re not doing their work.
Janelle: Sure.
Len: I was once interviewing someone who’d just finished a masters in maths at Cambridge, and failed to do basic arithmetic in response to a question. So I knew what the interviewee was capable in normal circumstances, but not in the stressful circumstance of an interview. And so even the best certification or qualification - or something like that - that someone comes into the room with on paper, can sometimes not translate in an interview, but can in work. I was wondering if you advocate giving people problems to take home and solve or something like that as part of the recruitment process?
Janelle: I don’t really do take-home problem solving kinds of stuff. I mean, I certainly see the benefits. But the main problem I have the take-home thing is that it takes too long, it’s too much work. In a market where everybody’s hiring, and there aren’t enough good people, the last thing I want to do is go an create more barriers to entry to hiring the right people.
And so we’ve basically focused on, how can you get the maximum amount of information to make a quality decision in a limited amount of time? Because the take-home thing has to be scheduled around life, it’s much easier to get people to show up for a scheduled one hour call. And then I basically do that with video sharing and stuff, so I can throw code up on the screen and ask people about it. We’re all about optimizing the time spent in that.
And then the other things I do, for the onsite, are a code pairing interview, and then something that’s more design-focused. So I tend to focus on core skills, as opposed to, “Do you know whatever trivia?” Like whether you know ordering of some… I don’t know? There are some things that are just - Computer Science-related kinds of questions, that are so far removed from what we actually use on a day-to-day job - I’m much more interested in people’s everyday problem solving skills. Can you refactor this code? Can you work with the people on your team? Are you an asshole? Those kind of things.
Len: That’s a really great and very practical answer. I don’t know how much of your work actually is recruiting people in Austin, but I can imagine the competition for good developers there must be really high.
Janelle: Yeah. We pretty much only work in Austin. We have some stuff going on in other states from clients that have multiple sites in most locations, but we’ve always been very central here. And then since our company is run by software folks - it’s kind of interesting how we ended up in software development.
New Iron used to be a product company that built a first generation web services stack. We’re out in California, and I thought, “Oh, we’re going to build this awesome product, and everyone will buy it because it’s so awesome.” Obviously that wasn’t a very good business strategy. And so we’re sitting around going broke, basically, and we got a call from an old client that we used to work with, and they’re like, “Hey, can you guys help us on this consulting project on the software?” And we’re like, “Money, no money.” And so, easy choice, right?
And so as we started consulting there, they’re like, “Do you guys know anybody else that can code?” And since we already had a vendor agreement set up, we just started talking and recruiting all our friends. And we did full-on interviews for everybody, not having any idea how the recruiting industry was supposed to work - that nobody does this kind of thing.
And so we started getting really good at technical assessment. And that sort of became our niche. Because it’s hiring people, and figuring out what skills you need, versus what’s available on the market - and having unrealistic expectations about hiring Superman, and that you can instantly take all these things that you’re deeply familiar with and say, “Oh, now you do this job.” There’s so many companies that have so many unrealistic expectations about it. Because hiring isn’t a first priority in a lot of companies, so people don’t necessarily put the time into it that it needs and deserves.
Yet it’s so important and causes so many problems when you hire the wrong people for the wrong things. And so we got a lot more involved in understanding what our clients actually are trying to accomplish as company, as a team, and help them to come up with a realistic strategy that they can actually find a gap they can fill, as well as the gap that they can sell. Beause in this kind of market, if you have a job for, “Oh, this is a Java development position like so many other Java development positions.”
Len: One thing one reads about in the press around recruitment is the way that employers or recruiters will sometimes look into a person’s internet history. And I was curious if that’s something that you recommend employers do?
Janelle: Internet history in terms of, like, what they post online?
Len: Yeah.
Janelle: It’s not something we do in that, generally speaking, we focus on whether the developer has the skills, is able to demonstrate the skills, as opposed to any kind of past history kind of thing. I think since we have a really good process for technical assessment itself, it’s like everything else is almost irrelevant. In that, if you’ve got one year of experience, but you can bring it when it comes down to sitting in front of a computer and solving a problem, you know what? I don’t really care if you’ve only been programming for a year. I’ll just think you’re awesome.
Len: That’s a really fantastic answer, I’m glad to hear that, that sounds very reasonable.
Moving on to the subject of your book - which, by the way, I really liked - you start with a pretty dramatic story about your experience working on a statistical process control system for a semiconductor factory, I think. And you mentioned the elephant in the room before. In that particular experience there was, I guess, a sort of invisible elephant there. I was wondering if you could talk a little bit about that experience and its impact on your approach to complex software engineering projects?
Janelle: Wow. Okay, that’s a big question.
So in terms of the experience, I was on this project with this really great team - really smart people. We had test automation, CI, great team, disciplined practices. And despite doing all the things that we’re supposed to do, our project basically crashed and burned and hit a wall. And by hit a wall, I mean we brought down production three times in a row, and then didn’t ship again for another year.
And the third time this happened - this is a 24/7 operation, so downtime is a really big deal. And the last release, the third time we brought down production. Not only did production go down, the rollback failed. And then the feature toggle that disabled the broken stuff failed. And so we were up at like three in the morning hacking out a patch fix to, just so we could get the software out of production while everything was down. It was the most miserable experience you can imagine.
And this was shortly after I’d been out of college. This was my second job out of college. I was kind of a little hotshot, and had my head in the clouds, and thought I was awesome and that I was pretty much infallible. And I had this great idea that would improve performance to basically flip the architecture inside out and do this massive set of changes. And, of course, I put that all in production all at once.
It’s just one of those huge lessons in your career that you never forget. And that really fundamentally changed the trajectory of my life. Especially since all these things, I was like a super TDD zealot-type at the time. And so I was convinced that our success was all about whether we were following the right practices, and whether we were doing disciplined TDD in our project. That was the thing that mattered.
And when I was doing all this great stuff, and I failed anyway. It was the most foundation-shattering experience that I could’ve ever imagined. And so we got together as a team, and we’re like, “What the hell are we going to do?” Because we’ve been doing everything right, but we’re failing anyway. And so we built this tool that could detect high risk changes in the code, because we thought the problem was technical debt building up over time. Same problems we always talk about are causing the pain.
We thought we could use this tool to let us know where we needed to do extra testing, by identifying the areas where there are changes by tech debt code. But what we found was that most of the mistakes were actually in the most well-written parts of the system. And so we started digging around in the data, trying to make sense of what we found. But there weren’t any answers that made sense and correlated with what we knew about software development, and what we’re supposed to be optimizing for.
The correlation we did find in the data was that a lower familiarity with the code tended to increase the likelihood of mistakes. And while that made some sense, it seemed like there had more to the story than that. Because when we had to work with complex code, it was really painful. And so we started down this path of trying to understand, “Well, what it is that makes development feel painful?” And so as we are working, we started collecting all this data on where we were experiencing pain, and then talking about the specific things that were causing the pain, and keeping measurements about how long all these various things took, and just writing up the things that seemed like our biggest problems on a whiteboard.
It wasn’t until we started down this road of collecting data, that we realized that we’d been essentially solving all the wrong problems. We had all this test automation, but the test didn’t catch our bugs. And we had well modularized code, but it was still really difficult to troubleshoot all the defects. And then we started looking at the specific things that were causing our pain, and learning with the data-driven feedback loop, we realized that most of the problems were caused by human factors that were, how a human interacts with the code while they write.
And so that’s where this whole technique of measuring development experience came from as a means of feedback. Because it was the only thing that seemed to really matter at the end of the day was whether it was easier or harder to do our jobs. And if we measured that, data became this unifying force on our team where it really brought us together in learning how to learn together.
And at the time I had this experience, I couldn’t put what I’d learned into words. But I knew that I wanted to teach it to other people. And so after I read Peter Senge’s book, The Fifth Discipline, about how to build a learning organization, I was so inspired by these ideas that I was like, “Alright, I need to figure out how to write this down.” Because this is the essence of what Peter Senge is talking about in his book – these five disciplines of a learning organization, and learning how to learn together.
Essentially, I found this really tight correlation between science and learning organizations. And then when we start using this scientific method-ish process to learn and examine the data and have a shared direction of better, that we’re all reaching towards, that suddenly it unifies a team in a really cool way. And I’ve been fortunate to have some really incredible experiences working with just amazing engineers, where we learned how to learn together as a team.
And so it wasn’t about what practices we were using. It was about whether we could identify the right problems to solve. And once I learned how to identify the problems, the rest of it was easy. I got a team of professional problem solvers. So that turned into the skill that I wanted to figure out how to teach. Even though it was hard - even though we didn’t have words in our vocabulary to describe the things we’d learned - I was determined to figure out how to do that, and to teach these things to other people.
Len: And when you talk about the data gathering, I know from reading your book that there’s very specific things around that. I was wondering if you could explain a little bit about what the data is, and how the gathering takes place.
Janelle: Sure. So I’m measuring, specifically, troubleshooting, learning, and rework. And we have tools that integrate with the developer’s IDE that are partially automated, partially manual. From a time-tracking perspective, it’s a lot different than managers wanting us to track our time in JIRA, because there are tools created by developers for developers to use to understand what their problems are.
So we’ve always kept management out of the data. It’s like, “We’re doing this for us, go away.” We’ve come up with better ways to automate things over time, but essentially - let’s say, when we’re writing code, we go through this cycle of, you write a little code and work out the kinks over and over again. And when you get to that point after you write a little code and you’re validating things, you’ve kind of got a prediction in mind of what you’re expecting the code to do. And at that point, there’s a decision point in our brain. If the code does what we expect it to do, we write a little bit more code. And if the code doesn’t do what we expect, we go into this troubleshooting mode.
And what I found is that, moving forward, modifying more code, learning - basically figuring out what you’re going to do, can take a whole lot of time, especially now that 90% of our software’s built from existing parts. Or, if you’re unfamiliar with the code, learning is a substantial part of our work that gets in our way. And on the troubleshooting side, we’ve got to troubleshoot the problem and then do rework to put things back in place before we can move forward again.
And so this flow of ideas between the developer and the software, and the friction in that flow - I’m measuring in terms of troubleshooting, learning, and rework, and then actual duration of those times. Then, we have a general rule on the team where anything that takes longer than 20 minutes, we talk about, and try and understand what the causes are that made this incident so painful. And it’s always a kind of gut-feel type factor. In some one case, 20 minutes is not much time. In another case it’s way more time than it should be. And so you kind of just talk about the things that are worth talking about. Don’t talk about - it’s just kind of noise.
Len: I find the importance that you place on pain to be really fascinating. I have a sort of specific question I’d like to ask about that. But first I’d like know if you could maybe define a little bit about what you mean when you talk about pain?
Janelle: So in this context, I’m relating pain to friction in idea flow. The reason why I use pain specifically is for a couple of reasons. One is that I want people to focus on the symptoms, as opposed to the cause. Because one of the problems we having our industry is automatically assuming what the cause is of the pain - like it’s this ugly code I have here, as opposed to focusing on the symptoms, and our experience, in working backwards.
And so, I use the word pain specifically to get people thinking about, “Well, what are the symptoms?” And then let’s have a richer conversation about what are the potential causes. Because usually most problems have multiple factors. And I think that’s one of the places that developers tend to get stuck on best practices, as it’s very solution-focused.
The other thing is that intuitively - just from experience - pain is associated with this idea of something that we want to avoid, and figure out how to have less of or reduce it. And from an optimizing, idea flow standpoint - that’s kind of the message I want to relay. There’s this thing that is a threshold thing that we want to keep in check to a level that we can deal with it. It’s not about eliminating pain. It’s about figuring out how to optimize and manage the pain so it’s at a tolerable level, and stays that way.
Friction - even though I’m associating it with pain - it often doesn’t totally correlate with what developers are using that word for now. For example, one of the perceptions that’s usually really far off in software development, is just our notion of what takes a long time. And one of the things that happens is when we’re busy working on a problem and doing stuff, time seems to zoom by really quickly. Whereas when we’re waiting on something to happen, like waiting for something to execute, it feels really painful, because we have to wait all that time for that thing to get done. And time seems to go really slowly.
And so when you start looking at things in terms of elapsed time, we feel like that thing that took five seconds was like an eternity, and that that was really painful. But then we’ve got all this time that we spent troubleshooting, that we don’t necessarily associate with pain - even though it takes an order of magnitude more time, in terms of human cycles. But because we’re busy doing stuff, and problem solving is fun even - doing things that are fun, we seldom associate them with being painful, right? And so I was trying to create this mapping visually, to sort of remap our pain sensors to something that is explicitly defined, that we could all share this universal definition of, What is pain? Being things that slow down the process of idea flow, even though it takes some remapping of our pain sensors.
Len: It’s really interesting to me. One of the reasons why I liked your book so much, was that you talk about addressing pain as a problem, and you were saying just now that pain is something we want to avoid. One of the reasons I find it so compelling is that, that’s so often not true in management practices.
My brother told me a great story where I had a kind of apple-falling-on-Newton’s-head moment. When he walked into a big box store, there were two employees - one of an earlier generation, and one of a later generation. They were handing out flyers or something, wearing little vests. The younger person started walking off, and the older person said, “Where are you going?” And the younger person said, “To the bathroom.” And the older person said, “Make it snappy!” What occurred to me in hearing that story was that pain is something that people often use in managing others - in a sense, positively, right?
Janelle: Interesting.
Len: They use pain as a tool to get the work done. I was thinking about what I’ve read about slavery, and the way there’d be an overseer watching people work, and cracking the whip, and making sure that they stayed active. And I just saw this line from that all the way to this box store, and these two people in vests, and in the way observation, and the infliction of pain is so baked in to various kinds of management practices. We don’t even kind of notice it, it’s so obvious.
I was interviewing someone recently, who said, in his first software job, he was a tester for the Australian government. [Editor’s note: this was actually at a call center, not as a tester for the Australian government. Sorry, Australia!] And whenever he left his station, he had to log why, including going to the bathroom and for how long - the humiliation built in to that.
Anyway, just to finish, the thing I find so interesting about managing software - and especially in a world where software is eating the world, and everything is being driven by software, it’s become this really important form of labor, but it’s invisible.
For example, imagine you’re in that long ancient tradition of the manager who watches, right? It doesn’t matter if you’re watching people sowing seeds or reaping the wheat, or working on a factory floor in an assembly line. You can develop all these practices around watching people. But with programmers, they’re just sitting there, right?
You can imagine being from that tradition, and then you get hired to manage programmers and you’re like, “Well, what’s there for me to do?”
What you’ve done in your book and your ideas is that you’ve like completely flipped it around, and abandoned this form of management - that’s so universal and so inappropriate for a world where software is important. People have attempted in so many ways to try and do metrics and things like that around the wrong things. And what you do, in my opinion anyway is, that you’re looking at exactly the right the thing, which is taking into account like the pain that people feel when they’re doing types of work, and then really thinking that through. So, anyway, I just wanted to say, I think that what you’re doing is actually quite revolutionary.
Janelle: That’s an interesting story and take on the management side of this. One of the other big target audiences I had in mind when writing my book was managers, and helping them to get a feel for what it’s really like to be a developer in this world. I don’t think - if you listen to the way managers talk and the way developers talk, it’s almost like we’re speaking two different languages. Developers use words like “beauty” to describe the code and “elegance” and “firefighting” and “explosions”. And all of our metaphors are in terms of art and war. And if you listen, it’s all about money and profit and bonuses and ROI, and all this money-related stuff.
And I think what we have to realize is that as software organizations, we’re basically building a business that sells profitable artwork. You need to think about the economy as a giant art contest. With software development, it’s very personal in that we very much invest our hearts and souls as software developers into our work. And then seeing your creation get stomped on by all this organizational dysfunction is an emotionally damaging experience.
And so there’s all this cynicism and struggle, and a feeling of helplessness that turns into contempt within our organizations. And we get these divided cultures, where you’ve got a management culture that’s all trying to raise morale through celebration. And there’s all this fake happiness, for a lack of a better word. And then defensiveness around not wanting to shatter the perfect illusion bubble. And so management is often really resistant to talk about problems and pain and things that are going on. Because then it seems like an reflection on them if they’ve got little red marks on their status updates, whereas–
Len: I can’t stop thinking right now, when you’re talking like this, there’s this infamous photo of the Yahoo executive team dressed up in Wizard of Oz outfits, that was being presented to the employees as this sort of like motivational thing. Like the leaders are all on board and sort of superstars, and then the total disconnect in the use of that image, and what’s really going on there.
Janelle: Yeah. I see so many managers that are really concerned about talking about the problems, and that they believe that it will bring down morale if they talk about the problems. But in fact it’s quite the opposite that usually occurs. Especially when you have data to have a conversation about, putting the pain on center stage, as opposed to fighting each other, and blaming other people for the problems - you have a beast to conquer. And then when you go and conquer the beast, then it builds camaraderie within your company, and it brings people together around seeking truth, as opposed to this culture of imagining the world that we wish we had and trying to will that into existence through celebration. Or the after work drinks and bitching about our jobs, and how nothing’ll ever change and how everything’s hopeless, which starts to breed this undercurrent of contempt in our organizations and, emotionally-speaking, people are very much going to war.
I see a lot of these problems in organizations across the industry, where our software problems create cultural problems, and have a huge effect on the lives of human beings. I’m hoping that these ideas catch on from a standpoint of a strategy where we can bring the art side of the world together with the money side of the world, and realize that we can have both.
Granted, it takes a lot of work to figure out how to get there and how to learn together to optimize the whole. But at the end of the day, you build something beautiful. And software organizations become about working together to bring something awesome into the world. And then when you’re competing in the world’s art contest, you make money because you built something awesome. At the end of the day, I think those are the kind of things that human beings really care about - it’s having the opportunity to do something that matters, and to experience that with other people.
Len: It’s sort of central to your vision, your insight, into what people are actually doing when they’re engaged in software development. It flows directly from your basic premise - that things like pain, and things like the human element matter because it’s a human activity. It’s so interesting that we live in a professional world, where talking about the human element immediately requires a little bit of defensiveness or something like that, right? Because that’s supposed to be of secondary importance, and just a problem to avoid.
But your idea flow metaphor puts the human activity - it’s sort of like ontologically at the heart of what is really happening, because, as I understand it, the idea is that - well, I’ll quote you here: “That interaction with the code as a stream of ideas, flowing between me and the software, is a correct description of what’s actually happening in coding. That I have an idea that I try to express it code. And then that code is then used to carry out various things. And you can see whether my idea was translated properly by me into the code, and then properly read by the machine reading the code. And then the next step is when someone else is reading my code, have I correctly flowed my idea from its origin within me into them?”
I just find that so compelling because that actually is what’s happening, that we need take into account. I mean, one can say the human side of things, but just that it’s consciousness that is engaging with the machine through code. And to try take and consciousness out of it, is to completely misunderstand what this process really is. And that’s why it flows naturally that concepts like pain actually become scientific metrics to measure, because that is actually what you’re looking at - is a process of consciousness engaging with something.
Janelle: Yeah. One of the interesting extensions - I mentioned early on, using mentorship, and creating a developer lab to test these ideas - one of the ways I’ve been looking at idea flow as mentorship being, I have an idea in my head about how to optimize productivity, say. And how do I teach that idea to somebody else? That’s another type of idea flow, of sharing that idea. And the thing that I learned from that is that the only real difference between communicating something and teaching something is whether we take responsibility for whether that other person actually understood the idea or not.
And so, likewise, I came up with a method of testing whether a person understood by testing their ability to think through and make trade-off decisions in a variety of staged scenarios. That led to my technical assessment technique, too. This whole underlying concept of “What does it mean for an idea to get from my head into somebody else’s head, and how do we come up with an explicit, sort of scientific, method for testing whether that has actually occurred or not?” - has given me a feedback loop for learning how to become a better mentor as well.
Len: On that note, you talk about communication, which becomes very precisely defined in this kind of circumstance. You talk about the importance of metaphors. I was wondering if you could talk a little bit about why you think metaphors, and good metaphors, are so important for this process.
Janelle: So, I had a couple key influences with that. One of them is a book I mentioned in Idea Flow called Metaphors We Live By, which is a theory on human understanding as a fabric of metaphor, and that essentially everything we understand is in terms of other things we already understand, and that we build up understanding through relationships of metaphors. And so one metaphor, how it relates to another thing through metaphor, and then how those things are similar and different.
The book goes through a whole bunch of a examples in our languages, and comparing cultures, and how when you have a different foundational metaphor, you get a fundamentally different understanding at the cultural level. All of this cultural difference analysis - it’s a fascinating book, and it fundamentally changed the way I started looking at learning and improvement and communication, in that I started to think of my brain like a shape toy, and that I’ve got all these shapes that I know in the world. When I see a square, I sort of shove that into my shape toy brain, if you can imagine that. And in order to be able to recognize something else from my experience, I basically need to define a new shape. And so one of the feedback loops I discovered was that by explicitly defining new vocabulary - like looking at my experiences and then saying, “Oh, this is this pattern” and making up all these new pattern names for types of mistakes or for different strategies or problems in the code or that make troubleshooting take a long time or whatever it is - I basically built a massive taxonomy of patterns.
But in the process of doing this, I realized that I started to recognize things in my experience that I never noticed before, and that there was this feedback loop of, the more patterns I could develop in my vocabulary, the more detailed my observations were from my experience, and the faster I ended up learning. It was like I had amplified what I could learn from each experience, as well as giving myself all these awesome tools and connections for synthesizing information I’d learned.
And there was one metaphor that had a huge impact on my thinking. So Ash Maurya - he’s got a book coming out really soon called Scaling Lean - I went to a workshop of his like three years ago, when he first started working on this book, and he introduced this metaphor he calls “the customer factory”. So Ash focuses on lean startup practitioner-related material.
Essentially what he’d done is codify a method for practicing everyday science, and then mapped lean manufacturing to kind of a customer experience model. So if you imagine your business is like an amusement park, and you’ve got people lining up to the entrance of the amusement park, when they want to use your product and then, their first experience of taking the rides - that kind of thing. And then he came up with this way to measure and model the flow of different experiences through the business, and measure that and optimize what he calls “happy customer flow”.
And what was so fascinating about this, is it was like an experience business - sort of an experience-based supply chain model. And since all the stuff I started doing was in developer experience, as I started looking at the problems, I figured out how to build a supply chain model, mapping the same kind of metaphor idea to software development. But rather than the process flow that we normally think of as the supply chain, I was using the software dependency supply chain with experience-based data on top of it - how you experienced changing a particular software component, for example, and modelling across software dependencies.
And once I figured out how to do that, and optimize idea flow across the idea flow factory as it were - since my background is in process control and supply chain optimization from the manufacturing world - suddenly it was like everything I had learned from lean manufacturing, like clicked together with everything from Lean Startup. That clicked together with everything I’d been doing with idea flow, snd it was like this massive synergy of patterns that suddenly went together, and they were all linked together through metaphor.
Whenever I discover a new metaphorical link, it’s like all of these new discoveries and ways that you can make associations just fundamentally shifts. And I think the major revolutions in our industry always come with some kind of metaphorical shift.
Len: That’s a really fascinating way of putting it. I mean, you used some great metaphors in there as well - like the brain as shape toy interacting with the world. And customer experience as being like an amusement park, and that’s just amazing. I think you sort of proved your point by using those metaphors so well. Because it’s sort of like, when you successfully use a metaphor, it’s like a shortcut to communicating not just a set of facts, but a conceptual structure around viewing facts - and understanding what you’re actually looking at in a practical way.
In your book, you also talk about addressing the root cause of project failure in the software industry, and this is something I think is one of the most important things facing industry, I guess, generally today - is that as everything becomes software-driven, understanding the issues in software development becomes more important. In particular, we all see in the popular press these catastrophic software failures.
One of the ones that first come to mind for most people would probably be - or at least in the West - would be the Obamacare roll-out and how that failed. Up here in Canada, we’ve had our share of these catastrophic failures as well. I was wondering, in the context of your concept of idea flow and your book, if you could maybe explain what you think the root cause of project failure is in the software industry, and then by extension all of industry because it’s driven by software?
Janelle: I think there’s a few different core problems, but I’d say the one that’s probably more of the root would be our inability to share knowledge as an industry. We’re sharing this solution-centric knowledge with one another, and what I found is it’s not enough. The things that we really need to know in our industry are largely tacit knowledge, and they’re largely learned through mentorship.
As the industry is growing, and more people are getting into the industry, they’re not getting the kind of education they need in school to know how to do these things. We end up with more and more projects with people that don’t have the skills that they need to do their jobs, and don’t know how to communicate these things to management. And management doesn’t know how to understand what’s happening on the software project to make it be successful.
And the combination of what I’m referring to is the “wall of ignorance” between the management world and the engineering world - and how we’re basically trying to explain and describe our problems being completely understood, being one major factor. The other one is not being able to share our knowledge as an industry, or have any kind of universal definition of “better” that we’re aiming for. So it’s like we talk about brevity and modularity and test coverage, but all of these things are really kind of a means to an end, and we don’t have a way to describe what better really means, as the center point in that conversation.
I think that’s one of the big things that idea flow brings to the table. It’s that it gives us a way to share our knowledge on how to do this stuff. Because these projects that fail - it’s just sad. Especially when the taxpayers get stuck with the bill, of just mismanaged projects from people that are running things that don’t know what they’re doing, when that knowledge of how to run a successful software project, is very much knowledge that is within our industry.
If we could do a better job at, one, sharing our knowledge, and then even taking responsibility for public service based projects, and potentially running these things as open managed projects by the industry - as opposed to having the government having no idea how to build software and then running these broken software projects. It’s a tax burden and stuff that is completely unnecessary, because we can solve these problems. It’s not - I should say it’s not that software isn’t difficult, it’s that sometimes the failures are so unnecessary.
Len: It certainly looks that way from the consumer end of things.
Janelle: It depresses me a little bit that we end up wasting so much of our money on things that - for example, if we ran healthcare.gov as an open source project, I think things would’ve turned out very differently. If you basically had one, we could’ve had the thing happen for free, basically based on voluntary support. And because it was something that’s important to the people - I’ll bet you, people do a good job at managing that project. Just because a lot of people care. And we can build societal support based on just humans caring about each other, and helping each other to build the stuff that we need.
Len: It seems to me that - well, with government in particular, there’s the challenge that being skilled at getting government contracts, and being skilled at building software, are two very different, and perhaps, incompatible things.
You talk about the “wall of ignorance” between sort of people doing software development and managers. I think that from conventional management perspectives, there are so many things about software development that are just new and counter-intuitive.
For example, if you’re a typical manager - and I’m going to just stereotype negatively here - you want to have more people underneath you. “I’ve got 10 people underneath me.” “Well, I’ve got 100, you loser,” would be the talk over the bar, right? But in software, it could be that having more people is worse, and actually lowers productivity, and lowers the quality of the work that you’re doing. And you can’t just have more people doing the planting or working the machines or something like that.
For example, a person just sitting there thinking, might be way more productive than a person typing or doing something active. In a very straightforward way, less is more, right? You want the smallest code base you can get, not the biggest. A manager might brag, “Well, my project has eight million lines of code.” And you’re like, “Well mine only has 10,000.” But it could be that what you’ve achieved with those 10,000 lines of code is superior to what the other group achieved with 8 million lines of code.
And so I think that a lot of the ways people measure success are out of alignment with the ways we ought to be measuring success, and that that “wall of ignorance” is partly there. You say ROI and money are the terms that managers are living with. And then the developers are living with - say beauty and efficiency, or something like that, and it just seems to me that one of the reasons those big, especially government, projects fail, is that there’s this concept that management and work are these fundamentally different things, and that’s there from the start. So the sort of the whole thing needs to be thrown out, in order for it to be improved.
Janelle: Yeah. I–
Len: You don’t need to agree with anything I said, by the way.
Janelle: I was just thinking in terms of management. I haven’t really run into any managers that were concerned about trying to measure lines of code as a measure of progress. I think we’ve moved past that era. What I see happening more is, the first problem you brought up definitely, of not realizing the impact of adding people to the project, or assuming that you can replace one developer with another developer. And not realizing the impact and the learning curve, in the effects of familiarity on a team.
Part of the problem I think is that - back to the metaphor of technical debt, that has us kind of thinking about the problems in terms of an interest rate, or like a predictable stream of interest payments over time. And when we think about interest, we think about, “Oh, we’re paying 10% interest, 20% interest”, and it’s this steady, predictable thing. It creates the illusion that we can throw more people at the problem, and distribute the cost across more people - when in actuality what’s going isn’t a problem with a predictable increase in cost, it’s a problem with chaos and a loss of control. And that metaphor is failing to explain the true nature of the problem that we’re trying to solve.
I realized this when I was in a business coaching session with Keith Cunningham, and I was talking about technical debt. And none of these people in the class knew anything about software. But I was trying to explain it in terms of these metaphors, and how technical debt was such a bad thing and what it was like. And the response was, “Well, that doesn’t seem so bad.” And I’m like, “What? Don’t you even want to know like the interest rate or something?”
And what I learned in that world, was that, the way these guys do math, and make decisions at the investment level, just seems so crazy, because it’s so far removed. But it’s like, “Okay, we’ve got revenue minus cost equals profit, and our goal is to raise profit by 10%. Well, how are we going to do that?” Well, you have to basically come up with an investment strategy of how you’re going invest money in all these different buckets to make this prediction come true.
And what I realized in this class was that what makes investment decisions more difficult isn’t an increase in cost, it’s a reduction in predictability. And that chaos and the loss of predictability is much scarier than an increase in cost. And when I started talking to management, I started framing everything in terms of a risk-based decision. As in, “the deadline will be here either way.” And the decision is about where we want to be when that deadline gets here. It’s not about - as soon as you start negotiating things in terms of time, you get back to that metaphor of “How can we throw more people at the problem to make it go faster?”
And then the other thing I still see is people thinking about developers like a commodity resource. And what’s fascinating when you look at the effects of a loss of familiarity on the team, I have one case study in the book I did with this team that was an awesome team. But basically it was like post-exodus phase, where all the original developers left, had this huge breakdown. And then they got new management which brought in some great people. But they had this code base that nobody understood.
And the lack of familiarity cost - 80% to 90% of their time was spent figuring out what to do. And the cost when familiarity walks out the door is insane. I mean a lot of rewriting your software is just to reestablish familiarity, because we can’t work when we can’t relearn the system. And you just get stuck in that state of 80%, 90% friction. And I don’t think - we’re talking about interest rates - you don’t think about stuff that’s taking up 80% to 90% of your capacity. It’s unreal when you start looking at how much time we waste for all these problems.
Len: It’s interesting. You talk about technical debt, getting us to see as nouns what we should be seeing as painful verbs. I really like that, and I was wondering if you could explain - on the subject of technical debt, a little bit more about what you mean by that?
Janelle: Sure. This kind of goes back to the metaphor mapping thing. We’ve got different metaphors that we map in our brain. We’ve got objects, which are basically thing-patterns. We’ve got spatial relationships and context-type patterns. And then we’ve got process-patterns, or things that happen over time. And all these different metaphors have these three basic types.
One of the things I realized is that technical debt is a noun. And the effects of it being a noun mean that we look for things that are noun-like in our experience. And generally we look at, “Okay, so this part of the code is a noun - this technical debt thing. And this is a painful technical debt. This is a low pain or a high pain.” But it’s all one-dimensional. We bucket our experiences in terms of high-pain and low-pain technical debt. But idea flow is a process of understanding and extending the software. And so pain occurs in the context of a process.
And once we shift from a thing-pattern to a process-pattern, suddenly we start seeing our pain in terms of a journey. We’re in these situations. We make these decisions to navigate around obstacles, and have to dig through different areas of friction and break down constraints. And you start to see the problem solving experience, and the true complexity of what’s going on, because we can start relating to journey metaphors, or things that happen in time, as opposed to just one-dimensional things.
If you look at the patterns in my book, they’re patterns of mistakes and patterns of causes of troubleshooting time. They’re not like any patterns we have in our vocabulary at all. They’re patterns in development experience, that I didn’t start to discover until I’d shifted my foundational way. I started thinking about pain as a process, and what were all the factors along my problem-solving journey that ended up affecting my pain?
Len: I’ve got one last question about your book. I was wondering if you could say, for example, to someone who is listening is or has or will be managing a software project - how they can use the concept of idea flow to improve their work?
Janelle: So would this be for a developer or for like a manager?
Len: For a manager.
Janelle: Okay. One thing I’d say is that the techniques I’m proposing in my book need to be developer-owned, in that the development team needs to take responsibility for identifying their problems, understanding what they are, and figuring out how to fix those problems. Learning how to do that and discovering the right problems takes time. If we don’t allocate time to figure out what the right problems to solve are, then we’ll inadvertently end up solving the wrong problems whenever we try and work on improvement.
And if management can agree to let the development team work on the most important problems to solve - whatever those might be, and the development team agrees to gather data to identify what the biggest problems are, and then share the things that they’re learning with management - we just set aside capacity to do that, and that becomes the contract between management and development. Then you can start working your way to a better place, and then shifting that ownership of making those decisions and gathering requirements and solving those problems to engineering. And we can start steering our projects.
We talk about the importance of empowered teams. But it’s not just about self-organizing and figuring out what we ought to do, because we’re all too busy to do that. And so we just shortcut all the structural things, and do what we need to do right now - and worry about all the process stuff later. Because we’d much rather write code.
I think we need to have that structure. And it’s worth putting some time into designing our organizational structure - more like its architecture, and having that be a first class responsibility - of understanding the communication dynamics in our organizations. And start creating more human systems architectural roles, if you will.
I’ve also got some diagrams of modeling using software as a metaphor for human systems design, and some of the design problems in our organizational structure that cause pain. I think using that as a template, even if you do nothing else, and take absolutely nothing else from the book - that one change will make all the difference in the success of the project.
Len: Thanks very much for that. That’s a really great answer. I was just wondering what the next step is for you? You’ve got the book and you’ve got these great ideas that you want to spread, and I was wondering what your next steps are?
Janelle: Wow, I’ve got so many next steps. Right now, I’m doing a speaking tour with No Fluff Just Stuff. I’ve got a five talk lineup that I’m doing, breaking down these problems of how to do data driven software mastery. And then the organizational side, how to build a learning organization. And then my business, to get time to work on it - I’m a very firm believer in that you just get up and do it anyway. And you don’t grow without stepping outside your comfort zone. So I’m learning my way through it. I’ve had things that I wanted to share for a long time. And this book, too, took me five years to write, and rewriting things over and over again, and failing to communicate. Because all this stuff is really abstract. And I can iterate a lot faster on stage too, I’ve found. But it’s kind of a challenge.
But the thing I’m working on right now is building a community at Open Mastery around these ideas, which is an industry peer learning network focused on data-driven software mastery - essentially companies and individuals that are interested in trying out these ideas. I’m working on spinning up local communities, where we can start working on implementing these practices in our organizations iteratively, and then coming together in a local community group, and talk about lessons learned, and codifying what we learned into patterns and principles.
And then we’re building a new vocabulary of patterns on Wikipedia. And we’re hanging out and talking over Slack. And we’re writing blogs, … we have every day, to build a community around - but my goal is to lead an effort in learning how to learn together, with the pain on center stage. And there’s so many problems in our industry with broken feedback loops between the education system and our economic system, and between management and engineering.
If you start looking at a lot of the problems and pain in the world, it all kind of comes back to broken feedback loops. And so, what I’m trying to lead is an effort in learning how to learn together - as an organization, and as a community and as an industry. And hopefully, if I can get this going, as a species. So that’s kind of what I want to do with my life. And I’m hoping that I can find a group of people that are interested in learning along with me. Because it’s a whole lot of fun, but it’s a whole lot more fun to do it with other people.
Len: Thanks very much for taking some time in your life to do this interview. I hope it helps spread your ideas, and helps you achieve that. And I really hope you succeed, by the way.
I guess I’d like to ask you one question about Leanpub, and that’s, why you chose Leanpub for your book? Obviously, something that took five years to write, how you distribute it, and how you publish it, is a very important decision. I was wondering if you intend to keep it on Leanpub as permanently, or are you looking for a publishing contract?
Janelle: So your first question of why Leanpub - I mean, being in software development and Agile and lean being so central to my life, I live and breathe “iterate”. And having a platform that was really easy to get started with, that I couldn’t get in my own way with all the excuses I would make of not doing this is just like - I remember first reading the Leanpub documentation, and how writing your book in Markdown without all these formatting things was a feature.
And at first I was upset that I didn’t have all these fancy tools. And then I started to realize, “Oh, I get what this means now,” of how much I get in my own way from writing my book. Because I’m sitting there fussing with what things will look like, as opposed to just writing the material. And I came to really like the flow and the way that the tools are organized in your content’s organized - it’s just really straightforward and easy to work with. And you focus on writing your book, as opposed to anything else. I’ve very much enjoyed that aspect of the experience. It’s been great. And then I can easily publish drafts and get feedback on things, which I very much needed. My book wouldn’t be anywhere near what it is today without five years of feedback. And some of it not the nicest feedback that you want to hear, but it doesn’t mean that it’s not what you need to hear. But yeah, I’ve been thankful for the tooling, and it’s awesome.
Len: Cool, thanks. I guess I have a tendency sometimes to ask two questions at once, but are you going to be looking for a conventional publishing contract and distributing the book through a conventional publisher at some point?
Janelle: I am. I’m working with O’Reilly right now, so I ended up taking my book and splitting it into two, because I just had way too much content. So what I’m working on right now is - I’m like 80% of the way through a second book, for which my tentative title is, How to Build a Learning Organization, which is kind of the learning framework side of Idea Flow. I ended up focusing Idea Flow just on how do you measure pain? And then the learning framework itself, I’m splitting into another book.
So one of the things that I’ve learned in trying to sell these ideas, is that I’ve got a two-sided audience, one of them being developers, and the other one being management. And so my second book, I’m going to focus more on looking at the problem from a top-down standpoint of what managers need to do to support engineers, and what does the organizational protocol look like from each side of the organization? And then, how do we start building an integrated organizational learning process, so we can create a steering wheel of sorts at the organizational level - which is a hugely complex problem.
But I’ve got an initial set of blueprints as a starting point, and then I’m using Open Mastery as a vehicle to get the community to take ownership of getting from point A to point Awesome. And so, let’s see. What was your original question, I forgot.
Len: It was, are you going to be looking for a conventional publishing contract?
Janelle: Oh, yeah. So I’m working with O’Reilly, and I’ll keep my book on Leanpub probably as long as I can. But my main goal is trying to get my book in front of as many eyeballs as possible, more than anything else. And working in partnership with a giant marketing machine has its advantages.
Len: I just wanted to say on that note, for us at Leanpub, when someone starts their book on Leanpub, and then ends up going in the end with a publisher, and retiring their book on Leanpub, almost nothing makes us happier. I mean that is just a fantastic outcome for the author and for us, because part of our goal is to open up that time when a book is normally hidden in stealth mode on a person’s hard drive, to the public and to people.
If all books were published while they were progress, and then get taken up by publishers at the end, it’s a win on all sides for everyone, including readers and authors.
I guess my last question is selfish. I mean, speaking to someone who thinks so much about improving processes, if there were one thing that you could ask us build or to improve about Leanpub, what would that be?
Janelle: Probably the thing that I feel like I’m missing the most, that I’ve been trying to make up for in other kinds of ways, is the ability to build a community around the book, that’s integrated with the tooling. I know there’s kind of email list setup, but when people buy the book - unless they check the little box of share their email. it’s really hard to create relationships out of that. And I’ve tried to come up with some other ways to work around that with signups through other kinds of tools to collect emails, so I could stay in touch with the community. But that’s probably the big thing I feel like is in my way as an author is, building, getting the community of people around the book talking to each other, as opposed to just this one-way communication. And I realize there’s discussion comments and stuff, but it just has a very different kind of feel to it.
Len: Thanks very much for that. You’re the second author I’ve interviewed just in the last couple of months to talk about community when asked that question. So that’s a very strong signal, and it’s actually something that we’ve been thinking really hard about. Hopefully, somewhere in the near to medium term, we’ll have a better community experience. As you noted, we have a sort of double-blind “email the author” process if people want to use it that way. And we’ve got Disqus comments if authors don’t turn it off.
But there’s just so much more we know we can do around that, including from our perspective, we’re sort of building the world’s first library of in-progress books. And what does that look like? And what is the importance of community around that? There’s lots of interesting questions around, for example, reviews, right? Back to 1996 Amazon. But what do you do with a review of an unfinished book, when a new chapter is added, or something like that? There’s an interesting aspect of time that’s involved. And it’s just so interesting to think around building community throughout that time, and around an unfinished thing.
Janelle: Yeah, I can see that being a really difficult challenge. I think about some of the early feedback I got that was just awful feedback - stuff that you’d never want to make public. Like, “I have no idea what this book’s about, or what you’re trying to say.” It was like, “You just need to rewrite this whole thing from scratch. It makes no sense.” I mean, feedback like that, you don’t really want to necessarily be a big public spectacle that you can never get rid of. I think there needs to be some privacy around certain aspects of feedback, especially - I don’t think you’ll get the kind of feedback you need in a lot of cases unless it’s private.
Len: That’s a really good point.
Janelle: It’s like there’s two different– Yeah, I’m not sure. It’s an interesting challenge you guys have.
Len: Well, that’s a really interesting. I hadn’t thought of that before. Because yeah - one of the processes that’s pretty commonly used by Leanpub authors, is to put their email address in the introduction to their book, and say “Please email me.” And then that is almost always a private communication. But I hadn’t put it together, that one of the reasons that’s actually - it sounds so clunky - but the reason it works so well is that it has this value of being private.
Janelle: Yeah.
Len: That’s really interesting.
Well, I think this is my first feature length interview! I want to thank you very much for your time, and for all your great answers to questions - and for your book as well. So, thanks very much for being on the Leanpub Podcast and for being a Leanpub author!
Janelle: Thank you for inviting me. This was really fun.
Leanpub Podcast Interview #34: Nicholas Russo
published Jul 20, 2016
Nicholas Russo is a network professional who has earned CCIE certifications in both Routing and Switching and Service Provider. A former Comm Officer in the US Marines Corps, Nick is also the author of the Leanpub book CCIE Service Provider Version 4 Written and Lab Exam Comprehensive Guide. In this interview, Leanpub co-founder Len Epp talks with Nick about his career, the duties of a Comm Officer on deployment in wartime, and about his book and his experience self-publishing on Leanpub.
Please note that the views and opinions expressed by Nick are his own, and do not represent the views of his employer, Cisco.
This interview was recorded on June 29, 2016.
The full audio for the interview is here. You can subscribe to this podcast in iTunes or add the following podcast URL directly: http://leanpub.com/podcast.xml.
This interview has been edited for conciseness and clarity.
Nicholas Russo
Len: Hi, I’m Len Epp from Leanpub, and in this Leanpub Podcast, I’ll be interviewing Nicholas Russo. Nick is based in the state of Maryland in the United States, and holds a Bachelors of Science in Computer Science and a minor in International relations from the Rochester Institute of Technology. Nick is a former Comm Officer in the Marine Corps, and recently he was one of the first people that passed the Cisco Certified Internetwork Expert, or CCIE, Service Provider, Version 4 Lab Examination.
Nick is the author of the Leanpub book, CCIE Service Provider Version 4 Written and Lab Exam Comprehensive Guide. His book is about achieving certification in the CCIE Service Provider track, which has recently been updated by Cisco to reflect changes in standards, techniques and procedures made by networking vendors. The book is meant not only for people interested in passing the related written and lab exams, but also for enterprise architects and other network professionals interested in the core routing concepts for any network.
In this interview, we’re going to talk about Nick’s career, his professional interests, his book, and at the end we’ll talk about his experience writing the book, and self-publishing on Leanpub. So thank you Nick for being on the Leanpub Podcast.
Nick: Thanks Len, I’m happy to be here.
Len: I usually like to start these interviews by asking people for their origin story. So, I was wondering if you could tell me how you first became interested in computer science, and eventually made your way to the Marine Corps and on to Cisco.
Nick: They had a relatively new computer science program when I started high school in the year 2000, and they had a bunch of new computers. I didn’t really know what computer science was, but I knew I liked computers as a kid. So I figured, “Okay, well I’ll go check this out”. And then I quickly found out that it wasn’t just putting cards in computers and playing games. It was about learning how to command computers to do things, and to solve real life problems.
I immediately saw value in that as a high school student, and I managed to do those courses for all four years of high school. That led me into the college degree of Computer Science, when I went to RIT to pursue that career. Along the way, in-between college, in between high school and college, I joined the Marine Corps in the Reserves. Partially - mostly for the adventure of it. I didn’t really have a great reason at the time, but it’s something I knew I wanted to do.
So I did the Reserves while I was in college studying Computer Science. While in the Reserves, interestingly, my job had nothing to do with communications at the time. I was simply an infantryman - like the bazooka guy, basically - which was a fun job, but not something you really want to take too far. Once I got done with college, I thought about that, and said, “I’m going in the Marine Corps, I’m getting commissioned as an Officer”.
When I went to training, at that point, I got to decide which way do I want to go. Do I want to continue to be an infantryman, or do I want to actually put my computer science skills, at least to somewhat of a use, and go into telecommunications? So I decided to go that route. On my second deployment to Afghanistan, which was in 2011, I got really interested in networking.
This was mostly the result of one of my fellow Marines. His name was Master Gunnery Seargent John Robertson, who’s one of my best friends, one of the most influential and smartest guys I’ve ever worked with. He could make anything work out of nothing. You see a lot of these stories, it’s kind of a different form of innovation, if you will. A lot of cool stuff comes out of Silicon Valley these days, people solving a lot of real world problems with new ideas.
This was the kind of “guy stuck on an island” innovation, where I have a nail and a shoestring, and I need to communicate 20 miles. It was that kind of stuff. I really enjoyed that, and I could see a lot of the ingenuity that came from the tactical side of communications. But then I became really interested in the large scale routing, switching and service provider technologies that were a little bit related to what we were doing in the tactical space.
When I left the Marine Corps, I worked for a company called Harris RF Communications in Rochester. They had a very large tactical radio program, networking tactical radio. So it actually worked really well. My experience with radios in the military, plus a little bit of my network knowledge, it was a good way of bridging that gap, where I could connect radios to a network, and extend my IP connectivity over a tactical radio net, so a guy driving around in a truck could access different network resources.
I moved on past that a couple of years later. Ultimately I went through a couple of other jobs, and now I work for Cisco as a Network Consulting Engineer - doing a lot of large scale routing designs and support for the Tactical Army, as well as other army garrison posts.
Len: That’s a great story. I think most of the people who listen to the Leanpub Podcast are still currently computer programmers, and I think the kind of work you described as a Comm Officer on deployment is something they might like to hear a little bit more about. Can you give me an example of what a typical day might have been, and what a typical task that you might be assigned to do would be?
Nick: What’s interesting about it is, computer science and networking - they’re different in many ways, but similar in others. The thing about computer science is you’re trying to take a machine that has no brains and give it brains, so that it can do something - solve some problem, generate revenue for a company. Or even something like, “Hey, sort this list of items using some kind of algorithm”.
In the Marine Corps, doing communication stuff, we have a whole bunch of items which are done - these radios have software on them, these routers have software on them. So the software engineering component isn’t really in play. However the systems integration component and the networking piece was really the major challenge there.
And as you can imagine, when you’re in a combat zone, your resources are limited. In addition to human factors, like stress and personnel shortages, you’re asked to do things like, “Hey, we need to communicate to these new sites. We’ve got these new outposts. These guys need connectivity”. That’s an important task, and I don’t have the resources either in people or equipment to do that.
So then you start to be creative with the tactical radio thing I was talking about. Like Harris for example, they had something, “Hey we can drop these small little [Manpack sized radios] out there, and extend IP connectivity to these guys, because we don’t have a giant truck that we can send out there for satellite communications”. So things like that, I think were really interesting.
And of course, you can’t just drop the radio in some guy’s lap and say, “Okay, turn it on”. That’s not going to work. There’s a whole planning detail that goes into that. Both from a spectrum perspective, making sure that you’re allowed to use the frequencies of that radio, as well as the configuration of the network side of it.
So doing that, I would say - on an interesting day, where we got to do something like set up a new site, or deploy some new services to some outposts - we would typically find ourselves trying to solve those kinds of problems. Ultimately, you think about it from a business perspective. That the business value is, we’re enabling a new site. Which is at the edge of our perimeter, ultimately providing security for the rest of the unit.
Len: I imagine that setting up a network so that people can communicate with their families back home, must be a really important part of the job now as well?
Nick: Yeah, that was another interesting part of the job, because typically in a tactical network, when you’re doing these networking things and enabling services for different units, you’re typically focused, number one, on the classified network. Because that’s where you exchange all your fire missions and that’s where you fight your battles. After that comes the unclassified tactical network, where you’re going to do things like logistical coordination and things that aren’t classified, but still protected. And then the last, and lowest priority of course is the morale network.
We were fortunate enough to actually have a pretty decent morale network. Mostly because the first two priorities were pretty solid. So we had a little bit of a morale station if you will. People would come by the main FOB, maybe on a monthly or every couple of months basis. We had a couple of computers set up in the corner where people could - it’s basically an internet cafe for free.
People show up, they hop on to check email, they do a video call, and then they go about the next couple of weeks being without it. So that was a pretty good thing to have as well. Obviously for morale purposes, and also because - it’s another network to maintain, so then you start to get into this habit of, “I have multiple different security enclaves, I need to be able to maintain all those different things”.
Len: You mentioned FOB, which I understand is Forward Operating Base.
Nick: Yeah, that’s right.
Len: Would you actually go yourself to FOBs?
Nick: I did a little bit of that. In my role as a Communications Officer, I was a little bit tethered by some of the leadership. Because they’re like, “Hey, we need you - you’re the bellybutton for us. When we need communications, we come and poke you”. So it was often times a lot of my subordinate leadership again - John Robertson was one of the instrumental people - he personally did a lot of that himself, and also helped me delegate that to some subordinate leadership within my unit that I was in charge of.
But what’s interesting is, the chances that I did get to go out there were kind of a real eye opener, because sometimes when you’re sitting behind a desk, you’re like, “Hey, you guys go out and set this system up, and it needs to be up by tomorrow”, and when it’s not up by tomorrow, you start to get upset. You’re like, “Oh come on, I needed that up now”. And when you go out there and do it yourself, it’s like, “Oh, that actually wasn’t so easy”.
That’s where I started to gain the appreciation for networking. It’s like - I sit here, and I think their stuff is easier than it really is, because I’m looking at a diagram with some routers and switches, and I just have to draw a line - and my job is done. Then I realise it’s never like that.
Len: It’s really fascinating. I was thinking as I was going over your book about the routing and switching part of computing. I imagine this might kind of annoy you at some point, but it’s one of those things that everybody who uses computers and communications devices just takes for granted is going to work. It’s the black box underneath it all.
I was wondering, if you were to explain, say to someone non-technical, what it is that you do - just briefly, how would you do that? And explain what the importance is of the technology that you work on? Now at Cisco, I guess.
Nick: Sure. So my job here at Cisco, it’s similar to what we kind of already do. You can think about my old role when I was in the Marine Corps. I was a Communications Officer in an infantry battalion. I supported communications for that battalion. Actually I was the operations guy, in a way - the Marine Corps fielded me all this equipment. They gave me 10 of these com vehicles, we’ll say, and I’ve got to supply reachability to 12 units. Okay, that math doesn’t work out so easily, so then it’s up to you to think of ways to solve that problem uniquely.
The role I’m in now is, I work to support one of the program officers that actually builds those trucks. So I support them in large scale networking. Because from their view, they’re fielding hundreds and thousands of these trucks across the entire US Army. So you can imagine the network that it takes, and time and resources - what it takes to maintain, plan, operate a network like that. It’s pretty substantial. So I work on that quite a bit to help them determine, What kind of architecture do we want done? How can we make this scale? How can we make it manageable?
And then the other important thing - the people running this network are soldiers. And nothing on them - they’re extremely courageous, smart, tough guys and girls. But a lot of them, they’re not professional engineers a lot of the time. So we always have to consider, this has to be real easy. Kind of like smartphones and all these new smart devices we have. Anyone can use them, but the tech inside of that thing is just overwhelmingly awesome I think. It’s crazy, it’s complicated, but it’s awesome. And somebody had the brilliance to take all that tech, and put such a simple interface on it and make it resilient. And that’s one of the biggest challenges that we have in tactical.
Len: I was wondering, on the subject of technology, what would some of the recent developments be that have had to be incorporated into your engineering processes?
Nick: Some of the recent developments - I mean, we try to always keep a thumb on a lot of the recent developments that happen within the industry. So for example you hear terms like SD-WAN and Software-defined networking, and just general terms like that, like cloud computing. We try at a minimum think through how we would want to deploy these things in a tactical environment.
I think there’s a lot of promise for those technologies, specifically things like SD-WAN, where I might have slow satellite links and fast line-of-sight links. Of course, all these things are built on the fly, and change very constantly. So some kind of intelligent system that can route traffic, based on different metrics and other user-defined criteria, would be extremely important. We’ve continued to keep a pulse on things like that.
The other thing that’s always fun in a government environment is, appropriating money to buy these things, field them, test them. That cycle can sometimes be very long. And you think about small startups, a smaller company of a few guys, they can make a couple of decisions and change the direction of their company incredibly over the course of a few hours, where a bigger ship is always harder to turn.
Len: On the subject of technology, drones and UAV’s are obviously on the top of many people’s minds who read tech news. I was wondering if that’s part of the communications infrastructure that you’ve worked on?
Nick: It is a little bit. We’ve talked about having kind of like an aerial tier in a sense. So you can kind of imagine, I’ve got a bunch of trucks on the ground, and maybe those trucks can’t see one another. But satcom bandwidth can be very expensive, and sometimes prohibitive. So if I can have some kind of aerial drone to serve as almost a re-transmission or a relay point - we’ve considered things like that as well.
Those things are continuously on our radar, because deploying capabilities like that, both obviously from a surveillance perspective, for intel feeds - which has been around for a long time - but also for radio re-transmission systems. Those are highly relevant for tactical.
Len: On the subject of your book, as I understand it, at the time you passed this latest certification, you were one of only 10 people or so in the world to have done so. You can correct me if that’s wrong! I just wanted to ask what some of the changes are that are reflected in the latest version that required a new certification test?
Nick: That number I think is kind of a scientific guess - a wild guess based on what I’ve talked to a couple of other people in the group about. Long story short is, every couple of years, most vendors, when they offer a certification test, they’ll come up with, “Okay, we’ve got all these new features that are relevant in industry. We want to make sure that we certify people on these topics”. But then after three, four, five years some of those things may lose their shelf life, they may not be relevant. So vendors try to do a good job, and cycle those things out.
The CCIE Service Provider version three to version four, at least in my opinion, had a relatively sizeable number of things that were added and only a small number of things that were removed. So ultimately, what you end up with is a very, very large topic domain. And that’s what drove me to write the book.
What’s interesting is, for a test like this, there’s training materials out there, there’s documentation. There’s a lot of information. There’s people to talk to. But a lot of times what’s really hard to find is a consolidated set of labs and technical explanations, to a sufficient level of depth that would allow you to succeed in passing the exam. There are companies that exist out there - very good ones in fact - that base almost their entire business model on the CCIE and passing those exams. And they’ll have professional instructors that do videos, they build labs, they have workbooks, they have training sessions. It’s a massive - I wouldn’t say massive, but it’s a sizable industry.
Obviously my book doesn’t add the human element, per se. I’m not doing boot camps and stuff like that. But I do cover all the topics in the blueprint, and a lot of people have contacted me looking for materials. And of course, now that I’m finally able to offer something, I think that there’s a lot of interest in it.
I think just in general, it’s kind of a human thing - “Hey I really want to achieve this thing, but I don’t have guidance, I don’t have a mentor, I don’t have information. I just don’t know what to do.”
So people typically do one of three things really. Either number one, they wait for it. They wait for someone to do it. Which is fine, and basically all those people are my customers now, and I’m more than happy to support them now that they have my book. They waited a couple of months, they got something.
The other group of people are people who just quit. And that’s totally understandable. “I don’t have materials out there, I know I’m not going to succeed”. Maybe they’ll try to go for a different task, maybe they’ll go back to college instead; maybe they’ll spend that time with their family, instead of studying.
And then the third type, which I think would be me, is the guy who goes and writes his own book on it, in lieu of not having any other materials. And that obviously takes a tremendous amount of work. I know you see the page count is almost 3,000 pages. There’s almost a million words in this book. It took me about seven months and 1,000 hours of continuous dedication to work through, to document these technologies and verify everything several times, so that I had a very clear understanding of what was in it.
Because my logic is, if I’m going to try to market this to my customers, I want to ensure I’m providing them a quality workbook, that’s going to accurately reflect the topics on the blueprint, as well as all the ins and outs of the weirdness. Because of course, when you deploy technology, it never works the way you think it’s going to work. So you always want to have that explanation of how you’re doing things, and especially why you’re doing things.
Len: You mentioned in a blog post that you actually failed the first time you took the test. I was wondering if you could talk a little bit about what you learned from that experience, and helped you pass the next time?
Nick: Yeah, that’s the interesting thing about these exams. I think they say the second attempt is common, because it definitely is. I took another CCIE in 2014, the Routing and Switching test, which I also failed once. I passed that on my second try, like this one. And taking it the first time, I went in there highly confident, my book was already done.
I’d already written the whole book, I knew everything in the book. I was at the top of my game. I walked in, I got through the first couple of sections pretty well. And some things later on in the test, I found extremely difficult. I was pressed on time, I was tired, I was hungry. All those human elements start to eat away at you as you’re trying to accomplish this very difficult task in a very limited amount of time. And at that time, the time I first took the test, I didn’t know a single soul. I’m sure there were some in the world, but I didn’t know a single person that had passed the test.
By the time I passed it the second time, I knew about two or three people [also had]. So it was heating up a little bit. I’m sure to date there might be 20 or 30, but it’s still a relatively small number. And I think the reason for that small number, it’s not so much because of the impossibility or the difficulty of the test. I think it comes back to the fact that people just don’t have the information that they need to pass it. And I’m trying to help break that, so I can bring some more knowledge to people, and ultimately help raise the popularity of people achieving this exam.
I know there’s a lot of people who use these technologies, and use similar technologies in their everyday work. People who work for large service providers, the large tier one carriers in the States. You’d think of Sprint and CenturyLink and AT&T. These guys have massive, massive carrier networks, and a lot of the technologies in this book are very, very specific to that environment - and also for large enterprises as well. In my opinion, having a mastery of those technologies is how you build large scale networks that are scalable and efficient.
Len: I noted that you talk in the book, I think in the introduction, you say that the book is obviously useful not only for people who are taking this specific test, but for people who want to be experts, or are experts in their field, and just need information about this somewhat undocumented subject.
One detail I liked from your blog post was, you mention it’s good to read - and I think this applies not only to the exam the book is about - you talk about how it’s important to read the entire exam, all the questions at the start. I was wondering if you could talk a little bit about that. Is that so you can prioritize questions? How would you go about prioritizing them if that’s the reason?
Nick: Yeah, that’s a great question. The thing that’s interesting about these tests is, and a lot of people will say this, and I tend to agree with it: a lot of getting through that exam is the strategy and how you go about it. Like I said, when I took my first CCIE, I failed it the first time, but I knew all the technology. I was extremely competent. Same thing when I failed the service provider test the first time - I knew all the technology, the book was done. I was extremely fast on the command line, I knew what I was doing. But it was a strategy thing that I didn’t have nailed down. And the thing about the strategy, as you know - you’re asked to design a network either from scratch, or you have some kind of a brownfield deployment.
For example, you have a network that already exists, and you’re asked to make significant changes to it in an extremely short amount of time. So you have to look at it and say, “What does the network look like now? What are all the things they’re asking me to do, and how can I put these tasks in the proper order?” So it’s both a prioritization thing, like, How can I get the most points as fast as possible? But it’s also a matter of which tasks make sense to do in which order, because sometimes what you might end up with is, in the beginning they have you do some basic routing tasks, and then in the middle of it, they have you do a bunch of value added service stuff. Maybe like IP multicast or quality of service or MPLS Traffic Engineering.
And those things are important. You’ve to do them to get the points. But they might have you come back and do some routing stuff later in the test, when it would’ve made more sense to do that upfront. So you kind have to think about, What is the overall objective? What does the network need to look like when I walk out of this room today? And that’s what I need to focus on from the very beginning.
Len: I was wondering if you could talk a little bit about why you chose to self-publish the book, and why you chose Leanpub as your platform?
Nick: The self-publishing thing, it’s - I’ll keep this brief, just in the interest of not boring everyone. But I wrote this book, and I finished it a couple of days before I joined Cisco. And because I did that, I talked to my company, and we worked it out and said, “Hey, I wrote this book before I came here, but now I work here and I’m trying to publish it”.
Typically when you work for a company, what you develop at that company is property of that company. So we went through a little bit of a thing there. Everything was great, they were extremely cooperative. It just took a little bit of time to get through the approval process. And then at that point, I was like - well, it’s June, I wrote this book in January and there are people failing this test around the world.
The test was introduced in 2015, and I figure the test has about three more years of shelf life on it. If I spend another six months working with a professional editor, it’s going to cost me a lot of time and a lot of money. And quite frankly, a book with a million words, it’s going to have some grammatical errors, it’s going to have some spelling errors. But it’s a technical book, and all the content is accurate. I think that’s what’s really going to matter for my customers, and those who have put their testimonials up on the website will certainly agree that the content is certainly there.
I figured I would rather push this out now, get it published quickly on a place that I think is reputable, and people are familiar with. And especially Leanpub, I think a lot of the information - I look at a lot of the books on Leanpub, they’re written by people who are coming up with new ideas about software and APIs and things like that. And it seemed like a very - I guess, for lack of a better term - a very tech-friendly environment, where I could post something that I came up with, and it would get viewed by people with similar views, and people who would potentially be interested in it - even if they’ve never heard of it before.
Len: Thanks very much for that, and for choosing Leanpub. Your case is one of the reasons we built Leanpub - so that people who are, say, on the cutting edge of something, and an expert in something, who’ve made something that has a kind of time sensitivity to it. Also, something that’s so useful to the people who need it, that the occasional grammatical error, or lack formatting finesse is actually irrelevant to them.
This is actually an example I’ve often used, just sort of in general. If you approach a publisher and say, “There are a thousand people in the world who are expert enough to understand my content, or to be interested in it in the first place”, You’re pretty unlikely to get a publisher unless you’re an academic, or you’re going to an academic press or something like that.
But in your case, you can publish a book, and provide a lot of value to your customers, but also get something back yourself. And because we pay a 90% royalty [minus 50 cents per transaction - eds.], then it can be worth it, even if you in the end you don’t reach a great number of readers.
Your book is really interesting in a very specific way, because, as you say, it’s nearly 3,000 pages, nearly a million words, and so you’ve set the minimum price at $200 and the suggested price at $300. I just want to explain to everyone listening, one of the reasons this is so interesting, is that Leanpub books have no digital rights management or DRM on them, which means technically you can make as many copies of them as you want and distribute them. And you can also get a refund with just a couple of clicks, no questions asked.
Your book so far I think has one return or something like that. So people are voluntarily paying, obviously, is my point, and I was wondering how did you decide to set the minimum price to $200? Was it based on instinct or some research?
Nick: The thing that’s interesting is I checked a couple of competitors out there. And just to be clear, I looked at some of the competitors who were offering workbooks and despite some of those other workbooks being much, much, shorter - like on the order of 600 to 800 pages, and being relatively incomplete, those books cost about $200. So I figured for my book to be a little bit more expensive, that’s appropriate. But for them to be the same price, there’s no contest. At least that’s the message that I’m trying to put out there. You can buy a 3,000 page book that’s known to be complete for price X. Or you can buy another book that is a quarter of the length for the same price. And you can choose based on that.
The $300 price was my initial idea for it, but I figured if I’m going to launch this book - at least for me personally, I think getting the exposure as an expert in this field, and this is my first book that I’ve ever published - being able to do that, and to call myself a professional writer with a publication, and to put it out there - in my eyes was more important than making that extra 50% on the royalties by selling it for 100 bucks more.
I think what’s interesting is, I didn’t really expect to see a whole lot of people voluntarily paying more. But there actually have been several, which is very cool. People I think that have said, “Hey this book is 3,000 pages, and if I spend 300 bucks, that’s 10 cents a page”. And you can imagine the amount of technical detail you’re getting for just that one 10 cents. You’re getting a whole page worth of extremely detailed stuff and explanations. Entire topics on here might be anywhere from 10 to 100 pages. So for a dollar to $10, you’re getting an entire topic that describes a technology that may not be documented sufficiently in any other study guide.
Len: You mentioned in an email the other day that your book is already up on torrent sites. Every author loves their books and puts a lot of work into them, but yours in particular is special in that regard. Given that you’re a first-time author, I was wondering just how it feels to have your book put up on torrent sites, and how you’re choosing to mentally, or even procedurally, deal with that?
Nick: It’s one of the things to be expected in the certification world. This is one of the the warts on certifications in general. With enough Googling and with enough searching on the net, you can find answers to probably any automated test out there. And that’s just really unfortunate. Typically when you meet guys that kind of cheat their way through these tests, you can see through them like glass. But nonetheless, I like sharing information. I’m certainly willing to do that. I feel like I’ve set a fair price for the book.
And the thing that I also believe, and this is actually, I kind of stole this from one of Cisco’s values - one of the things we say in Cisco is, “Intensely focus on your customers”. I do that at work with my Cisco customers of course, but I also decided to make that kind of the flagship of my publishing career. I get personal emails from a lot of my customers who use the email link on Leanpub, after they bought the book, to ask me a question about, “Hey on page 2,000, I have a question about this command. How does it work?” And I make every effort to answer that question within a couple of hours. I’ve already taken on to try and mentor - some people have asked me to help me mentor them through their study journey. I’m willing to do all that for anyone who wants to get on board with the book, ask questions about it.
I think that’s extremely important to being successful, basically in whatever you do. I’ve heard a really great saying, “When you focus on making money, you ultimately fail. But when you focus on your customers, you ultimately make money”. So even though making money’s not really the goal here, I just - getting the exposure, becoming a professional writer, publishing a book and knowing that I had the spine to go through and write this thing and publish it and take it from cradle to grave - I think brings me a lot more benefit than the money.
Regarding the torrent-site stuff. I’m kind of half-hearted chasing that down, in terms of infringement, things like that. I don’t want to make that the cornerstone of what I do every morning when I wake up and I roll out of bed, and I start searching torrent sites. I’ve got better things to do with my life. I’ve got more books I want to write. I’ve got customers that I want to help get through this test, and I think that’s the most important thing to stay focused on.
Len: That’s great, both of those answers I would say are also textbook Leanpub principles, but expressed probably much better than we’ve ever managed to express them ourselves in the past. And we like trying to do that. So thanks very much for that. I mean, engaging with customers, and focusing on the kind of work that you as an author ought to be doing, are two things that are central to what we’re trying to deliver for people as well.
One specific question I have regarding engaging with readers and feedback through email and things like that is, Do you plan to incorporate any changes that people suggest to you, or corrections that they find into your book, and publish new versions?
Nick: That’s a very interesting question. I want to say yes, but I must say no. And the reason I must say no is - as I mentioned, I [finished] this book about a week before I joined Cisco, and anything I do now today would be a Cisco thing. However, I’m working very closely with the service provider program manager for the CCIE. She and I talk probably on a weekly basis, and I told her, any errata to the book, if people want to see other technologies, maybe something I’ve missed or - if there’s any corrections or supplements to do, I’m going to try to work with her and just publish official white papers from Cisco that try to cover those gaps, while at the same time, improving the product that the PM is offering, additional full scale practice labs and other things that Cisco is going to be able to use to help people get through this exam.
Because now that I wrote this book outside of Cisco, and I can collect that money not related to Cisco, at the same time, the real goal is to continue to keep people interested in this track, and assist Cisco in doing that as well. So now I’ve transitioned from, “Hey, I’ve published my own book, I’m running that on the side, but I also want to transition to assisting the Cisco Program Management Office, who runs the CCIE service provider program, providing them ideas for labs, and simply helping them actually build the labs - as property of Cisco. And it’s something that they can either market or give away, or build in their own study group.
So I would definitely encourage any feedback and corrections. Please send those to me, I’m happy to consider those kinds of changes for any kind of future documents that we publish for Cisco.
But the thing that’s interesting though is, the one thing that’s a little bit orthogonal to the way Leanpub works - “Publish Early, Publish Often” - well for me, it’s “Publish Once”. Which is a little bit odd. Ideally I would go back and fix the problems, but changing a - if I change a period to a comma, then I kind of enter a gray zone with legal stuff. And at this point, it’s probably not worth doing.
Len: I just have one more question, which is, we take customer development very seriously at Leanpub, and now that I’ve got you here, I wanted to ask, If there was any one feature we could add, or one problem we could solve for you, what would you ask for?
Nick: I was thinking about this yesterday. This kind of goes back to that one return. Again, I don’t think one return out of 35 sales is anything to be concerned about. However, the thing I think is interesting - the two-click return policy, I think is great. I think what would be potentially beneficial is two clicks and a mandatory drop down that maybe just says, “Hey, pick a reason. You don’t have to comment, but just give me one out of these five reasons - I over-drafted my account or the book wasn’t what I thought it was”. Or, “I bought the book, but I couldn’t download it for four hours and I got frustrated and I want my money back.” Just a basic reason. You don’t have to justify it, Leanpub isn’t going to grill you on it. The author’s not going to grill you on it. Just something for a record.
I think that’s valuable, because if the number of returns starts to become significant - for example if I see 10 returns in a week, and they all have the reason of, “I couldn’t download the book”, that might point towards a technical problem. Or it might be something like, “Hey, we all bought this book, and it wasn’t what we thought it was”.
Well, maybe those 10 people are part of the same organisation, and they were briefed by someone - maybe a friend of mine - a third party, “Hey you guys need to check out this book, it’s great”. Maybe he misspoke, and set expectations incorrectly. And now we ended up with a bunch of returns.
So just having that reason I think would be beneficial. And then of course a comments box, so that people could expand on that if they want to. That would help authors I think, just to understand what are the big muscle movements that justified the return. I think there could be some value in that.
Len: Thanks very much for that suggestion. Just for our listeners, what Nick is referring to is our two-click, 100% Happiness Guarantee return policy, which lets people easily get returns on Leanpub purchases. When they do that, they’re prompted to optionally leave a comment. We don’t ask for feedback in any other way. And we don’t require a comment either.
And that means that sometimes authors like Nick end up with mystery returns, where someone returns the book without saying why. And that can be unsettling. I’ll talk to my co-founder Peter about your suggestion, because I think it’s very good and could really help authors a lot, and maybe even help readers feel like they’re participating in a process at a deeper level as well.
One comment I would make in my experience watching Leanpub books and authors and returns over the years, is that usually if there is a problem with content, one or two people might return it without commenting, but if there is a real problem, someone will eventually comment. And often what they’ll do, if they understand how Leanpub works - I mean in your case this wouldn’t be the case, because you can’t change the book - but instead of returning it, they’ll contact the author through our “Email the Author” form, or on Twitter, and say, “Hey, can you fix this problem on page five?” Or something like that. And then the author will do it, and update the book and let the reader know. And then the reader feels like they’re part of the process, and it’s all a very happy thing…
Just before we go, is there anything else you wanted to say or an issue you’d like to address?
Nick: No, I just want to, I’ll just thank Jeremy Filliben. He’s a Cisco Certified Design Expert, or a CCDE trainer. He’s out of Delaware in the United States. He was actually the one who introduced me to Leanpub.
He and I were chatting on Slack one day. This was about a month and a half ago. I was complaining to all my friends I was having issues with publishing, I was having issues with the book. After writing this big book, I was right at the finish line, just trying to crawl across and get my book out there. And I was having a lot of different non-technical personal related issues with people and process and things like that.
He says, “Hey you should check out Leanpub”. I’d never even heard of it. I went to the website, I read through how does it work. And I read the FAQ page. And I was like, “Yeah, these guys look pretty cool”. And then, when I was approved to do the book and I was ready to go, I had the book in hand, and I said, “Okay, I’ve got multiple different self-publishing options here”. I kind of weighed those out, I talked to Jeremy again, and ultimately went with Leanpub for my publishing.
Len: Okay, well thanks very much Nick, good luck with the book! Thanks for being on the Leanpub Podcast, and for being a Leanpub author.
Nick: Alright, my pleasure. Thanks Len, I appreciate it.
Len: Thanks.
Leanpub Podcast Interview #33: Amir Rajan
published Jul 07, 2016
Amir Rajan is an indie game developer based in Dallas who is best known for having made the bestselling iOS app for the popular text-based game A Dark Room. He’s also the author of the popular and aptly-titled Leanpub book Surviving the App Store. In this interview, Leanpub co-founder Len Epp talks with Amir about his career, the experience of writing a bestselling iOS app, about his book, and about self-publishing on Leanpub.
This interview was recorded on April 27, 2016.
The full audio for the interview is here. You can subscribe to this podcast in iTunes or add the following podcast URL directly: http://leanpub.com/podcast.xml.
This interview has been edited for conciseness and clarity.
Amir Rajan
Len: Hi, I’m Len Epp from Leanpub, and in this Leanpub Podcast, I’ll be interviewing Amir Rajan. Amir is a Dallas, Texas based indie game developer and independent software consultant who’s experienced working with a number of platforms and languages. Amir also works on several open source projects, and speaks at conferences and user groups, and he blogs at amirrajan.net.
He’s best known for developing the iOS app) for the popular text based game, A Dark Room, which was originally created for the browser by Micheal Townsend (Amir also released an Android version recently: A Dark Room - eds.). Amir’s app hit number one in the App Store, and it’s been downloaded over 2.5 million times. You can follow him on Twitter @amirrajan.
Amir is the author of the Leanpub book, Surviving the App Store: How to Succeed as an Indie Developer, which is a very well-titled book. It’s full of lots of great tips, a very practical and valuable approach to helping people get started in game development for the App Store.
In this interview, we’re going to talk about Amir’s professional interests, his books, his experiences using Leanpub, and ways we can improve Leanpub for him and for other authors at the very end. So thanks, Amir, for being on the Leanpub Podcast.
Amir: No problem, it’s a pleasure.
Len: I always start these interviews by asking people for their origin story, so I was wondering if you could tell me how you first got interested in coding and being a developer, and how you ended up being a game developer and working on A Dark Room?
Amir: I guess a lot of people should be able to relate a little bit to the story. I learned how to build software through just building video games. I think my first thing was, I tried to build a chess board with The Knights Tour. I think I was a sophomore in high school at that point in time, and I had Turbo C++ and I was like, I want to try to build chess.
Pro tip: when you’re just starting to get into game development or any kind of development, chess is probably one of the harder problems you can pick up.
I got my first computer when I was 10 years old. It was a 386 with a turbo button. I played games on there like Scorched Earth. That was one of my first games that I remember. And then I had Chessmaster 3000. And that’s what really got me into it - just being around video games.
I had the Sega Saturn, the NES, the Atari. I just grew up with that.
And then I just did my thing in high school. I found that I really liked programming, and got a degree at University of Texas at Dallas doing computer science and software engineering. And during that time period, I just finished my degree, and then went into the corporate world. Beause that’s what you’re supposed to do, right? You have all these dreams and aspirations of, “I’m going to build video games,” and all that stuff, and then reality hits, and it’s like, “Well I actually have to make money”.
I became a software developer at that point. I did that for 8 years, and then I went on a sabbatical, and saved up enough money to take some time off. I downsized from a two-bedroom to a one-bedroom. My wife was okay with it. Our standard of living really didn’t change, because I made sure to have a really solid foundation.
I took some time off, and A Dark Room was like - hey, I want to do iOS development, and learn the platform, and this is a really easy, minimalist text-based RPG that I can port over. It wasn’t, but it starts out really simple, and then it becomes really intricate. I ported it, and I was really happy with it. I put it in the App Store, and then I became obsessed with the marketing aspect of it, and building a user base – watching my baby grow.
It went viral, and I guess that’s all she wrote, right?
After that I was able to get a new physician. On the sheet they make you fill out, it says “Occupation”. I was able to put down “Game Developer”. That was a pinnacle moment for me, that I could stop calling myself like a “Software Engineer” or “Software Consultant” and call myself a game developer.
It’s so fun, because you’ll be part of conversations - before it was like, “So what do you do?” And I would say, “I’m a software developer, software engineer”. And then that’s where it stopped, right? Because they’re like, “What does that mean?” My wife would do the same thing. Like someone asks her, “Oh, what is your husband doing?” She goes, “I don’t know. I think he does something with computers”. You always get, “Hey, I’ve got this computer problem. My internet’s not working” or something. Now when I talk with people, it’s like, “Oh yeah, I’m a game developer”, and it completely breaks the ice, and you can start talking about really cool things from that point.
Len: That’s a fantastic story, and I’ve got to say, I can actually sympathize. My first corporate job was working for a company that’s now called Dealogic, in London. I was doing research on mergers and acquisitions basically. But the company at the time was called “Computasoft”. All I had to do was say that name, and people glazed over, not interested.
Amir: Yeah, they’re like, “We don’t care.”
Len: It was just amazing. You don’t even need the full word “computer” in it.
Amir: And it’s so interesting, you start saying “I do software development or programming” [and they reply] “So you’re in IT, I hear that’s not a great industry to be in.” Suddenly they know everything about IT, and they can just group it into this one, umbrella discipline.
Len: I don’t know if this is a tangent, but it’s actually a very interesting topic, and very important, I think. For example, there was a recent exchange I saw on Twitter where a relatively prominent journalist based in New York tweeted about - I think the guy’s the co-founder of Vox, and he had written a Medium post about digital publishing, basically. And then someone sent out a tweet saying like, “How can this guy, a tech administrator, talk about digital media?” To which of course he sort had a really snarky reply. Marc Andreessen retweeted it, saying, “What’s a tech administrator?” ironically. I replied to that with an image from Peter Thiel’s book of the Solyndra guy next to Elon Musk.
Amir: I’ve got to see this post.
Len: There’s a weird preoccupation that people who are unfamiliar with software have, where it’s kind of scary to them, because the person who knows how the computers go, actually runs everything, and knows how everything works. It’s very unsettling to people aren’t familiar with software, but have a prominent position from a kind of legacy, institutional structure.
So they take the opportunity to sort of goad the new media guy, for actually knowing how the computers go, as though that’s somehow evidence that he doesn’t understand the real work of publishing.
Amir: I can empathize with that too. It’s like, when my car breaks down, I take it to the mechanic, and he’s like, “Oh you’ve got this and this wrong.” And you feel so disenfranchised, right? You know that this person has the upper hand. You’ve got to trust them and have that kind of rapport. So it’s frustrating.
I hope in the future more people are more literate at manipulating a general computing machine, and doing something with that. I guess there will come a time.
By extension, I think game development does a really good job of that. Beause when I released ADR to iOS, I had teenagers email me saying, “I want to get into game development, this is one of the first games where I really enjoyed it, and I saw that you don’t have to have crazy graphics, or 3D models or anything to actually build something that people will enjoy playing.” And they could visualize; it was like, “I can create a button. I can put some text on a screen.” These aren’t difficult things, I mean they’re familiar to some extent. And that’s what kind of got them into development.
I had people actually email me, “Here’s this like Python console-based game that I built.” And it’s so awesome. I play, and it’s like, “Oh this is the coolest thing in the world.” To be able to inspire people from that perspective has been pretty fun.
Len: It’s got to be really rewarding to have a community built around it, and to have people emailing you with their creations. And to be able to be a game developer yourself and create things.
You brought up your sabbatical, and I found that really interesting. I was wondering – have your employers since then cared about you taking a sabbatical? Was it something that they asked questions about? Or is it like – they’re just not relevant. You’re a good developer, and that’s it?
Amir: For me, the sabbatical was actually a learning sabbatical. I was primarily a .NET C# developer. And during the, I would say 14, 15 months off, I ramped up on a whole lot of tech, so became really proficient with iOS. I did a lot of Ruby development, primarily for build automation. So I got into Ruby.
I say I’m a Ruby-ist, because I got into Ruby using Rake to build .NET projects. Usually you hear Ruby developers say, “I did Rails.” But I have very little Rails experience. All my Ruby experience is through building iOS games, and doing build automation for .NET.
But during that sabbatical, I ramped up on so much stuff – Node, Angular. I did a lot of Knockout back in the day too. JavaScript, a lot of runtime things.
I finally went into my OSX partition. I had a Mac, but I always used Bootcamp, and I loved the device, and the system itself, I didn’t ever use the Mac partition. So this was an opportunity for me to do that.
When I talk with employers, I say, “Well I took a two year sabbatical to really ramp up on other technologies,” and sell myself as an integration specialist. So not only do I know .NET, I know Rails, I know Node. I can do QA stuff, I can do mobile development. I can speak a lot of different languages with regards to - real languages - interacting with humans.
The fact that I created a product, it makes me a little bit more able to communicate with the business end, saying, “I understand that you’ve got these limitations, or you’ve got these things that you’re trying to accomplish. Let’s see if I can figure something out for you.” So it’s worked out pretty well for me.
The other side is that I’ve kind of become unemployable from that perspective too. Because you start valuing your time so much more. You realize that 9 to 5 on a salary leaves very little for the person, from some perspective. Especially with the market as it is today. I think taking the plunge, and taking the risk, maybe doing independent work, having some kind of side hustle is so valuable.
Some employers are like, “What do you mean you don’t want to go salary?” Or, “What do you mean you don’t want to work 40 hours a week?” It’s just kind of weird for them hearing that from a potential employee.
I do contract work part time. I’ll do six months of contract work, and then when I get the itch to actually do game development, I’ll go back to game development. And go back to contract work and flip-flop. If I find a really interesting project, I might do a year contract. Or if I want to do game development - right now I’m in the game development phase, I’ve laid out my next eight months, and that’s what I’m going to do. And then hopefully, go back to “contracting” afterwards.
Len: That’s a really great story, and it sounds like a pretty cool position to have built for yourself.
I was wondering - I’ve been interviewing Leanpub authors for a while now, and they’re mostly developers or involved with software one way or another. One thing I’ve noticed is that about half studied computer science. I think you graduated about 10 years ago?
Amir: Yeah, December of 2006.
Len: Knowing what you know now, if you were talking to you about to graduate from high school, would you recommend studying computer science?
Amir: I think personally I needed that structure. I loved programming from the get-go. Around 2001, the emphasis on having a computer science degree was much, much heavier. Especially if you didn’t have a public portfolio. GitHub didn’t exist. You couldn’t publish something to an App Store.
I think in this day and age, if someone was like, “Oh I want to get into computer science and programming,” The first thing I would ask them is, “Well do you have an app?” Or, “Show me your website,” or– ‘Cause it’s so much more accessible these days. So I think if I was to graduate high school now, my answer would have been, “Build something, get something out there and show me what you made.”
It’s weird, because you have some more formal professions, like nursing or accounting or business, that I think you really need to have really strong fundamentals that are taught in college. At least for the kind of software that I build, you can hack a lot of things together and get away with it. If I was really interested maybe in like compiler design or distributed systems, or network protocols, I don’t think I could get away without having a formalized education. But at least for the business apps that I build, the crowd apps, product pages, games - you could probably get away with just building something, and then just being hungry from that perspective.
Len: One person that I interviewed recently who’s in software testing said that he would still take four years to study and build things, but he wouldn’t do it at university. It’s this concept of taking a chunk of time at the beginning of your career that you devote specifically to becoming something that you’re not yet.
Amir: Yeah, and recording that.
Len: It’s still an important phase to go through.
Amir: I think another important thing is to just document that in some way, to show to people. So when they say, “Oh you don’t have a degree,” it’s like, “Well, I have 10 apps in the App Store.” Or, “I built these websites.” Or, “I bootstrapped this company,” or, “I helped this mom and pop shop set up a checkout center.” And all those things perk up ears, especially from a junior dev, when you hear that kind of stuff.
Len: Turning to A Dark Room, or ADR as you call it. There’s so much to talk about. It’s been written about on the Huffington Post and the New Yorker and places like that. And you’ve got this great post on Reddit as well, where you talk about what you did, in addition to your book of course, which we’ll get to in a bit.
But I wanted to ask you, there’s a really interesting feature of your success with the game, which is that there’s no reason you can attribute to its taking off. As you said at the beginning, it just went viral. It’s interesting, because playing A Dark Room, unexplained things just suddenly happen, and change the game for you, right?
I was just wondering if you wouldn’t mind talking a little bit about what’s that experience like? When you work on something, it’s been out there for a while, and then inexplicably, it’s suddenly a hit. I mean, what’s that like?
Amir: So the short version is, you’re not prepared for it. I don’t think anything I’ve done in my life could’ve prepared me for getting 20,000 downloads a day for effectively a little bit over two weeks. You wake up in the morning, and you see, because you can go to iTunes Connect and see how many downloads - and you’re at the number one spot, you’re like, “Oh, 20 000 downloads.” It’s like - I’m going to get a 6 figure check from Apple this month. And it’s just surreal.
I lost sleep when it went viral at the end of March in the UK. I have some ideas about how some of the virility occurred. It went viral in the UK at the end of March, and then it took 12 days for it to go viral in the US, and then it stayed there for 18 days. Then it finally fell from the number 1 spot. I did a release, and then it went to the number one spot again for two more days. During that time period, I didn’t sleep. It’s really difficult to put into words.
Len: Can you maybe explain a little bit about, like, how did your day change? Were you getting inundated with messages from people and requests from the media?
Amir: The requests from the media came when I hit the number one spot. That’s when Huffington Post and New Yorker and those people started contacting me. And of course, you say yes to everything, because you’re like yes I’m going to try to push this strain as long as I can. And then you wait.
For me - I actually wrote it in the book as an analogy - it’s like you buy a lottery ticket, and you’re watching the TV, and they give you five of the six numbers. And all five of the numbers are correct. So you might make some money, right? You’re like, “Alright, I’ve got this much in the bank.”
But that sixth number is where your life changes. It’s where you don’t have to worry about that kind of stuff anymore. And the TV cuts off on the sixth number, and you have to wait for some other media outlet to tell you what that sixth number was.
For me, during that time period - I’d do the math, and I’d say, “Okay, this app needs to stay at the number one spot for 62 days, and I never have to worry about money again.” Obviously that didn’t happen, but for that 18 day period, that’s what I was mentally going through. To stay grounded and realistic and say, “This is going to end,” or “This may not happen,” it takes mental effort to not get away with yourself.
Every moment, like every 3 minutes I was checking the app store, “Am I still number one? Am I still number one? Am I still number one?” That didn’t go away during that time period.
At the number one spot, I got about 300 reviews a day. Just reading the reviews, it was nice. It gave me something to do, while I’m toiling away going, “Oh gosh, what’s going to happen?” You can’t release a new version of your game, because you don’t want to lose your reviews, and you don’t want to rock the boat. So I’m in this limbo going, “Okay, I can’t release a new version. I can’t make any changes. There’s bugs that have been found, and I can’t do anything about that. So I’m going to just read reviews and refresh the App Store page, because I have nothing better to do with my time right now.” It’s like that kind of feeling.
Len: You refer to negative reviews in your Reddit post. I was wondering, just thinking about it generally that nowadays with social media and stuff, when you become well known for any reason, it seems to me everyone just suddenly encounters negativity. It’s just a feature of the experience. I was wondering if that happened to you. Did you encounter people saying, “This is a fraud,” or, “This is BS,” or anything obviously, totally, false?
Amir: Yes.
Len: What was your experience like with that, and did you do anything specific to deal with it?
Amir: That was interesting. Negativity is like death and taxes, you’re going to get it. Especially when you’re in the limelight, and you’ve got that big target on your back.
So what happened in the UK – The UK market is, I would say a fifth of the US market. Hitting the number one spot in the US took 20,000 downloads. Hitting the number one spot in the UK took about 5,000. Coincidentally, getting above the fold, like the top 10 games, is significantly less also, right? In the US, it probably takes about 4,000 to 5,000 downloads to get to the number 10 spot in games. In the UK, it was only like 800 to 900 downloads to get there.
What happened was, I got to the 800 to 900 downloads, and I was above the fold. That’s what slingshotted me up to the number one spot in the UK. Up until that time period, my target category was RPG. So the reviews were generally good, because people wanted an RPG, and they downloaded an RPG and that’s what they expected.
But suddenly, I’m in the top 10 games with real games. You’ve got Minecraft just above me, and I’m like, “What’s going on here?” And people - it changes. Suddenly someone downloads the game that’s looking for, I don’t know, like just a regular arcade game. And they’re like, “What is this crap, how did this get here?” So they left one star reviews.
The interesting thing with the review system is the way the reviews are sorted, it’s by the number of stars and then by length. But you hit some threshold where the length becomes more of a factor in the review. I got a few people leave like these really long, one star rants, and they became the top reviews on ADR’s page. It created this herd mentality. Suddenly people that would download this game that had this negative connotation to the game already going into it. They’d play it for a minute and go, “This is stupid. Everyone’s right.” And then they would leave their one star rants.
It snowballed. It got to the point where about 33% of my reviews were 1 star reviews. Initially that was really rough. I would go on Twitter and search for people talking about A Dark Room and try to directly address their issue, saying, “Look, this is a real game I promise. Look at all the things I’ve done, look at my blog. This is not a fake scam game.”
But I think, in retrospect, you can get used to it. Even on the Reddit posts posts that I did, there were comments like, “Oh this guy’s just doing marketing for his own game,” even though there was like a lot of good content. Then there was another one, where it’s like, “Why are you shilling your game out? You’ve already made $800,000 or $700,000. Why do you have to shill your game?” You’re lose either way, right? Because if I made my stuff anonymous and said, “I’m not going to tell you the name of my game,” then I would have gotten negative comments saying that, “I don’t believe you, you’re just blowing smoke up our skirts with regards to what you’ve done.”
So you lose either way. I think with regards to coping, you just get kind of used to it. After the first drove of negative reviews, I found that I was still getting positive reviews. Tou just get used to it, and it rolls off your back after a while.
Len: That’s a really interesting story. You say that you even got negativity about posting these long, really helpful, detailed pieces of advice for people to succeed in the App Store. Even then you get negativity. But what also happened is, I saw someone had tweeted about it. And then, well I mentioned him already, but Marc Andreessen retweeted it. I haven’t had experience with success in the app store or anything like that, but one thing I’ve remarked is that, if you succeed genuinely, smart people, all the smart people, see that.
Amir: Yeah.
Len: And so the negativity is coming from - no reason to say the negative term, right? But it’s coming from not those people.
Amir: Right.
Len: That’s one way at least I’ve found of dealing with it. It’s like there are some people who are just not reachable. That’s just what that person is. If they’ve got a problem, the reason they’re negative, and in the way they’re negative in the first place, is because they’ve got some kind of underlying problem. Thinking about it that way, it’s like, “I can’t solve their problems for them, right? This isn’t my problem. This is their problem.” Anyway, I find that that sort of helps.
Amir: Another thing that I think helped quite a bit for me was, I try to practice empathy. It’s funny, because I was like 28 when I started the sabbatical. And I turned 30, and I don’t know, something clicks in your head. You suddenly stay young, but become more mature from that perspective. So I think back to [when I was] 27, 28. I was like, “Man, I was an idiot and asshole back then.” And then you grow older, and you’re like, “I can do better at it just being more empathetic.” So you see some of the negative reviews, and you’ve got to just think to yourself, “Man, they might be having a bad day, or they might have tried doing the same thing I did and really, really struggled with it.” They just have some resentment that came from their own struggles. So there’s no telling what they could be going through in their life. And I think that really helped also, to ease that.
Len: That’s interesting; that’s another sort of conclusion I’ve come to about dealing with people through a product. It’s that you don’t know what their day’s been like. You don’t know if they’re at a bad point in their life. You really don’t know anything about them. And so someone who’s relating to you negatively might - yeah, you become a lot more empathetic.
One thing I’ve also found too is that treating everybody uniformly in a friendly manner - if there’s a negative interaction, and you come back with that empathy, almost every time you actually get an apology. I think that might be a little bit different on Twitter, because Twitter’s so short form.
Amir: Yeah, the brevity, it just leads to–
Len: But definitely dealing with people over email - in my experience in a number of projects - maybe they just skipped lunch?
And then sort of again, there’s sort of another way of coping with it, where their issue is their issue, not mine.
On the other side of things, what about the positivity? You must have been elated, and it must have been different having people contact you with like, “Hey, here’s my game.” What was that like? Did you have any kind of impostor syndrome thing happening?
Amir: I’ve struggled with impostor syndrome. Even with the new games that I build, I’m like - I can’t top A Dark Room. I’ve got the prequel out there. For those that have read - Flatland by Edwin Abbott (it’s on the Gutenberg Project) - if you haven’t read it, it’s about an 80 page book, and it’s about a two dimensional shape that lives in a two dimensional world. It’s actually a satire around the Age of Versailles, the Versailles time period. It’s told from the perspective of a mathematician, about how this civilization interacts with each other, and how women are treated, and how different caste systems are created, within this two dimensional world.
I built a game around that, that’s my third game I’m building. And I’ve tried desperately to say, I need to get out of the shadow of A Dark Room because I need to prove myself that this wasn’t just a one-shot wonder, or one-hit wonder. So I struggle with that impostor syndrome all the time.
But man, the emails - a lot of that stuff - it is the most rewarding thing I’ve ever experienced. I’ve done a lot of open source, especially during my corporate time. You get occasional thank you’s and, “We really appreciate what you’re doing,” and stuff. But I’ve gotten emails with people saying, “I loved your game, I want to get into game development.” I actually got reviews in the App Store, there was one review on - and the reason I mention A Noble Circle was because it was on that review - the review said, “Today was my 17th birthday, and I don’t think I would have made it past today if it wasn’t for this game.”
How do you internalize that? He was on the verge of saying, “I was going to end my life, and this game kept me from doing that.” When you see reviews like that, it just really puts things into perspective, that you could potentially be changing someone’s life.
Another thing with A Dark Room is that it’s actually playable via voiceover. So people that are blind can actually play the game. It hit the number one spot, and I got an email from a brother who was saying like, “Hey man, I really appreciate it, that you made the game accessible to the blind. Because my sister now, at school, can actually talk about the game with her friends. And she doesn’t feel left out around that kind of stuff.”
So those kind of things - I hate talking about it, I feel like I’m bragging about it, but those kind of things are just immensely rewarding. You can’t put a price tag on that kind of stuff. It’s been humbling, really humbling from that perspective. From what you were saying, with being genuine - I try my best, absolute best to be genuine with all my interactions and whatnot. Hopefully that shows through whenever I write or talk; even in my games, I hope that shows through.
Len: I certainly think it does.
I’ve got a particular little hobbyhorse about how, often when someone has had a success, and they’re asked about it, the story ends up being, “Well I was working in the movie store, and then I was directing Reservoir Dogs.” It’s not the best example, because Tarantino actually has explained how that happened in detail. But often people will tell their story about how they became a success, and you’ll think back on it and be like, “Hey wait a minute, there’s this huge gap.”
But what you’ve done is you’ve filled in that gap. There’s a really humorous moment at the beginning of your Leanpub book, where you talk about how there’s so many questions to ask. Like, “What should my stack be? What should blah, blah, blah, when I’m getting into developing an iOS app?” And then you go, “Well there are lots of things to choose from, but here’s one answer.” And you just list a place to start. The joke is that it ought to be so obvious that people should do this, but most people don’t.
When you’re desperate and you’re starting out, you’re like, just tell me what step one and two are. Then we can move on.
But if I have to know what I’m doing before I start, then I’ll never start. Anyway, I noticed, between your book and Reddit posts and stuff like that, you’re obviously trying to give people that. You’re giving people that equipment right away, which is very generous and certainly something no one has to do.
Len: You’ve mentioned A Noble Circle a couple of times, one of your games. I noticed that you’ve written about that too. You were publishing it while it was in progress, right? So the first version was kind of, as you say, barely playable.
Amir: Right.
Len: I know that there are in-progress games on Steam, and it’s interesting, because at Leanpub we’re into in-progress book publishing. Even Kanye West is getting in the game. He was in-progress publishing a song.
I’ve never asked someone who’s done this with a game before what that process is like. Do you get a lot of feedback from people? Do you have a plan for your releases? Where like it’s like you set up a surprise release in advance? What’s it like in-progress publishing a game?
Amir: I guess I just realized this idea of incremental development - and this kinda goes back to my corporate experience, because I don’t want to say that my corporate experience wasn’t helpful to me - during my corporate work, I got all the like Agile, Big A Agile stuff and everything, and it helped, because I understood what it meant to actually incrementally build something.
There’s so much pressure for game developers to have this phenomenally perfect release. But we have these facilities to allow us to release incrementally, so why aren’t we using them? That was kind of the idea that I had with A Noble Circle.
So my first version was barely playable, and I made it free. I was like, “Here, this game’s barely playable. I’m going to put it out there.” Then I put a bit of developer commentary at the end, saying that, “Look, the App Store’s broken, or I think it’s broken. We have a lot of ads, we have a lot of in-app purchases, and everyone’s gone for this cash grab without really taking into consideration that you may not need to do that. So what I’m going to do is, I’m going to release a game, and I promise to update it at least monthly. You’ll see these updates come through, and if you get it now, you get it for free. Eventually it’s going to cost money, but right now I don’t feel it’s worth money. So just take this journey with me. Not only are you going to get a completed product, but you’re going to see the story of how I got there.”
And that’s been really great. You get negative feedback, something like, “The jump mechanic’s stupid,” or things like that. I listen to that and I incorporate it. But then you have people that say, “Yes! There’s another update, I can’t wait to try it out, and see what he did differently, or what changes did he make.” They love that aspect of it. because they replay the game and go, “Oh yeah, he added this new thing.” Or, “I like the changes that he made. They’re already adding a new chapter.”
It’s wonderful. It’s still not done. But I’m charging a dollar for it now. I released a free version, A Noble Circle - Prologue, that had that a much more polished version of the initial experience. There’s an interstitial to say, “Here’s what I think is wrong. Here’s what I’m trying differently. Go buy my other game.” Or, “Go buy the full version if you want to continue the story.”
I hear game developers say, “I’m going to start a Kickstarter.” But I quasi did that myself. I just did my Kickstarter in the App Store, and said, “If you like what I did here, buy the full version, buy A Dark Room buy The Ensign, you have different options. And if you didn’t, no problem. I mean, you didn’t lose anything, hopefully you played a little bit of it, and maybe you can give me some feedback on that.” But generally speaking, all the reviews have been very positive. This approach has been really exciting for the end user, and for me too, because I just enjoy doing it.
Len: Can you get refunds in the App Store?
Amir: You can, but it’s tricky. They don’t advertise it, but on the receipt that you get, it’ll say, “You’ve purchased this game”, and there’s two links. You can gift the game to someone, and you can report a problem with the app. If you click the “report a problem” button, you can actually get a refund, and there’s a dropdown list of why you wanted to get a refund. Things like, “I accidentally bought it,” or, “It wasn’t what I expected it to be.” Generally speaking they do give the refund. But they don’t advertise it. They say, “All sales are final. No refunds.” But you can get a refund if you go through that process.
Len: That’s really interesting, because as you pointed out, there’s all these tools to publish incrementally. And that can be various types of products. But it’s especially, obviously, suited to digital products. I have this view that we’re working out what commerce is like around things that aren’t finished.
Amir: Yeah, and that’s why I like Leanpub too, from that perspective. I can put something out there that I’ve written. How it started, was I had the blog entry. And I had all this blog information in there. The first step was, take that blog, put it in the book, and then write a postmortem on all the postmortems that I wrote. That worked out really well, because I could reflect back saying, “During this time period, I thought that this was the problem. But this was really what was happening during that time period.”
Len: I actually have a couple of questions about that that I’ll ask in a couple of minutes. The theory that I have is that, especially if a digital good is published in-progress, but also self-published, where there isn’t what you could call a triple-A company behind it, then putting forward the fact that it’s easy to get a refund is a very important step in establishing a positive relationship with a person who’s buying the thing, and the person who’s making the thing.
Because then, there’s this positivity and this openness to it right from the start. It’s like, “We trust you.” Your first encounter with us, is us trusting you and asking you to place some trust in us. It’s like, “you’ve released something early and unfinished, but I’m trusting you that you’re going to do what you say you’re going to do. My theory is that surfacing the ability to easily get a refund for a product you can download actually drives sales, especially in the long run.
Amir: You can see the other side of that coin. I try not to worry about it, but there was an indie game development company that built a game called, “Firewatcher.” It’s on Steam right now. If you haven’t played Firewatcher, you can get it on PS4, you can get it on Steam. I don’t know if it’s on XBOX One. They call it a walking simulator. You’re basically this guy that’s at Yellowstone National Park, and you’re making sure that there’s no forest fires. But there’s this whole story and this relationship that you build with another person that’s also watching a fire in a different quadrant of Yellowstone, and uncovering your own past, what happened to you and why you’re there, what happened. There’s no way to die. There are no objectives you can fail at. It’s basically a walking simulator, where you pick up things and talk on a walkie-talkie with this other person.
What happened on Steam was that it was selling for like 15 bucks. And this person was like, “Yeah, I got this game. I beat it in about three hours, and I want a refund, because I could do something else with those 15 bucks. I mean, I loved the game but what should I do?” The developer actually responded to that entry. At the end, the person said, I’m not going to get the refund. But it’s really weird that even in this digital age, we still haven’t found a way to assign value to something that you can’t touch and feel.
That’s something that that everyone in the App Store struggles with. Especially premium game makers. It’s so difficult. I can buy a $700 device, but I can’t sell a game for 99 cents, without people feeling like they’ve been cheated if they don’t like it. I’m hoping as time goes on and we get more digital things, people just are okay with it. But that’s been a struggle that I’ve seen personally myself.
Len: It’s in that broader class of the way we’re in engaging new forms of commerce now, that have been enabled by certain types of technology. There’s no common sense yet. It’s so new we haven’t had time to develop. Remember how people used to leave their ringers on their mobile phones on, when they were walking around? You basically never hear a phone go off anymore.
Amir: You don’t hear the custom ringtones that they had?
Len: When they first came out, people would do that. It takes a while for codes and conventions to be built.
One thing we’ve definitely experienced at Leanpub is, we surface to people that refunds are two clicks away. That means that yes, if you want to, you can buy a book, and download it, and then you can get your refund. I mean obviously we don’t encourage that. But it’s rare.
Amir: Yeah, and you try not to sweat that either.
Len: Well you can’t, I mean with BitTorrent and whatever. This is my personal opinion. People in this industry obviously have many different opinions about it. As we’re talking about it, people are still working it out.
My view is that, especially if you’re a creator, if you’re creating things, you should spend your time creating things. Even if it were possible to kind of diminish the pirating of your work, you’re not the right person to be doing that. You should be writing your book. You should be thinking about your novel. You should be thinking about your game. You should be working on your song or whatever it is, and just really genuinely leave that out of your head space, and concentrate on being better at what you’re doing.
Amir: For me it goes back to the empathy aspect, too. If someone downloads my book or pirates it, it’s like, “Maybe they can’t afford it.” Even with my game, I had people on Reddit, when I would give away promo codes, say “Hey, this promo code doesn’t work in South Africa. I can’t use it here. And I really can’t buy your game either. Because there’s no really good means for me to buy something. Because I don’t have a bank account or credit card.” Or it’d be a kid: “I’m 13 years old. I get an allowance, I can’t do this.” So sometimes I just give them the benefit of the doubt: there’s a reason why they can’t shell over the cash.
Len: I couldn’t agree more. It’s interesting - the best-selling Leanpub book in 2015, in terms of copies and revenue - which are two separate calculations - had a minimum price of zero. The book that made the most money, you could get for free.
I like to say, this isn’t us being precious. It’s a serious business model: giving that flexibility to people is actually beneficial. Definitely, my personal belief is that spending time and energy in attacking people is really bad. That is counterproductive activity.
Amir: It’s just not worth it.
Len: It’s interesting you’re saying about how they kind of hide refunds at the App Store. It used to be, when I was a kid, if you like got your remote controlled car for Christmas or your birthday, and it didn’t work, and you took it back, you had to engage in a negotiation, and prove that you didn’t do it. You stand in line, and they treat you in a sinister fashion, like maybe you were trying to hoodwink them or something. The last time I returned something was at a Best Buy or something, and they’re just like, “Okay, got the receipt?” And then I handed it over, and they gave me my money back. They saved themselves so much money, and made a happy customer, by not being jerks about the exchange of money and goods.
Hopefully we’ll get to the point where it’s just common sense that being negative to everybody - like DRM on an ebook for example - that treating all of your potential customers like you think they might be bad actors is just a terrible approach. The people who are kind are going to win the hard ass war.
Just moving on actually, I wanted to ask you a couple of questions about your book.
Surviving the App Store is a great title by the way. You did a really interesting thing where you had this very early release, where you had a promotion, where you can get it for a big discount if you buy it now, but there isn’t any content in it yet. I was wondering how that worked out?
Amir: It worked. It was the craziest thing. I did the free thing. I had just one page, and it was like, “This is kinda what my table of contents would be.” I had a whole “Hello World”. I set the price to free, and $4.99. I tweeted about it, I had the cover. I did my cover in Photoshop, and I’m terrible at it. You can actually see the progression, I’ve got some tweets with the old cover on there. I’ve got some feedback on that too, which was cool.
But I put it out there, and I was like, “Somebody talk me out of this please.” Somebody talk me out of writing a book. And then right off the bat, I got like insta buys. I got like five or six people paying me the five bucks for the book. I’m like, “You guys are crazy. I don’t have any pages, and you’re buying my book. What’s wrong with you?” And then from there, I was like, “Okay well, let me take my blog. My blog is still up, so you can read at least the initial response that I had. It’s like a 14 month diary of everything that I went through. So you can still read that.” And I was like, “Okay, let me copy and paste that in there, in the book. And then read through it and then have a postmortem on my postmortems.” That was the first exclusive content that I added to the book.
I kept it free during that time period, and just said, “Here’s the update to the book.” I kept on getting good feedback and good sales from it. It’s surreal that I can sell a promise - that I built up at least some credibility that says, “I promise to do this. I have no pages. Give me $5.” And people did it, and it was just awesome. It’s a really great feeling.
Len: I was wondering actually, I mean the answer might be in there, but why did you choose to use Leanpub rather than any of the other platforms you could’ve used?
Amir: I think that was the one that I heard the most about. The GitHub integration was my primary point - that I could just have Markdown files and have the GitHub stuff. It just worked for me, so that’s the model that I went with.
Len: Okay so the Markdown was a selling point?
Amir: Yeah, Markdown and the GitHub integration was the big selling point for me. And a part of me was like, “Oh I can make the repo public”, but I decided to keep it private, during my editing phase and whatnot, and they can get the book. If you really tried hard, you could probably go through all my Reddit entries and glean some aspects of it, but at that point it’s like, email me and I’ll give you a free copy of the book, just do that.
Another interesting angle that I had with the book was - and I’m still playing around with it, especially if I get some more traction - is to keep it free. Right now it’s $4.99, the minimum price. But I might make it free again. Because it’s free publicity for my games, and that’s where my income really comes from, is the games themselves.
Is there a service that I can set up, where if you email me your iTunes receipt, and I see that you’ve bought my game, that I could give you the book for free? Or some variation of that? There’s no public API to get sales information from Apple. Those are the weird things that I’m thinking about right now. How can I drive more sales to my game? And then you get a lot of the - the 90%. The other 90% for free. So just letting my wheels spin on that. I think it’s doable. I think it is actually really doable. But I just have to see if I can figure it out.
Len: That you could do it the other way around, where if someone bought your book, you could give them the game for free?
Amir: I get 100 promotion codes per release. So I could do that 100 times, but that’s where it stops, right? I was thinking, I could just start up a daemon process and send AWS instance where you email this daemon process, and then it passes the receipt. And then that gets you a coupon code, gives you the coupon reference, so that you can download the book for free.
And then you start thinking about, well they might be able to forge the receipt. And it’s like you can’t start thinking about that kind of stuff, you just have to concentrate on those things.
I think I emailed you before about this - a means to say, have you subscribed to my newsletter, and get a discount on the book, and just facilitate that through Leanpub - that’d be great. There are different ideas there too. I use MailChimp right now, and trying to figure out ways to integrate potentially - if you subscribe, then send off this subsequent thing to give you the coupon code for the book. Yeah, different ideas there too.
Len: My last question related to that is - the theme of your book and your posts is dealing with Apple, as an iOS product maker. I was wondering if you could just, for a couple of minutes, talk about what’s that experience like?
Amir: It’s definitely, don’t call us, we’ll call you. There’s two sides of it. The editorial team is this untouchable group. Developers don’t have direct access to them. So I have a liaison to the editorial team that will advocate for me. The good thing about these liaisons, they’re human beings. You can talk to them and send out a quick email saying, “Hey man, I’m doing this.” You don’t have to worry about being really formal with them and everything.
But it is basically a black hole. You send things over, and you get very little feedback coming back on the other end, outside of the Thursday during lunchtime in Cupertion where you get to see if anything really happened. But I guess, they’re in a position where they can do that.
Len: When you say editorial, is that both approving the app or a new release, and positioning in the App Store?
Amir: This is strictly for featuring. Curation of featured content. As far as the ranking system, it’s pretty much completely calculated off of the number of downloads you have. So from what I’ve seen, it’s about a four to five [day] moving average of your downloads. So getting 600 to 700 downloads a day on the premium App Store, will put you in the top 150 games. Getting 6,000 to 7,000 downloads a day in the free App Store, will put you in the top 150. You could calculate it very easily, if you have the entire range – which I do!
But yeah, the entire feature process is 100% curated. There’s no automatic engine. There are people that look at apps, interact with developers. Of course interact with Triple A companies. And they figure out what they want to feature, and what they want to push. They try to curate that content. I think that’s where the value is in the App Store. I haven’t done much Google Play yet, I’m going to - I’m porting the game over right now.
But as far as Apple goes, yeah, there’s a lot of work that goes into getting good content out there. And frankly, it’s good content. So when you see something featured, it’s worth buying. I can say that pretty strongly. At least if it’s not a Triple A company, right? If it’s some unknown company, if it’s not SquareSoft, Kim Kardashian, King, Warner Brothers - buy it. You’ll enjoy whatever it is.
Len: That’s really great to know that detail, and that they’re doing a good job over there. Sometimes, as a non-expert consumer, it’s hard to know.
My last question is, if you could ask us to build one feature or solve one problem that you encountered at Leanpub, what would you choose?
Amir: I think it’d be the newsletter thing. If I can collect an email for a discount - huge. I think that’d be pretty big for me.
Len: So what do you mean? Can you say maybe a little bit more about that? Because currently we have a feature where, when someone’s buying a book, they can choose to share their email address with you. And you’re saying you want something more than that.
Amir: Right. If they share their email address, I give them a discount.
Len: Oh I see, yeah got it, understood. Yeah, that’s something we’ve thought about before.
Amir: That’d be really cool from my perspective. Because I’ve got the blog, and the more people that I have on the blog, I can subsequently have different entries around game development. I can talk about my Twitch channel if I want to start that up, screencasts, and of course, cross-promote my games. So that’d be pretty valuable and I do believe I give good content outside of the book.
Len: That’s really interesting actually, on the theme of things that have changed because of technology and the types of products that we can create and sell to people, and that people can buy. There’s a certain category of author who comes to us thinking that by default you get a customer’s email address, which to me is just like, that would never occur to me that, if I’m buying a book that the creator of it is going to get my email address.
Amir: Yeah, that’s personal.
Len: Yeah, exactly. Like I said, we’re working out the codes, right? There are some people like - especially people who live in the newsletter marketing world -who often are like, “That’s what I’m building, my email list.” We’re hearing things about Bernie Sanders’ email list basically is this hugely valuable thing that the Clinton team is trying to get him to hand over and stuff. There are people who think from the email out, right? It’s something we encounter. But definitely giving people the option to share their email address with an author is something we really like. Because we want to encourage people to establish a relationship with each other.
Amir: And out of all the subscriptions, only one person has opted out, right? So everyone’s been really good about giving their email.
Len: On that though, you actually have to opt in.
Amir: Yeah, yeah. Sorry.
Len: That means everybody but one person has actively opted in. So that says a lot!
Thanks a lot for taking your time to talk about this. It was really interesting chat. And thanks for being a Leanpub author too!
Amir: Oh I love it.
