The Leanpub Blog: On Writing, Publishing, Self-Publishing and Ebooks
published Mar 09, 2017
Mike is the author of three Leanpub books, Python 101, Python 201, and *wxPython Cookbook. In this interview, Leanpub co-founder Len Epp talks with Mike about his career, his books, and his experience self-publishing on Leanpub.
This interview was recorded on October 25, 2016.
This interview has been edited for conciseness and clarity.
Len: Hi, I’m Len Epp from Leanpub. And in this Leanpub podcast, I’ll be interviewing Mike Driscoll. Mike is a computer programmer, who writes the popular blog, “The Mouse Versus the Python,” at blog.pythonlibrary.org. He has been programming in Python since 2006, and has been a technical reviewer for Packt Pack Publishing since 2009. He also blogs occasionally for the Python Software Foundation.
Mike is the author of three books available for sale on Leanpub, Python 101,” Python 201, and wxPython Cookbook. *Python 101 was written mostly for beginners learning how to program in Python 3. Python 201 is the sequel to Python 101, and is meant primarily for intermediate level Python programmers. And wxPython Cookbook is full of great tips and tricks for using the wxPython toolkit.
In this interview, we’re going to talk about Mike’s professional interests, his books, and his experience in self-publishing, which is really interesting. So, thank you, Mike, for being on the Leanpub podcast.
Mike: Thank you very much Len.
Len: I usually like to start these interviews by asking people for their origin story, so I was wondering if you wouldn’t mind telling us about your path to becoming a programmer, and the kind of work that you’ve done since?
Mike: Sure thing. When I was in high school, I decided that I really wanted to get into computer graphics. And then I realized that I wasn’t that great at drawing. So I decided that instead I’d look into computer programming. And the first two years are really rough. I took computer science in a community college, and didn’t really get it. And then in my third year, everything just started to click. And I was like, “I can do this, this is going to rock.”
And then the dot com busted, and I couldn’t find any work. So I started doing website work for an auction company, and eventually got hired by local government to work in Python. Basically I learned Python on trial by fire. They said, “Figure it out, or get fired, basically.” And I learned it and excelled at it, and that’s been my career ever since, is doing Python.
Now, while I did do the Python, I started realizing that to help me cement it into my brain, I could start writing about it. So I was using blog software - it was kind of a brain dump at first, to help me remember this is how I did something in the past - and I don’t want to forget that later on. Which frequently happens if you don’t use a library for a while.
So that’s why the blog started, and eventually my readers got to be so many and so numerous, and they just all wanted to know, “Hey, can you start turning these into books?” And that’s kind of how the book writing got born.
Len: One thing I like to ask programmers in interviews is, if you were starting out today, would you get into it the same way that you did? For example, would you go to college and study computer science now? Or would you start another way?
Mike: Well there’s definitely pros and cons to each path. I think you have a much more structured way of getting into programming if you go the college route, because you’ll get the algorithms and the math that you need, most likely. On the other hand, if you take the path of running the programming language itself, I think you’ll better understand the language fairly quickly, because you’re learning it while you’re using it.
Most of the time in the classes that I took, you had a semester to work on, really, rather lame programs or projects that really didn’t teach you a whole lot. It just taught you the concepts and the syntax, but not how to actually use the language. So, like I said - there’s pros and cons.
You definitely need the algorithms, if you’re going to get into like, [an] engineering degree. You need the math for that too. But if you want to learn a language - a lot of the time, I think just diving in and starting a couple projects to figure out the language, is probably quicker at picking up the language.
Len: And was there something that happened in your learning, in that third year, something specific that happened that helped you to turn the corner? Or was it just a gradual build of knowledge and experience?
Mike: I believe I was taking a C++ class, and something about the way that the instructor taught it just clicked with me. And I think the previous two years, all those concepts and knowledge that I was learning - everything just kind of melded together. And I became, “Oh, this is how it all works.” And I started understanding all those abstract concepts, and how to apply them.
Len: I was curious how your blog became so popular. Was it organic growth, it just started happening? Or were you promoting it various places?
Mike: Originally I believe it was just organic. I ended up joining a place called Planet Python, which is an aggregate for Python blogs. People would occasionally tweet my articles, and post on Reddit. But for the most part, I didn’t really advertise that much. And then eventually I started adding - I think it’s called TweetFeed, which is actually going out of business, unfortunately, this year. They would automatically tweet my articles for me, so I didn’t have to always remember to do that. But other than that, and a couple of postings to Hacker News, I don’t do a whole lot of advertising myself.
Len: That’s really interesting to know. You write in Python 101, near the beginning, about how learning the basics isn’t enough. And in your book description you talk about how you switch perhaps more quickly into intermediate-level content, than most beginner books might. I was wondering if you could talk a little bit about that? About why learning the basics isn’t enough?
Mike: Sure. I’ve read a lot of Python books and a lot of programming books. But I’ve noticed in a lot of these beginner books - you learn the syntax, and then not how to apply it. So when I wrote my own book, I wanted to get past just the basics. I wanted to get into the stuff that’s actually kind of interesting. Interesting to me, and hopefully interesting to my readers.
The other thing that I noticed that was missing from a lot of these beginner books, is that they don’t tell you how to write a program, and then distribute it. So if I wanted to write a module and distribute it to the rest of the Python community, most books don’t tell you how to do that. Or I want to create a program, and turn it into an executable or an installer for Windows, for example. Most books don’t tell you how to do that either.
So I decided, I’m going to fill that gap, and show how to get into Python quickly. Show them what kind of libraries there are already in the standard libraries - you don’t have to install anything. And then show you how easy it is to install stuff, create your own modules, how to distribute them - and then create your own executable at the very end.
Len: I was wondering if you wouldn’t mind talking a little bit about what the wxPython Toolkit is, and your involvement with that?
Mike: Yeah, so the wxPython Toolkit is a cross platform user interface. It’s a wrapper around wxWidgets, which is a C++ library, kind of like the Qt toolkit. The nice thing about wxPython, is that it typically will take the native widgets on whatever OS it’s written on, and actually use those widgets, instead of drawing a facsimile like Qt does. So what you end up doing, is you actually have the very native widget on each platform, and it looks correct. And there isn’t any kind of weird or wonky-ness you’ll sometimes see in like TkInter or Qt.
I got involved in it, because I was actually converting some VBA code - that was on top of Excel and Access into wxPython. And so I learned how to do that. Then, I worked in the community a lot, and they helped me figure out how to use the toolkit. And then I just gave back, by helping out other new people who wanted to use it as well.
Len: Helping out is a theme in the work of yours that I found online. You’ve got screencasts, for example, for your book, and videos helping people out. And your first book, Python 101, which I believe you published in 2014, had a Kickstarter campaign associated with it, and for your subsequent books. I was wondering if you could talk a little bit about that? What was your experience like starting a Kickstarter campaign for the first time?
Mike: It was quite interesting. I’d never done that before, but I had supported some Kickstarter campaigns in the past, and it occurred to me that doing the Kickstarter campaign would be a good way to gauge whether or not my book’s contents would interest other people. So I created a table of contents, and posted that as part of my Kickstarter, and explained all the different parts I was going to cover in each Kickstarter, and why I thought they were important.
Kind of like I just told you why I wrote the first book the way I did. People really responded to that, and by far, Python 101 brought in the most backers, and the most funding of any of my projects. I really enjoyed that process. I got to learn who my readers were, what they’re interested in, and what kinds of things they might be interested in in the future. They’re always giving me feedback through my blog or by email as well.
Len: I noticed that, like many people who publish books on Leanpub, you have a section at the beginning of your book, where you include an email address for people to contact you, and you ask them for feedback. Has that method worked for your books? Do you think it’s helped improve the content in your books?
Mike: Yes, I think so. Most of the time they don’t contact me through that method. They actually contact me through my web - through my blog.
I do get contacts through those email addresses as well. And I’ve had several bug reports mentioned. Most of the time they’re just silly typos. But occasionally there’ll be an example that I just didn’t test well enough, and I have to modify. But overall, it’s been a really good way to connect with my readers.
Len: You mention in, I think the video on Kickstarter for Python 101 and for Python 201, that you intended to do some advertising with some of the funds that you raised. And I was wondering if you could talk a little bit about your experience advertising your books?
Mike: Well for, Python 101 I didn’t get to do as much advertising as I wanted. But for Python 201, I’ve done a lot of soft advertising. I’ll do promotions through my blog mostly, but also send out emails to my email list that I’ve gained over the years. And I’ve also been looking into trying to find some targeted ads, [that I could use on] Facebook or Twitter.
But my research has indicated that a lot of those methods don’t return investment very well. You have to use like less than 10%, for example. So if you had $1,000 funding, you’d want to use maybe $100 to advertise, to get any kind of return on investment using those methods. And that doesn’t get you very much advertising using that kind of quote or quotas or percentages. So I haven’t done too much that way.
But I discovered something kind of interesting last week, where I posted a promotion, where I give away my second book for free for 48 hours. And that brought in a lot of new readers.
Len: I was watching that happen. You got something like 16,000 readers in just a couple of days. It was really great to see. I wanted to ask you specifically about that. Was there a famous person who tweeted about it? Or was there something special that happened that you noticed?
Mike: I don’t believe there’s any famous people. In fact, when I originally did this, I was just going to post it on Reddit - and there’s a Python subreddit on there. So that’s where I posted it. And I think that’s where the majority of the people came from, from what little analytics I can gather through my previous publishing websites. But it just kind of organically grew from there. And then I decided to also promote it on my blog. And I do get a lot of people going through my blog. So I think the two combinations brought in a lot of readers.
Len: You also actually have print versions of your books available, or at least of your first two books. I was wondering if you wouldn’t mind telling people how you went about making those print books? I believe they are for sale on Lulu and Amazon.
Mike: Yes that’s correct. When I wrote Python 101, I was using something other than Leanpub to create them. I had a home brewed script written in Python, that would actually generate the books into PDFs and MOBI and EPUB versions.
Well eventually, I needed to figure out a way to put the PDF up on Lulu to generate the book. And so I’d have to create versions of the book that didn’t have the cover, and sometimes other information, because Lulu doesn’t want the cover part as part of the PDF. You have to upload it separately. So I just stepped through that process with whatever book I was using. And occasionally, you had to generate the book and cut off the front, and get Lulu to process it. And then add it on on the back end.
Overall, I think it’s gone pretty well. They have a global reach program, which allows you to publish through Amazon and Barnes & Noble and a bunch of other websites, as well as internationally. And they have really stringent rules. So if you don’t put in the ISBN on the right page, they won’t accept it. There’s lots of little gotchas that I didn’t know were even there, that I’ve had to cross, and make sure that I always add it on if I recreate the book again. Just little things you don’t even think about.
Len: And what have your print book sales been like?
Mike: Not that great really. The print books usually sell a handful a month. I probably get maybe 10 to 50 the first month a book is released. And then two or three per month after that. So the bulk of my sales are obviously electronic. People want the digital versions or the Kindle versions, on Amazon for example.
Len: You’ve both written technical books, and you review them as well, and I was wondering - I wanted to ask you what your opinion is about, if you have one, about trends that you see happening in technical book publishing, going forward. Where do you think the industry’s going to be in 10 years?
Mike: The big trend I’ve noticed lately, is that the big companies like O’Reilly and Apress are going for niche markets. They’re writing really targeted books. So, for example, O’Reilly put out a SQL alchemy book, which is just for the SQL alchemy library in Python. Packt Publishing really does this a lot. Where they have all kinds of books on scientific computing, Pandas, Django - they even have a Turbo Gears book. They just have all kinds of little sub-library books, and you don’t normally see that.
Well like 10 years ago, when I was first starting this, this journey, you couldn’t just go out and buy - I want a book on this little library that has 10,000 users, or whatever. And now you can. And that seems to be the trend. Because even O’Reilly’s doing that, and some of the other big companies are trying to do that as well.
Len: Your books are available for sale through your blog on Gumroad, on Lulu and Amazon, and also on Leanpub. I was wondering if you wouldn’t mind talking a little bit about why you decided to also publish on Leanpub?
Mike: I’ve seen other authors through Planet Python that have mentioned using Leanpub successfully. And I thought that opening up another channel of revenue would be worthwhile to try. So that was my main impetus to trying them. And so far it’s worked out pretty well. And I actually like the way that Leanpub generates the books.
I think it looks a little bit more professional than my home brewed version of generating the books. So I’m probably going to start cutting Gumroad out a little bit. Just because I think Leanpub does a better job of tracking sales, and the book quality seems to be a better than what I’ve done myself.
Len: One of the techniques that our authors have for increasing sales, is to make packages on Leanpub, so they sell their books along with their videos. And that’s a trend that I’ve seen. I think O’Reilly specifically bought a company - I think in Canada, last year or the year before - which is all about making videos. And I was wondering if there’s a reason you haven’t made a package on Leanpub with your screencasts yet? Or if that’s just something you’re not interested in doing.
Mike: Actually, I was going to do a package, but I didn’t realize I could use the videos. I didn’t think the screencasts could go with… I thought Leanpub was mainly for books. I didn’t think I could combine the videos with the books.
Len: Oh, so that’s our fault for not communicating well enough. Yeah, you can actually create packages with - you can sell the book along with digital content. So that can be code samples. It can be spreadsheets. And it can also be videos. One of our very popular data science books is sold in a package that’s doing quite well, along with some very large video files, that are really popular.
This packaging together of text content and video content seems to be one thing that’s becoming quite common in the technical publishing space.
One thing that I noticed right away that you do very well in your books, is you have fantastic covers. I was wondering if you could talk a little bit about how you managed to get such great covers? I imagine you used some of the funds from your Kickstarter campaigns to hire graphic designers?
Mike: Yes, basically. For my Python 101 book I actually funded the cover art myself beforehand. And I actually did that for Python 201 too. But basically, I went and looked for someone who could draw a design that was based on the idea of a mouse versus a python, which is my blog title. And I took that idea, and created this idea of a classroom - Python 101 - where the mice would learn about pythons.
And the first guy I used was really great. I liked working with him, and he did a good job. But then he ran through some bad - basically a bad year, and couldn’t do anymore work for me. So I actually ended up having to find some other artists for my next two books. And I think that’s actually kind of good, because it gives them kind of a fresh look, when you look at the different covers, because each book is done by a different artist.
So for, Python 201 I hired a nice Russian lady to draw the art, and she did a really good job, and got lots of compliments about that. She’s actually done work for another book of mine that’s not published yet, but I’m looking forward to announcing next year.
And then the other lady I got for wxPython Cookbook is from the Ukraine, and she also did a really good job. But as you can see, if you look at each of the covers - they’re all very different in their styles.
Len: Yeah, I’ve got them all up on my screen right now. They are very different, but they share the quality of all being excellent. Was there a specific service that you used to find them?
Mike: The first artist, I actually ended up asking my brother, because he knows a lot of artists. And he just recommended this guy. And for the other two, I went on Adobe’s website. And they run a program that you can look up artists on. I can’t remember the name of it off the top of my head. I found it, the website was called Behance, it’s an Adobe affiliate. Basically artists can go on there and show their work, and people can contact them through Behance, and find out if they’re available to do commissions.
Len: Thanks for that, that’s really useful. I’m sure a lot of the self-published authors listening will be happy to hear about a source of high-quality covers. I mean it really does make such a huge difference, I find, for sales.
My last question is - if there were one feature we could build for you, or one thing we could fix, or something we could improve, or something that we’ve missed that you would like us to do on Leanpub - what would that be?
Mike: The one feature that I’ve noticed I need for my current book is image scaling. I’m doing a lot of screenshots for the wxPython book, and doing it with restructured text, I can tell a program to rescale the image any way I want to. But there doesn’t seem to be a way currently with Leanpub, to actually scale it programmatically. So I have to find a different way to do it. And the ways that I’ve tried so far, haven’t worked very well, so….
Len: Okay, thanks for that. I’ll communicate that to the team. Images is an area where we know that there’s more work we can do to improve things, and that’ll be really good feedback.
Thanks very much Mike, for doing this interview. And for also using Leanpub to publish your books. We really appreciate it.
Mike: Well thank you for creating such a wonderful service, and for having me.
published Feb 21, 2017
Alex Lancaster is co-author of the Leanpub book Python For The Life Sciences: A gentle introduction to Python for life scientists (you can also learn more about the book here). In this interview, Leanpub co-founder Len Epp talks with Alex about his career, his books, and his experience self-publishing on Leanpub.
This interview was recorded on November 7, 2016.
This interview has been edited for conciseness and clarity.
Len: Hi, this Len Epp from Leanpub, and in this Leanpub podcast, I’ll be interviewing Alex Lancaster.
Alex is an evolutionary biologist, engineer, writer and consultant based in Boston. He completed his doctorate in Computational and Genomic Biology at Berkeley, and has worked in R&D in the broadcasting and IT industries in the US and Australia. And he’s also helped research in faculty positions in academia, including a research position at the Whitehead Institute at MIT, and a faculty position at Harvard Medical School.
Along with his colleague Gordon Webster, Alex is co-author of the Leanpub book, Python For The Life Sciences: A gentle introduction to Python for life scientists. The book serves as an excellent introduction to computer programming for biologists, including those who have never written a line of code.
Along with the book, you also get copies of code samples that you can learn from and adapt to your own specific research.
In this interview, we’re going to talk about Alex’s professional interests, his book and his experience self-publishing through Leanpub.
So, thank you Alex for being on the Leanpub podcast.
Alex: Thanks Len, happy to be here.
Len: I usually like to start these interviews by asking people for their origin story. I know from your bio that you studied both physics and electrical engineering before you got into evolutionary biology, and I was wondering if you could talk a little bit about your path through all these disciplines, and how you ended up at Berkeley?
Alex: Yeah. How long have you got?
I started life thinking I would be an astrophysicist, basically. It was where I was originally when I was an undergrad. And actually I spent about a week in a radio telescope down in Canberra - a while ago now, shall we say? Another century. And I realized that that wasn’t really going to be it for me for the rest of my life. Astrophysics has changed a lot since, but there was a lot of sitting in very quiet, desolate places, pouring over data, and it sounds very glamorous on the outside - but the reality of the day-to-day just turned out that it didn’t really appeal to me.
So, trying to figure out what to do, I decided to finish my engineering degree, which I started with. But I was always interested in evolution from a very young age, I think [from] when I picked up Richard Dawkins’ The Blind Watchmaker, which was written sometime in the 80s. I was just fascinated with the idea of these biomorphs, which were these little creatures that he had built evolutionarily on a Mac. It was nothing to do with real biology, but it was very - basically you could construct these creatures from this very simple genetic code. And it sort of always stayed with me.
So I always sort of followed along, thinking that if I could get training in physics, then I could move that over to biology at some point. But I didn’t want to go back to academia straightaway. So after I’d finished up my undergrad, I went and worked in the software industry for about four years.
I started as a design engineer at the ABC - which is the Australian Broadcasting Corporation in Australia, it’s like the equivalent of BBC - in their R&D section for a while. And I cut my teeth on coding. I did a little bit of hardware stuff, but I rapidly realized that software was where it was at. And the web was growing, and - those were very early days.
And so I basically sent myself around the world doing software contracts. I went to the UK for about a year in the mid-90s. I worked a little bit in the banking sector, a little bit in the telecommunications sector, building my tool bag. But I always had this idea that at some point I’d come back and do grad school.
And then, sometime in the late 90s, I decided - I stumbled across - well I actually I hadn’t knew about it before, this place called the Santa Fe Institute in New Mexico. I had been following what they’d been doing out there, and they are sort of really on the cutting edge of complex systems and biology and all that stuff. And I thought, “Well that’s a great place - I should somehow get myself a job there.”
So I ended up downloading, I think, a very early version of the Swarm software in the late 90s, and basically playing around with it. I ended up moonlighting - while I was working in a bank, on their website - building these models, computational neuroscience models, with some folks that I knew in Australia, that I had found in Australia. And back then it wasn’t as easy to find collaborators, so I had to go to conferences and chase down physical papers and stacks and stuff like that.
I helped build a bunch of models for those folks. And in doing that, I learnt this software package, called Swarm. And then it turned out there were a couple of job openings. So I applied, and I got it. And so I found myself in Santa Fe, New Mexico in ‘97, not knowing a soul, wondering quite why I’d gone there. And I was part time in a PhD program at UNM, University of Mexico in Albuquerque.
And I was doing that for a while, trying to do it part time. But I was really having a lot more fun at the job, working at the Institute. So they hired me as one of their software developers, and I was sort of able to work with a lot of researchers. That solidified my interest in biology, basically, in moving back towards the evolution side of things.
I ended up postponing my program, and then I reapplied, ended up going to Berkeley, and studying population genetics and theoretical models of biology for grad school. But all the way along, I was interested keeping up my software skill. So I always had a foot on the computational side, and a foot in the biology.
Then I ended up doing the standard academic track, of doing a postdoc. I did a couple of postdocs, and then I was faculty at medical school briefly for a couple of years. And I just decided that the way - well, a lot of things have happened since I’ve moved here, but one of the things that have happened in academia, as you may know - it’s become a very tough environment to do more “out there” research.
And in the way that I see academia going for probably a while now - it’s really accelerated in the years since the crash, I think. So I decided that it would be more interesting to try my hand at some kind of hybrid career. And at some point, Gordon and I met, and we kicked around this idea of creating this company. And we really got going about a year ago. And that’s what bought us to Amber Biology.
That’s probably an overly long-winded answer to a shorter question. But that takes us right up to the present. I’m happy to go back into any of the eddies that you found interesting there. That’s how I got here.
Len: That’s a really great answer, thanks very much for that. I’ve had a bit of a - in conventional terms, I suppose - meandering career myself. So it’s really interesting to hear from someone who follows their curiosity, where it takes them. Which is, it sounds like, what’s motivated you.
Alex: Pretty much, pretty much. I’ve never been one for really mapped out career plans. And in a funny kind of way, I think the way things are moving now, that notion of the career plan is becoming somehow less relevant. But that’s something we can definitely talk about if you want.
Len: I have a doctorate myself, in English, not in biology, but I could talk about academia and things that have been happening there forever -
Alex: I’d love to hear your experience in that too.
Len: But actually I was wondering - when it comes to academia and the sciences, this is a topic that one sees in the news recently, about the difficulty that young scientists have getting tenure, and the importance that’s placed on getting published, regardless of necessarily the relevance of the publication. Is this something that you’ve had direct experience with?
Alex: Yeah. I would say that it’s more than just the people getting tenures. People getting the tenure track positions in the first place. The bottleneck is I think even greater there. And you have a lot of very highly trained, highly motivated people who are competing for a very, very limited number of slots.
The slots are certainly not increasing, and if anything, they’re probably decreasing, because universities are oftentimes cutting their budget, and they’re often looking to supplement the people that they do bring on.
It’s one of these things, it could be overstated, but I certainly think that at the level of the higher administrations, there is definitely a push towards finding sort of faculty and research areas that are sort of highly fundable, because a lot of the costs of running a university have been sort of shifted toward federal grant money, especially in United States, and so that puts a lot of pressure on those administrators. And that gets translated down into in the department. And I think now in the department levels it’s probably - the picture’s a bit more mixed, because I think that most people there really want to hire people that are doing interesting things. And I think that, in general, most people want to do the right thing, and are interested in intellectual balance, and the usual things that academia’s known for. But they find themselves under a lot of pressure.
So I think that that combination… it’s sort of a system, the pressure to publish in prestigious journals, and how they’re ranked in terms of grants, has a tendency to factor into the grant making decisions. And so that feeds back to the faculty. So there are a set of interlocking factors, you might say, that drive the system towards a setup where you want to minimize risk and maximize return.
That militates against people doing more unconventional and risky approaches. And it also militates against doing smaller scale, and actually cheaper research. Which is sort of a strange thing, because oftentimes these little side rivulets can be the things that can actually drive science forward. And you really don’t know where the next big discovery’s going to come from. So yeah, definitely the publishing part is part of a larger network of problems, but it’s definitely a big driver.
Len: It’s really fascinating to me, to watch, in North America, what I call “admin creep,” like mission creep, but happening at universities, where tuition costs are rising and rising. The cost of running a university is increasing. And yet there’s this budgetary pressure on professors and scientists, and people doing research. And so costs are going up, and yet there’s this squeeze. And it’s -
Alex: No, absolutely. And in fact, we were pretty much talking about this for almost a full day on Saturday at the Ronin Institute’s first Unconference. I don’t know if you might have seen that on my blog?
Len: Oh no, I didn’t see it, but I know what an Unconference is. Actually if you could describe that, that would be good, I think.
Alex: So the Unconference - I’d actually never done one myself before, but it was pretty cool. The idea is that the topics and the areas that get discussed, and the talks are effectively self-organized by the participants. And the way that they did it on Saturday, I was loosely involved in organizing it.
There were three seed speakers at the beginning. They spoke for about 10 minutes each. And then that generated a list of things in people’s heads, and people would write down on a piece of paper a topic they’d like to discuss. And then you put those pieces of paper around the room. And then people walk around and identify the things they’d like to discuss.
And then we proposed topics that were similar, you kind of merge. And then out of that, we got about three distinct groups. And we have a discussion for about an hour and a half, I think, or an hour or so. Really interesting group of people. And then we break for lunch, and then it’s a repeat in the afternoon, then summarize it at the end.
It makes for a really interactive kind of format, as opposed to a traditional conference, where everybody’s like half paying attention, and on their laptops and that kind of stuff. It was perfect for the kind of thing that we were trying to do, which was to really generate a robust discussion around the future of scholarship in general. And not just in sciences, but in humanities as well - to try and think about ways that we could do things that don’t necessarily involve the traditional kind of institutions that we’re using.
So that was big, that was great, because we got - there were people, as I said, from like theology and English literature. And there were a lot of biologists represented there, because they tend to be over-represented in the Boston area. But yeah, that was really interesting.
Len: That is really interesting. Do you feel that there’s a pressure building to push people to a model of education that’s not university-based?
Alex: That’s an interesting question. That may happen. I mean, I think there’s certainly room for a lot of different paths to getting knowledge, that don’t involve going through the ivory tower. I think there’s sort of a realization - I kind of get the sense, that sort of realization that, when you get a scarcity of positions, or a scarcity of - what’s the right word I’m trying to think of?
Basically a scarcity of educational good, shall we say? Everyone’s so focused on getting into sort of colleges, so they can get top jobs, and so on. And I think what happens is that people attach a sort of monetary value to that - to that luxury good. And then oftentimes, that becomes the goal rather than the education.
I always assumed for many years that that was a byproduct - that should just be a byproduct of getting education. But I do think there’s a lot of pressure to getting, sort of, credentialism, I call it. I don’t want to be the pot calling the kettle black, beause I played that game too. But I realized the limitations of that sort of thinking. And so I do think there’s a realization that maybe having everyone go to college in the standard way won’t necessarily work for everybody and may not be desirable in all cases.
Especially with the student loan thing. I feel like that’s the next bubble, that’s the next scary bubble that people haven’t really confronted - is the student loan crisis in the United States. Because if everybody’s told they have to get this kind of education, but then they go through life saddled with all this debt, that they always feel they need to get the kind of job that can support that debt. Then that cuts down on career options that people can pursue.
Len: Yeah, and very crucially, one of the interesting aspects of student loan debt in the States, which is over a trillion dollars, and greater than credit card debt, is that you can’t -
Alex: Can’t go bankrupt.
Len: You can’t go bankrupt. It’s incredible. I was listening to a podcast interview by Ezra Klein, with Joseph Stiglitz recently - the Nobel Prize winning Economist. And he was saying two of the most consequential decisions that the United States made in the last 20 or so years, is - one was that if a company went bankrupt, rather than the people working for it - having the primary claim upon the assets, it was people who held derivatives in the company.
And the second one was that if you go bankrupt as a student, you can’t clear your debts. And this can happen. It can happen where, if there’s a student debt that’s associated with a parent, the child can actually die before completing the degree. And if the parents go bankrupt, they can’t clear the debt. It’s very perverse. And when you add into that the importance that’s placed on the rank of the university that you’ve attended, and when you think about the pressures that are on someone who’s 17 or 18 - You know if you don’t get into one of these universities at this age, you can still get ahead in life, but you feel like you’re behind. And you will be, in a sense - in the conversation, behind, your whole life - you really will be, if you don’t get on that track, if you don’t go to Berkeley, or Harvard or something like that. And the pressures are extremely intense.
One of the topics that comes up on this podcast, just because of the type of people that often publish Leanpub books, is, if you want to become a computer programmer or software engineer, developer - should you go to university in 2016?
I was wondering what your opinion is about that very specific question - if your goal is not to get an education, not to become an educated person, but to be a developer and work, do you think that people in the States, should get university degrees in computer science?
Alex: It’s a good question. My sense is - not, if that’s just the one thing you want to do. Like if you know that’s the one thing you want to do, and that you can educate yourself in the areas, in other ways, then I’d say that probably - at least it shouldn’t be a necessity. I mean I do think that it’s kind of silly to force people into…. the more general answer to that is we’ve still got a kind of one-size-fits-all system, that doesn’t really take into account the nature of people’s individual, quirky career paths. There’s an expectation that there’s a set of norms that you follow. And if you’re off those norms, then you’re probably a little bit weird, and you’re probably some kind of person who’s failed.
Which is kind of weird. Because at the same time we laud all the college dropouts - like Bill Gates and Mark Zuckerberg, for starting all those things. But at the same time, they’re kind of the exception that proves the rule a little bit. I feel like, in general, I would say also - growing up in Australia, I never felt that same pressure in the same way. Partly because the culture is different. Now things have changed a lot in the 20 or 25 years since I was an undergrad. But there was never a feeling that -
There were always options, if you really wanted to do those super-professional things. But there was a sense that you could get ahead if you didn’t finish school or you didn’t go to grad school. Maybe you went to college, but you didn’t really necessarily want to build a career in whatever thing that you studied. But it was a period of your life, and it didn’t define you in the same way that I feel that it defines people here. Or at least it feels like they’re defined by that experience.
That’s a part of a cultural difference. And also the fact that we do have student loans, but they work totally differently. That’s, again, a long winded answer. But yeah, I would like to see a world in which we didn’t push people into career paths which they either don’t want, or aren’t really a necessity. Just a general openness to like people finding a different way to whatever their passion is.
Because I feel like - ultimately - that’s the thing that matters. And that’s the thing that’s going to make people productive members of society. It’s not to say you have to do it this way. But, figure out ways to support what they do - rather than sort of pre-defining it for them, and map it out. Because things are changing so fast anyway, that I feel like almost any career advice these days is going to be like five years out of date.
Len: And where did you grow up in Australia?
Alex: I grew up in Sydney, in the suburbs of Sydney. I was there until like the mid/late 90’s. And I went to the UK and I came back. I bounced around before I come to the United States.
Len: You did the walkabout.
Alex: I did the walkabout, yes. And we’re sort of known for that - we tend to like go overseas and then come back.
Len: When I was living in London, I always had an Australian roommate. Which meant I always had at least three Australian roommates, because people would always be visiting. Including parents on the couch for two weeks - that kind of thing.
Len: But I saw - who was it? Paul Rudd. Not Paul Rudd. [Note: This might be Len’s funniest gaffe ever - eds.]
Alex: Kevin Rudd.
Len: Kevin Rudd.
Len: I worked for Macquarie Bank for a couple of years in London.
Alex: Oh really? I worked for Macquarie Bank for about six months.
Len: No kidding.
Len: At 1 Ropemaker Street?
Alex: One right at the stock exchange there in Sydney.
Len: Oh, in Sydney.
Alex: Yeah Sydney, yeah not–
Len: Oh pardon me, I worked for them in London. But I had my training in Sydney. Yeah, that’s funny.
Alex: I worked on their first website ever.
Len: Oh really?
Alex: Yes, I was on the team that helped build the first Macquarie Bank website. It was about ‘96 I think?
Len: That’s fantastic.
Alex: Running Perl objects and stuff like that. Yeah, it was an interesting learning experience too. I realized that banks weren’t really my future at that point.
Len: Yeah, I realized that after two and a half years. It took me a bit longer. But it was an exciting experience. Especially working for an Australian bank, trying to make its way in London. It was quite curious.
Alex: Which side of the banking pot were you on?
Len: Investment banking. I was doing mergers and acquisitions. It was pretty interesting. In fact, actually quite a few of my colleagues - one of the curious and great things about working for Macquarie, was that people weren’t - there were fewer business school graduates than you might get at other investment banks.
Alex: Oh that’s interesting.
Len: People were - for example - one of my colleagues, who was brilliant, had done maths and had gone to London on a holiday. And the job he got was being an investment banker for Macquarie Bank, or Macquarie Group, as it came to be known.
Len: I remember the word “holiday” played a role in his passwords and stuff like that. But there were a lot of people - a chemist from Perth and people from all over, from all kinds of different backgrounds. This was in the mid-2000’s, and it was a really interesting, interesting time. I’m sure things have developed a lot since then.
I wanted to ask you about Swarm - it was one of the first open source agent-based modeling tools. I was wondering if you could talk a little bit, for people who might not know what those are, what that is, and why Swarm was important?
Alex: Swarm came originally from folks, before my time, at the Santa Fe Institute in New Mexico. And there, there were… I sometimes call it a think tank. They don’t like to use that term. It’s a small, private, non-profit research institute that’s dedicated to what they call a science of complex systems.
And that, in practice, means people building computational and mathematical models of all kinds of systems. From natural systems, like biological systems through to model the economy. And even to things like archaeology. And so you might think of it as a sort of set of language primitives that a lot of these models are built in. Sort of in the language of agents.
Where, rather than like a lot of typical mathematical models, where you sort of have an equation that describes the aggregate behavior of a bunch of individuals, using say differential equations and things like that - you build models where you represent the actual individuals, as code. And the natural representation is - in computer science terms, is that of an object.
And so, there was basically a push. People basically found they were building code structures that were almost identical to each other. And then there was a realization that, “Well maybe we should have sort of a common platform that we could reuse, and then build our domain-specific stuff on top?” So you’d have a library that you would then write your code for your model in, that would call functions from that library.
And that’s basically how Swarm came about in the mid 90’s. It went through a number of iterations. But I wasn’t involved in the prototype. I came into the project a little later. And we were at the point then where we were now interacting. So my job was actually interacting with the scientists at the Institute, and visiting people to talk about the science, and think about how we could translate that science into the model, and then work with the other people. Building the kernel to create the right infrastructure.
So it’s kind of a bit of a translator between a lot of different disciplines. To try and sort of figure out, okay, how do we represent these things, that would work for the largest number of people, and the largest number of kinds of scientists. And then also figure out at what abstractions you would want to use, that’d be generally useful, and what abstractions that are very specific - say economics or sociology or things like that.
The fundamental notion is that you have a bunch individuals that all interact. And they have a set of rules, and they have state. It’s like SimCity or something. You set the things up, then you sort of let the thing go, and you see what happens.
So Swarm was the first tool kit to do that. It’s inspired several others. And there’s still the SwarmFest meeting, that we started back in the late 90’s, it’s still going strong. It’s been for about 20 years, and I hadn’t been for 10 years, and I went again - the first time in many years, 10 or 11 years - two years ago. It’s sort of great to see that the community is still sort of out there, trying to push the boundaries. Because it’s still, in some sense, it’s still kind of a little bit on the edge. I am surprised that it’s not more mainstream actually.
Len: And why do you think it’s not more mainstream than it is?
Alex: Probably because models are complicated. I think, there was probably some early overselling. This happens a lot in areas of science where everyone’s very excited. They start making promises that they probably can’t deliver. And so there’s been a little bit of a backlash, to complex system approaches in general and agent-based modeling is one of them. So I think that had some role.
They are more complicated to analyze than just traditional models. You don’t have the same set of tool kits you can use to do sensitivity analysis. And I think it sort of dovetails a little bit, with the problem with academia in general. There’s just not as much - I feel like there was more appetite in the late 90’s, to just try new stuff in general, in science.
It could just be that I’m getting older. Or it may be in my head. But my sense is that it’s harder to do stuff that’s a little on the edge now. People really want to see something like return on an investment, whatever that means in science.
I think it’s unfortunate. Because I still think that it’s important, for people doing stuff, to do stuff that doesn’t always work, and might fail. Just because it hasn’t worked yet, doesn’t mean it will never work.
Len: It’s interesting that the theme of stress in academia’s coming up. Because it’s something I think about a lot. I mean - Einstein goes for a walk, and sees a workman on a scaffold, and imagines him falling down, and has a great insight that changes the world. You can’t possibly quantify Einstein going for a walk.
There was something that happened a few years ago in the UK, called the Research Assessment Exercise. Where basically, an incredible amount of professors’ time was wasted in assessment of work, under the futile illusion that you can quantify research. And you end up with the people who should be doing the forward thinking, subordinated to - to put it crudely - politicians who are trying to make a point to people who are skeptical about higher education fundamentally.
It’s very cart before the horse kind of stuff.
Actually, that’s probably a bad metaphor. It’s kind of like - people who really don’t have an appreciation for what happens at higher levels of research - being skeptical about it, because they don’t see progress. And you need to give people the time to have the revelation in the shower. And to pursue paths that may actually ultimately be fruitless. Because that’s what cutting edge thinking is.
I was actually wondering, what was the work you did for your thesis, for your doctoral thesis back in the day?
Alex: I kind of switched gears a little bit away from the purely agent-based stuff when I was working in immunogenetics, or immune system genes - it was actually fairly empirical. I was looking at all these data sets coming from different populations around the world, where they would go up and genotype sets of genes called HLA genes - human leukocyte antigen. They’re basically involved in the immune system, and they’re the things that help detect - like when you have a bad pathogen that’s invaded your body. And one of the questions that have puzzled evolutionary biologists, and people who study population genetics - why this region of the human genome is incredibly polymorphic. Why are there so many alleles in different variants?
And so, I was working on trying to quantify that, that nature of that variation, and then building tools to analyze it and see if we could basically measure the strength of selection at the level, not just in the whole gene itself, but also at the level of the individual residues, amino acids in the actual 3D structure of this molecule.
So, I’m going off on a tangent here. But it’s effectively trying to figure out if we could use the population data to get at functional questions about how evolution has shaped the sort of nature of these molecules. And so there were some statistical analysis, and a fair amount of coding. I built a pipeline for that, that’s still used today, called PyPop - Python for Population Genetics.
I was also developing this methodology for figuring out like how to use things like Monte Carlo, Markov chain stuff to better analyze this data - it’s basically on that sort of interface. That is really quite classic computational biology meets evolutionary biology. It’s where I was at. And I’ve still got colleagues at projects that will probably see the light of day eventually.
But it really got me interested in the - it sort of cut my teeth on learning one specific biological system really in a lot of detail. Because the complex system stuff is great, but you can often find yourself going off into abstract speculation. So I feel like even though it’s not really what I still do on a day to day basis, I still think it was a valuable, for someone like me - I will always like to be trying new things, it’s good to have the training in one area. I think of it like the fox and the hedgehog, if you’ve heard that analogy? Which is why I call one of my blogs, “the curious foxhog”, because it’s like, I feel it’s good to have deep training in one area, but at the same time it’s good not to get tunnel vision. So yeah, that was where I came in on that stuff.
Len: And is that related to evolutionary systems biology? Which I’ve read you’re involved in - or is that something different?
Alex: It’s part of it. After I left my grad program, I started working in models of evolvability, but specifically related to prions. And it turns out that prions are this interesting mechanism for storing variation that can be released when an organism’s in a moment of stress. It’s kind of a fascinating, and I think of that as a classic example of evolution systems biology. Because systems biology, I think of it as like the mechanisms and the networks that are ultimately sculpted by evolution. And that’s why - I think ultimately, people that are trying to integrate sort of….
Evolution for a long time was very - evolutionary theory’s very abstract, and it didn’t refer to any sort of real systems. Just they would have these models about, consider these two alleles and play around with it. But now we have a lot more data, and we can say things like, “Well, now we know this trait is generated by these networks. What are the different evolutionary paths that you might take? Will the system take?” And so that’s kind of what I think of as evolutionary systems biology.
But my PhD wasn’t really in - it didn’t really exist even as a discipline. But, again, it was the training I needed to get into that area later.
Len: This is kind of a selfish question, because it’s a preoccupation of mine, but what do you think of evolutionary psychology?
Alex: I don’t really think about it a lot. I used to read a lot about it back in the day. I’m a little bit skeptical of it in general. I’m always a little bit skeptical of explanations that involve pre-defining our notion of what’s fit in the environment. And I think the problem that I see with evolutionary psychology, or some of it anyway, is that it tends to overestimate the role of competition and the survival of the fittest side of evolution.
Whereas evolution as a whole, includes all kinds of - not just competitive processes, but cooperative processes, and symbiosis and mutualisms. And all these rich dynamics that I feel with some of the evolution psychology stuff, it’s a little on the simplistic side.
It also interfaces with these arguments that come more from the political and economic side, that can easily be used to justify some of the existing power structure…. There’s always a danger when you go into nature and say, “Oh look, it’s done this in nature, so it must be - that must be right” - you know what I mean?
Alex: That’s my main problem with some of the evolutionary psychology. Having said that, I don’t have a problem in principle with studying - using evolutionary principles in all kinds of areas. In economics and psychology. I really like the stuff that David Sloan Wilson works on, which is integrating economics and evolutionary thinking, complexity thinking. I’ve been reading his Evonomics blog quite a bit lately, and that involves psychology.
But yeah - so as far as the classic evolutionary psychology stuff, again I haven’t looked at it in a while. But I felt like at least the version of it in the 90s, early 2000’s, was always a little bit iffy to me.
Len: The fundamental question I have about it, is how do you do experiments? If you can’t do experiments, it’s not science, and it seems, to put it crudely, and so you see things - like for some reason, I think probably now, the former editor of the Science and Technology section of The Economist, loved evolutionary science just-so stories. And you see this in the science press generally.
I mean to think they actually did this in The Economist once. But they said that they’ve proven that women have a preference for pink. And that this is cross cultural. And that it’s probably because when, in the olden times, in the long, long ago, women had to search for berries. And so they were selected for experiencing pleasure when seeing color in nature. And this was in like the 2000’s. This wasn’t in like 1810. This was in like 2010. [See the last paragraph of this article from 2007 - eds.] It just seems like - I mean, the stuff that surfaces in the press, is obviously going to tend towards nonsense. But how can one possibly run experiments on human psychological evolution?
Alex: That would be my problem with it too. The just-so nature of it rears its head in those circumstances. And the danger is it can easily reinforce people’s preconceptions and scientize something that really shouldn’t be scientized. To coin a term.
Len: That’s a great word. On the subject of science and programming, which is what your book is about, I wanted to ask you how important is it for scientists these days to learn how to program?
Alex: I think it’s pretty important. Especially as… the more quantitative disciplines like the life sciences are becoming rapidly…. But I also don’t think that one should assume that that’s the only thing you need, and that everything should be - that you should drop your pipettes and just do coding. I have done pipetting once but, I know that I’m never going to be a great bench biologist.
I think getting your head around, going a little bit beyond spreadsheets - I mean, we say this in the book blurb - is going to be really important. And it’s interesting actually. Especially with the SwarmFest, I’m meeting a lot of people who are not even scientists, and the digital humanities people are really picking up on the programming side of it too.
And so yeah, I think it’s important. I think also, you have to simultaneously keep in mind that programming is ultimately still just a hammer, and you don’t want to make everything a nail. So on one hand, I’m like yeah, people should get used to computational and quantitative thinking and all that good stuff. But at the same time we shouldn’t get rid of people who work in museums and that love collecting specimens.
There’s room for all of those skills and people. So I get a little bit nervous when people say, “Well everyone needs to learn to code, because everyone’s going to be coding in the future - I don’t think that’s true.
Having said that, I think another reason to code, for everybody - and scientists in general - is that it’s good to know the thinking. Because a lot of the systems that you’ll be interacting with are going to be engineered - which means knowing the fact that there’s code behind that, what does that actually mean, and what the limitations are, just to be a generally scientifically literate citizen.
There’s a great book by Douglas Rushkoff, called *Program or Be Programmed. I don’t know if you saw that? It’s really small. I always give it to people who wonder about programming, because he has a good spiel about, “Yes you should probably code, but you don’t have to. But you should definitely know what’s involved as this becomes more part of your world kind of thing.”
My position is maybe a little bit more nuanced than you might expect, just because I also realize the limitations of the data-driven, metric-obsessed kind of thing that we often can get ourselves into. But at the same time, I feel like it’s good to know those things - know how to analyze and use code, even if you don’t do it as your full time job, it’s for no other reason than you know what’s going on behind the scenes. And you know code is being deployed. So I think from that perspective, I think, yes, people should learn to code, even if it’s not their only thing.
Len: And my last question is - why did you decide to publish your book on Leanpub? I should say - by the way, Python For The Life Sciences is a really great book. A ton of work went into it. It’s really well done.
Len: I mean that. I see a lot of books. I was wondering why you chose to publish it with us?
Alex: We kicked around a lot of different things when we started our consulting firm. And one of the things that Gordon and I both agree on is, we don’t love gatekeepers. We love the idea of people doing things from the bottom up. And so we thought about approaching a publisher and proposing it. But we just felt, that’s going to add a lot of stuff in front of us, and let’s just write the damn thing and see where it lands.
And then when we started looking around - actually a friend of mine on a Slack channel that I recommend, he’s always got cool things, he writes a lot of cool things about programming and science and complex systems, his name’s Bill Tozier vaguery.com is his website - he just mentioned it in passing. I was like, “Oh, I should check this thing out.” And I went over there, and I said, “This is really interesting.”
So we were initially thinking, “Oh well, we’ll just sort of put it up and see what happens.” We really thought that… fit our general ethos. And obviously you have a really great revenue model, which I think is really good. And to be honest, looking at the way Amazon works, and the big e-publishers, they are starting to act more like rentier-type models.
Len: Like what sorry?
Alex: They’re sort of monopolizing the market now. And so they’re able to set the - like able to set monopoly prices. And I would rather support, in general, new, emerging businesses and organizations that are trying to make a way to make a living without necessarily having to create a massive infrastructure. That’s why Leanpub and self-publishing seems the way to go.
And that said - if a major publisher was to pick us up tomorrow, I guess we could continue doing both. But… for example, with our consultancy, we’re trying to build a sustainable business. We don’t necessarily want to be taking over the world. We’re not after world domination, we’re after - earn enough money to keep paying the bills, so that we can do the cool science or research or art or whatever it is that we’re doing. You know what I mean?
So that was part of it. I like it when I see other people doing cool new things. I’m always like, “I want to support that thing.” Even if it means like, okay, I don’t get the massive return in the immediate. I feel like in the long run, we’ll all be better if we do that.
Len: Thanks, that’s a really great answer. In many ways your book is the classic Leanpub book. And I mean that in the sense of books that we love to see. Because I think both your minimum and your suggested price is $34.99. It’s 304 pages. And if you’re in the life sciences or a biologist, and you’re looking to learn to code - I mean, this is a book you get great value from. Definitely worth 35 bucks.
Alex: Right, and we think so too.
Len: One of the interesting things - we’re still learning about this new model of self-publishing. But Amazon decreases the royalty rate that it pays when you go over $9.99 for a book.
Alex: Yeah, we noticed that too.
Len: I mean from 70% to 30%, right? [Note: On Amazon it’s actually 35% royalties for books over $10, not 30%–which is still terrible compared to Leanpub’s 90% minus 50 cents royalty rate - eds.] So they’re basically saying, “All ebooks are interchangeable, and the price should be less than $10.”
Alex: Right, exactly. They drive you towards that.
Len: I think that that’s probably an appropriate price point for novels. And I still think if you are writing a novel, you should have it on Leanpub as well. Because you will make more money, because it’s a 90% royalty. But if your book is worth more than $9.99, and I don’t just mean, because the price is more than $9.99. If it’s actually worth more than $9.99, you should not be publishing in a place that’s meant for novels.
Books like yours, books like so many other Leanpub books - well I mean, not specifically your book, but like for other types of books - they can change the amount you can charge people for the work that you do. Because you’ve learnt something new, and you’ve got skills that you didn’t have before. They can help increase the skill level that you have. And something like that, people are willing to pay more for. And authors should earn more from, I think.
Alex: Right. It was basically - a little under a year, I mean, not full-time, it wasn’t like a 40 hours a week, 7 days, 24/7 type-thing. But we put a fair amount of… and the other reason I liked it, the way we did it was that, because we had the freedom to do it at the pace we wanted, we could write the book that we wanted to write. Not the book that someone else had decided like fit the right market niche. So we could take our time and make it a little bit lighter, and put in jokes and things like that, that might not have passed muster, or may not have been into some other publisher’s conception of what they wanted.
So we liked the idea of having a book that has a bit of personality. Beause textbooks can be kind of on the dry side. But that was another nice thing about it. And also the iterative nature of it, because it just goes with the whole software development philosophy that we kind of use in our own consulting. The sort of Agile type thing of - release it, get feedback, improve. I feel like that’s a really good model to….
And we didn’t fully embrace it. We didn’t do chapter by chapter. But we’re certainly going to upload new versions. So I feel like that that approach is really, it’s still pretty new for the book publishing world still, I think. But you see how effective it is in software. At getting a better, more robust, if it’s done properly of course - kind of result. So that was another thing that was cool about the Leanpub thing. I liked that. I liked that idea that - doing anything creative or anything new. Get something out there, it’s not perfect. Improve it. Improve it. Do it in public. Improve it, get better. I’m learning this, just myself by taking improv classes. The same kind of thing. Like you’re not that great at the beginning, but you - when you get in front of an audience, you get better, so it’s a bit like that. I mean, just like not taking the risk, exposing yourself. People might not like every little bit of it, but that’s okay - you get better at it. So I feel like it fits with that sort of philosophy.
Len: Just before we go, I feel like I would be remiss if I didn’t ask someone living in the United States the day before the big election what’s the mood like in - well you’re in Arlington, Massachusetts - you were saying before we started this interview - just north of Boston, I think. What’s the mood like down there?
Alex: In the greater Boston area?
Alex: I think - what’s the thing, your acceptance, grievance, with the stages of grief. I feel like we’re at the acceptance phase now, with, just in a sense of like, whatever will be will be. Everyone’s just so over it. So much, so many - I don’t know. I can’t read anymore editorials. I can’t read fivethirtyeight.com. I feel like almost everything has been said that is going to be said, and it’s just going to be up to the voters now.
But yeah, people are concerned obviously, I think, definitely in our community. Because we’re around a lot of academics and scientists and people who - what the results will be for all of that. But that’s something I think that definitely comes up when we’re in conversation with friends and colleagues.
Len: Well thanks very much for that, that’s very well said. It’ll be an interesting artifact, because this will come out after people know the results.
Alex: Yeah what the result was. May I ask you what your - what’s the view from Canada?
Len: I could talk about that for a long time, I suppose. I think that probably most Canadians are quite concerned that Donald Trump could win, to put it straightforwardly.
Canada’s a more complicated place than Canadians pretend it is. But the spectacle over the last like two years of the campaign we have one border, right? And it’s with the United States. And to see someone who appears to be driven by no sense of responsibility, and not constrained by reason - within a hair’s breadth of being the President of the United States, is something that I think people are concerned by.
Knock on wood - when this comes out, we’ll all know.
Alex: Hopefully we will be able to breathe that sigh of relief.
Len: I think people here are in a sense, complacent. I mean, we just kind of take it for granted that Americans will make the reasonable choice. I don’t want to speak on behalf of all sorts of people who disagree with me, and generalize dramatically, but like, it’s - we’re watching. We’re always watching what’s happening in the States - we’re definitely watching now. I’m just getting a memory of when I stayed up late in 2000, to make sure that Al Gore won Florida, before going to bed. And woke up to a different world.
Alex: Yes, yes, I remember that election well. I had only been in the States for a few years at that point. And that’s when I started paying attention to politics…. I think it’s true, I gave a massive generalization, to say that academics and researchers tend to try and go about their business, to pretend that like nothing’s going to affect them. Or it’s like, it’s above them, or….
I think this election has certainly made people sit up and pay attention. Because even in the other elections of other Presidents, candidates, there’s a general agreement that probably the NIH would continue, and NSF always - they’re not about to abolish them, but sort of all bets are off with Trump. I think that’s what makes people really scared. Because it’s like, you really don’t know. It’s like a lot a lot of the things that you take for granted, just could not be true anymore.
Len: I hadn’t actually thought about that, but that’s the National Institutes of Health and the National Science Foundation - nd I hadn’t thought about that.
Len: Thanks very much Alex. I guess–
Alex: Yeah on that cheery note!
Len: On that cheery note, thanks very much for taking the time to do this really fun interview. And thanks for publishing your book on Leanpub.
Alex: Oh, you are most welcome, it’s been really fun.
published Feb 17, 2017
Gordon Webster is the author of the Leanpub book Getting Started With Python In The Lab: An Introductory Python Tutorial For Life Scientists and co-author of Python for the Life Sciences: A gentle introduction to Python for life scientists. In this interview, Leanpub co-founder Len Epp talks with Gordon about his career, his books, and his experience self-publishing on Leanpub.
This interview was recorded on October 12, 2016.
This interview has been edited for conciseness and clarity.
Len: Hi, I’m Len Epp from Leanpub, and in this Leanpub podcast, I’ll be interviewing Gordon Webster. Gordon earned his PhD in biophysics and structural biology at the University of London, and has worked with life science R&D in both Europe and the US. He’s currently based in Cambridge, Massachusetts.
He has both academic and commercial experience, and is the author of a number of patents, in addition to scientific articles. In his profile, he writes that his “career path has reflected his belief that the most interesting and potentially promising areas of research lie at the intersections between the traditional scientific disciplines,” and I’m sure we’ll get to talking about that in just a bit.
Gordon is the author of the Leanpub book, Getting Started With Python In The Lab: An Introductory Python Tutorial For Life Scientists, and more recently - along with Alex Lancaster, he is co-author of the Leanpub book, Python For The Life Sciences: A gentle introduction to Python for life scientists.
Python For The Life Sciences is a great introduction to computer programming, written with the interests of biologists in mind - in particular those who haven’t written any code before. Along with the book, you get code samples that you can learn from, and even use for your own research. The book covers topics including biochemistry and gene sequencing, molecular mechanics and agent-based models of complex systems.
In this interview, we’re going to talk about Gordon’s professional interests, his books, his experience using Leanpub, and at the very end, ways we can improve Leanpub for him and other authors.
So, thank you Gordon for being on the Leanpub podcast.
Gordon: Oh thank you for having me.
Len: I was wondering if you wouldn’t mind telling us a little bit about yourself, and what I like to call an interviewee’s “origin story” - how you first became interested in biophysics and structural biology, and how you got to where you are now.
Gordon: I think my interest in biophysics started with seeing three-dimensional structures of DNA and proteins and stuff like that. I remember being very captivated by that intersection of physics and biology. And so I went into biophysics - kind of related to the thing you mentioned a moment ago, about the fact that I really enjoy things that are on the boundaries of two different disciplines.
So the idea of using physics to study biology, actually really appealed to me. There’s a sort of a certain mindset and methodology to physics that doesn’t always work, I have to say, in biology. It’s an incredibly interesting area.
The other thing that’s spurred my interest in biophysics was computers. I remember in the 1980’s, I got a home computer. I was completely hooked from the minute I started writing BASIC on a home computer. All through college, I always pursued projects and electives where I had a chance to do computing. So that’s always been a big part of my career too. Biophysics is a very computational, quantitative, numerically-intensive field, and so the computer stuff has always played a very large part in that.
Len: And what is the difference between what one might conventionally understand to be biology, and biophysics?
Gordon: I paint this picture of a spectrum. At one end of the spectrum of biology you have evolution, and field biology - studying species and animals and the way they interact, and all this kind of thing. And then there’s all the classification, taxonomy and botany and stuff. And then at the opposite end of the sort of spectrum, you have the almost atomic and molecular biology.
I call it the study of dead stuff. And it gets kind of ironic, that when you get to the very small scale in biology, down to atoms and molecules, nothing really looks like biology anymore. Because you’re essentially studying things that are governed by the laws of physics and chemistry.
And it isn’t till you get further towards that first end of the spectrum that I described, where you start to look at organisms and reproduction and survival and evolution, and populations of organisms and the dynamics of those populations - that you see anything that you could really call biology. So it’s kind of interesting that at the very small scale, a lot of the stuff that biologists study really looks like chemistry and physics.
Len: That’s really fascinating. I wanted to ask you, what was the subject of your research for your PHD at University of London?
Gordon: I studied structural and computational biology. There was a great interest at that time in finding ways of shutting off certain gene sequences. And we didn’t have the kind of technology then for developing these, like silencing RNAs and technology that’s out there now of that sort.
People were very interested in looking at drugs that could bind to DNA, and actually close down a certain gene, essentially by binding to the beginning of the gene, or the gene promoter - and shutting off that gene. The goal was always to try to be able to control gene expression, so that you could - for example - cure cancer, or other diseases that had a genetic component.
Len: I’m sure probably some of the people listening to this podcast have heard about CRISPR and how powerful that is. I was wondering, since I’ve got you here, if you wouldn’t mind maybe explaining a little bit about what that is, and why it’s so important.
Gordon: CRISPR is an interesting system - it’s sort of enzymes or a gene editing system that people have found in organisms. It’s not human made, it’s not invented. It existed in nature. And now there’s a number of companies who are trying to essentially patent it, and develop it for use as a gene editing tool.
So the former dogma of biology’s always been that once you’ve established a gene sequence in a cell, that it’s there forever and that there’s not much you can do about it. You can put things into the cell, maybe to switch it off. But then those things need to be there all the time.
The difference with the sort of CRISPR approach, is that now you’re basically going in and looking to edit the genes themselves that are in the cell, so that you’re interfering with the cell’s processes at the genetic level, which is something we’ve not really been able to do before.
Len: And what do you think some of the new applications might be, that people can make of this?
Gordon: I know that obviously people are very interested in disease. So some of the genetic diseases - there are genetic diseases where people are born, for example, without the gene that codes for a vital enzyme for example, that processes carbohydrates in the cell.
There are some people that have deficiencies in processing certain kind of chemicals that are essentially vital to growth and life. And those people often don’t live very long. They often die as children. I know that there’s a lot of interest in trying to fix those genes, whereas previously, all you could do was try to intervene with drugs and things like this.
Now there’s an effort to try to fix those kind of diseases - again - at the genetic level. So that’s something - again - that we’ve never really been able to do before. There were attempts sort of in the 90s. I mean, you probably heard about gene therapy, which was in the 90s. People were trying to do gene therapy with viruses. And viruses also have a very interesting kind of gene editing capability.
So for example, a lot of viruses, when they invade cells, they’ll splice their own gene sequences into cells, and co-opt the cell to produce more virus, instead of producing what the cell wants to produce. And so people thought that maybe viruses could be a way to do gene editing, and a lot of the gene therapy early on, was done with viruses.
And that field is still going. It’s not dead or anything. But I think that the CRISPR thing is an advance beyond that, in terms of having much more control over the way the gene is edited. The problem with the viruses, I think, is that it wasn’t always very easy to control where the virus would put the gene that you wanted into the cell.
Len: My next question is kind of personal, a little bit selfish. I lived in London for a few years working, and I studied in the UK at Oxford doing my doctorate there. And I always thought Oxford was the perfect distance from London. It was just far enough away that it took some time to get there. But it was close enough that you could still go there and enjoy London.
But I always wondered what it would be like, because there’s so many great universities in London - what it would be like to actually be a student, with all the fantastic distractions of London life around you. What was that like, studying, doing your doctorate in London?
Gordon: It was awesome. And you’re right that it was - it was sometimes not easy to - to focus on work, when you had all that stuff. But you have to also bear in mind, I mean - I was there in the 80s. So I mean this was the era when people like [The Clash])https://en.wikipedia.org/wiki/The_Clash_ were playing at the Hammersmith Palais. It was an incredible time to be young, and also to be a student in London. And I absolutely had a marvelous time there. Maybe sort of too good a time, sometimes. But yeah, it was - it was fantastic. I just had a really, really great time. It’s a wonderful city.
Len: I wasn’t there in the same era you were, but like, just going to Camden any given night, you can find fantastic bands playing. It’s just so amazing.
I wanted to ask you about Amber Biology, which is the consulting firm that you have with your co-author, Alex Lancaster. When did you set up your consultancy, and what kind of work do you do?
Gordon: I created the consultancy about three years ago. Originally I had a partner who was somebody I used to work with when I was working more in the mainstream of the pharmaceutical business. He was somewhat engaged at the beginning, but he had a day job and he didn’t really want to give up his day job, and he ended up kind of becoming a silent partner. And in the end, I guess, the company was kind of moribund for a few months. In the end, I persuaded him to relinquish his partnership, so that I could work with Alex, because Alex was very interested in being actively engaged in Amber Biology. And so we had a change of partnership last year.
They finally got all the paperwork through in summer of last year, about when we started on the book as well. And then essentially, we’ve been building the business. The business had been going for three years, as Amber Biology. But Alex and I have been working together for about a year and a half now. So it’s about a year and a half we’ve been doing it together.
And the kind of work we do is all computational biology. So anything you can do in which biology can be done on computers. This includes a lot of things that are - I mean, when you talk about biology and computers, a lot of people immediately think of bioinformatics. It’s the big area that everybody’s heard of. People think about gene sequencing and genomics and gene analysis. And that’s certainly stuff that we do as well.
But both of us have a background in modeling and simulation in biology, and that’s an area that we are really keen to pursue. There’s a whole backstory here, and we can get into that if you’re interested. But I would say it’s still very early in biology for people doing modeling and simulation.
If you think about physics and civil engineering and things like that, simulation and modeling are a main stream of research. In physics, for example, people model the movement of stars and planets, using sort of gravitational models. They plug the observations from telescopes into them.
And then when you have a deviation of the model from the observations, that’s actually interesting. This is an example I like to give. Where models can be wrong, but still informative, and that is that if you’re studying a binary star system - you plug in the Newtonian Gravitational Model, and you find that it doesn’t match the observations.
What that often tells you is that there’s hidden mass there that you can’t see with the telescope, and there are one or more planets orbiting one of the stars. And so the deviation of the model from the observations, gives you a clue as to how much mass is missing and where that mass is.
And that kind of thinking, that kind of mindset of using modeling and simulation is really prevalent in physics and civil engineering, similarly. I mean, you want to build a suspension bridge. It’s going to get built in CAD in a virtual sense, before any steel or concrete gets built in the real world. And then all the pieces get tested in CAD, and there’s feedback from the physical testing of all the pieces of the bridge, back into the computer model.
That’s the kind of place that we would like to see biology go. But it’s still extremely early, and most modeling in biology right now is exclusively the confine of people doing, for the most part, theoretical biology. And those people are often people who have backgrounds, for example, in computer science, and who are doing this kind of thing that you talked about earlier - of straddling different disciplines, and bringing computer science ideas into biology. This is the area that we’re really interested in. But like I said, it’s very early in biology right now.
Len: I really liked that analogy. I found it in something that you wrote - I think - on your blog. I mean, I don’t know if you used this example specifically. But I think it was Neptune was discovered because people saw deviations from the expected movement of another planet. So they derived from the deviations, from the model they had of the way the whole system worked what must be going on. What you’re saying, I think is that biology is, given our current understanding of it, too complex, to have a whole model in the simple way -
Gordon: Right, exactly.
Len: I mean people think physics is really complicated. But even physicists will tell you in some ways that it’s very simple. And it reminded me of the story of Vulcan, which was this planet people thought existed, because they saw deviations in the movement of Mercury that they couldn’t explain. And it took centuries until Einstein, to figure out that there wasn’t - well I mean, people realized from observation that it wasn’t there. There was no planet there causing the deviation, so it must be something else. And then it actually took a fundamental change in the entire model, to understand why Mercury was moving the way it was. And I guess what you’re saying is - biology is so far from even having a kind of model of the first type, in the first place, that getting to that second step, isn’t there yet. You have a blog -
Gordon: Yes, that’s exactly right. And the other issue is that people who have not had a lot of experience with modeling, which is true for the most part in biology - they tend to think of modeling like weather forecasting. The idea is you have this very big, very complete model with, essentially, data points for everything…
All the data are very well represented, very - very complete. And then you run the model and you make predictions. This idea that an incomplete or partial model could be of any value is something that - I think most people in the biological field tend to dismiss modeling, because of these kind of fears. Because of the complexity. Well, how could you model the inside of a cell, because there’s just too many moving parts?
Len: You have a great blog post called, “Big Data Does Not Equal Big Knowledge”. I’m sure everyone’s heard talk about Big Data by now, and I was wondering if you wouldn’t mind talking a little bit about what you were getting at in that blog post. You talk in particular about how visualizations - or the type of visualizations that people often get from data - are not necessarily as useful in the life sciences, as they might be in other fields.
Gordon: With these numerical quantitative approaches, it’s a little like the kind of demographic data mining that political campaigns and advertisers do. It’s like sort of looking at trends in the data. And I think there are lots of areas where that kind of approach works really well.
And in biology, I mean, you can do it too. I think I give the example of dose response curves and things like this. Where you have a relatively simple system with not too many variables operating under the surface. And the problem with the examples - for example, where this kind of stuff has really failed dismally is in areas like gene expression and genomics. So people were sure that once the human genome project was complete - I remember, I think it was Watson, was saying, “Oh within a couple of years of this, cancer will be a thing of the past, and we’ll have a handle on all of the disease genotypes,” and so on.
And really, what we learned from that, is that we don’t really understand the genome as well as we thought we did. So, having the human genome sequence is a bit like having the physical location of any neuron in your brain. It’s still a long way from explaining consciousness. I mean, yeah you could map the brain in the greatest detail, but it still doesn’t exactly tell you how the system works.
And I think with Big Data, what people are trying to do, is say, “Well I don’t really understand what mechanisms are under the hood here. But if I look at the data under one set of conditions, under another set of conditions, and I carefully weight the data, so that I’m not comparing apples and oranges, then basically if I can see some significant differences in the data, those may point to where the problem or where the issue is - whatever the thing is I’m trying to investigate actually lies.”
And it’s a valid approach in a lot of ways. I mean it’s not crazy. And some of the low-hanging fruit has probably been picked in that approach. But, for example, patterns of gene expression, or patterns of phosphorylation in the phenotypes of cells - those things are so complex, there’s so many different moving parts.
And it might be, for example, that what you’re looking for isn’t the biggest difference between one set of genes expressing in another, but maybe some pattern of differential expression, that might be buried in all the noise that you cut out, because you think it’s not significant. But it might be some recurring pattern of 10 different genes, all of which have very small but significant deviations when you look at them all together.
So these are the kind of things that Big Data is trying to uncover. And the visualization thing is also usually - you apply a lot of filters to the data. You try to pull out the differences in the data, in the way that a sound engineer would try to filter background out of a recording. As you were saying earlier about your software for doing audio filtering.
And I think that the problem is that, it’s an effort to sidestep the complexity of the biology. It’s partly driven by this fear that, “Well, I could build a model, but how could I ever build a complete model?” It’s always going to be a partial model at best. And so that probably isn’t going to work.
Len: You mentioned earlier, failure, and you just mentioned side-stepping. That leads me into my next planned question for you, which was - I wanted to talk to you a little bit about Theranos. I’m sure a lot of people listening have heard about this company that’s turned into a pretty catastrophic failure in the health sciences area. I know you’ve written about it on your blog, The Digital Biologist, which is the reason I’m bringing it up.
And I wanted to ask you, how can something like this - if you could explain a little bit about what Theranos is, and how it failed, and how can something like this happen in the sciences? I think it’s a question that a lot of people have. The lay person associates science with rigor. And there appears to have been this huge fraud.
Gordon: Right. I mean, I think the one thing I would say is that - yes - that lay people do tend to think of scientists as being almost kind of like Mr. Spock - that is logical, and everything is kind of decision making, devoid of all that other human baggage like emotion and ambition and greed and all that stuff.
And the truth is, it’s really still very much a human activity. And the application of the scientific method - there’s this kind of ideal view of it, if you look at the books on the philosophy of science and Karl Popper and all this kind of stuff. There’s this very idealized, sort of Platonic ideal of what the scientific method is. But when you start to combine science and commerce, then all that human stuff, it still plays a role. And honestly, it plays a role even in academic research.
There it’s not so much about money, but about prestige and ambition, and people in academic research sometimes stray because they want the result that they want, because they know it’s going to get them that Professorship, or the prize or the prestige or the recognition within the community that they want. And so, the number of cases of academics going off the rails - even over issues about prestige and standing in the community - are well documented.
And when you start to think about that in the context of the Theranos thing, I mean there, the stakes are even higher. You’re talking about ambition and prestige and standing, but also about billions of dollars and entire careers. And so the human stuff definitely plays a role in science.
Len: It’s really interesting. In particular, I remember when I first heard about the company. I looked into it, and I saw that– I mean, this is currently on its [corporate] board - but it’s also, in addition to being very human, and a business with lots of money at stake - it was extremely political.
Currently you’ve got a former Wells Fargo CEO. A company that’s also in the news these days. And a retired Marine Corps general on the board. And on its lists of Counselors, currently it includes Bill Frist, the former US Senate Majority Leader. And Sam Nunn, a former Senator - and Chairman of the Senate Armed Services Committee. And incredibly, to top it all off, Henry Kissinger was involved.
Do you think that one of the reasons they could get away with their self-representation was, all of these powerful people were behind it, and that that may have deterred people from seeing the truth early?
Gordon: Yes. I liken the Theranos problem, and problems that a lot of biotech and pharmaceutical companies have, generally, with the kind of problems, for example, that NASA had. I understand that one time there was kind of a management culture - you had a lot of people managing projects who are driven by deadlines. And as you said, political considerations - who weren’t really engineers, and didn’t really understand the risks and understand the complex systems that they were building.
And the Challenger disaster was an example where this kind of management culture essentially overrode the culture that should’ve prevailed at NASA. Which is one where - in my opinion - for those kind of projects, you need engineers who understand the systems that are being built, and the risks inherent in those systems. Those are the kind of people that should be running the project.
And at Theranos - and not only Theranos, but other biotech and pharma companies too - what you often have is kind of a management mindset where you have people who maybe did an undergraduate degree in science, never really done a lot of research. I don’t want to slam MBAs here, but there’s definitely a - you see a lot of MBAs in high places in the pharmaceutical industry, driving R&D, who’ve never really done any R&D themselves.
And so I think that you have this culture now, where there’s this management culture - people go to business school, they get an MBA. They feel that it makes them qualified to oversee all kinds of human activities - whether or not they really understand the risks and the processes inherent to whatever it is that the company, or the organization is making.
And I feel like with Theranos, you have a similar kind of thing. I mean they didn’t publish any data. Everything was just like radio silence, in terms of actually validating the technology. And they held out for a really long time. I mean - obviously now, we know, thanks to the Wall Street Journal’s reporting - that that was because, essentially the stuff didn’t work.
But it it would’ve behooved them to have taken a more rigorous approach, and known before they had gone down that path of wasting all those millions of dollars - that this technology wasn’t going to work. And somebody else might’ve intervened, or the R&D might have been done differently. Or they might have pivoted much earlier, as they’ve pivoted recently to this new thing, when they’re no longer a provider of blood tests. Now they’re going to be, as I understand it, a developer of hardware for this?
Len: It’s really interesting, what you said about MBA’s and the concept of overseeing. That subject has come up on some of the interviews that I’ve done for this podcast, repeatedly. And I think it’s partly because a lot of the people that I interview are software developers. I could talk about it for a long time. But the theory behind a lot of management, I believe is based on being able to oversee. Being able to see what people are doing. And it’s deep, it’s deep in the structure of the way MBAs are taught. So deep that people don’t even know it’s there.
For example, if you’re managing people laying bricks, or if you’re managing people building a house - you can have a kind of abstraction around watching. You can see whether people are having a pint, or laying the bricks. You can see whether people are hammering. You can hear whether they’re hammering the nails, or whether they’re playing cards.
But a scientist? I mean, you can see her at work in the lab. Or a software engineer, you can see him sitting at a desk with a computer. But you can’t see the work. Because the work is mental. And it seems to me that this is a real problem in an era where software is eating the world. We need a new way of thinking about management. And the thing that keeps coming up in these conversations is, that you need to have domain-specific expertise, in order to manage people who are doing work that involves primarily things you can’t see.
Gordon: Absolutely. And the other thing about R&D generally, is that it’s a very non-linear process. I think the management mindset has arisen out of these kind of industries where you have a production line, and you have this chain of processes from A through Z. And you go A, B, C, D. And then if there’s a bottleneck at D, you fix that bottleneck, so the thing works better. But it’s all very much a a process of box checking and crossing T’s and dotting I’s, and you have a defined process. I feel like that management mindset works really well for that.
So the kind of people who manage well in the pharma business, tend to be more, I feel, on the regulatory side. Where, once the drug gets through this sort of R&D phase, and it’s now being in development and clinical testing, I feel like the process there - t’s still not completely linear for sure. But there’s much more of that kind of production line mindset there.
But R&D - it’s iterative, it’s non-linear. You start an experiment. You see something really interesting. It can take you off in a whole different direction. No amount of management deadlines can mandate that nature is going to behave in the way that you want it to. “Well, we must have an answer from this particular cell line before December, so we can tell the investor something.” And if the cell line doesn’t want to behave as you expect it to, then you’ve got all kinds of questions you haven’t answered.
This comes back to the modeling again. I see modeling as very much an adjunct to this experimental sort of method, where you’re basically helping to determine what the next experiment is that you really ought to do, to answer the questions you need to answer.
I wrote a piece on this on LinkedIn. I likened a good R&D team to a jazz ensemble, rather than an orchestra. And I talk about this quite a lot in that thing too. I have certainly worked at companies where I have seen this kind of management decision-making going on. And it’s really based more on what the company needs to do, but without a real understanding of what the company is actually able to do.
I’ve worked quite a lot in software development too, and I’ve also seen that in software development where you have a similar kind of situation, where you have people who’ve never really written or tested code - overseeing a software development effort.
One company I worked at, I ended up having to leave, because they felt that testing was an unnecessary waste of time and money. And this was not an ideal world, I was told. And in an ideal world, we would test everything - but here, we just don’t have time to do that. And so it all turned really sour for them, because - of course - a lot of the software they were rolling out, just didn’t work - it was embarrassing, and I was embarrassed to be a part of that effort.
Len: I’ve spoken with a couple of professional testers about that very issue, and so I have a little bit of a sense of how frustrating it can be to be part of a project where people are just profoundly mishandling it under a false view of efficiency.
Speaking of writing code, I wanted to ask you - what was the inspiration for you and Alex to write, Python For The Life Sciences, which is a book devoted to helping people who haven’t coded before, to learn how to do so?
Gordon: The biggest single reason is that, well, partly it stems from what we said earlier about the fact that modeling and computational approaches are still relatively non-mainstream in biology. What that translates into is that there isn’t really much in the way of a computing component in the core life sciences curriculum. So most biologists can go through college, and pretty much avoid using computers - other than maybe for writing their articles, and maybe using Excel spreadsheets and stuff like that.
If a few of them are lucky, they may get to have some kind of training in MATLAB, or R or something like that. But for the most part, a lot of biologists graduate and start doing research. They go into grad school, or even become postdocs, without ever having really done much in the way of computational research. And what you see in labs is people doing endless calculations, still with hand calculators. People are using Excel to process all their spreadsheet data, painstakingly copying stuff into tables.
Nowadays, there’s more lab automation. So a lot of the lab instruments produce data that’s ready to be visualized in Excel. But Excel - it’s a great tool for what it is, but it’s not really the tool for most quantitative biology. There are certain things, for sure, you can do with it. But being able to write code gives you the opportunity to look at your data in ways that’s just not possible using hand calculators and spreadsheets. And so that was really our major…
So when I worked, for example, at one pharmaceutical company, I remember that the really the only numerical piece of software in the entire company that was used by everybody from the financial people in the accounts department - to the scientists at the bench, was Microsoft Excel. And that was in a company doing the kind of quantitative work that drug development is. This is the progressive aggregation of knowledge. It just - it struck me as kind of bizarre that in such a quantitative field, where data and numbers are so prevalent, and more so now than ever, that you have so many people working in that field who just have no real way of using computers to their full potential.
Len: And why did you choose to focus on Python?
Gordon: Because it’s, I think in the book we call it, the Swiss Army Knife of programming languages. It’s a wonderful language that you can just start using right away. I trained also in Java, for example. And Java requires you to use the kind of object-oriented paradigm for programming right from the outset. So there’s a steep learning curve there for anybody who’s not familiar with object-oriented programming.
And it’s also kind of a sledgehammer to crack a nut. If you just want to write some small scripts to open a file and read some data in, and reprocess the data in a different format, or find some patterns in a sequence, or something like that, you don’t really need to be writing object-oriented code all the time. So I like the fact that Python gives you that option to just jump in and start writing the procedural code that we all used to write, when we were writing in C and Basic and stuff.
Or for more complex applications, you can scale it up and use that object-orientated programming paradigm, to help you - to organize all the moving parts. And write applications in the large.
Len: You mentioned earlier, and I believe I read on one of your blog posts, that the book took you about a year to complete.
Len: Or to get to the state it did, that it’s in now. Was it your plan from the beginning for it to take a year?
Gordon: No. I think the book ended up being much bigger than we thought it would be. I think it was going to be a little 50 to 100 page thing about biocomputing with Python. And almost like a get-you-going tutorial. But then, it just blossomed, mainly based on both of our previous experiences in modelling and using Python in our own research. And, “Oh, wouldn’t that be cool? Remember when I did that stuff with the robots?” And, “Remember when I did that stuff with next generation sequencing? We should include some of that.”
And so, there was definitely - I guess in the software world, you’d call it feature creep. But we’re very happy the way it. We were glad we did it. It’s much more of a full-fledged book than I think we imagined in the beginning.
Len: It looks great. And I wanted to ask you - you didn’t use the Leanpub workflow to make your book, rather you used our “Bring Your Own Book” feature to upload your book, so you can sell it on our bookstore.
I was wondering what tools you used to make your great-looking book?
Gordon: The entire book was actually built and edited in Google Docs, because we needed a collaborative platform. And I use Macs, and I use Linux and Windows as virtual machines on my Mac. But Alex is a Linux guy. We couldn’t really use something that was primarily in the Mac world as a tool, and so we settled on Google Docs, and it worked really, really well - until we got up to about 250 pages. And then you start to see the limitations of trying to edit large documents in a web browser.
I’ve got to give the Google people credit. Google Docs is a great tool. But once we reached pretty much the maximum size that’s practical for a Google document, around the 300 pages mark, we already started to see that it was unresponsive sometimes.
And the other issues that we had were - when you create a PDF out of the Google Doc, it does some silly things. For example, all of the internal links point back to the original Google document, and not to the new PDF. So if you have a link in your new PDF to page 100, it will actually point to page 100 in the original Google Doc. Which is kind of absurd. I mean, if you’re exporting to PDF, you would hope those internal links would remain internal.
So what we ended up having to do, was to save the entire document as a .docx file in Microsoft Word format. And then we used the Mac Pages program. Well - initially, let me say - I tried using Microsoft Word 2011. Which is the version I happened to have on my Mac. And that does not preserve the links.
When we first published the book on Leanpub, all the links inside - the external links, were dead, because Word didn’t handle those properly. And when we put it into the Mac Pages program, then it did a good job of exporting the document. And also, there were some other issues with Word. The images would stray. It didn’t really know how to place images where we’d placed images in text. The images would stray into the margins of the page, and look kind of ugly. And you ended up having to go and do a lot of fixing of the positions of the images and stuff like that.
So in the end, the workflow was - Google Docs, save as .docx, import into Mac Pages, fix any kind of page formatting stuff that we needed to fix, and then export as a PDF. And that worked for us.
Len: That’s quite a journey. Thanks for all of the details.
I wanted to ask for any other self-published authors listening: both of your books have great covers. And I really like the one for Python For The Life Sciences, where the sort of strand of DNA is the snake, presumably a python.
Len: I was wondering, do you have any advice for people about how to find a source of good book covers?
Gordon: I used Keynote to make that cover. Which is kind of the Mac equivalent of PowerPoint. I find that to be a really versatile graphic design tool. I don’t claim any great expertise or knowledge in graphic design, but Keynote is actually a really great tool if you want to blend some images together, and make some simple shapes. If you look at the cover of the book, it’s all fairly simple shapes, and takes a bit of playing around with gradients and colors to get it right. But yeah, Keynote - it’s a fantastic tool for putting together designs. That cover was completely designed using Keynote.
Len: You have a section at the back of your book where you ask for readers to send you any errors or omissions they may find, and just send them to you via email. Have you had any responses like that?
Gordon: Not yet, no. But, I mean, one of the things that attracted us to Leanpub - we both have software development backgrounds. We really like the iterative publishing model. It’s liberating to be able to get a book out there. Not to have to worry that every little typo is fixed, every diagram has the right caption.
Obviously we did our very best. We didn’t want to put something out there that looked sloppy or half-finished. For our own pride, as much as anything. But it’s great to know that if - there are always errors, and it’s great to know that with the Leanpub model, you have a way to go back in, fix the errors, upload the book, and all your readers are able to benefit from that too. That’s a really nice feature, something that really attracted us to Leanpub.
Len: Do you plan to make a print version of your book?
Gordon: We actually do, so actually - here I can show you on the video. Here’s a proof copy. With the cover. I actually expanded the cover, so that we’re on the back and it has the spine. We went to a local book store. They have a machine called a Gutenberg Machine, obviously named after the old printing press. And it does a really nice job of printing books on demand. It’s a black and white copy now, so the interior of the book is black and white. The machine unfortunately doesn’t do color. But we are exploring a number of places right now, where we might be able to produce physical copies of the book.
There are people who still like physical copies of the book. And I think also for libraries and schools and things like that, there’s still a place for having a physical copy of a book.
Len: If there was one feature we could build for you, or one problem we could fix, what would that be?
Gordon: I think putting the book together and collaborating on the book is something that it would be great to have - a more fully fledged tool for doing that kind of thing. And not necessarily in a web browser. I mean it could be like an app. For example, I’ve made some photo books previously. And a lot of those photo book, online services - they have an app you can download to your desktop. And you can actually build the book in the app, and then it publishes it to the website for you. So you’re not being forced to work in a browser with all of the limitations that entails.
So I feel like it would be great to have something. And also, a tool for creating a book, that would allow you to immediately go into multiple formats. PDF, MOBI, eBook - ePub, sorry. All this sort of thing.
Len: We do have that if you write your book using Leanpub.
Gordon: Right, right.
Len: We automatically produce PDF, EPUB, MOBI, and you can make a website if you want to. But around collaboration, that’s something where, it’s this huge area, where we’re definitely going to be doing work at some point.
Gordon: I can give you some tangible examples of the kind of problems we faced. We had lists of topics and things we wanted to cover. And we’ve put them all out on this whiteboard, that’s actually behind my desk. And then you’d have things like - well, okay - I wrote the chapter. Do we introduce Matplotlib in chapter four or chapter six? Oh, I think you introduced it. So some parts of the book, we would have explanations for things, where somebody had already introduced it previously in the book. And we’d have to move the explanation back in the book.
Had we covered all the topics? It would be really nice to have almost a kind of a meta book assembler. So that you could assemble the book in a kind of outline manner, with all the topics you want to cover. And then as people are working on it, you could tick off the topics and where they first appear in the book. And all that kind of thing. It’s more of the structure of the book, like a way to collaboratively define and keep track of the structure of the book as you’re working on it.
A lot of the features of most sort of book editors, are focused very much on layout and putting images in the text, and the markup and, what’s bold and where the links are. And chapter headings and tables of contents and stuff like that. Which is great. You need all that stuff too. But I don’t see much in the way of meta kind of - do you know what I mean? I don’t know if meta’s the right word, but…
Len: I do - I do know what you mean. Thanks, that’s very clear and that’s really interesting. That’s a really great observation too. I mean, especially where there’s so much emphasis placed, in so many writing tools, on formatting - but not on structure. When presumably, when it comes to the reading of a text, or most texts for most purposes, the structure of the text is far more important than the formatting. So that’s a really good observation. Thanks for that, we’ll process that internally.
Gordon: Alex and I both used - I don’t know if you’ve ever used TeX or LaTeX? They’re these sort of markup languages for creating typeset text. But something along those kind of lines, that way, you can really define the meta structure of the book as well.
I mean, like you said - the structure of the book is really important. And then to be able to, in essence, kind of apply stylesheets, like that CSS kind of model, where you have the structure of the book, and you say, “Okay, a chapter’s going to have a header and a footer. And it’s going to have this block of content at the beginning that describes the chapter, and maybe a picture and all that kind of stuff.”
And you lay that all out, and then you can just - “Okay, let’s look at it in this style. Let’s look at in this style.” And yeah really, really decouple the content and the structure of the book from the layout.
Len: That’s a request that we’ve had from some of our best authors in the past. And it’s something that we’re thinking about. It’s really - conceptually it’s very consistent, as an idea - it’s very consistent with Leanpub’s approach to writing. Which is that - when you’re writing, you should be writing, and you should consider formatting to be a separate process.
Len: Like for 99.9% of books, that’s the appropriate approach. And separating those things too conceptually, is very important to us.
Len: Unfortunately, I think our time is about up. And I just wanted to say - thank you for a great interview, and for making such a great book.
Gordon: Oh thank you.
Len: And for being a Leanpub author.
Gordon: Thank you very much, it’s been a pleasure. We loved it. I’m sure it won’t be our last one.
published Jan 24, 2017
Cal Evans is author of five Leanpub books, Signaling PHP, Iterating PHP Iterators, Going Pro, Culture of Respect, and Uncle Cal’s Career Advice to Developers. In this interview, Leanpub co-founder Len Epp talks with Cal about his career, his books, and his experience self-publishing on Leanpub.
This interview was recorded on September 15, 2016.
This interview has been edited for conciseness and clarity.
Len: Hi, I’m Len Epp from Leanpub and in this Leanpub podcast, I’ll be interviewing Cal Evans. Cal is based in Jupiter, Florida and has been working for the past 15 years with PHP, MySQL on OS X, Windows and Linux. He’s worked on projects of widely ranging size, including multi-million dollar applications. Cal also builds websites and is a popular conference speaker, delivering both talks on technical subjects, and also motivational speeches. And I believe he also likes bourbon, so if you ever see him at a bar, please feel free to buy him a shot.
Cal is author of five Leanpub books Signaling PHP, Iterating PHP Iterators, Going Pro, Culture of Respect, and most recently, Uncle Cal’s Career Advice to Developers. A little bit later, I’ll be asking Cal some questions about his books.
In this interview we’re going to talk about Cal’s professional interests, his experience self-publishing using Leanpub, and ways we can improve Leanpub for him and other authors.
So thank you Cal for being on a Leanpub podcast, and for sitting through that intro.
Cal: Not a problem, I’m happy to be here.
Len: I usually like to start these interviews by asking people for their origin story, so I was wondering if you could tell us a little bit about how you became Cal Evans.
Cal: Sure. But before I say that, you listed the five that I actually have released. You skipped over the like 10 books that I’ve started, and never released - including such developer-centric books as “Oh crap, why did I do that?”, reviewing my old code. But no, back to my origin story.
I started programming when I was 18 years old. That was way back in the early 80s. I started programming, I got married the same year. You know what that means. I ain’t going to lie. I got married. I got a computer. I spent all my time programming, because I’m a geek.
I started working on a Commodore 64, and all of a sudden I discovered that people would pay me to do this - and it was surprising to me. So I started coding for a living. I’ve had various other career paths, short career paths.
I used to do live concert videos. I used to direct and edit them. And at one point, I ran a printing company - but old school, offset press. But for the past 15 years I’ve done MySQL and PHP. And for the past 10 years, my focus has been on developers, building better developers. And that is my goal, to help people become a better developer.
On one of my profiles, it says “I don’t want to change the world, I just want to help you become a better developer.” So, that’s what I am. I’m old school. As long as people still keep paying me to do this, I’ll keep doing it.
Len: And what are you working on currently?
Cal: I am the manager of training and certification for Zend. So I handle, amongst other things, the official PHPs, or the official Zend PHP certification: Zend Certified Engineer. I also manage the team that built all the training, and builds and delivers all of the online training.
And then, of course - like every good developer, I have in addition to the books, a couple of side projects. Nomad PHP, which is - we get together twice a month now, we have two meetings every month. We get developers together online, and I’ll have a conference speaker come online and give one of the conference talks, because not everybody can make it to a conference. So we get together and we do this twice a month.
And then three or four times a year, it really depends on my mood, we will do what we call Day Camp 4 Developers. I get five speakers, and we all get online and we’ll have teams get online, throw it up on a projector, order pizza. And we’ll just spend the day focusing on one topic. Recently, we’ve done modern techniques for building applications in PHP, things like that. So, those are my side hustles.
Len: And how did you get into conference speaking in the first place? I’m interested to hear about that.
Cal: The very first time I was ever asked to do a conference talk was, I believe, right around 1995, and I was doing FoxPro, and I was so scared that I actually told them no, I can’t do it. And I went on to do one the next year, but I co-presented with somebody. And then, I just kind of ignored it. It wasn’t really interesting to me, until I went to work for Zend the first time. This was in 2005, I believe, and I was the “community guy.”
We didn’t have developer relations or developer advocacy or evangelism. I was just the community guy, and I was going a long right nicely. I built a website for Zend called “Dev Zone,” and I posted on there. And all of a sudden, my boss calls me and says “Hey, in three weeks Apple is having their Apple FileMaker conference in Orlando. There’s going to be 1,500 developers there, and you’re the closing keynote.”
Okay. So I had three weeks, not only to put it together, but to get my head around the fact I was going to get up in front of people. And let’s just say I was not awesome, okay? I still don’t think that I’m awesome, but I really feel that I shortchanged these people for what they had to pay to be there. But I did survive it.
Len: And did you do any kind of reading about presenting or speaking anything like that? Or did you just jump right in?
Cal: No, no, no - I’m way too arrogant for that. I’m a developer, I don’t read the manual. So, I just figured that I could do this, and I put together a presentation. I put together a demonstration, because this was FileMaker and this was when Zend and Apple had worked together to build the gateway for PHP and FileMaker. And so I put together a little thing using the now-defunct Netflix API. And it was really fun, and I learnt my very first lesson of presenting at technical conferences. Which is, “If your presentation requires the internet, make sure you have backup slides because when I -“
Len: Oh my God, yes.
Cal: - when I did my run through, while everybody was at lunch - man, everything clicked. It was wonderful, because I had the entire network to myself, because everybody was at lunch. I get up to present, and I go in there - and there are no IP addresses left in the network, and they didn’t have a dedicated speaker network, so I had no internet whatsoever. So, I would point and describe and say “If we could see this, you’d say ‘I got to get me some of that,’” so -
Len: I’ve done a fair amount of pitching. And yeah, you learn very quickly that you need to have all the equipment with you. I think I ended up with like a 24-foot HDMI cable, because the situations that you find yourself in, are so totally unpredictable. And you don’t want to be caught unable to do what you came there to do. Somehow the tech becomes your responsibility.
I was wondering - you talk about, on your profile - about “management by walking around.”
Cal: No, no, no, no, no. “Management by walking around” was either Hewlett or Packard. That was what they were famous for. Mine is “Management by wandering around.” Vastly different, vastly different.
Len: Sorry, I got that wrong. Can you maybe explain the difference to us?
Cal: I ran a team - when I coined that phrase, I was running a team back in Nashville, Tennessee. And this was - this sounds so stupid. It was around the turn of the century. And I had put everybody in cubicles. So I had these really high cubical walls. I tried to give them as much privacy as possible. And I realized that even though I had 15 people working on three different teams, and the team leads knew what was happening, I wanted to get a feel for where everybody’s head was.
And so literally, I would wander from cube to cube. And I don’t mean down the line. I would go here, and then I’d go visit her and then him. It’s all over the place. One of my developers brought in two buckets of Legos, and that’s how they would think problems through, was play with Lego. Usually if you couldn’t find me, I was over there at their desk playing with the Legos.
But that’s what I started doing, and I learned that I can spend five minutes with somebody without interrupting their flow. Because if I see them in the flow, I’m not going to bother them. But I can spend five minutes with somebody, and if I do that every two or three days, I know where everybody’s head is, and I can take the temperature of the team.
My team leads knew the project, and knew how things were running and all that - I wasn’t worried about that. I wanted to make sure I wasn’t burning people out, that people were feeling good about the project. Things like that.
Len: In your book Culture of Respect, in addition to finding and hiring people, you talk about keeping them. I was wondering if you could talk a little bit about what you say in that book about, How do you develop a good culture? What is a good culture for keeping developers in the medium term, or even the long term in your company?
Cal: Really I talk about that? I really need to read that book! No, I’m kidding. In line with the “management by wandering around,” one of the things that I am just really huge on is - I didn’t realize there was a term for it until later - what is called “servant leadership.” I was the first person into the office most days. I was the one that fired up the coffee pot. We worked right next to this huge Target Superstore. And so, I would go over to Target once a week, spend 50, 60 bucks on candy - and I had bought everybody a candy jar. So, on Monday mornings, I would wander around. And if your candy jar was empty, I would take your candy jar, take it back to my desk, and fill it up, and put it back on your desk. And I made sure that there was always coffee made.
That team - I’m not proud of the fact - but we got to the point where we were behind the eight ball, and we worked some long hours. And there’s only so much pizza that one human can consume. So I started catering in from some very nice restaurants in the area. And my food bill was three, four thousand dollars a month, for about two months while we were doing this. But I was asking a Herculean effort from these people. It was important to me that I showed them that kind of respect.
I also sent flowers or appropriate gifts to all of their significant others. When the project was finally finished, everybody got gift certificates. I think most of them were to The Melting Pot a high-end fondue restaurant, enough to cover a nice meal for two, and things like that - to show not only them, but their family and their significant others that me and the company really appreciated what they were having to go through.
Niceties and little things like that, that’s like a having a foosball table or a kegerator - it’s not going to make the difference. But the respect - the fact that I took the time to do this. I didn’t say, “We need to go do this.” I didn’t assign somebody to do this. I took care of making all these things myself - to show them that I respect what they do.
And quite honestly, that was a team - we were running Java and Oracle, and I know a little PL-SQL, and I can read Java, but I couldn’t do their job. I couldn’t dive in and help them. So, I did the next best thing. I tried to take care of everything else. That was also the office where we had two doors into the developer area. Both of them had combination locks on it, and if you weren’t a developer or my direct manager, who was the CIO, you did not have the combo. Even the COO and CEO had to be escorted in and out.
Len: That’s really interesting. I’ve never heard of having locks like that. But what a comfortable space that probably provides for people. Knowing that you can’t suddenly sort of look up, and there’s the CEO wondering why you’re playing with Legos. And then you’ve got to explain.
I was wondering, approaching the subject negatively - can you maybe talk a little bit about the worst workplace culture you’ve seen or worked in, or - I was going to say the worst example - the best example of poor management you’ve ever seen?
Cal: It’s funny, because I’ve actually got a blog post on my blog called “Good Boss…Bad Boss”, in which I break down four of my bosses. Two of them were my mentors, and two of them were Satan incarnate.
One of them was - I was working for my parents’ company. I’m the boss’s son, so it’s obviously not a great situation to begin with. But I’m the only computer person in the entire company of 40. I’m the computer guy. And we were using an accounting system – I believe at this point we’d migrated to FoxPro system. But the sales manager kept saying, “These numbers don’t look right, these numbers don’t look right.” I said, “You’re saying these numbers don’t feel right. I can show you the line items where this data is coming from, these numbers are right.” And she looked at me in front of everybody and says, “I don’t think you’re a good programmer. I don’t trust these numbers,” and walked away.
Even though I’m the boss’s son, there’s limits to what one person can put up with. It was at that point I decided it was time to make a career change, and get out of the nest, move out of mom and dad’s company. I had enjoyed my time there, but it was time to go. That was the absolute worst, because this person had no concept of how to treat people. This person was an old-school command-and-control manager. I don’t know anything about managing accountants or sales people. Maybe that’s how you manage them? You don’t manage developers that way.
I’m famous for making enemies by saying “If you’ve never been a developer, you have no business managing developers.” And this person had never managed a developer. They didn’t understand deadlines or anything like that. And so it came through, and they were a horrible manager.
Len: Yeah. It’s interesting. I was talking to an author named Janelle Klein recently on this podcast about issues around this, at a theoretical level like that. One of the images I like to use to convey the difference - not having been a developer myself, I mean I was kind of hazed by having to internationalize Leanpub when I started - is that, a lot of management practices are actually based, since ancient times, on visual cues.
So as a manager you can just, whether it’s stacking bricks for the pyramids, and you’re working with people under horrible enslavement, or bricklayers in Victorian times, as a manager, you can stand and watch, and you can see progress - are the bricks getting stacked? But with developers, that’s completely gone.
All those ancient instincts about ways that - whether they were ever good or ideal or not, did work. All those ways of managing people, like watching what they do, just completely blows up when your “worker” sitting in front of a screen typing away.
I mean, even if you do look at what they’re doing - and I’m just sort of supporting your point - if you’ve never been a developer, well you have no idea what you’re looking at.
But there’s other things you won’t understand as well, like - say they’re on Slack. You don’t go like, “Get back to work, guy.” That’s work. Hacker News, that’s work. Facebook, that’s work. You want your developer to have a wide net of information that they’re receiving and engaging with all the time.
Cal: I had another manager when I was working in Nashville that - this was my last FoxPro job, and he had been a FoxPro programmer. But if you don’t know FoxPro - FoxPro started off as a procedural language, and morphed into an object-oriented language. It’s a wonderful way to learn object oriented concepts. Because I already knew the language, I could control the concepts. He considered himself a very good FoxPro programmer, but I was an object-oriented programmer, and he considered me just a little better than him.
Well, he gave me this task to do, and it took me about two weeks, because this was some deep stuff, it was a compiled language - we did nested inheritance, and all this. Because you didn’t pay any penalties for it at that point.
And so, I was digging through this legacy code, and rebuilding it. And he came up to me one day, and he’d just had enough. It was a Friday. He needed to blow off steam. So he yelled at me for two hours.
Because he asked me “When’s this going to be done?” I said, “I have no idea.” And so he yelled at me for two hours. And I went back to my desk totally energized, okay? Because I was feeling the burn at this point. And I finished it up about an hour and a half later. I finished the project up. I didn’t know that was the point. But he had no concept. Even though he was a developer, he was the old command-and-control, “You do what I say, work harder.” That kind of stuff. Of course, I left that company. And I was told that six months later, he was escorted out of the building by security. HR had him escorted out, because he explained to a female business analyst that he could train a German Shepherd to do her job.
Len: Oh my.
Cal: Yeah. We all got a kick out of that. We got together for a developer reunion one day, and we all got a kick out that one.
Len: Wow. I guess it’s easy to judge from a distance, but if people have one flaw - I mean we all have flaws - but if a person has one kind of flaw that manifests itself in aggressive behavior, they might have others.
On that note, actually in the book Culture of Respect, you talk about building a character sheet for potential hires, like you would playing Dungeons & Dragons, and I found that really interesting. I was wondering if you could talk a little bit about that idea and what the division is between what you call soft skills and hard skills.
Cal: You know, it is like Dungeons & Dragons. I’m so deep into marketing, that these days it is a persona that you build. But no, it’s a D&D character sheet that you are building. And this forces the hiring manager - not HR, okay? I’m not a fan of HR. In my entire life, I’ve met one person who works in HR that I did not absolutely hate–and she did get on my nerves sometimes. But the hiring manager has to sit down and think through, “What do I really need this person to do? And what are the skills I’m looking for, and what are the traits I’m looking for?” Because there’s a huge difference.
Skills are hard. Can this person code in Java? Can they do C++? Can they do PHP? This kind of stuff. Traits are: Can they communicate their ideas with others? Can they have other people communicate their ideas to them? Because I’ve worked with developers who could tell you exactly what they were thinking, in ways you could understand. But if you explain to them that they’re off base, and you need them go this way - they would go ballistic on you.
So those are the differences, and I urge managers to sit down and think it through. Because if you don’t, what you end up with is what we call the “kitchen sink” job ad. “You need 15 years’ experience in PHP 7.0 or better. You need Photoshop, you need HTML, you need to be able to stand up your own Linux server.” All of this stuff. Which sounds real great - but really all you needed somebody to do, was to manage this application that you’ve got running on a server - or maintain this application.
So, think it through, don’t ask for everything and - I rail against HR, because HR usually likes to add things in. “This job requires Photoshop.” Well no, it’s a developer. He’s working, she’s working in a command line. There’s no Photoshop equivalent of the command line. So no, we don’t need this person to be able to do Photoshop. “Well it’d be nice to have.” No, not really, it wouldn’t. It’s just going to limit the candidates you get.
To those people - honestly, if you’ve got a kitchen sink ad - you’re not going to cast a wide net and get everybody. What you’re going to get is those people who will apply to anything. Because the people that you actually need, see all of that and understand you have no concept of who you actually want, and so they just pass over.
Len: I’m curious about your thoughts about interviewing people. Once you’ve got the ad out there, the proper ad, you’ve managed to fight off HR from corrupting the process - and there you are in the interview with the person. I was just curious about what some of your thoughts are, about what to do and what not to do.
Cal: There used to be a site, a long time ago, called Freshmeat. It was a sister site to Slashdot. I hope people still remember Slashdot. I actually wrote an article for Freshmeat one time called, “Nerd Herding” - it’s now up on my blog - and I talk about this very thing.
My process is this. I put out the ad, I do work with HR. We get the legal requirements covered, but I don’t let them add things like skills and traits and all this.
I would say, “Hey, I got your resumé. Thank you. Why do you program?” - things like that. I would look for some insight to the person. Honestly, if they gave me anything at all, that would usually warrant a phone screen. So, that would usually filter about half of them. So out of that 150, I got 20. Out of 20, I got 10 people that I would sit down and call. I would talk to them on the phone for five or 10 minutes, and just get a feel for the person. I already know that the person has the technical skills - or at least is saying that they have the technical skills. And I know that they are a little bit insightful, because they responded. I just want to talk to them.
But out of the 10 people that I would talk to on the phone, eight to 10 of them would usually end up in an interview with the team. At that point, I turned it over to the team. I told everybody I hired, “I don’t have to work with you, I just have to manage you. These are the people you have to work with, they’re the ones that you’re going to have to impress.”
So we would do the team interview. I would call everybody into the room. This is a very expensive way to interview, but it’s still less expensive than making a bad hire decision. I would bring everybody in, juniors up to my architects. We’d sit down in the room. I would not say a thing after I introduced the candidate. And they would go around the table, just asking questions. As long as it was legal to ask the question, there were no filters that I put in place. And they would ask until they got finished. I didn’t stop them.
I think the longest one was an hour and half, and we were literally interviewing a rocket scientist. He worked at - what is it in Huntsville, Redstone? Anyhow, he worked at the NASA Center there in Huntsville. He was literally a rocket scientist, and it was fantastic. We ended up hiring him, he was great.
Then I would thank everybody. I would escort the candidate back out the front. The team would stay there, and then we’d sit and talk. “What would you think?” You think this, you think that. And then we would take a vote, right then. While it’s fresh on everybody’s mind, I’m going to take a vote.
If I had one person say “no,” then immediately, that candidate is no longer viable. Now that seems very harsh, because you’ve got junior programmers that have the same weight as my architects, and the juniors had to go first. I didn’t want them “me-too-ing” one of my senior developers. So we went around the room, everybody got a vote in hiring over 50 people in this way. I had two that we just walked away from, because somebody voted “no”. Because by the time you get to that point, everybody is either “yes” or “no, this is not going to work.” When I say I had two we walked away from, I mean we had two we walked away because we had one vote “no.” Usually, it was pretty obvious by the time I got back to the room, the mood of the team, and whether this was going to be a viable candidate or not.
I built some great teams using this methodology, because by the time the candidate got on board, they already knew everybody. The people were comfortable with them already. We had already started that break-in period. And so you get right down to getting to work, and building those bonds, that esprit de corps that is so vital on a development team.
Because when you have to work with someone on a project that is overdue - I hate to use the analogy “death march,” because that kind of minimizes what a death march really is, and what we’re doing is long and uncomfortable, but it’s not really a death march - but when you get into one of those situations where you’re working long hours, you’d better be comfortable with the person in the next cube or two cubes over - or tempers are going to start fraying. And I never had that problem.
Len: That sounds like a really fascinating process. I mean I can just imagine the impact it has on people, if they see someone that they’ve all endorsed, maybe having difficulty; they might be more motivated to help than otherwise, knowing that you’ve got a kind of collective buy-in.
Cal: That was the thing. The team had buy-in on each hire. So the team was committed to each hire. I did not bring people in and say, “Here’s your new team mate.” Other than the very first person. The first person on any team, I’m the one that hires them. After that, it’s a collaborative effort.
Len: In your latest Leanpub book Uncle Cal’s Career Advice To Developers - which I gather is the text of a talk?
Cal: Yes, the text of a keynote that I gave.
Len: You write about how - I guess is more from the individual, rather than the team perspective - you write about how the job will never love you back, and I was wondering if you could talk a little bit about what you mean by that?
Cal: This was advice given to a friend of mine, Samantha Quiñoes, up in New York. he said her high school counselor gave her this advice: “The job will never love you back.”
I’ve worked for San Francisco startups. I’ve worked in that culture, I’ve worked in the, “We’ve all got to pull together and do things.” And the team I was building in Nashville, we were working so hard on - I was there, almost the entire time. I like to be the first one in. I hated to do it, but I wanted to be the last one out. Now we had some people working till two, three in the morning.
I let them stay. But I was there literally 14 to 16 hours a day, towards the end there. And the COO would walk out at two o’clock to make his tee time. And you’d see the sales people wander in, and wander out for early lunches and stuff like that. And it didn’t hit me until Samantha and I got to talking at a conference one time - because we get to hang out with each other at conferences - and she said that, and it just struck me - that’s the problem. Everybody tells you, you’ve got to love your job, you’ve got to give everything towards the company. Well, the only people that’s actually going to make anything off this company are the founders and the investors, okay?
You’re not going to get anything but your paycheck. So when it comes to your career, you have to be a mercenary. You’ve only got so much time to trade for money. So you’ve got to make sure you’re trading it for the most money you can get - money and benefits. And if you love the company, and you think it’s wonderful - hey, that’s great. But if you think that the company is going to love you back, it’s not. You’re not family, and if you want to put that to the test - be the slack-ass cousin who doesn’t work for two weeks, and see if you’ll still get the paycheck. Not going happen. You’re not family. You’ve got to treat it like a business.
Len: That’s a great lead up to my next question, which is - you’ve got a great line in the book where you say, “Some days, I’ve been the windshield, some days I’ve been the bug.” I really like that image, and I was thinking a little bit about it. Then I realized, in this context, this might be an interesting ambiguity there, that I’d like to ask you about, especially if you’re working in start-up land. You can interpret that as being, “Sometimes you’ve got to do really unpleasant types of work.” And this is actually a part of being responsible in your role. Is that what you meant, or were you talking about something else?
Cal: Actually that comment is much more of a day-by-day comment. Some days, you’re going to knock it out at the park, and some days, the bat’s just going to hit you. I have a version of that, that says, “Some days, you’re the windshield, some days, you’re the bug. If I’m the bug today, let me be an armor-piercing bug.” Because I want to blow through.
But you’re right. There are times when you just have to power through it. And I have another talk, which I believe is actually one of the books, called “Going Pro.” And in it, I talk about a similar topic, in that there are times when you, as a team member just cannot find your happy place on that team. For whatever reason - they’ve made a technical decision, you can’t get along with your manager - whatever reason, you can’t find your happy place. As long as the paychecks are clearing, you’d be an adult. You give your 110%.
But you start looking. Because if you’re not happy - you owe it to yourself, and to that team - to go find another job. You are not doing that team any favor by hanging around if you’re pissed off. Because if you’re pissed off, you’re not happy. It’s going to affect your work, no matter how much you think it’s not. So, you give your 100% while you’re there on the job. You don’t slack, you don’t, “I’m going to goof off today. They’ll never know, what are they going to do, fire me?” Well, no. You’re an adult. Do the right thing. But start looking around, and as soon as you can do it - exit gracefully.
And I don’t mean get so pissed off that you rage quit. Rage quits feel really good for about a day. I know, I rage quit a McDonald’s one time. The most pointless gesture I’ve ever made. But rage quit feels really good, until you call your buddy who - you knew it was a sure thing, because they’re looking for somebody just like you, and then your buddy says, “Yeah, my manager was talking to your previous manager. You kind of left them in a lurch. We can’t talk to you anymore.” All of sudden, that rage quit don’t feel so good anymore. So be an adult, do your job. But if you’re not happy, you owe it to yourself, and the team, to find you something else. Don’t be the bug everyday.
Len: I just love that image.
You mentioned it at the beginning that I left out of my introduction your unpublished books on Leanpub, and I was wondering if there’s one in particular that you’re more focusing on now? Or if you even have a plan for what your next book might be to release?
Cal: I am almost finished with my next book - I don’t know if it’s on Leanpub or not. My wife actually handles all of that now. Used to be, I did it, and that’s why I did Leanpub. Because Leanpub is easy enough that even I can concentrate on writing, and not have to worry about the book production. It’s a wonderful platform. But I don’t know if she’s put this one up yet. I have full intention to put it up there. It’s called Spin a Good Yarn, and it is everything developers need to know to build presentations. I don’t build slide decks, because my slide decks suck. My slide decks are white backgrounds with black text. And I upgraded on my last one, and actually put a picture of myself and a picture of one of my books. I mean, that’s a graphical upgrade for me.
But everything else that you have to do, from coming up with the concept, to writing the abstract, to getting a title, that will actually convince the conference organizer to bother to scan your abstract. To the practice, to how you do all of this. And then what do you after you present? How to be a good speaker for a conference, and not just a good presenter.
I will never ever do this again, but I recorded an audio version of it. And oh my God. Editing audio - I grew up around audio, but having to edit an hour and half worth of audio and take out - it just seems like I take these huge breaths on the microphone - having to edit all this out is a pain. But I got the book, the audio, five or six videos to go with it and all of that, and I’m putting it up. It’ll be at - and this is the worst URL I could find - spin-a-good-yarn. Or you can just go to my blog, and there’ll be a picture of the book up there.
Len: You’ve partly answered it already, but why did you choose to self-publish on Leanpub?
Cal: Well, that’s two questions. Why did I choose to self-publish? I have actually published through a traditional publisher before, and I got 20% of my book sales. And I just felt like I deserved more. One of the reasons that Culture of Respect is not on Amazon is, because of the price, I would only get 35% royalties. No. Amazon doesn’t deserve the 65% royalties. They didn’t do anything worthy of that. So, that’s only available on Leanpub, because you guys have a good platform and it’s fair. But the reason I self-publish is, I know I can probably sell more. I don’t know that I could make more, and I just like the control.
Why did I use Leanpub? I got to researching what it would take to do it. And I actually had systems set up, and I was playing with them - on how to create a Kindle [Note: Cal means a MOBI format ebook file - eds.], because that was my focus. I wanted to create a Kindle book, okay, and it was just a pain in the rear end. And yeah I’ve got KindleGen, and I’ve got all this other stuff. And I can edit the XML - what is it, the OPF file, or whatever it is - by hand. I can do all this stuff. But that’s not what I want to do. I want to write books.
And then I came across Leanpub, and I don’t know if I found you because of Google search, or somebody introduced me. But about the time that I found you, all of my friends found you also. For a while, I thought all you published was computer books, because every time I turned around, somebody’s publishing a computer book on Leanpub. I’m like “Okay, this is the platform I need. And I have published - I think you said five - I’ve started three or four more, and I have plans for at least three or four more next year.
I have looked at doing my own thing. while Leanpub is a wonderful platform, I want a little more control over things like stylesheets for PDFs. Where possible, I want to make it look really nice. And my wife’s a graphic designer, and she says, “Well this is all we can do in Leanpub.” And so I looked at several other platforms. I’ve even got a droplet out on DigitalOcean that has Pandoc installed. And, yeah that took a weekend, and some PHP code…. I can publish the Markdown, and it will– I’ve got stuff that’ll convert it to HTML and PHP, that will create the OPF. I’m like, “This just isn’t worth it.”
Leanpub gives me most of everything I need. I can either figure out the rest, or live without the rest. I don’t have anybody screaming at me, “If only your PDFs were a little more colourful, I would buy them.”
Len: That’s a really interesting challenge for self-published authors. To what degree do you focus on - in the end, after you done writing - on formatting? Or do you just go ahead and start selling?
One feature we developed, we launched a few months ago was, upload. So one way you could use Leanpub is to write your book in-progress on Leanpub, and then when you’re done, we have an InDesign export feature that you can use, so then you could take it, and then get it to your book designer. And then you can actually switch writing modes to the feature that we launched a few months ago, which is, “Upload your book,” so you can actually upload a PDF and/or a MOBI, and/or an EPUB file that you’ve made yourself.
We’ve got quite a few very good-looking books that people spent a lot of time, working even with teams of people, to get them to look really nice. And then they can upload them to Leanpub. And they can do that as many times as they want. And then they can take advantage of our high royalty rate, and a lot of our other features. Like email the author, email your readers - coupons, bundling - things like that, that aren’t really available on Amazon.
I just wanted to actually address that point that you brought up at the beginning of that great answer, which was that, for books that are priced higher than $9.99 on Amazon, they drop the royalty rate from 70% to 35%.
For people who aren’t familiar with it, there’s a whole discourse around ebook pricing. And there’s been a controversy since the Kindle came out, basically around this. But Amazon - it’s at a complex and evolving sort of position. One way of describing their position is, ebooks should not be priced higher than $9.99. They want that idea in people’s heads. And when I say people, I don’t just mean readers - I mean authors and publishers as well. Which is partly where the controversy comes in.
And so, if you’ve got a book that’s worth $9.99, and you’ve got a book that’s worth $11.00 on Amazon, you’ll make more money from the book that’s worth $9.99 per sale, from the book that’s worth less. So, it’s an interesting strategic move on Amazon’s part.
That’s one of the things that does make selling on Leanpub different. One of the reasons Leanpub is popular with people who write technical books, is that they are often books that ought to be worth more than $9.99.
I mean you can imagine - if reading a book gives you skills, that means you can now command a higher price for your consulting per hour. How much is that book worth to you? Well, lots.
And we’ve actually even got one book right now - that’s about getting a technical certification, that’s got a $200 minimum price. And people are buying it. This is a book that’s got no DRM, a self-published ebook. But it’s worth that much money to a lot of people.
Cal: Well, I fully support authors being able to charge whatever they want, and I fully rail against Amazon’s control of the market. But, I’m happy to put my $9.95 and below books up there. I’ve got series of books called, “Learn One Thing Books”. They’re short books; specifically, technical books. They focus on a single topic, and they’re $9.95. And what I do is, I publish them on Leanpub - take the MOBI, and shove it up on Amazon. It’s nothing deep or anything. I don’t recreate it just for Amazon. I take what you give me, and just put it up there. But I did not know that you had the “upload” thing. That’s really cool. I’m going to have to start doing that, because my wife is a designer.
I usually start in Google Docs, because that’s what I’m most comfortable in. And [my wife will] take it, and she’ll start breaking it up into text files, and putting it on Leanpub. But now that she can produce it, I think she will start using that. That works a whole lot better, and I don’t have to fire Pandoc. Because yeah, I got friends that use Pandoc, and they love it - but, that’s not what I want to do. I don’t want to manage yet another system.
I thought about actually writing my own, and I’m like, “Oh my God, why?” Just more code to maintain.
Leanpub is not a perfect platform - I’ve had disagreements with you all, although that was early on - and I think you’ve resolved most of the issues I had. It is not a perfect platform. It is a platform that has served me very well. And I’m very pleased with the service I get from you guys, and will continue to publish my stuff on Leanpub. And hopefully, help put a few pennies in your pocket, so that you can keep doing this.
Len: Thanks, I really appreciate that. And we appreciate you being a Leanpub author. I think you’ve been around for quite some time in Leanpub’s lifetime. so we really appreciate that.
My last question is about self-publishing. Is interacting with readers something that you do? Is it important to you to get emails from readers, or get communications from readers with questions about your books, or suggestions, or anything like that?
Cal: I don’t get a lot of interaction with readers over email. Occasionally, I will get tweets. I love the fact that you put a tweet in the front of the book. “Tweet off that you’ve bought this book.” I’ve had people do that, and it’s awesome. But, I go to a lot of PHP conferences, and most of my audience is PHP developers. Even for my non-technical [books], that’s mostly who reads it.
And I get to sit down at lunch and talk to people. And they say “Okay, you said this, but what did you mean?” I get to talk with them, and they gave me feedback. I had somebody write me today. “I’ve got one of [your] books,” and he gave me a long list of - the book is on creating brown bag lunch programs. He says, “I like your book. It has a lot of good points. But here’s how we’re doing it, and I think your readers could benefit from that.” And so, I’m about to go - hopefully in October - go into a revision cycle, produce a new version of that - based on Jeremy’s feedback.
Len: Thanks it’s interesting. I just love to hear about the different approaches that people take in the way - that publishing books can be part of a wider kind of environment that you’re operating in. Talking at conferences and things like that.
Well, thanks very much for your time today, Cal. I really appreciate it. And thanks for being a Leanpub author.
Cal: Hey, thank you for the platform.
published Jan 16, 2017
Thad McIlroy is an author, speaker, and publishing industry expert who blogs at thefutureofpublishing.com. In this interview, Leanpub co-founder Len Epp talks with Thad about his wide-ranging career in the publishing industry, the state of the publishing industry today, whether self-published authors should submit their books to subscription services, some recent controversies in the book publishing world, and the future of publishing and book publishing startups.
This interview was recorded on January 13, 2017.
This interview has been edited for conciseness and clarity.
Len: Hi, this is Len Epp from Leanpub, and in this podcast episode, I’ll be talking with Thad McIlroy. Thad is an author and consultant who writes and blogs at thefutureofpublishing.com. His consulting work and experience reaches into many aspects of publishing, from print and the origins of desktop publishing, to analysis of the latest tech, digital book publishing, and supply chain optimization and other areas.
In addition to consulting, writing books and blogging, Thad is also an accomplished speaker, who has spoken at hundreds of events, from conferences to corporate meetings.
In this interview, we’re going to talk about Thad’s career, his interest in tech, and his recently published report, An Authoritative Look at Book Publishing Startups.
So, thank you Thad for being on our podcast.
Thad: Thank you Len, glad to be on board.
Len: Thanks. In these interviews, I normally like to start by asking people about - what I call their origin story. You’ve got a great one, that ranges widely across the publishing industry, and I was wondering if you could tell us a little bit about how and why you first got interested in publishing, and an overview of your path to where you are today?
Thad: Sure. I can go back centuries - I’ll only get briefly into that. But I came out of, well, a literary family. I mean, not that highbrow a literary family, although Kenneth Grahame is one that I can point back to. That’s fairly literary, but my father was–
Len: The Wind in the Willows, right?
Thad: Yes. We have a very direct lineage. And my great uncle, who is also an author, he spent time with him as a boy. That was Lawrence Hill Grahame, who wrote a number of books for young adults at the turn of the last century, which were best sellers, and award winners at the time, and totally unreadable today.
And then my father was a novelist and had two novels published, he was working on a third one when he died. He did a lot of radio work at the Canadian Broadcasting Corporation, which you well know, the CBC. And so that’s sort of the milieu in which I grew up. My father always said to me, “You’re not the creative one in the family, you’re going to be an engineer. It’s your sister who’ll become a writer.” And she does write, but she’s not published.
But my interest gravitated to book publishing. Where I really started in the industry was as a book seller in Toronto in the ’70s, and working for independent book stores and then working for chain stores. That’s where I fell in love with the book industry, and that’s really what’s launched me forward to today.
Len: And did you grow up around Toronto?
Thad: Yes, I was born in Toronto, and lived there for the first 30 years of my life, then moved to San Francisco.
Len: I’m curious about your first entry into the publishing industry. You worked for a bookseller, and I believe early on you edited a book about a Canadian Prime Minister as well?
Thad: Well, that was after I started my first publishing company. So in the third or fourth year of my book selling career, I had gone down to San Francisco. It was for the annual convention of the American Book Sellers Association, which is now the one called BEA [Note: It appears they have rebranded themselves as BookExpo - eds.].
And in those days, it was a huge hall, where all kinds of publishers, all shapes and sizes, would display their wares. And I was there with a friend at the conference, out of interest. And saw all these small presses with nifty books, where it’s like, “Oh you can’t get those in Canada.” And then it’s like, “Well maybe I could start a little company and distribute some of these small presses?”
And so I did. I started what I called Virgo Press, because my partner and I were both September babies, so Virgos. And we started this as a distribution company out of my bedroom, basically. And then a friend came to me and said, “I’ve got this amazing idea. I think it’d be a big best seller. How to Win Canada’s Lotteries was his idea.
And this was when they were still pretty new. They were [a phenomenon]. It was still something that was fresh and exciting. They didn’t have them every single place. They didn’t have 27 different kinds of tickets you could buy, and so on.
And so we published that book, and it was a bestseller. We wrote it pseudonymously. He, I and one other person wrote various chapters for it. And then I toured on behalf of the book. And so that was my first self-publishing experience. It launched what became Virgo Press, which ended up being a trade publisher of some substance. We ended up with 15 staff, and about 60 tittles published over about a three year, four year period.
And then ran into a situation where the bank decided they didn’t want to be financing us anymore and called in our loan. I couldn’t replace the financing, and went bust. But I had had this wonderful four year experience in my early twenties running a fun, small general trade publishing company out of Toronto.
Len: Wow, I didn’t realize it was that early on in your career, that it was in your twenties that you were founding a company and writing bestsellers already. That’s fascinating.
You mentioned the Book Expo America, the BEA Conference, was originally in San Francisco?
Thad: No, it used to travel around more, before they settled on New York much more recently. And then they tried Chicago last year, and that didn’t work out. In those days, they would move it around, as most conferences used to in those days, where it would be West Coast one year, East Coast one year. Then its centre would be Chicago, usually one year. And they’d just circulate that way. So it just happened that year was a San Francisco year.
Len: For those listening who might not be into the minutia of publishing conferences, there are currently two big conferences in the United States - the BEA conference and the Digital Book World conference, which is actually just coming up in a few days after this interview - and where Thad will be giving a couple of talks [The talks are “5 Tools You’ll Wonder How You Lived Without” on Tuesday, January 17, and “How to Thrive in an Era of Constant Change”on Wednesday, January 18 - eds.]. But the reason I asked about San Francisco is that these conferences tend to be focused around New York, so it’s interesting to know that there was a West Coast presence for the big, big players at a certain time.
I think I read on a bio about you somewhere that you may have been one of the first people to publish a book that was created entirely with desktop publishing software tools, and I was wondering if you could talk a little bit about that experience.
Thad: Sure. That was great, great fun. And I stumbled into it. It was not like I had a vision and thought, “Ah, I see what’s coming here, and I’m going to be the pioneer.” At the time, after my publishing company went bust, I became a freelance journalist, and on the side, with a couple of partners, we started a small book packaging company.
To call it book packaging … from the industry, they’re the people in the space between being an author and being a publisher. Instead of just writing the manuscript, you also contract for all of the editorial design and illustration for all of the books you do. You deliver what we’d call camera-ready copy to the publisher, so it just goes straight to press on their side.
They don’t have any of the developmental expenses, you take that on yourself. So you get a higher royalty, as a result of their lower overhead, and it gives you control of the project where, in many cases, because of the particular nature of the book, you want that kind of control. You don’t want to give that up. So I was working on books of political cartooning - it was fun, I’ve always loved political cartoons, and I edited two books, one on Pierre Elliott Trudeau, and one on John Diefenbaker, where I told their life stories with a focus on their political stories, in cartoons and in words. And so on one page would be a famous cartoon, usually I would have about 30 cartoonists in each of the books, with representative cartoons at each stage of their career matched up against quotes, either from one of those Prime Ministers or about them. It was a really fun way to go through and get a history lesson, without having to read a big long text.
The Diefenbaker one - as American Presidents have their own Presidential libraries, Diefenbaker has his own library and centre in Saskatoon, in the spot where he’s buried. It’s a great institution, if you’re interested in John Diefenbaker and in Canadian history, and that’s where I tracked down the cartoons.
While I was there, the Director of the centre said, “Next year, the embargo on his personal papers is ending, and they’re going to be open to researchers. But very few researchers know that. So Thad, you’ve got an opportunity where you could edit John Diefenbaker’s personal correspondence with his wife, with his brother, with his two wives, with his mother, and do a volume.” So I ended up doing a volume called, Personal Letters of a Public Man.
But most of his letters were handwritten, and in those days, to get something from handwriting to typesetting, meant [you had] to hire at great expense a typesetter, who was willing to take the effort to decipher the handwriting, and put it directly into the typesetting machine. So it was a very expensive per hour cost.
However, that year was the release of “DTP”, [Desktop publishing](https://en.wikipedia.org/wiki/Desktop_publishing. The Macintosh was maybe a year old, the first Mac.
Len: So this was about 1985 or so?
It came to me as a suggestion to get hold of a Mac, which I did, and type up all the letters myself, with the notion that I would then give the disk to the typesetting company, and they would translate it into their machine language for the typesetting.
As it turned out, I didn’t check whether that actually worked. And when I got all these letters typed, there were hundreds of pages, and I went to the typesetting companies we knew in Toronto at the time, they’re like, “We don’t know how to read one of these diskettes. We don’t have any way to translate it. Sorry, but if you want this book published, we’re going to have to do what it was in the original thing. We’re going to have to retype them all on this Linotype system, or this Compugraphic system that they were using.
And it was like, “Oh man, all of that for nothing.” But my art director, who we worked with on this project, said, “I was just reading in Popular Mechanics that you can actually output from Microsoft Word version 1.05 directly to this laser printer. If we do that at 125%, and then condense it, shoot it down to 80%, that’ll tighten up the density of the type. And we can use that instead of typography, instead of a traditional typographic machine.
Thad: And so we did, and the book was published. Doubleday was the publisher, and we showed it to the editor and my art director at Doubleday, without telling them how we’d done it. We showed them a page spread, and said, “What do you think? Is it okay - running heads, folios, the typography?” They were, “Yeah, fine, sure.”
And so out came the book, with a little credit on the cover page saying, “This was produced with the LaserWriter, and on Microsoft Word,” because PageMaker actually wouldn’t be out at that time. That was just a few months before it appeared.
And when that was published, Apple heard about it and they called me, they got in touch with me and said, “Do you realize what you’ve done?” And it’s like, “Well, no. We just did this and–” “You’re the first ones who have actually done this. We were hoping that this would happen, but it’s never happened.”
And so we got together, and they ended up connecting me with a company that was going to become Canada’s largest reseller of this equipment and technology. I became the product manager for this line, and so converted from being a journalist, a publisher, into a tech really - but a tech in the publishing industry. So that’s where my career transitioned into where it is today, where all of the work I do is at that intersection of the culture of publishing and the technology of publishing.
Len: That’s a really fascinating story. I’m very surprised to hear it, and it’s great to hear that Apple found out and were aggressive and watchful enough to see this, and reach out to you.
I wanted to ask though, what did Doubleday do when they found out?
Thad: Nothing. They didn’t know what it meant. They were fine with it.
Len: And you ended up in Vancouver at some point, and I wanted to ask how that happened?
Thad: I went to San Francisco when I turned 30 and lived there for 15 years, wanting to be closer to Silicon Valley, and also get away from the damn winters. And it seemed great. The city’s fantastic, and it was so close to all of the technology of publishing. I thought that would be a great opportunity. I’m a dual citizen, because my father was born in New York.
And so that was a great move, and I was there till early in the new millennium. And then my mother got cancer. She lived in Toronto, and I moved back to Toronto to look after her for what ended up being three years. And then found myself like, “When will I go back to San Francisco, or what am I going to do now? I don’t really like Toronto. I got away from here, I don’t really want to be back here.” And a friend had an opportunity with a house in West Vancouver, right on the ocean. And so I moved out to Vancouver, and I’ve been there since. Very much in love with the city.
Len: Thanks for that answer, it’s great to learn how all these things connect from the past to the present.
Thad: You’re in Victoria.
Len: Yeah, that’s right - I’m in Victoria, British Columbia. I’m sort of relatively new here. I’ve learned that all winters in Canada are not bad.
Len: Just 99%.
Thad: Where did you move there from?
Len: I moved here from Montreal. I’ve moved around a bit myself as well.
Thad: Plenty of tough winters there.
Len: Yeah, and before that I was in England for a few years, so I’ve seen a little bit. The joke I tell to my American and British friends is that I now live on a Canadian island in the Pacific Ocean. Just something most people haven’t even really heard of.
Len: Your latest book is called, Mobile Strategies for Digital Publishing: A Practical Guide to the Evolving Landscape, and it came out, I believe, in 2015?
Len: I wanted to ask you - we don’t need to go into it in depth, but what are some of the mobile strategies for digital publishing that you write about in the book, and what’s your general opinion about the current state of affairs? I guess that is a big question.
Thad: The core kernel that is most important on this topic is this: that everyone has to recognize that the smartphone is no longer an adjunct, it’s no longer sort of, “Oh I have a smart phone as well.” Or, “I work on a computer, and I have this smartphone for when I go out.” They have to get their mind around the idea that people have smart phones, and then on the side they have tablets and computers. The smartphone is the centre of the universe for an ever-increasing majority of the people that are owners of electronic devices.
And so, for publishers who have always seen the smartphone as some kind of globally adjunct, to be, at best, accommodated. My message is: No, start by thinking about the smartphone. When you’re developing content, you should be thinking, “How is this going to display if someone’s reading it on a smartphone?”
And then you can figure out what it’s going to look like as a printed book. Because you know how to do that, you know how to make printed books. You’ve been doing that throughout your career. But do you know how to make something optimal for a smartphone? Of course they don’t, and they’re still not getting their mind around it fully.
At the time there was this distraction around apps, and we all thought that maybe the way to go, the way to embrace the space was via apps. And that hasn’t turned out to be true. A lot of companies have done some interesting innovative things around that space, but that hasn’t been the answer. And so is it in a browser? Or do we just accept that the default apps - Kindle app, Kobo app - that that is the interface by which books are communicated on smartphones?
Well, as of today - yes. Is that the way it’s going to be over time? I don’t think so. There’s a lot of transition still to take place. But my overall message to book publishers is start with the phone, and work backwards from there.
Len: And do you think publishers are heeding that message?
Thad: No, not at all. Not at all.
Len: And why do you think that is?
Thad: Publishers are - I don’t - there’s a - I’m trying to say it nicely. Publishers are not known to be technology-adept. It’s not been a technology industry, even though it is now a technology industry - which is a big argument I try and push to people. And in one of my presentations next week at DBW, I’m trying to convince the audience, in the same way you should be thinking of mobile first, you should be thinking technology first. You should be thinking of your publishing company as a technology company that also does this interesting artistic craft, these cultural artifacts. But this has to take place via technology.
But publishing companies are not in that mind space at all, to their great detriment, because they’re losing market share to self-published authors who are technologically adept. That’s not the only reason they’re losing market share, but that’s a significant part of it - until they get their minds fully around technology, they’re going to continue to be losing sales, as they’ve been doing over the last few years. And they’re going to continue to be outclassed by newcomers.
Len: That’s really interesting. It reminds me of an experience I had at the BEA, Book Expo America conference in New York, I think it was in 2013, when there was this panel of grand eminences, including the CEO of one of the Big Four, Big Five - do we say Big Four or Big Five publishers now?
Thad: Big Five.
Len: Big Five still, okay.
Len: I’ve heard both. So, a CEO of one of the Big Five, on one of the “Top 50” lists of the most important people in America kind of thing, was on the panel. And there were the heads of some other organizations. And I remember, the CEO or sort of higher-up of one big company actually picked up an iPad that he had sitting in front of him, and like turned to his right and waved it in the face of this big CEO, and said, “These things are real. It’s not a science experiment. They actually exist.” And there was, nonetheless, this wall, just this wall between him and the CEO of the big publishing company that was not penetrable. It was amazing to just see it. I mean it’s one of the things we all know, but to actually see it happening, in front of a big crowd as well, was pretty amazing to me.
Thad: I can well imagine. Yeah, they still see them as toys. It’s unfortunate, really unfortunate.
Len: It’s interesting what you say about technology as well. I mean, I’ve got this theory that there’s sort of two big cultures in corporate America right now. One is the sort of old time-y one, you might say, where domain-specific expertise is not necessarily required, or might even be frowned upon. And so an executive is supposed to be an executive. They’re supposed to be good at business, and it doesn’t matter what the business is. They have these universal skills that they can use, mostly networking and influence peddling and things like that. Those are important and powerful things.
And on the other hand, you have the domain-specific expertise leaders. A classic example from right now would be Elon Musk. Someone who is actually not afraid of typing and doing work, and literally getting hands-on. He doesn’t just go to the factory to put on a decorative hat and intimidate the workers, but actually really knows what’s going on.
It’s a really interesting question with software eating the world, as Marc Andreessen famously said - is it possible to succeed in business with no domain-specific expertise at the top anymore?”
Len: You don’t think so?
Thad: It’s not. No, not at all.
To me it’s like - forget it. If your senior management are not technology-adept, if they’re not comfortable and fully aware of what technology can do for them, then they’re not managing the company anywhere near it’s full potential, period. I don’t even want to argue with you about that. And not you - with any of them.
Len: I understand.
Thad: It’s unequivocal at this point, from my point of view. They are losing competitive advantage every day they fail to put that expertise in at the highest levels of the company.
Len: And I imagine that as a consultant with technical expertise, this must be something where there are lots of people out there who I’m sure are aware that this is something they require, and that they need to find that expertise somewhere.
Thad: Yes and no. As someone who’s been consulting now for 30 years, it’s never been so difficult, it’s never been so challenging to get clients because of these problems where the easiest way to deal with the fact that they’re not technology adept, is just to ignore it. And so to ignore it, is to ignore that there’s consultants out there that can help them. I’m very fortunate that I am keeping busy. But it ain’t easy. It’s certainly not like they’re lining up outside my door.
Len: That’s really fascinating. I mean, the years keep ticking away, and the big publishers keep not responding.
I was wondering if you wouldn’t mind talking a little bit about some of the big news that’s been happening in the publishing industry - at least for insiders, the big news over the last, let’s say year and a half or so, about data and the analysis of ebook sales versus print sales?
One thing one hears from one side, is a celebration of declining ebook sales. And one hears from the other side no celebration because, first of all, on the other side, people like ebook sales, as one would think publishers would. But also, there is no decline in ebook sales, is what some people are saying.
I was wondering what your much more informed than my own position on that issue might be?
Thad: It’s a great question, and a complex issue. Let me try and give a really short answer, which is hard for me to do. But I’ll try and do that, otherwise I could sort of head off for 20 minutes on this. But you can then ask for some clarification.
Short answer: Yes, ebook sales are declining for the Big Five and for, as it turns out, a pool of about 1,200 publishers, who are measured regularly by the Association of American Publishers. There is a measurable decline. The factor that seems most certain to be the reason for that decline, is the fact that the prices have gone way up, as a result of a legal thing that went on between Apple and the publishers, and the Department of Competition in the US.
So they gave the publishers the ability to up the prices, and they did. And ludicrously so, in my view. But they did, and coincident with that, the sales have gone way down…. There’s one other issue, but that seems to be the biggest one.
At the same time, self-publishers are growing, growing, growing. They’ve reached some kind of a plateau recently. But the growth has been enormous over the last decade in self-publishing. And so all of that, 97% of self-publishing activity is digital, not print. And so all of where the big publishers are saying, “Our sales in ebooks are down,” while these other people who really are your competition, and you’re then….
Part of what they can’t get their mind around is that, “Well this little person, this little self-publisher, they’re not my competition.” Well, en masse they are your competition, and there are tens of thousands of them. Hundreds and thousands of them. And they are succeeding where you are failing.
And how are they succeeding? Both in pricing, of course, but also in understanding the technology, and where the technology intersects the marketing. They understand that far better than the large publishers.
And so that’s the real story of ebooks. No, they’re not declining overall.
Len: I watched an interview with you on YouTube, it might have been with Joanna Penn… where you talk about - there’s a kind of irony where, Amazon is obviously gigantic, and is a pre-occupation of publishers and self-publishers. But actually books - which it was known for initially - are a very small part overall of what Amazon does, and so the irony is that although books and ebooks, they’re a smaller subset of book publishing generally, are, to the rest of us, a huge industry - the book publishing industry is like 150 billion dollars a year worldwide, it’s bigger than other forms of media - but for Amazon, they’re so big that books are a small part of what they do. And yet, they somehow just effortlessly dominate all these other companies. And these companies have become almost entirely reliant on, or existentially dependent upon Amazon doing its job well.
I was wondering what your position might be - if you were CEO of a Big Five publishing company, or if you had been for the last 10 years, what would you do with respect to Amazon?
Thad: Yeah, that’s the huge elephant in the publishing room. There’s a lot of antipathy towards Amazon, and understandably so. But forget that. Almost all of my clients, Amazon is now their largest single customer, and growing. The numbers suggest that Amazon controls about 75% of ebook distribution in the United States.
So most of my data - I’m very US-focused, even though I’m Canadian-based - my career was built in the US. So you’ll forgive me that my figures generally are US-referenced. In some cases, I’ve also got Canadian data. But generally speaking, as we’re used to the publishing industry, in Canada it pretty much mirrors the structure of the US company, with some very significant differences. But anyway, to use US data is not to greatly mislead about Canada….
Len: We’re based in Canada, but basically our we have the same issue with our audience.
Thad: In the US, [Amazon has about] 75% control of ebooks - about 50% control of all book sales - physical and digital. So it’s like, the contest is over, Amazon won. And you can cry about that all you want. But it’s not going to help you at all. Amazon won.
A colleague of mine, Ted Hill, who’s running the DBW conference next week, has a lovely way of putting it, where he’s able to provoke a response or a thought about a response. “What if Amazon was not merely your biggest customer, what if they were your only customer? How would you run your publishing company if they literally took over the rest of the market?”
Which - in fact, I mean, if you track the trend - at some point, that’s not a completely ludicrous thing. But as a brain exercise, it’s really important to think through if there is only one distributor, and that distributor is Amazon. And we know how they behave. What does that do to publishing?
Well it’s not all negative, because Amazon’s a fabulous marketer. And you’re suggesting in what you say - yeah, books are an insignificant portion of their revenue. But part of what makes Amazon so awesome, awe inspiring, is that markets they don’t even care about from a fiscal perspective, they run them as if that was the only thing they did. And so in book marketing - they continue to be incredibly innovative, incredibly aggressive.
And it’s not getting any easier to work with Amazon in the same room, because you have to pay attention to every little movement they make. All of which are designed to sell more books. But also for Amazon to sell them at the expense of any other company.
They’ve decimated Barnes and Noble. Barnes and Noble, it’s just a very sad last few years for them. One can be critical of Barnes and Noble, and there’s lots of reasons to do so. But if you or I were running Barnes and Noble, we would’ve run it into the ground too, because competing with Amazon is a mug’s game, you just can’t do it.
However, from a publisher’s point of view, it’s like, embrace the beast - you’ve got no choice. And for the future of your company - you’re not single-handedly going to stop them. If anything stops them, it’s going to be a groundswell - a very innovative groundswell that none of us perceive at this point. But it ain’t going to be you trying to single-handedly work against them.
So for all of my clients, I say, “Embrace them wholeheartedly. Put on a big smile, even though you don’t feel it. Because these are the people that are selling your books better than anyone else in the world.”
Len: That’s a great, and in my experience, pretty original answer. It’s just so fascinating - you mentioned Barnes and Noble, and the Apple and the big publishers controversy from before. I mean it’s just - it really is in it’s own way, a kind of fascinating comedy. I remember reading a quote - I think from the president of Barnes and Noble, blaming declining sales on the election in the US, because people are afraid, and they’re just staying at home watching cable all day.
I mean, of all the things…. I think there was an article in Publishers Weekly, where they referred to the price fixing scam that was happening at many levels of the publishing industry as - I think it was “a government imposed price reduction.”
Len: The willfulness of it is the thing that really fascinates me, because it it indicates that underneath, people kind of know what they’re wrong about, and why they’re wrong. But there’s something about what’s happened in the last 20 years to publishing that a certain type of person just can’t face up to. And what’s interesting is that it’s not like some kind of deep economics that you’re watching, or like business strategy. It’s something psychological, even at positions of prominence and responsibility. It’s down to some personalities, and their own preoccupations and, I mean, what to you is like something you accidentally discovered, which was the empowerment that comes from new technology, to other people, was Armageddon Day.
They just see the things that they associate with publishing, which were material. Like your typesetting, and stuff like that falling away and falling away and falling away. And they feel like - I think at least my view is that they feel like we’re losing literature, or we’re losing knowledge, because we’re not doing things on paper anymore, and -
Thad: It’s preposterous.
Len: Yeah, and there’s this really interesting conflation of the subject with the material.
Thad: The artifact. Yeah. We have to get away from the artifact. From the concept of the artifact. We have to look at each medium, however we like to do that, as an enabler. As one more opportunity to get the word out. To bring in new readers, to bring in new opportunities. These things are enablers. They’re not artifacts.
One thing you’re saying there reminds me of something a colleague told me 25 years ago. He said, “The only sustainable, competitive advantage is understanding and adopting technology faster than your competitors.” And the point of that with book publishers is, business is debugged. We know how to run book publishers in the traditional way. There’s nothing left to discover there. There’s nothing you can know that your competitor doesn’t know.
The only thing you can know that your competitor doesn’t know is how to, for example, use metadata more strategically than they do. How to maximize the efficiencies of the EPUB format better than they do. Understanding the supply chain, and how metadata informs the supply chain, faster than they do. Aside from that, you have no competitive strength. And that to me is sort of the summation of where we stand as an industry. Technology is the only thing that’s going to save you, let’s put it that way.
Len: On that subject - moving from the big to the small, I suppose - you very recently - just three days ago - released a report called, An Authoritative Look at Book Publishing Startups in the United States. I was wondering, it’s a very long list of companies that you’ve compiled there, and some of them are failed companies. Some of them are plugging along. Many of them were at least attempting to innovate in one way or another technologically. And I was wondering if you could talk - I mean, to begin with - a little bit about what the origins are of this report, and why you were interested in writing it?
Thad: Sure. About five years ago I was on this panel at Tools of Change on the topic of startups. And I was the contrarian on the panel. And the four other people on the panel were all like, “Thad, you’re just being very, very negative. These are amazing opportunities. These are some amazing companies. And they’re going to flourish. And the book publishing industry is fertile ground for startups. Your negativism is completely incorrectly placed.”
I can be a negativist. And so I was a little bit stung by that. But it got me to thinking, “Well, why don’t I go deeper on this, and really find out what’s going on here?” And it turned out to be quite an interesting rabbit hole. I found 900 companies, and started sort of hoarding, collecting.
Over the next five years - every time I heard about a new startup, I would download the information on that startup, if it came out of a blog post or something in Publishers Weekly. Or someone would tell me about it, I’d go to their website and get their mission statement, and add that to the spreadsheet. It kept growing - 300, 600. I did an interim report when there were 600. Then now it was up over 800, it’s like, “What am I going to do with this?”
Well, I decided in the end - I’ll distribute it freely to the industry so they can get a look at what the startup scene is. And I did some quantitative analysis of the data, to try and understand what areas these startups were working in. What kind of funding they got. Whether they’ve had any mergers. Whether they’ve been acquired. A couple of them had gone public. How many had gone out of business? About a third of them have. So that’s the knot of what’s in this report that came out earlier this week.
Len: Speaking of your negativity on that panel five years ago, I looked at your slide deck related to that talk, and I was curious - have your views changed about book publishing startups in the last five years? How would you characterize the current state of affairs facing startups?
Thad: Good question. As I was doing the report, the cynical side of me was reminded - some of these startups are so goofy. And not only is this startup number 231 goofy - that’s just a number - but then you find out that startup number 427 has got the same idea, and launched six months later. And so we have one goofy idea has not made this company successful, and another company’s coming in with the same goofy idea, and they’re going to try and do it too.
And so there’s a lot of that - as I was building this list, I’d find out about a new one, and I’d think, “Oh a new start up, going to add them to the list. Oh my God, their mission statement is the same as 23 others of these startups, a third of which are already out of business.”
So what I saw going on there, is - there’s a cult of startups, right? - that I’ve pointed out a little bit in the report. But we know it, right? I mean the media - whatever newspaper or website you’re on - as long as that company can say, “We’re a startup, and we’re looking to disrupt this or that,” the media eats it up, because the public eats it up, because there are so many glamorous stories around the magic billions that could be made out of thin air six months after startup.
I understand what the allure of that is. But people have to get down to earth a little bit more, and realize that just because they said they’re a startup, just because they’re enabled by the web, just because Mark Zuckerberg exists, doesn’t mean this is a good idea, or that this company’s going anywhere. So that’s the downside.
On the other hand, what you’ve got is a lot of smart people - committed, willing to put their careers and their livelihood on the line, to try and bring some real innovation to the publishing industry. And that’s a great thing. And that’s something that I keep reminding myself of. That bottom line is like, “Thad, forget about the dumb ones, focus on the interesting ones.”
And so I’m hoping in the months ahead on my blog to profile as many as I can that are the most innovative, the most intriguing. Even if they’re very, very small. There’s some that really do have some nifty ideas, and I’d like to spread the word about the best of them.
Len: One example of a type of startup that was backed by really smart people, and run by really smart people - and had talented staff, is Oyster.
I bring them up, not to pick on them, but because their particular approach was very interesting. It was a subscription-based model. I mean, in the end the end they had a book store, but primarily, their business was aimed at people who they believed would pay a monthly fee to have access to loads of books. There’s other examples. Scribd had something similar with romance books, that sort of went belly up. Oyster’s team - just for those listening - got acquired by Google, I think. Or many of their team, or some people on their team were.
Now the example of a subscription model that appears to be working is Amazon’s Kindle Unlimited, although that’s a very curious piece. I was wondering what your opinion is? Looking back on the last couple of years, do you believe in book subscription models? Do you think they’re something that can succeed for a business? And on the other side, do you think it’s something that a self-published author should put their material into?
Thad: Right. To me, at the time - it’s easy now retrospectively, because they have failed to say why it wasn’t a good idea - but at the time, before they failed, there was a lot of hype surrounding it. And I was thinking, like - subscription model, all the books that you would want to get hold of, isn’t that called the public library? And I think I’ve already got that, and I don’t pay an annual fee, and if my branch doesn’t have it, they have inter-branch loans.
So the only thing that made sense on the subscription model to me was, it’s kind of instant-access, not having to go through the queue at the library, or, for the popular books, you have to wait a couple of months if you want the ebook version, until it becomes available. Or even for the print version, sometimes you’re on a list for a couple of months.
So for instant gratification, a subscription model made sense. Except there too - it fell apart because no matter what, they were going to be missing some of the books you wanted. Beause they– Even if they had the Big Five, which they didn’t - but let’s say all the Big Five agreed to it, that’s only 50% of the trade books published by established companies, leaving out the self-published authors. Only 50% of the books published each year in the United States, come from the Big Five. So it’s another several thousand publishers that you need to sign up if you want the other 50%.
So it was never going to be a place where I could say to myself, “Oh, I’ll just go onto Oyster, because I can get it right away.” I can go onto Amazon, and I can buy it right away. So that’s already available to me, and it’s only a few bucks and I can have it instantly. So the subscription model to me was never a profoundly smart idea.
And what Oyster ran into was a particular problem, whereby they couldn’t get the publishers on board. And so they had to pay way too much for these books, which was not economically feasible, and hastened their demise. Scribd is still in business for various reasons. But virtually all the subscription ones that are on my list have gone out of business. And so to me, that was a combination of the publishers undermining them.
But also, the primary value proposition had not been fully developed. It wasn’t that compelling. I mean, is it our manifest destiny, as Kevin Kelly would argue, that every book is going to be online and available to us on a subscription model eventually? That somehow we’ll figure out the way to have one source. But would it be a public source? I don’t know. Will it be Amazon? Who knows?
But our manifest destiny, or the way it would best work for people is - yes, you could access any book at any time - in the moment that you’re interested in doing it. You don’t have to purchase it. You can have it, borrow it, with some sort of funded subscription model, or whatever it would be. That makes sense to me over time. How we’re going to get from here to there, I don’t know.
Len: And if you were advising a self-published author, or a group of self-published authors, would you recommend to them that they try Amazon’s Kindle Unlimited?
Thad: That’s another tough one. Go ahead.
Len: I was just going to say - yeah, for those who are listening - one of the interesting things about this service is that if you’re a self-published author and you put your book on there, Amazon imposes some restrictions on you, on what you can do with your book, and where else you can sell it, and how much you can sell it for, things like that. But also, as I understand it, the money for a subscription service just goes into a big pool. And an inherent and inescapable part of the process then, is that you have to decide how to divvy up the money, and that calculation is entirely up to Amazon. I think their latest way of doing it is to look at how many pages have been read in your book.
Len: And then this is something that immediately a lot of enterprising characters began taking advantage of. By figuring out, for example, that Amazon wasn’t actually counting the pages that were looked at. It was just looking at the highest number page that had been looked at.
So what people were doing was, basically, putting up more or less fake books that were hundreds of pages long - let’s say - and then having a link at the beginning of the book to the last page of the book. And then that would be sending the signal to Amazon that someone had read all 580 pages of “cat cat cat cat.”
And so there’s this inherent murkiness to what goes on in a subscription service like that. And you’re also susceptible to change. In the same way that internet companies can be screwed over by Google changing it’s algorithm for search results - like one day you can see your ranking fall dramatically. Similarly, you’re exposed to Amazon just deciding one day to count differently, and divvy up those funds differently.
Thad: Yeah, it’s a hornet’s nest, absolutely.
So let’s think of the downside of it. The biggest downside is they demand exclusivity. You cannot distribute your book outside of Amazon, if you want to be part of Kindle Unlimited. That’s a big restriction. And for certain authors of certain genre fiction, to get that extra boost just through Amazon, can offset the potential loss of sales from other distributors.
But if you’re an author of substance - if I can use that as a broad rubric to suggest the more committed fiction writers, and all of the non-fiction writers who are trying to produce books that really add to the canon - it’s an unacceptable restriction. There’s too many other ways that you will miss being able to connect with readers, if you stick with that restriction. So it works best for genre fiction. And it is a chunk of change.
I mean they put 15, 16 million dollars a month into that fund. So in fact, if you think about it on an annualized basis, it’s upwards of 200 million dollars, of what amounts to royalties that are going to the self-published author community. Economically, it’s very, very significant.
But the bottom line for me as a self-published author is - as well as all these other things - the books I have that I self-published, my books are technical, they’re not exciting mysteries. It’s out of the question that I would use Kindle Unlimited. It would just restrict my audience far too much.
So then, who are the audience for Kindle Unlimited - they do have to pay a subscription fee. In the old days, when there was only romances, where the only really consistent genre was, like Harlequin in those days, they would publish weekly, get a new book every week. There was a defined audience of people who wanted a new romance per week, and they weren’t particularly fussy as to which author. They liked a particular sort of sub-genre within romance.
Well that’s spread now, where there’s a lot of people who feel the same way about certain kinds of mysteries, certain kinds of thrillers. They like to get a book a week, or even more than a book a week. And this fulfills that. There’s enough good stuff in KU, that it’s cheaper to be a subscriber to that, than it would be to buy those books individually. But that’s a pretty narrow use case.
And KU, Kindle Unlimited gets a lot more press than it really needs to. I guess it’s become sort of emblematic of Amazon’s power, and ruthless use of power, and ability to - as you say - change the rules according to their whims, and to be exposed to scammers. So it is a very visible thing. But in terms of, really, its significance as a phenomenon for the book publishing industry, it’s much less than meets the eye.
Len: Looking towards the future, in a recent blog post, you wrote - and I’m going to quote you here: “Central to the future of publishing, is understanding where AI intersects with traditional book publishing.”
I was wondering if you could talk a little bit about where you see artificial intelligence intersecting with traditional book publishing and why you think that might be a powerful feature of the future?
Thad: The starting point for my interest in AI and book publishing - well it actually goes back a few years. Let me try and give a short answer and see how that works.
There’s a category of publishing technology that has to do with expressing the meaning. Rather than just saying - if we have a whole paragraph describing a politician’s education, just the education of that politician, and we’ve got 400 words within a much larger biography of that politician, that talks about her time in high school and college and the education that she gained, well, the abstraction of that is semantic. It would be xyz politician - education, college and secondary education. So there’s that abstract layer, in which we describe the meta meaning of a type of content. And so, when you get at that meta level, you have another layer at which you can process content. Where you don’t have to regurgitate every single word in the book.
And so we can get it to the point– Like if we can abstract it to that level, well then someone who’s interested in the education of politicians can do a search of everything that’s been classified at that semantic level, and be able to much more quickly find [what they’re looking for], because it’s not a whole book about that; they could find all of the sub-sections within all of the published literature to cover that particular topic.
And so it’s a potentially very powerful way of classifying this enormous, uncontrollable, multi-billions of words that have been published, and remain in print - from books that have been published over the centuries, literally now. And this then starts to suggest that there are data mining methods that might increase our access or understanding of published work. That was a starting point for me - trying to understand the semantics and how that would intersect with book publishing.
The next thing that happened is Google, Yahoo and Yandex … in Russia came up with a standard called schema.org, which is based on semantic representations of content, and allows those search engines to understand content better.
If you make the effort to explain what the content is at a semantic level, those search engines can do a better job of locating your content. So that put a big impetus on the publishers to start to get a handle on semantics as well.
So then, the next thing that happened that triggered me, was this book that you would’ve seen on my blog, where I reviewed it in a couple of posts, The Bestseller Code. The Bestseller Code is a fascinating book, where the two authors are both scientists.
It’s not just an exploitative look at some secret little method that they came up with, that’s the magic bestseller code - no, these people have done some very extensive analysis of the patterns of the words in bestselling books, of the sentiments that are aroused within, by those words, types of characters, the plot - all those sorts of things.
And they have come up with a formula, a code - the “Bestseller Code” - that identifies the commonalities in the books that have become New York Times bestsellers. And while they aren’t willing to offer it as a prescriptive - i.e. “Do this, your book will become one of those best sellers” - their interim report is really what the book amounts to, where they’re able to say, “We have in fact come up with a scientific quantification of books using the techniques that we now put under the broad rubric of artificial intelligence.”
Things such as sentiment analysis, text mining - you know what I mean. There are a whole set of secondary technologies that are thrown under the umbrella of artificial intelligence. And so this was the first really big, obvious use of the tools of AI towards a specific outcome.
And certainly as a publisher, you just have to start scratching your head and saying, “Okay, so could I use some of this text mining, sentiment analysis to evaluate incoming manuscripts and be able to make some kind of a qualitative assessment.” Based not just on human reading of that. Maybe you could? These are some of the questions that are posed by that.
So coming back to the question that you’re posing to me. What my feeling is, is that the science of artificial intelligence is coming forward by leaps and bounds. The publishing industry has enormous amounts of data in the form of text. And there is an intersection point between those two things that we’re just beginning to get a look at, but which I think is going to be quite profound in the years to follow.
Len: That’s a really great answer. It was sparking in me thinking about Netflix. Netflix is well known for having highly detailed metadata around its shows. It can be like, this person likes movies that are kind of like Quentin Tarantino movies, which have characters named Joe, who dies in in act three, when something falls on his head. And they could categorize the same show many different ways like that.
What people talk about, is how they’re actually using all that viewer information, both to encourage the discoverability of things that people will want to watch, but actually also to create new shows. They base their decisions for what to do, what shows to choose, and perhaps how they should be written, what should happen to the characters in them, based on this information that they have about people’s viewing habits. And it’s a fascinating future ahead of us, where that kind of data can be used to make decisions, and put it in front of people, and then iterate on it and see - well did that work, or did that not work? You can follow the experiment through and look 10 years ahead to what you want to do.
Thad: Yeah. The Netflix example is a particularly good one to spend some time with. My brief comment on that would be, I was initially quite thrilled to learn about what Netflix has done. And you’ll probably remember the Netflix Prize, where they offered a million dollars to a team of scientists that could improve the accuracy of the recommendation engine by, I think, 10%. So it’d get 10% better at successfully recommending a next viewing product to customers.
There’s a whole separate story that is a fun little story on how it played out. But it points to what you’re talking about. At Netflix, they have a lot of scientists on staff looking at how these things can be solved, towards the ultimate task of - can we develop shows, whole shows - where we plot them, develop the characters - which kind of actors are we going to hire? What’s the plot arc going to be? And can we create successful series on that basis?
House of Cards was the first poster child for that outcome. What I point out to people these days is, get a listing of Netflix series that have launched in the last three to five years, and find out how many of them are still being broadcast. They’ve had a lot of failures.
And but people keep talking about their successes. They’ve had a lot of failures. And this is no slam on Netflix. It’s only a reminder that this technology is still very much in flux, and that people are not robots. So you cannot predict with 100% dependability or anything near that number, using science. But you can get closer to it. And so the smart, the best users of the technology are those who fully understand its limitations.
Len: Thanks for that - on the note that people are not robots, it’s something I very much agree with, and it’s an optimistic thing - although I do like robots, too.
I just wanted to say, thanks very much for a fascinating interview, and for giving me and all of our listeners the time. Good luck at the Digital Book World Conference in New York this week, and all the best from me.
Thad: Thanks so much. I enjoyed chatting with you. You are better informed than anyone I’ve spoken to in the last several years, in terms of preparing for the interview. I’m really pleased, because it just made it a lot more fun that you’d taken the time to look at things and ask such smart questions.
Len: Well, thanks very much – and I’m going to leave that in.
Len: Okay, thanks Thad.
Thad: Great, my pleasure.