Leanpub Podcast Interview #20: Paul M. Jones
published Nov 09, 2015
Paul M.Jones is an internationally recognized PHP expert. Paul is the author of the Leanpub books, Modernizing Legacy Applications In PHP and Solving The N+1 Problem In PHP. In this interview, Leanpub co-founder Len Epp talks with Paul about his career, the origins of his interest in PHP, and his experience self-publishing technical books.
This interview was recorded on July 13, 2015.
Paul M. Jones
Len Epp: Hi I’m Len Epp from Leanub, and in this Lean Publishing podcast, I’ll be interviewing Paul M. Jones. Based in Burns, Tennessee, Paul M. Jones is an internationally recognized PHP expert who has worked for organizations in many different sectors, including medical, non-profit, educational and military organizations. He is the lead developer of the Solar PHP framework, and lead on the Aura for PHP project, and was a founding contributor to the Zend Framework. Paul is a regular speaker at technical conferences worldwide, and blogs at paul-m-jones.com. In a previous career, he was an operations intelligence specialist for the US Air Force.
Paul is the author of the Leanub books, Modernizing legacy applications in PHP, and Solving the N+1 Problem in PHP. In Modernizing legacy applications, Paul explains how to get messy legacy PHP code in order, using a series of specific steps to turn it into an organized, modern, testable application. In Solving The N+1 Problem, Paul explains what the problem is and how to discover and solve it in your app using PHP.
In this interview, we’re going to talk about Paul’s professional interests, his books, his experiences using Leanpub, and ways we can improve Leanpub for him and for other authors. So thank you Paul for being on the Lean Publishing Podcast.
Paul M. Jones: Thank you for having me.
E: I usually like to start these interviews by asking people for their origin stories, so I was wondering if you could tell me how you first became interested in programming?
J: This is actually one of my favourite topics. A lot of people go through life and never find their true calling. I got my true calling very early in life. When I was just shy of 13, my dad brought home a TI-99/4A Texas Instruments computer. If anyone out there knows what one of those is, you know exactly how long ago that was. I sat down with that thing for a couple of weeks, and by the end of that time I knew what I was going to be doing for the rest of my life. I started out programming in TI-BASIC, when you had to save the program to a tape cassette recorder. And then you had to play it back to load it back into memory. So that’s how I got started, and it’s just been - it’s been a wonderful experience since then, learning how to program, learning the craft and the skill that goes along with it, and all the communities that go along with it as well.
E: Was your father a programmer himself?
J: No, my father, first he was an Army Chaplin, then he was a Methodist Minister. And then he became a certified financial planner, which is sort of a minor version of a stock broker. So he himself was not especially technical. I ended up having to help him out a lot with the various computers that we ended up buying for his businesses. I remember we had an Apple III for a long time. And I was in charge of helping him set up a lot of VisiCalc stuff to begin with.
E: It’s interesting, a lot of–
J: For good or bad, I still have to help him out with a lot of computers.
E: It’s interesting, a lot of people I’ve spoken to, their first introduction to computers is playing games. But it sounds like you just dove right in and started messing around with the machine?
J: Oh course there were video games at the time, but they were stand up, coin operated things. You did not download the latest version of whatever it was to your iPhone. If you wanted to play a game on your home computer, most of the time what you had to do is go buy a magazine - a physical paper magazine, look for a code listing in that magazine, and then type it in by hand from the magazine into the computer that you’re working on, and then save it.
J: And then try to figure out how to debug it from there.
E: Do you remember what the first game was that you played on that computer?
J: Oh wow, that’s a great question. I remember one of the first ones was a text adventure-style game. Again, I don’t remember specifically what it was. I think that was one that worked on both the TI and on an Apple II that I had access to at school. And then after that of course, you ended up buying games. But we were not super wealthy back then, and games were pretty expensive - comparatively speaking. So if you were going to buy a game, you either had to shell out 20 or 30 dollars for it, and wait for it to come in the mail and then put it in. So it was easier to spend the time than it was to spend the money.
E: And how did you end up with a focus on PHP in your career? Can you explain a little bit about your path to that specialty?
J: Yeah, when I was in the Air Force, I did some programming there was well. I worked with databases a lot, worked with FoxPro. And then around 1994, we got a copy of the original Mosaic browser, that we used on internal classified networks. And of course to program that, you would have to write up pages using plain old HTML. So I started doing that, and realized I liked it. When I got out of the Air Force, I continued working with web pages, that kind of thing, and trying to attach databases to them.
One of the first database systems that I worked with, attaching it to the web was Filemaker Pro - again, that should give you an idea of how long ago this was - using, I think they called it CDML, Claris Dynamic Markup Language. That was fun. I did that for one of the colleges that I was that I was working at at the time. And then I heard about this thing called MySQL, which was a real SQL database system - and this language called PHP that you could use to interact with that, and then generate a webpage from it. And that sounded interesting, so I started on that. That was in 1999, and I’ve just kind of kept on doing it since then.
E: You mentioned that you were coding when you were in the military as well. I saw from your bio that you’ve worked for a number of different types of organizations, and I think people might be interested in knowing if there’s any sort of stark difference between, say, coding in a military organization and coding for a - say a company or a non-profit. I guess it would depend what sector they’re in?
J: Yeah exactly. And my programming work in the military was sort of secondary to the work that I was doing. My primary work was as a - it’s called an operations intelligence specialist, as an enlisted guy. But they put me in charge of training other people who were in the organization. And then to keep track of a lot of that, a lot of that training - I ended up with, like I said, FoxPro and a couple of other things. So the programming for that was, again, secondary to the job. It was in support of that job. Programming there did not strike me as especially different than programming for any other client on the outside. But then that’s because it was a - sort of a small, local client compared to the rest of the military. It was just from my organization.
So I have not found any dramatic differences other than the need to make sure that whatever you did in the military stayed in the military. There was no open source, anything like that. So everything you did had to be kept under wraps. Whereas out here, in the civilian world, you’ve got all this open source stuff, which is fantastic to work with. And you can share both your solutions and your problems with everyone else, and have everyone else look at what you’re doing. To tell you what you’re doing wrong, and maybe help other people figure out what they’re doing wrong. So there was that difference.
E: That’s really interesting. There’s obviously been quite a bit of press lately about US personnel hacks and things like that. Do you have an opinion in general about the security of data in the US systems?
J: We’ll we’ve all heard of Murphy’s Law - anything that can go wrong, will. There are military variations on that law. One of them is, “Always remember that your weapon was built by the lowest bidder.” Unfortunately the same thing is true for government data programs - anything that’s related to data, it’s built by the lowest bidder. So I think this is an example of where that comes back to bite you really fast.
E: That’s really interesting. When the ObamaCare website fiasco unfolded, I had this joke that winning a government contract is an entirely different set of skills from actually making a website or doing anything. So I always thought of the problem in terms of just procurement. But that’s pretty straightforward criticism when you say that the lowest bidder wins.
J: Yeah, that’s pretty much it. In fact when you talk about ObamaCare, it doesn’t matter what your political affiliation is, how you felt about it at all. As a project management case study, it is fascinating. It’s almost a list of everything you could possibly do wrong, all combined into one project. There’s a guy named Arnold Kling, who is an economist. He’s blogged about that in the past as well, if you’re interested in an economist’s point of view, from a guy who’s not only an economist, but was also a programmer and built websites during the boom. He’s got a series of very interesting criticisms about it. So does Megan McArdle. I think she works for Bloomberg now, and is also a technical person. She wrote a lot about the ObamaCare fiasco, again from a production point of view. All really good stuff.
E: And do you think that it’s possible that - to put it broadly, the government has learned a lesson from that in the future? Are changes to procurement processes just so difficult to change?
J: It’s my guess that the inertia is not in favour of things getting better. None of the incentives are right. With government, again, this is an economics thing. When you are taking someone else’s money to spend money on something that you yourself are not personally vested in, then you’re not going to be careful with how much money you take. And you’re not necessarily going to be careful how you spend the money. And you’re not necessarily going to hold anyone, or any of the right people accountable for how it gets spent. And that’s regardless of where you stand politically, it’s just a set of economic incentives.
E: Moving back to PHP, I know that it’s gone sort of up and down in terms of reputation in the last ten years or so. What’s your take on that, and how things have changed just in the last couple of years?
J: Depending on how you felt about PHP ten years ago, it had nowhere to go but up in terms of security. There were a lot of security flaws in the language itself. The primary problem was not the language itself, in my opinion. The primary problem was that people who were drawn to PHP were not necessarily professional programmers. They were not people who had security concerns at heart, they just wanted to get something done quick. Get it on the web. Because either it was going to make them money, or they needed to do it for their own internal organizations. Ao they weren’t necessarily thinking about security concerns right off the bat, because they weren’t targets.
And then a couple of years later, suddenly everyone was a target. With cross-site scripting exploits or SQL injections they can take over your machine. That’s the point at which the community started standing up and paying attention to those kinds of things. You started seeing things like the filter extension put in place. I think that was Pierre-Alain Joy who did that, which was a great boon. But in addition to that, there was widespread education, if by no other means, then by word of mouth among developers, that these kinds of security flaws existed and you needed to watch out for them. So I think that security concerns have been addressed by the language, but it’s been primarily addressed by the better education of the people working in the language.
E: Okay great, thanks. You’re the lead developer for something called the Solar PHP framework. Can you explain a little bit about how you got into that and what it is?
J: Solar is actually an older one at this point. Way back when, in 2002 or 2003, I became more aware of a project called PEAR, that’s P-E-A-R, a collection of libraries. And I wrote a small, what I called the foundation, not a framework, because back then the word framework was a bad word to use in PHP communities. So I called it a foundation, and it was a collection of libraries from PEAR, to do all the basic things that you needed to do, like database connection, authentication, caching, logging - stuff like that.
I realized when putting this together from those other libraries, that none of the libraries really worked the same way; even though they were all part of the same project, they were all from different authors under one banner. So they didn’t look the same way, they didn’t feel the same way, you didn’t call them the same way. So, with that in mind, I decided that I wanted to start another project, where those individual libraries would all look and feel the same way. This was right around the time that PHP 5 came out. So I figured it’d be a good time to make a break with that older system, and start putting together a new set of libraries.
Solar originally stood for Simple Object Library and Application Repository. But it ended up being this monolithic thing, where you downloaded everything all at once, and used all of it whether you wanted to or not. That turned out to be the standard for framework projects at the time. So that was the origin of it, that’s how it got started. A lot of people got interested in that and started contributing as well. It was nowhere near as popular as say the Zend framework or Symphony 1 at the time. But it did have what I would call a small but committed community, and I learned a ton from writing it, and I learned a ton from the people who provided patches and helped to work on it as well.
E: And the Aura for PHP project - that’s a newer thing that you’re working on?
J: That’s exactly right. Aura is essentially Solar version 2. One of the problems that we had with Solar first of all, was the name. It’s S-O-L-A-R. At some time thereafter, the Apache S-O-L-R - SOLR project came out, and people started confusing it with that. So when Solar the PHP project went 1.0, we started discussing how we were going to do the 2.0 project. The first thing we decided was that we needed to change the name. So we settled on Aura as sort of a pun on the name. Au is gold, a sun symbol, and Ra is a sun God, so Aura. And we decided that the new project should be more - should adhere more closely to the original ideas that we had had when we started. That is a series of individual libraries, not something that’s made to deliver, there’s a monolithic framework. So our primary goal was to take the SOLAR stuff and split it off into individual, independent, decoupled components, where they would be independent not only from a particular framework, but also independent of each other.
So those were the driving principles behind starting Aura as version 2 of Solar. I think it’s worked out pretty well. We moved away from a universal service locator to using dependency injection and providing a dependency injection container that you could use if you wanted to, and then the various versions thereafter of the individual libraries, we’ve been able to split those into even smaller components. For example, the Aura SQL component used to be a database connection, a connection manager, an SQL query builder and a gateway and mapper system. We’ve actually split that out so that in version 2, the SQL query builder is its own thing. It’ll work with any database connection at all. You don’t even need one, you can just build the queries at random if you feel like it. And the SQL connection portion just does that. The mapper and gateway then split off in their own components as well. So it’s been a story of reducing the size of the individual packages so they can be recombined in any way you like.
E: Great, thanks, that’s really clear. I was wondering what motivated you to write Modernizing legacy applications in PHP on Leanpub?
J: So the first motivation was, I wrote a talk called, “It was like that when I got here.”, subtitled, “Steps toward modernizing legacy codebases.” The motivation for that talk was that I had been in several organizations where we had these really old codebases - they were tough to work with. And over the course of several years, over several different organizations, I came up with a list of steps and notes for myself on how to reorganize these codebases and make them easier to work with, so that we could add features more quickly, fix bugs more easily and isolate things more easily.
So, the talk came out of a generalized version of a story that I heard over and over again, when you go into an organization and you look at the codebase, and it’s horrible. You ask the people who are already there, “How did it get to be this bad?” And they look at you and they say, “I don’t know how it got to be this bad, it was like that when I got here, and we’ve just dealt with it since then.” Of course that’s a lot of suffering in our daily lives. That’s a lot of pain and anguish. And you end up having this relationship with the codebase that is sort of adversarial. When you walk into work, you’re always kind of scared. “What’s going to break today after I try to fix something?” You spend a lot of hours late at night trying to make sure that things are going to work, because if you touch one piece of code over in one place, then something else somewhere else breaks. And it’s not really your fault, but you’re the programmer, you’re supposed to know what’s going on.
So after writing that talk and going through some of the initial steps of what I had done to modernize these legacy codebases in my own work, I gave the talk the first time, and it was well received. I gave the talk a second time, it was well received. The third time I gave the talk, some people came up to me who had attended the first one, and they said, “Yeah we’ve done it all, we’ve done everything you said, what’s next? Because it’s not really where it needs to be yet.” Well, I had this huge list of notes already, and it turned out that the timing was right for me to sit down and take all of those notes and compile them into hopefully a good “how to” instruction manual on exactly how to follow these steps, follow these principles, and through a series of baby steps, end up on the other side with a codebase that has gone from the spaghetti mess to something that is auto-loaded, dependency-injected, unit-tested, layer-separated and front-controlled. Based on the feedback, I think it’s been a success in those terms.
E: It’s great to hear when things develop from talks like that, and getting feedback from people that they like it, and then wanting to share it with a wider audience over time, and at any time when they want to read it.
E: I was wondering, in the introduction to the book, you define legacy applications, but you make some comments specifically about the nature of PHP legacy applications. I was wondering if you could maybe say a little bit about that?
J: Sure. First of all, the word legacy carries a lot of baggage with it. Normally when we think of legacy code, we think of something that is merely old. It’s five years or ten years old or was developed according to old principles, or it was merely “there before I got here.” And so by definition, every programmer who comes onto a job looks at all the code that was already there as legacy - no matter how good it might actually be. But in the work that I had done that led me to write the book, I discovered certain patterns cropping up over and over again, as to how these applications had been constructed. And some of some of the points in those patterns included things like globals being there, or there being evidence of having attempted to rewrite it using a framework more than once.
So you walk in, and you’ll see a codebase where you can see evidence they’ve tried to apply one framework, and it never really finished, and then another programmer came in later and tried to apply another framework, and that one never really finished. And so you’ve got this codebase with a mess of idioms in it from different systems. Or, there is a poor separation of concerns between what you’ve got; this is especially true in PHP. Where you’ve got a page script that sits in the document root, that has a lot of includes in it to execute logic. And those includes, when you combine them all, end up combining the concerns of the model and the view and the controller all into the same scope. We’re all sharing each other’s variables. So that the typical PHP application in those cases - it ends up looking like what I call an “include oriented” system, rather than class oriented or object oriented, or even procedural. It’s include oriented, and it’s page-based, because you browse to these pages that are sitting directly in the document root, and each page is responsible for setting itself up and tearing itself down. Those and other factors are the terms that I use to determine whether or not something has a legacy application in PHP land, or not.
E: And you talk about how the particular way that people come to PHP in the first place has a specific impact on legacy applications in PHP.
J: That’s exactly right. One of the great things about PHP is that you do not need to be a professional programmer in order to use it. If you’ve got a business idea, or if you’re just working for your organization, you’re not necessarily a technical person to begin with. You hear about this language, PHP. You know that you can type in a few lines of code and get them actually running on a server. And that’s fantastic. It allows you to make money very quickly. So the great thing about PHP is that pretty much anyone can use it.
But the great thing about PHP is also the terrible thing about PHP. And that is that anyone can use it, whether they are professional or not. So the people who write these programs, even if they are junior developers who will end up being professional programmers - PHP allows you to do a lot of stuff that maybe you shouldn’t be doing in the first place, like combining concerns, that kind of thing. But it lets you get up and running so that you can get something productive on the web. And then you turn away and you go to do your next thing. Unfortunately, what’s left behind is something that’s kind of a security and maintenance headache, because you weren’t necessarily thinking about, in advance, good architecture. You weren’t thinking about whether you’d be able to test it automatically or not. Testing, who needs that? I can look at the page, and I can see that it’s working - clearly it’s alright. And that’s great, until your first SQL injection hits, or until your first cross-site scripting problem hits. And then you’re left with this horrible mess. Reading a book called The Inmates Are Running the Asylum, by Alan Cooper, I found what I think is the right analogy for this. When you look at these codebases, it’s like looking at a dancing bear. When you look at a dancing bear - you are not thinking that maybe it’s pirouette is off balance, or that it’s plié is not a full extension. You’re just amazed the thing dances at all in the first place. Looking at codebases is a lot like that. You wonder how this ever worked.
E: That’s a really good image. You also talk about how, when people are faced with the situation where they’ve got this terrible legacy code, that often they have a desire to rewrite the whole thing - which I guess in the context is understandable. And you also write specifically about developers who have the desire to become one’s own customer. I was wondering if you could explain this? I mean I think it’s clear, but I was wondering if you could explain a little bit about that - that image of wanting to be your own customer, and why that’s a problem?
J: So every developer, first of all - no developer ever looks at anyone else’s code and says, “This code is fine the way it is, we’re going to leave it alone.” The first instinct of every developer when they look at someone else’s program, no matter how good it is, is, “This is not the way I would’ve done it, it needs a rewrite.” So because we have that first initial instinct, every time, we need to be suspicious of that. It turns out that that instinct is very self-serving in a lot of ways. We’re not necessarily looking at it to rewrite it, to make things serve the customer better - or for the program to be better. We’re doing it because we want it for ourselves. And if we want it for ourselves, that means that we get to be the customer then - “I get to determine what the needs are for it. And it should be done the way I want it to be done.” So we use ourselves as a reference point, instead of using some external concern as a reference point.
Treating one’s self as the customer includes things like, “Well, I want to use this new framework X. Because that’s the newest, hottest thing. So clearly it should be rewritten in that.” Or, “The way this has been put together is not the way I would have put it together myself, so I am going to be the determinant of what is right in this case - regardless of what any external concern is, and do it the way I would have liked,” rather than looking at it and saying, “Well, it’s serving its purpose pretty well. Other people are paying for it. Maybe we should mostly leave this alone, and only tinker with it around the edges. Or, “Make sure that it keeps working the same way, but just improve the quality of what’s going on under the hood.” The neat thing about being your own customer is that, if you’ll pardon the phrasing, it feels very sexy to do a complete rewrite, because then you get to do it “the right way”. The way it should have been done.
The problem is, that makes you very optimistic as to how well it’s going to go. The joke that I make about rewrites is: You’re going to estimate that it’s going to take a certain amount of time. Of course everyone’s got their favourite estimation techniques - normally it’s, pick some estimate, and then double it - and that’s how long it’ll really take. So you’ve got some buffer. But when it comes to a rewrite, it’s not enough to just double it. You also have to convert it to the next higher units. So that if you think it’s going to be a 6 week rewrite, it’s really maybe about 12 months. That’s because there’s so much going on in the system that you’re blind to, that your optimism blinds you to. And then when you’re most of the way - when you’re 12 weeks into what was going to be 6 week rewrite, you realize how long it’s really going to be. You start cutting corners, you start taking shortcuts again. And you end up with a system that’s just as bad as the old one, just in different ways.
E: And I think you mentioned Netscape as an example in your book. I think that - you say it took them 3 years or something to do their rewrite?
J: Something like that. I don’t recall. I grabbed that one from Joel Spolsky.
J: The idea is that Netscape looked at their codebase and said, “We need to completely redo this.” And maybe that was true. But they went out of business during the rewrite. They’re Mozilla now. They had to completely change everything about their business in order to make that happen. And if you’re in a position where you can afford to do that, maybe a rewrite is for you. But most people that I know do not have either the money or the time to actually go through with that kind of thing. So I hope that this book, if nothing else, will save people from thinking that a rewrite is going to save them. Nuking it from orbit feels great, and refactoring feels a lot like work, and so we don’t want to do that. But it turns out that refactoring, even though it feels like work, has the benefit of actually working. Whereas a rewrite, most of the time, is just going to blow you up.
E: Recently, I think it might have even been just over this past weekend, you held an online boot camp, walking people through the modernizing process. Can you explain the motivation for setting up that boot camp? How you set it up and how it went in the end?
J: I was invited to speak at the Zend conference. I don’t remember if it was last year or the year before, where they wanted a tutorial session on basically working through the book. So I put it together, and it ended up being something like 380 slides worth of information or something like that, for a three hour slot. I presented it to a full room; I was very happy about it. Unfortunately, I had to cut out about a third of the information in it, and I had to skip one entire chapter, because it’s just too much to go over at once. So I figured that if I could present it online to a committed audience over the course of a weekend, rather than in one three hour session, trying to shove everything in there, that that would be a better way of delivering the information. And it turns out to have been true. We worked through, essentially, the same set of information, but I was able to spend a lot more time showing examples and working through - basically, doing limited code examples while doing it. And people could ask questions while I was doing it. It took the full eight hours to get through it, so I really don’t remember quite how I got through any amount of slides in three hours previously. But it was very well received.
E: And do you think it’s something you’ll be doing again?
J: It is something that I would love to do again. This was a good first run, but I saw some mistakes and flaws in how I did the presentation, and there were some things missing, so I had to sort of hem and haw and add those in on the fly. I expect the second version of it to have a lot more to it, and to go a little more smoothly. Again, it was something I very much enjoyed, and I think the people that attended liked it as well.
E: Great, fantastic. I have a question about your second Leanpub book, which is called Solving the N+1 Problem in PHP. I was wondering if you could just explain what that problem is, and why it’s important?
J: Yeah, so the N+1 problem essentially is a “number of queries” problem when you’re putting together your objects. It’s not something that applies just to PHP, this will happen in any language. In fact when I first encountered it, it was happening as a series of stored procedures in Postgres. The idea is this. If you’ve got, say, a blog post or a series of blog posts, and you want to get all the comments for all those blog posts - generally what happens is developers will first get the list of blog posts. So they have to get a list of 10 posts. And then they loop through that list to get the comments on each post, and attach the comments to the post objects. What happens in that case, is you end up making 11 queries. 1 to get the original set of 10 posts, and then 1 query for each of the 10 posts that are in that collection. So you end up with a total of 11 queries. Maybe that by itself is not such a big performance problem, but when you’re putting together a collection of say 20,000 objects with five relations each - you end up with 200,001 queries, and a webpage that doesn’t load for three hours. So that’s a pretty serious performance problem. And it doesn’t even need to be in terms of 10,000 or 20,000 objects. It could be in terms of 20,000 requests being made against your system. If you can reduce by an order of magnitude or more, the number of queries that are being made coming out of an application as scalability concern, then you need a lot fewer machines in order to scale up.
So that’s the basis of the N+1 problem, where the 1 is the initial query, and the N is the multiple number of queries that have to happen to populate those other objects. The book is about how to recognize that that’s going on, why it happens in the first place, and how to solve it - and then some automated ways of solving it after you’ve figured out how to do it by hand.
E: Okay great, thanks, that’s really clear. I was wondering - just switching gears a little bit to your process publishing the books and things like that - I was wondering if you could talk a little bit about how you found out about Leanpub in the first place, and why you chose to use us to publish your book?
J: So Leanpub was highly recommended by a colleague of mine, a guy named Chis Hartjes, the Grumpy Programmer. He had published once through a traditional publisher, and then again through Leanpub - and had nothing but praises to sing for Leanpub. A joke that I have made, and I think he’s heard this before is that, I figured if Chris Hartjes could write a book on Leanpub, then by God so could I. And so that’s how I’d come to Leanpub in the first place.
E: Okay thanks. If you have more to say, please go ahead.
J: No, no if you’ve got more specific questions, I can answer them.
E: Sure, I actually do have a couple. But before I get a little technical, I would like to ask what your opinion is about the book market and how it’s evolving now? Maybe even specifically the computer book market. Do you see more authors doing what you and Chris chose to do, to self-publish programming books? And are there market- and technology-based reasons for doing that? Or is it something with the publishing industry itself that’s driving those choices?
J: I’m going to take the easy way out and say there’s a series of different concerns all playing back and forth with each other on that one. One of the things that people in general like, is to feel like they have been chosen. So when a publisher comes to you and says, “We have seen you, we like you, we want you to publish a book with us,” it has that aspect of feeling like you’ve been chosen by someone else. It makes you feel good about yourself. So that right there is a very powerful driver that favours the traditional market. Because you get that sense of other people having recognized you. It’s a status symbol if nothing else. So that’s one driver in favour of traditional publishing.
The primary driver that I see in favour of individual publishing or self-publishing is that you get to keep more money. And frankly I am a cheap little man, with a mercenary heart, and I like that idea. It also means that I get to be in control of - for good or bad, in control of everything about the process. I get to choose the art, I get to choose the font. I get to choose everything about the book. I get to choose the process for writing. I get to choose in all but the broadest general sense how it’s going to be published. So for people who have very - I’m not going to call them people who are control freaks, I’m going to call them people who are control enthusiasts - for the control enthusiast, self-publishing is a wonderful, wonderful thing.
E: I’m not sure if this was important to you, but one thing in particular with technology books is that having control over timing seems to be really important to people, especially if they’re talking about something that’s meant to solve a problem that people have right now. I think in particular it’s hard for people who think in a technical way to say, “I’ve got the solution, it’s out there, to solve the problems that exist right now. But now I need to subject myself to an arbitrary process. That means it’s going to take a year or two for my solution to get out there.” It seems like there’s just a certain type of person that finds that to be an unbearable situation.
J: I completely agree, and as a follow on to that, it may be that your solution, when written down, is only 30 or 40 or 50 pages. It’s going to be tough to find a publisher who’s going to find that a profitable concern to follow. Whereas with individual publishing, you can do small one-off pieces that address as narrow or as broad a topic as you like. But if it’s small, it’s probably a relatively narrow topic, but being highly focused, you can spend all your time working on that one thing - get it out, get it out of your brain and have it published. And that’s a fantastic thing, in fact that’s the N+1 book that we talked about before. I think that’s a pretty good example of that kind of thing. It didn’t really fit in the modernizing book, even though we reference it. But it’s not something that would be a full-sized book on its own either. So publishing it as, I think it’s like 60 pages, something like that, publishing it as its own stand-alone thing turned out to be really - I mean, it’s really nice to be able to do that. You just can’t do that with a traditional publisher. Not at a profit anyway.
E: And did you publish either of your books in-progress? That is, publish the first version before all the chapters had been -
J: I did, and that was another real benefit of self-publishing, and specifically with the model that Leanpub presents. I was able to write, I think, the first three chapters, which frankly were the hardest three chapters to write, with one exception, and then put them out there on Leanpub to say, “Hey, this is here, if you’re interested come and get it. If you like, please give me feedback so that I can fix typos and address other concerns.” The feedback was phenomenal, just right off the bat, and it wasn’t even finished yet. So, first of all, being able to publish it in installments in that way was super helpful to me as an author. But it was also super helpful to me as a perfectionist. Because you’ve got to get a lot of feedback from a lot of people who are paying attention, who will all find some small detail that you never noticed, and you can put it in and have it go out on the next iteration. It’s fantastic.
E: And how did you receive feedback? Did you explicitly encourage it in your book or on your landing page?
J: So again, one of the wonderful things about the Leanpub process was, as you publish, as you release iterations, you are able to send an email to everyone who had already bought it and say, “Here’s the next version. If you notice problems, here’s how to contact me.” So people would send me emails. In fact I think it was only by email at the time, or someone would say on Twitter, “Hey I noticed on page X that you’ve got this right here, and that’s not quite right. You should probably say this instead.” That communication mechanism, just from hitting the publish button, was fantastic. In addition to that, there were several people who knew me or already, knew my email address, and so could buy the book or download a sample. And they gave me feedback just prior to –
E: That’s really interesting, I have question related to that. So, just for anyone listening, the way our emailing readers feature works in Leanpub, it’s that when someone publishes a new version of their book, they have the option to send a message to all readers. And that message goes to you by email. But you don’t actually see the author’s email address, and the author doesn’t see your email address either. So, we facilitate communication without revealing email addresses to people, although it is possible for you to share your email address with the author. My question to you Paul is, did you feel that that was a loss that you didn’t have email addresses for all your readers? Or was the sort of - not exactly double blind - but was the sort of blind communication system that we have, did that work just fine for you?
J: It worked just fine for me. But that was mostly because when people would email me, or when they would contact me through the Leanpub system, they recognized that it was essentially a double blind system. So, if they felt like giving their email addresses they would, and if they didn’t feel like it, they wouldn’t. But in every case, I can’t think of a single case where this wasn’t true. Everyone wanted to provide their email address as part of that communication. So I did not feel like I was missing out on anything. If nothing else, it felt more valid that they were not required to put an email address of some sort, that they wanted to actually have this communication back and forth. And that was very satisfying, if that makes sense?
E: Okay thanks, that’s really clear and really good feedback for us, to know that that works that way. I was wondering, as you were saying about control enthusiasts, a lot of authors who use Leanpub are very opinionated about their writing and publishing tools. I was wondering, from your perspective, if there’s anything that stood out that you think we could improve?
J: That’s a really good question. I can’t think of anything offhand in terms of the tooling that goes on. I just use a text editor and I work in Markdown. I use Markdown for everything personally anyway. So that was not such a big deal to me. Having the Markua superset Markdown was also very nice. If there was one thing that I think could be improved in terms of things on Leanpub’s side, and I think I’ve mentioned this to Scott and one of the other guys at Leanpub, is that it would be nice to be able to publish the sample separately from the main book. So that if you want to make a change to the sample or add or remove things from it, it would be very nice to do that independently. I am aware that the backend production process for those two things is probably closely - they’re probably closely tied to each other. And so, decoupling them from each other might be a very difficult thing to do. But even so, as an author and as the person doing the publishing, I personally would find that pretty useful. Having said that, that is the only thing that I’ve run into that’s been even mildly inconvenient. And everything else about the Leanpub publishing process has been very straightforward and very easy to use.
E: Okay thanks very much for that. That’s the way Leanpub works right now - if you want to update your sample book, you have to hit the “publish a new version” button - which, if you haven’t changed your core book, won’t change anything. But it’s an important - there is a distinction obviously between updating your sample and updating your book, whether you do them both at the same time or not. And it certainly would make - there would be a sort of logic to having them be separate processes. Perhaps one day we’ll do that, but right now, yeah, they’re pretty closely intertwined, those two, the generation of those two things.
The only question I have left is, are you planning on writing another book, and is there one in the pipeline right now?
J: There’s nothing in the pipeline right now. I do have some general vague ethereal ideas about what I would like to do next. One of them is to write a book about action domain responder, which is a refinement of the model for your controller pattern that applies more specifically to web applications than it does other kinds of applications. That would likely be a very small book.
Another one that I’ve got in mind is, there is a culture of what are called “preppers,” people who like to be ready for emergencies, that kind of thing. I have considered the idea of writing a book specific to my experiences in putting together my own little stash of stuff, and how I went about doing it in a step-by-step way, so that it didn’t have to be this gigantic expense all at once. It could be this small step-by-step thing, sort of like how the modernizing process works for applications.
E: That’s really interesting to connect those two things, that would be great to see that.
Okay, well, Paul, I’d just like to say thank you very much for your time and for being on the Lean Publishing podcast - and for being a Leanpub author.
J: Thank you very much for having me.
This interview has been edited for conciseness and clarity.
– Posted by Len Epp