Mark Watson, Author of Practical Python Artificial Intelligence Programming

The transcript below is unedited output from OpenAI Whisper.

Hi, I’m Len Epp from Leanpub and in this episode of the Front Matter Podcast, I’ll be interviewing Mark Watson. Based in Sedona, Arizona, Mark is a consultant specializing in deep learning, machine learning, knowledge graphs, and general artificial intelligence software development. He’s the author of over 20 books on a wide variety of topics including AI, deep learning, Java, Ruby, machine learning, JavaScript, and more. You can follow him on Twitter at Mark underscore L underscore Watson and check out his website at markwatson.com. Mark is the author of a number of Leanpub books. His latest is Practical Python Artificial Intelligence Programming. In the book, Mark covers deep learning, symbolic AI, and a wide range of tips and techniques for using Python both in experiments and in production. In this interview, we’re going to talk about Mark’s background and career, professional interests, his books, and at the end, we’ll talk a little bit about his experience as an author. So thank you very much, Mark, for being on the Leanpub Front Matter Podcast. You’re welcome. Thanks for inviting me. Thanks very much. We were talking a little bit before we started recording that Mark’s been around on Leanpub since 2013. So this meeting between us is long overdue, and I’m just really excited to have him on. So Mark, I was wondering if you could talk a little bit about your origin story, as we call it, where you grew up, where you went to university, and how you found your way into a career in AI. Yeah, great. Yeah, so I was raised in Berkeley, California. My dad was a physics professor there. Because of my dad, I got access when I was, let’s see, it must have been 1964, I got access to a basic timeshare system on the ARPANET. And so it sort of gave me the idea that you can program. I didn’t really even know what a computer was when he sat my older brother and I down and let us play with it. But I always sort of liked gadgets and making things. I had a Charles Babbage style mechanical computer when I was really young. And my older brother taught me to play chess and go. So when I wasn’t playing vacant lots with my friends, I had a pretty wide range of interests even as a little kid. And when I was in high school, I took a Fortran programming class at UC as an extension class. And it was really great, it really clicked for me. We had punch cards and you give your deck to somebody on staff and an hour later, you get your printout back. And I thought that was just the coolest thing ever. But then, you know, it was years before I had access to computers again. And as a physics student at Santa Barbara, UC Santa Barbara, and I thought, you know, I’m going to take a programming class in the engineering department. So I take this class, and it was literally writing a program and toggling in the program on the front panel, like an old Altair computer. I was not very impressed, but I got a bachelor’s degree in physics and totally enjoyed it, but really didn’t want to be a physicist. And I guess a friend of the family suggested a company SAIC was just starting in San Diego. And because I had a physics degree, and I knew Fortran, it was an easy hire decision for them. And I discovered that I absolutely loved programming, it just was just, you know, didn’t even seem like work for me. And I’ll fast forward a little bit to, I guess, the late 70s, I read a book called Mind Inside Matter by Bertrand Raphael, it’s about artificial intelligence. And I was just hooked, the idea that you could somehow make a computer and in some way, mimic human intelligence, I just love that. And in my spare time, I wrote the chess program that Apple gave away on the early Apple twos, and I wrote a go playing program for the Apple two that I sold, ran ads and sold it. And then I started learning Lisp on my own time. And kind of a funny story that I, by some really random coincidence, I crewed for the founder of our company had a big ocean racing sailboat, so I crewed for him and his family as navigator, and got to know him. And during that time period, a bunch of us in the company, we only had like 20-25 offices at the time, it was not the big company that we later grew into. We got really interested in AI, and we wanted hardware Lisp machines, which were expensive ticket items, right? And the company saw the bill and went no way, but the founder of the company and the corporate treasurer who had gotten his graduate degree at MIT and knew Lisp had liked me, they decided they were just buying me the darn thing. And they gave me quite a bit of time, usually about maybe 30% of my work week, I can work on literally anything. So I did some commercial AI products, expert system for the tool for the Xerox Lisp machine for the Macintosh. I got really interested in artificial neural networks, and I sat on a DARPA advisory panel for a year for neural network tools, and for me, like I was just a programmer, my business card that Mark Watson programmer on it, right? And on the DARPA panel, I was with all of the famous people who were the pioneers in neural network theory. But the thing is they were all academic, and I implemented all of their algorithms, and we sold a library called SAIC Ansem. So in a practical way, I really understood their work fairly well. And around the same time, my company was building a bomb detector for the FAA, and the statisticians that were doing the modeling for that were using a linear model to try to see if there’s a bomb in luggage or bomb, no bomb kind of thing, sort of like Silicon Valley hot dog, no hot dog classifier, and it was basically a failure. And somebody on the board of directors had seen some of my demos with neural networks and was aware of what I was doing, and they sent me up to our office in the Bay Area that was doing the contract. And in about a day, we plugged in a back prop model and it worked. And so then I was, and this was in the late 80s, so then I was really hooked on this. And then by kind of accident, those different opportunities come up, I really moved away from AI and I did two large distributed systems, one for DARPA, it was a nuclear weapons monitoring system, worldwide sensors, and then another thing for PacBell and a bunch of other RBOCs for fraud detection. And so even though there were AI people on those projects, it was really an engineering project. And I kind of skipped something very important. Near the beginning, after they got me the Lisp machine, I wanted to very much get some government contracts and do AI work and not have my company pay for it out of internal research and development funds. And so I got the idea that I should write a book and I met through social connections, Bill Gladstone, he had a big agent company in Del Mar, close to where my wife and I lived with our family. And to make a long story short, he sold a contract to Springer Berlog, and I had this book idea common. Now, tell me as a publishing professional, if you think this is a bad title, the title of the book was Common Lisp Modules, like libraries. And my editor, she was a no, no, no, no, your book is about neural networks and chaos theory. So she named it Artificial Intelligence in the Era of Neural Networks and Chaos Theory. I learned a lesson. And so anyway, just writing one book really opened things up for me and opportunity-wise because I only had a bachelor’s degree and was in the wrong field in physics, not computer science. And so that was great. That really kind of kickstarted my career. And I wrote, I guess, about 10 books for McGraw-Hill and Springer Berlog, Morton Kauffman, John Wiley. And it was always kind of fun. I mean, like when the girl and the kids were sitting around watching TV at night, I wouldn’t as often as not be in the same room working on a book, which I found a little bit more interesting than TV. And so anyway, I got to be pushing 50 and my wife and I decided we really didn’t want to be in the rat race anymore. And so we moved from San Diego to Sedona, Arizona, beautiful place up in the mountains, bought a little tiny house here and thought we would kind of retire. And I had an idea that I would do some writing and whatnot. And so for years, I got out of the habit, I think my last book was for a press that was published like in hardcover form. But I really got tired of having to do what a publisher wanted me to do as far as content and whatnot. I was always pretty agreeable, but I didn’t really feel like I had the freedom. And it was, I think, around 2003, Lawrence Lessig started Creative Commons, Creative Commons license. And I had written a book that did pretty well. It was my first Java AI book, and I supposed to be write a second edition of that. But I started working remotely for a startup company, which totally got my juices flowing. I didn’t really want to do anything else. So I ended up not finishing that book. And the publisher, Morgan Kaufman, I cannot believe how generous they were. They let me keep the advance and they turned back the rights of the book to me. You know, just wonderful people. And so when that startup went bankrupt, as so many of them do, I thought, oh, I’m going to finish that book. And I wrote it, released in a second edition with the Creative Commons license and Lawrence Lessig made me the second Creative Commoners. They would feature for a couple of months, the first guy was Lasseter, I think his name was, he was a movie director who did Waking Life and a bunch of really cool movies. And so anyway, I found myself giving away this book. And I would, for the five or six years that I kept track of it, I was getting three to 400 downloads a day of the book. So plenty of good, good feeling of that, you know, people seem to be appreciating it. And I didn’t know a lot of email and feedback and whatnot. And I guess I kept going in and out of this period where I would like not work for six months and go hiking and kayaking or whatever you do up in the mountains. And then I’d get really bored and go back to work. And in 2012, 2013, somebody at Google invited me to go there and work on it, it was a fairly short term project I got, you know, my wife, I went to Mountain View and enjoyed the good food and all that stuff and enjoyed their tech. And when I got back home from doing that, I decided that I really liked writing. And that’s about the time that I discovered Lean Pub. And one thing I really like about it was my last book before Lean Pub, I wrote using LaTeX, and there’s just all this overhead. And I would spend, you know, two hours messing with stuff for every 10 hours I spent writing just seemed like a waste of time. And so I really kind of fell in the groove with Lean Pub where there was like, there was no overhead to it. There is nothing to figure out. And so for quite a while, I basically just wrote, and I eventually took a job at Capital One several years after that, running a deep learning team. My wife and I, for a few years, went to Urbana-Champaign, Illinois, where that team was located. And that was an awesome job, but, you know, eventually came back to Sedona and started writing again. And that’s, I hate to say synopsis of my career because I kept running on. Yeah, no, no, no, that was a really fantastic story. And there’s a lot I actually want to try and ask you about, and I’ll try and do it in as coherent a manner as I can. But just before we sort of maybe go back to the beginning, and I guess that’s sort of what I’d like to do is sort of the theme of like commercial AI throughout the decades would be what I’d like to through your personal, very hands-on experience with it. But before we do that, though, you had this great line, I think, on one of your recent blog posts where you call yourself, you’re currently a gentleman computer scientist. Yeah. You laugh out loud because that’s just an awesome way of putting it. But I believe you’re currently, you do some consulting for a South Korean company called Mind AI? Yeah, that’s correct. Okay. I originally actually did, I started with them about nine months ago when I did one large technical task, they had this huge common Lisp application, and I did some work on breaking it up so they can horizontally scale it. But when I finished that, I didn’t really feel like being a developer, you know, I’m in my 70s, right? So I get to do what I want. And so right now, I’m really just an advisor, which really means I spend a lot of time on Zoom. Yeah, I know, it just sounds really, really just sort of like, you know, you get to live in the mountains, and you get to do this cool work with a company like, you know, thousands of miles away. And it just sounds amazing. But one thing I wanted to, I just sort of took to go back to the beginning, one of the sort of themes that’s run throughout the podcast over the years, since so many, not all Lean Pub authors are sort of programmers, but or people in tech, but many, many are. And so it’s become a bit of a kind of like, jumping back and forth kind of time capsule for people’s first experience with with computers, and all around the world as well, right. So you know, I remember interviewing someone who was from Crimea. And he remembered sort of having to, you know, do write programs out by hand when the power was out, you know, kind of thing. And this was this would have been like in the early early 21st century, you know, or Jerry Weinberg, who, I believe, like, I think he, he was the first computer he ever met, you know, that he was hired to be a computer. And but so your first experience with with a computer, and by the way, a very common experience is a parent, often, often bringing the computer into the home, and then the sort of person just becomes sort of enamored with it. But in your experience, you were taken to ARPANET. That’s just amazing. I mean, what was like, just for people who are interested in the history of computing, what was the what was the room like? If you can remember, what was the interface like, you know, I’m pretty sure is on the Berkeley campus in the physics department. And there was a teletype with yellow paper tape. And when something would work, we can punch out paper tape that could then be fed back in that was the local storage. There is there was no storage, I won’t call it in the cloud. And I didn’t get a whole lot of time there, between my dad, my brother and I, we all wanted to fiddle with it, right. You know, my dad’s 101. And he still is a is a computer geek. Oh, wow. He did he does video editing, he likes to do I work before we moved to Sedona, I had like a midlife crisis job where I went worked at Angel Studios on Nintendo video games. And I was a tech lead on a virtual reality project for Disney, which was really quite grand, big motion platform, dinosaurs, wraparound screens, reality engines, and whatnot. And yeah, funny personal story. Like, I skip around jobs a lot, but I tend to work with the same people over and over again, like, like last year, I had six visitors in Sedona, people visit me that I used to work with, including somebody who is my manager at two companies 40 and 45 years ago. You know, so I tend to even though I tend to job hop, I do have loyalty to people I work with. And every time I change jobs, my dad would always be really upset because he always thought my last job was the best one. But when I went to do the the entertainment stuff, that he thought was pretty cool right off the bat. Well, I mean, it’s it’s it all sounds really cool. I mean, you know, getting on being in DARPA and stuff like that, and you know, being invited around to all these different kinds of projects. But yeah, I guess I guess when I let’s let’s maybe try and talk, I guess one of the things that I find so fascinating when I was researching, you know, your your background and your books for this interview was, you know, what was the, I guess the question I want to ask was like, what did when when you were sort of getting into AI, say, in like the early 1980s, was there a sense of constraint that people had? Or was it just a sense of like, you know, wonder, wonderful openness? I mean, you know, for example, given given the, what we would now view as very limited kind of computational kind of capacity for the machines that you were using in the incredible expense that you would say that, for example, you would have to sort of be be sort of friends entrusted by the CEO to get a company to actually buy you the expensive machines that you needed. Was there was there a sense of like, Oh, my God, where are we going to be in 40 years that people had like like you that were actually like hands on? Well, I had drank the Kool Aid, I really believe that AI was going to be a real thing in 10 years back then. And and that is despite the fact that project wise, we would have small successes, and a lot of failures. You know, like I remember, my company bought one of our remote offices, a connection machine, the first connection machine. And everybody was so enthusiastic about that. But it turned out it was very difficult to write code for. And and and just, you know, many sort of AI type projects really didn’t pan out. And I, I was, I was really pretty lucky, because nobody ever got mad at me when a project didn’t work. And sometimes they did. Yeah, and I guess I guess to kind of drill into the detail, though, for example, like let’s say, let’s let’s have all the many interesting things you mentioned, you know, using kind of AI to do to see if there’s bombs in suitcases or in luggage or in I don’t know, pallets going into airplanes that you were doing in the mid to late 80s that you mentioned, what what what do you mean? What do we mean when we’re saying AI and that kind of very, very commercial, like, I mean, it couldn’t be more practical kind of context. Yeah, well, I would say the use of AI in that context is more marketing than substance. Because if you look at the actual code that I contributed to that project, it was probably 250 lines of C++ code. And, you know, so it was it was like nothing, I mean, is like, you know, a lot of effort put into learning, you know, which kind of models to use and whatnot. And so that was just, that was like, like a gift that basically just something worked out really easy, like, about like, like a painful example, like, there were multiple divisions in my company that wanted to get a contract for an AI battlefield advisor, and system, we were a defense contractor. And it was like, we spent just tons of money trying to end and I was a very small cog in that wheel, but we really put a lot of effort into it, and didn’t get a contract. Some other big, bigger defense contractor got the contract, and, and the whole thing was a failure. I mean, they didn’t really get anything out of it. And so, you know, so there was there was a lot of excitement, and I can’t remember exactly when it was, but it really, people really call it the first AI winter, and people just sort of lost, lost interest in it. But another success, I mentioned that my company was, was a lead on DARPA expert system to detect nuclear, illegal nuclear testing. And my part of that was distinctly not AI. Even though that was about the time my first AI book was published, I was like straight up engineering. We had 38 data acquisition stations around the world. And back then, the internet was horrible, right? Unreliable satellite connections. So just going from the US to Europe might be two second latency, if it was like three hops or something. And so my part of that was, was engineering, but we did have a couple people and we did we teamed with, I forget what the name of the company was, but the expert system worked really quite well. And that was all done in common list. So that was a big success. And then I’m having a hard time remembering the failures. They were there. It is interesting to think about the sense of potential and just excitement. You mentioned very straightforwardly that sometimes when people talk about AI being in a project, that’s marketing. And that’s definitely true to this day. I actually know someone who works for a relatively well-funded startup, not in anything AI related. And there was a sort of grumpy investor who was like, can you make this AI somehow? So that’s still new to this day. And I can imagine when we talk about marketing, particularly in those contexts, it’s not just kind of like necessarily cynical marketing. It’s a kind of positioning, right? Like particularly if you’re a defense contractor or even a military person, you might want to be able to say we’re cutting edge. That actually isn’t just kind of cynical or something like, or just about money even. There can be very serious and important reasons to do that. But I guess like let’s say for you, you just mentioned the illegal nuclear testing work that you did. That wasn’t AI, but it was programming. And actually that was an AI project, but my part was a straight up engineering on it. But your part was straight up. Okay. And so what would the… For someone, let’s say someone who’s never written a line of code, but obviously knows that there’s such a thing as computer programs, right? What’s the difference between a kind of, let’s say a vanilla computer program and an AI computer program? You know, it’s almost what problem you’re trying to solve. I mean, I spent a lot of time doing natural language processing, trying to extract structured data automatically out of text, newspaper articles, documents, whatever. And that was really kind of hit and miss. And this is really important, I think, to the broad topic of artificial intelligence. And that is scaling things from little demos that people look at the demo and they go, that is fantastic, to scaling it to a real world problem. And then it failed. I often on for probably 15 years, mostly off, but you know, cumulatively years of my time, I was spent doing conventional, symbolic, natural language processing and little bits of success, you know, failure, poor, you know, things that just wouldn’t scale. And when we entered the world of deep learning, suddenly things that were very difficult, things like anaphora resolution, that’s like resolving pronouns and text to proper nouns, you know, a paragraph or two back, that’s a pretty important problem. A friend of mine at Google, she got her PhD at MIT just on that one thing and very difficult. You fast forward to maybe four years ago when Google first did their transformers paper and one of the first transformers was called BERT. And BERT basically solved anaphora resolution. I mean, BERT took advantage of the problem with machine learning and deep learning, getting training data. They had an infinite amount of training data because what they would do is they would take text from the web, from published books, and they would randomly remove words from the text and put in a marker token and then train it as a sequence model as it’s reading the text character by character or token by token or word by word, it would train it to estimate probability distribution of what words would go in that empty slot. And so for the first time ever, there was basically three training data. You didn’t have to do any human labor involved in it. And so BERT, one thing, the transformer models were a very clever idea and all the free training data you wanted, suddenly this very difficult problem that I worked on, my friend had gotten her degree doing it, suddenly it’s a solved problem. And in between jobs, I spent about a year writing a natural language processing library that I sold, and one of the things I worked really hard on was summarizing text. And I came up with an algorithm I thought was really pretty clever, I would categorize text by what classification was using something called a bag of words, and words that had the most evidence for a category, I’d say, oh, those are important. And so in a sequence of texts, there’d be like these islands of words of how important they were, and I would just simply do extractive summarization, and customers liked that, it worked pretty well. But then fast forward to, see, I think I got access to GPT-3 maybe a little over a year ago, and then suddenly I could give a model that somebody else trained, I didn’t have to do anything but call an API, and I gave him my credit card, and I could give as much text as I wanted within bounds, and it would summarize. And I could say how long I wanted the summary to be. And so I still believe that we are a long, long ways away from general artificial intelligence, and I don’t believe that deep learning gets us there, but suddenly it’s solving all of these really useful and interesting problems. And so I feel like I’ve been a kid in a candy store the last couple of years, just because I love it when there are problems that just used to be unsolvable, now high school kids can do it. And in my Lean Pub book that was published a few weeks ago, I made a comment that I have a U.S. patent on assigning human readable names to clusters when you do k-means clustering, and it was just some ideas that my team and I kicked around, and we didn’t really do much with it, but we got a patent for it. And so then like five years later, I was reading some documentation, I forget which thing was it, for Hugging Face, some examples, and they had an example, Jupiter Notebook, for doing k-means clustering and determining what would be good names for the data clusters, which was just something that I worked on by hand 10 years ago, five years ago we did some brainstorming and came up with sort of a, it was sort of mediocre, but good enough to get a patent, which is pretty common, may I say, and so to me it’s just exciting that we can do stuff that we didn’t be able to do before. Yeah, I know, it’s super fascinating, and I love how when I asked you sort of what’s the difference between a normal program and AI, you immediately went to a certain kind of scaling, but also just language and symbols and symbolic AI, which I know you mentioned in your book is good old-fashioned AI and things like that, but when we’re talking about, there’s a kind of scaling you mentioned that sort of started to happen that I think probably sort of programmers might be more familiar with around 2010 with big models and stuff like that, right? So when people think about scaling and technology, they often think, well, instead of serving 100 people, I can serve a million people with my app now or something like that, but there’s another kind of scaling, which is the kind of thing that, if I hope I’m using the term kind of correctly, but like with AI, the example I always like to give of like just the, my background is a bit in sort of philosophy and English and stuff like that, right? And like if I can go to you, okay, look, here’s Napoleon and here’s Wellington, and it’s the morning of the big battle, and then Wellington’s army goes like this and Napoleon’s, no wait, hold on, this is symbolic AI and this is natural language processing, you know, and they’re different in these different ways. And the fact that like our minds can sort of like turn my fists into these different things and we can understand what I’m doing and flip like that to something else. And then it could be any arbitrary thing that I could start associating my hands with and talking about and building something around. AI is the kind of programming that’s sort of meant to be able to handle kind of things that you throw at it like that, and it has to have, we immediately go to sort of these anthropomorphic metaphors, some kind of understanding, you know, but what we really mean is like, you know, effective understanding, right, at this point. And then so what’s the wonderful thing that’s been amazing to see and just in the interest of time is sort of moving on to where you did, you know, the sort of exciting time we’re in now where you can ask, you know, chat GPT, which for anyone listening who hasn’t heard about it, basically there’s this company called OpenAI that has this sort of software out there that’s very good at answering prompts that people can give it through a relatively recently released sort of chat style interface. And you can do things like, you know, I mean, given my background in English and philosophy and you know, sort of stuff like that, the first thing I did was, you know, write an essay about Hamlet on love, and it wrote in 30 seconds or whatever, a sort of like, you know, B minus first year essay on that. And it’s just amazing to see, but you know, people have been doing so many things like getting it to sort of mimic a Linux terminal, you know, kind of stuff like that, like, you know, and it’s not because it knows anything about that. It’s just because it’s been, it’s got this, been trained on this huge database of interactions and questions and, you know, outputs and stuff like that. And I guess I wanted to ask you specifically, cause you’ve talked about this, but how does, how is chat GPT useful for programmers? That’s, that’s a, that’s a good question. The company in South Korea that I’m an advisor for does chatbots, AI chatbots, and I’m not, actually, I probably shouldn’t talk too much about our work, but, but believe me, we have lots of discussions about our mostly symbolic approach versus chat GPT and, you know, it’s like, you know, chat GPT is one of those things that you don’t really understand how powerful it is until you use it and just a real couple of real quick anecdotes. When it first came out, I was reading a paper, like I wrote a Haskell book for Lean Pub, but I’m not a good Haskell programmer, but I love the language. So I kind of wrote a book from a beginner’s mindset about something that I really enjoy, but I was reading a paper and they had some Haskell code that, you know, I can admit this, right? I couldn’t really understand it. It used Haskell lenses and I thought, I really want to understand this. And I thought, you know, I’m going to try something. And I, it was only about 30 lines of code and I, I wrote it into chat GPT and their web interface, I wrote, I explained the, what the following Haskell code does, colon, and then command V paste in it went, and it, it clearly was, was using things that were not in the train model. It was clearly making some web searches on its own because it gave me information about the libraries that were being used. And it took a while, took like 45 seconds to give me a response. And I was looking at the code as reading with chat GPT output and I really felt like I understood the code. And later I went back and then did a few web search web searches and then kind of made sure I really understood it, but you know, being able to in one minute, get some background on some code that I did not understand. And I’m a pretty good programmer. I’ve been doing it for a long time, was just an eye opener for me. And the other thing that just absolutely amazed me was I, I was just kicking the tires and I said, please write a poem in the style of Elizabeth Bishop about my pet parrot that escaped out the window. And it wrote a really nice poem and it’s been 20 years since I’ve read Elizabeth Bishop. And I think the style maybe was sort of, was sort of there. And I thought this is really cool. And then a friend and his wife were over a few days later and I was showing her this and she was very skeptical and she had, can’t remember the name of her favorite poem, but a modern poet, she said, could you do the same thing? And I said, well, chat GPT is not always running. I’ll see if it’s running and it was, and so I did the same thing. And she said it was very much in the style of her favorite poet. And so even though chat GPT is not intelligent, it’s not general intelligence, just being not only being trained on most available text, open text, but also part of it that I don’t fully understand because they don’t talk about it, that it obviously is doing some web searching and there are models that have been around for about five years, squad type models, where if you give it text, they’ll answer a question and they’re clearly doing that behind the scenes and the Bing search demo two days ago, clearly, that was a wonderful demo, clearly they’re going out and doing general searches, getting text and then using chat GPT to process that. But to circle back to your question, what it does for a programmer, I think the best answer is explaining that arcane piece of Haskell code that was beyond my ability to understand without doing some research on my own. And it’s just so, I mean, that’s just so amazing too, because even though, like, you know, I mentioned before, you know, people who might not make, they know what that there are such a thing as computer programs, but not might know what they are. And now like anybody can put some code in into this machine, just as you say, command V, paste it in there and like get a little bit of an explanation, you know, you still use it as a beginning point, you know, for looking into things, but that’s just amazing. I actually have to share one sort of related anecdote to yours. So when I first discovered that I was doing this, you know, write an essay on Hamlet and love, write an essay on Heidegger’s being and time. And it did that in a friend of mine who’s an English professor was like, ask it to write an essay, write about Heidegger’s being in time in the style of a Celine Dion song. And it did. It did like within 30 seconds or so. And like, you know, it kind of it had a chorus, you know, it was using terms from the book kind of, you know, in a way that sort of made enough sense to be B minus again, you know, kind of thing. But it, you know, anyway, it’s just it’s just incredible. But the idea that, you know, just to be very specific about, like, the idea that just anyone can put in some code and have an explanation, a prose explanation come out as a beginning to understanding things. But you know, then it goes the other way, right? That you can like, of course, and the people have been doing this, you know, for a while now, but like, you know, typing out some prose and going write me a program in this language that or a set of code that sort of like, you know, will actually carry out these these instructions is also just incredible just to think about where that’s going to take us. Yes. Yes. Yeah. And we could we could talk about that forever. But just in the I guess, in the interest of time, just moving on to the next part of the interview where we talk about your book a little bit specifically, the book is Practical Python Artificial Intelligence Programming. It’s your most recent one. I think it’s your seventh on Lean Pub. And I was just wondering if you could talk a little bit about who the who the who the book is for. Yeah. So so this book was was was different in the sense that usually I mull over books for a long time, and I write a lot of program experiments, code and kind of play with ideas. And then I write a book, but I’ve been so blown away in the last four or five months by GPT three and then chat GPT that I really wanted to do something quickly. And in a couple of chapters near the end, I sort of apologize, I dear reader, I will in a future edition fill this material out. But I just I had a feeling that I wanted to just get like an idea there. There’s very little code. Maybe my books are just full of code and that it’s more as almost more like mentoring advice. Like if somebody says that they’re interested in getting into artificial intelligence, broad field, right? Really broad field. And so what I was hoping was that if somebody needs to do like recommendation systems or if they’re interested in natural language processing or I love the semantic web, favorite topic of mine, that I expected the average reader to probably, you know, focus in on maybe 25 percent of that book that that interests them. And I felt like I was in a real hurry because I felt like what I was writing would would be obsolete much quicker than usual. I mean, I mean, typically like my Java book, I think the last two editions of that I’ve done on Lean Pub, I think I think maybe I’ve discarded three chapters, just material I thought was old. And that’s one thing I love about not printing physical books is to be able to drop material when you think it’s no longer relevant and slide in new material. I mean, I mean, can I admit that I put out like, I think, three updates to my new book in the first five days it was out and none of them was anything that that I felt like had to have all the readers be notified that there was an update. It was just like, you know, so I got a really nice series of emails from one person in particular I didn’t know that it was really jazzed about something. And he said, oh, by the way, I found a typo, you know, so the typo was fixed and it was a wonderful feeling that people who read the book for free online, which I love having that available or got a copy or the book reader would have that fix in it. And you know, it’s just it’s just that fluidity that I love. And then typically, with Lean Pub, within about three or four months, it seems I usually do something that’s pretty much of an upgrade. And then I click that box to notify readers that, yeah, you probably want to grab a new copy. And then I actually go look back in history, but then it seems like that I usually let them sit for a year and then rework them again. So the seven books that I’ve written for Lean Pub and with three updates, which I love, you know, it’s more like a living thing. I remember I wrote one Lean Pub book on a language that I don’t really like, but a lot of people have paid me to write code in it and that’s Clojure, which is a Lisp language. I much prefer Common Lisp and Scheme. And this wonderful guy I’ve become long distance friends with, a Russian married to a German woman living in Germany, got my book and I think I was probably pretty open, at least on social media, that like, you know, Clojure’s not really my language, but I’ve had to use it so much for work that I wanted to take some of the stuff that I usually do in Common Lisp and just hit a new audience. And Alex rewrote a whole bunch of my examples and said to use them. And so next time I updated the book, I took out some of my inferior code examples and put his in, you know, with a nice thank you to him, right? And so it’s really a different style of publishing, you know, I don’t know if you’ve ever written a book, you know, with McGraw Hill or something, but it’s a very rigid process you go through. You gave me the best gift a podcast guest can give, which is you naturally segwayed into the next part of the interview for me, which is going from, you know, talking about the content of your book and who it’s for and stuff like that into the kind of process by which you wrote it and you’ve done your other books. And I just, it’s really actually, I mean, in so many ways you’re sort of like the author we built Leanpub for, right? Because, you know, I mean, you had that story, your first book was with a sort of conventional publisher and they gave you, you learned some things from them, like, you know, maybe having a title that’s sort of more evocative with some keywords from the moment is really important. But then you, but then at some stage late, having written more books, you then did one of the first creative commons kind of books out there. And then, and one of the things you mentioned was like, you’re getting hundreds of downloads a day and feedback from people, which is super interesting, right? And like, you know, in, in the, in the sort of, in as it were the kind of olden days, which still exist for some, you know, book publishing processes, your book goes out there and hopefully you get sales and get on bestseller lists and maybe you do readings and stuff like that. But, and of course you get letters and things like that, but, but the idea of getting, first of all, that boost to one’s career from one’s first book, once you’re established, then being able to sort of like have the resources and the wherewithal to put out a book maybe for free, but, but that serves this purpose that hundreds of people a day discover and can use. And then to sort of like kind of go galaxy brain and sort of move to Leanpub where like, I want to be able to sort of like update it, you know, five times in a week. And you know, I want to, I want to sort of get, get feedback from people where they’re helping me actually improve my book and they’re giving it to me because they want my book to be better. Or actually one, one experience we heard is like, people as readers are just really bugged by something and they’re like, they’re almost mad at the author, you know, like fix this here, I did it for you, you know, things like that. And it is, it is really amazing that that world that one can be in where you sit with not paper books are awesome and serve great functions and stuff like that as well. But once you really embrace that idea that you can have a book that’s sort of updatable and changeable and you know, in a sense, kind of like collaborative, but, and like in particularly the sort of rather radical thing that you mentioned, even, even for a Leanpub author, which is deleting, deleting chapters because they’re outdated. I think that’s something that a lot of people, most people would much rather kind of update it. Um, but to just note, you know, that’s not relevant anymore. That’s that’s, I don’t know if I’ve actually had anyone on the podcast sort of mentioned doing that, that before. Yeah. Well, you know, what, what, what, what are the examples that I cut was, was a wonderful example is a natural language interface to a relational database and, uh, Peter, Peter Norvig, who’s really well known guy at Google and AI, uh, he was a tech editor on the early edition and, uh, he, he said, he said, Hey Mark, uh, that, uh, the problem is that that example was so good. It makes all your other examples look, you know, and, and that’s, and that’s an example I ended up, uh, uh, pulling out because I thought that what was a really cool technology demo was not relevant to what I thought readers would like to do. And I thought, is there any way to modernize that example? And I went, no. And so pull the plug on it. Yeah. But you know, it’s sort of like almost like an open source, uh, ethos, uh, and, and, and a way, um, you know, usually, um, you know, one of the really nice things about being a writer is I I’ve gotten to meet all kinds of people I would never get to meet if I didn’t have written some books, um, uh, and, and usually readers are really nice to me. I mean, like, you know, the first 10 books I wrote for regular publishers that were printed in paper, uh, one was a UML, uh, book, uh, um, unified modeling language. Uh, and I got the, I got hate mail from a reader because I had, uh, put a longer program listing in a appendix, uh, and he was making a guesstimate of the number of trees that I had murdered, you know, so you do get a little bit of pushback, but it’s, it’s very rare. And I did something that maybe I shouldn’t have done, but, uh, about maybe it’s like a little over two years ago, I thought, you know, I’m using a creative commons, um, uh, no commercial reuse, but otherwise reusing with attribution, you can give it away. So somebody buys the book, I have a PDF, they can give it to their two colleagues at work or their brother-in-law and with the creative commons license. That’s okay. Uh, that’s, that’s perfectly legal and okay. And, and, and I don’t mind doing that, but a few years ago, I got the idea that wouldn’t it be cool if like some student wanted to use part of my book for a paper or, or somebody was writing instruction manual for something and that they could take some of the texts and reuse it, change it any way they want and just simply put an attribution back to the website. So if somebody might see the attribution, right, purchase it. Um, and I let that sit for, I think I did that for a little over a year and I then decided that that was a little bit too far because I really do like, uh, getting credit or things. And just, just a few people were, were very bitter that I did that. And I asked him, I said, Oh, were you taking advantage of that? You know, tell me the circumstances, you know, do something to make you happy again. And they said, Oh no, we didn’t work doing it. We just liked the idea of it. Yeah. And so I got, I got some pushback on that, but you know what people are entitled to their own opinions and, you know, and whatnot. Uh, so, so I went back to the, uh, the share alike, uh, no commercial reuse. In other words, somebody could not take something I published on lean pub and sell it elsewhere. Yeah. Thanks very much for sharing that. Actually. That’s one of the great sort of details I really liked. But when we talk about the sort of like life of the working author, right, is actually, you know, the, you know, one of those details is actually dealing with, with negative feedback sometimes. And in the case where you’re sort of doing what, what most people would consider to be kind of like experimentation and changing things up and taking things away and adding new things, uh, you can get people who just get just, they just sort of generally don’t like it. They’re like, this is kind of confusing. I mean, to this day on lean pub, still we get, you know, I say still as though, as though everyone knows who we are and what we do, but you know, we do get people who are like, I bought a book and it’s only one third finished. What’s, what’s wrong with you? No. Uh, um, and we say, oh no, you’re the authors publishing, publishing it in progress and you, and you’re going to get all the updates for free. And all of a sudden people can go from like giving you the middle finger to being like, oh my God, what a cool idea. I can’t wait for the next chapter or else they’re like, screw you. Give me a refund. And which is, which is fine. You know, if that’s not what you’re expecting, but, but when you’re doing, if you’re experimenting with sort of different publishing models and different licenses and things like that, you’re going to get pushed back from people from time to time. But you know, one, one way to fortify yourself is to think they wouldn’t be pushing back if they didn’t care in some way. Um, well, uh, Mark, thank you for, uh, talking to, uh, me and to our audience, uh, on this episode, uh, at length. Um, and so, well, it’s such an interesting story that you have to tell. And it’s such an interesting time that we’re in. And I think you really captured that excitement. Um, but just before I let you go, um, the last question I always like to ask, uh, on these episodes, if the guest is a, is a lean pub author is if there was one magical feature we could build for you as an author, or if there was one terrible thing that after all these years, you’ve been still shaking your fist at lean pub about that we use for you. Can you think of anything that you would ask us to do? Yeah, as, as a matter of fact, um, this hits me about once a year, I’ll, I’ll go through a manuscript. I’ll just read the whole book and start making little changes on the whole thing. Just do some rewording and I will make it. So the manuscript is no longer legal markup and it takes me a little bit of time to find out if I wear what’s chapter in which location. So there’s, there’s no like error code, uh, you know, like you, you might be inserting some file and then the market for including an image is, is misformed or something. And so, um, the way I do that is I do a binary search and I think I publish a preview book with half the chapters, you know, and then find the half and then half it again, half it again. Um, and, uh, sometimes that’s like 10 minutes and one time it took me like an hour well over an hour. So I love the platform and, and if, if I were implementing it, that would be a real nuisance to do because you’d have to, uh, track and I have not looked at the source code to the markdown, uh, system, but you’d have to track the location and when there’s an error sort of make that visible, but I, um, since I run into that once a year and it probably costs me an average of like maybe 20 minutes of my time. This is, this is not a serious complaint, right? But if there’s one thing that I, that would change, that would be, uh, be wonderful. Yeah. Thanks. Thanks very much for sharing that. Just for anyone listening for whom that might sound a bit mysterious. Um, so when you, when you read a lean pub book, you write it in plain text. Um, uh, we, we actually do have an upload mode for people who are like, screw all this. I just, I’ve got my own way of doing it. I want to upload a PDF and an E pub and then that. So we do have that. But if you’re writing a book in, in lean pub, as it were, you write in plain text, uh, since you’re writing in plain text, that means that there’s no, you can’t go command B to make something bold. You have to put the instruction in there saying like, make this bold. And we have a sort of syntax for that. Um, uh, you know, the, the way I always like to explain it to people is like, it’s, it’s imagine you were writing on an old fashioned typewriter and you want it to indicate something should be in italics. That’s what the underline was for, um, was so that when you handed it over to someone who was going to make something out of it, um, they know, oh yeah, that’s, that’s, and so that’s actually a markup syntax is just underlining. And anyway, but, um, but what that means is you have to give this plain text manuscript over to lean pubs, book generators, and they have to create ebook files out of it. Um, and if there’s a certain category of, as it were mistake in that markup, then that causes our book generation process to fail. And what lean pub doesn’t do. And I actually, I actually do think this is a, you were being big sort of very generous by saying it’s not the biggest problem, but like that actually is it, even if it only takes a little while to figure it out qualitatively, the book generation process failed and no information about why that’s actually, that actually sucks quite a bit. And we do have a kind of hidden feature, uh, where you can get a kind of barfed out kind of error message thing. But really what we should, there’s two things that we should, and we’ve got designed the reason I can speak about it at length like this is because we’ve had this feedback before. Um, and there’s, there’s two things really that we should have one day, uh, one of which is robust error reporting, so you don’t have to kind of do what you were describing, which is the same way we debug things, which is like, cut it in half, see if it fails, cut it in half, see if it fails or else what you can, one thing we recommend people to do is they make a book in our browser editor and then just paste one chapter in at a time, hit preview, see if that chapter runs. Um, but so a, we should have error reporting it’s line 52 in document five, uh, that caused the problem. Uh, that would help people greatly, but also real time rendering. So like having a side by side, um, so you can see what it’ll look like in the, in at least in the PDF or something like that is something we’ve had lots of people ask for too. So thank you very much for that really good feedback. We really appreciate that. And in so far as lean pub is usable is because of great feedback from authors like you about things like that over the years. Uh, well, Mark, uh, thank you very much for taking time out of your day instead of being hiking out in the beautiful mountains of Sedona to talk to us. And thank you very much for being a lean pub author. We really appreciate that. You’re, you’re, you’re welcome and, and, and, and thanks for inviting me this, this was a, uh, was, was, was fun except for maybe the game part of talk too much about myself perhaps. Oh, no, no, that’s, that’s, that was wonderful. I enjoyed, I enjoyed it. And I’ve got to say as a podcast host, when, when you, when you, at least for me, when you’ve got a guest who, you know, they can tell a good story, well, carry on as long as you like. Uh, so thanks very much for that. And as always, thanks to all of you for listening to this episode of the front matter podcast. If you’d like what you heard, please rate and review it wherever you found it. And if you’d like to be a lean pub author yourself, please check out our website at leanpub.com. Thanks.