Helen Anderson, Author of Big Data: From A to Z

Len: Hi, I'm Len Epp from Leanpub, and in this episode of the Frontmatter podcast, I'll be interviewing Helen Anderson.

Based in Wellington, Helen is a Business Intelligence consultant who works on high quality business intelligence and database solutions. She leads projects that use AWS to deliver empowering services to users, and is an advocate for data analysts.

She also is a popular blogger and speaker and mentors people who are new to the tech industry. You can follow her on Twitter @helenanders26 and check out her website at helenanderson.co.nz.

Helen is the author of three Leanpub books: AWS: From A to Z, Big Data: From A to Z, and SQL: From A to Z*.

In this interview, we're going to talk about Helen's background and career, professional interests, her books, and at the end we'll talk a little bit about her experience as a writer, and now as a self-published book author.

So, thank you Helen for being on the Frontmatter Podcast.

Helen: Thank you for having me.

Len: I always like to start these interviews by asking people for their origin story. I was wondering if you could tell us a little bit about where you grew up, and your career trajectory so far? I believe you've actually written about career switching and how your journey hasn't really followed a traditional course?

Helen: I'm based in Wellington, New Zealand, which is about 200,000 people in the CBD area. So it's not exactly a huge place.

I'm currently working for Xero. We are an accounting and business platform for small businesses. My role at Xero is to produce BI reports and data analytics for whoever needs it within the company, and also to mentor those who are new to the data industry. So, those who are new to SQL, who've maybe just graduated from using Excel dashboards for data analysis, and to give them a bit of a helping hand along the way.

Len: And did you study tech in university?

Helen: So, I haven't had the most traditional background, I guess. I studied marketing and management at university. I wasn't looking to get into a tech career. But I have sort of fallen into it, I guess? I started off in supply chain analysis with Excel dashboards and lookups and pivot tables and Microsoft Access, which is not as sophisticated as what we use today. But that sort of set me on the path to where I am now - using databases and BI tools to deliver solutions for people.

Len: I'm actually going to be asking you about supply chain analysis in a little bit, just taking the opportunity - when we get people who are knowledgeable in areas that most guests aren't, it's fun to talk to them about things.

But before we go on, you've mentioned business intelligence, BI. Could you talk a little bit about that? Just imagining an audience that hasn't heard of it before?

Helen: Yes, sure. The BI function is to take data from all over the business and external - that could be things like production databases, CRM systems. And then external, so publicly available data. And we create data pipelines to gather all of that information, create data models and then visualizations, so the sales and marketing, product teams can see how their product's doing, or see how they're performing in comparison to last year. All their KPIs, or externally - we're sort of the information hub for the business.

Len: And for internal information?

Helen: Yes - traditionally, BI's an internal function, but with data science and big data coming along as well, sort of changing the scope. Predictive analytics and natural language processing is certainly something that BI has been more involved with.

Len: I've got a question to ask you about data analysis in just a minute. You've already actually mentioned a couple, but what are one or two of the biggest changes you've seen in business intelligence in that industry over the last, say, five years?

Helen: I think the scope is certainly changing - with changes in technology and what people want from data, that's certainly changing. We've gone from just creating glossy dashboards for execs and answers for sales and marketing to - yeah, predictive analytics and more data science-y solutions. As statisticians have come on board, it's became a bit more sophisticated - and it's certainly changing every day, which is great.

There's always a role for data in business. And I think the roles that people take on are changing as well. It used to be the business intelligence consultant would do everything from talking to stakeholders to building ETL pipelines to get the data that they need. And then all the way through to supporting them to use their dashboard or their solution.

But now we've got data engineers who do the ETL side of things - moving data around and making sure that it lands in the right place. And then visualization experts. So it's no longer one role with many hats, we've got experts who are able to use the new technologies that are coming on board, and really have a specialization in that area.

Len: Before I ask you about data analysis, I wanted to ask you about supply chain analysis, which I know you mentioned you worked in. Just to begin with, what is supply chain analysis?

Helen: I guess it's really going back in time. My first role out of university - in my first few years of my working life, was in supply chain - exporting apples from New Zealand. So ,not Apples with iPods and things, but actual fresh produce.

The aim of the game was to look at what the grower predicted that they would be growing this year, based on historical information, and what sort of growing techniques that they were putting in place. What kind of size range that they were expecting to grow. And then looking at what the customers were expecting, and making that work - translating what the grower wanted, what the market wants. And then making it work with all the shipping schedules. You can't just make more apples, so it was really important to get that right - and put a bit more science behind it, rather than gut feel.

Len: It's really interesting, when I was preparing for this interview, I was reminded of an old friend of mine who's a French engineer, and he works for a company, or worked for a company that made offshore oil platforms.

He talked about the challenge of making sure like the screws from the factory in Nigeria get to the port at the right time and stuff like that. It's an incredibly intricate and complex field.

One of the reasons I was looking forward to asking you about it - a bit cheekily, because it isn't what you do currently in your work - but, supply chains have been in the news lately because of things like Brexit, and US tariffs on Chinese goods, and things like that. So people are seeing in the headlines there's going to be disruptions to supply chains.

I think in a lot of people's minds, that might be like, "Oh well, you put it on the boat - maybe it'll just wait in the harbour a little longer?" Or, "The Chinese will find another place to go with their goods." So I know it's a view from 30,000 feet kind of question - but just as someone who knows a bit about it, how difficult - what would be the effect of a disruption? Like all of a sudden, you can't get your goods across the Channel into Britain from France.

Helen: Yeah, it's always a risky business. Supply chain is solving a puzzle. You have to make sure that all the moving parts are going to fit together. And having a plan B is always useful.

fter I did the work in the fresh produce industry, I worked with Timex in the UK. So mid-price watches. And that was always a challenge. Because watches are so intricate and there's lots of little pieces that go to put it together. Yeah - it was a challenge to make sure that all the right stuff was getting to the right place at the right time - and having a plan B if there were shipping disruptions, weather disruptions. Things like public holidays. If you've got a public holiday going on and you're expecting things to arrive, then you've got to have lots of backup plans. Which is great life skills - I guess, if you're a junior in the supply chain world.

Len: Just speaking as an outsider who likes to read about it a little bit from the fringes - was there a shift - I don't know, maybe 20 years ago - to "just in time" manufacturing? Which is the only reason you can do that. So, like you're always manufacturing exactly what's needed in the current schedule. The idea is you don't want to be stockpiling unused stuff. You want everything to go out the door as soon as it's made. And so the way things are made, and decisions you make about when to make them - are highly contingent on this sort of rapid flow of goods.

Helen: Yeah, you have sort of rolling forecasts. So every 12 months you've got a picture of what's going to happen 12 months from now. But you're always changing things on the fly and making sure that if there is an influx of demand, then you can figure out how to make that work.

I guess it's the same way with cloud computing, which is what I'm involved with now. Once upon a time you were racking and stacking your own servers and having to put investment into that kind of thing. And now we can use AWS or other services, where you are just using what you need. I think the "just in time" wave of innovation that came through - I think it's like the 80s, maybe? Out of Japan that, yeah - that's revolutionized how supply chains work, and had a knock on effect to other industries as well.

Len: We've already talked a little bit about data analysis and data science. You mention in one of your books that part of your role involves onboarding and supporting junior data analysts. I was thinking - starting around 2015 at Leanpub, we actually had a huge explosion of sales in books on data science. This even led us to partnering with a team at Johns Hopkins University to provide a Leanpub-powered MOOC platform, where our most popular course set is called Cloud Based Data Science.

I was wondering if you could talk a little bit about what a junior data analyst does, but also why there's been such an explosion worldwide in the popularity of data science as a profession. Because that's-- I mean I've heard a little bit, and interviewed some people, but I don't know if I've really pinned down an answer to that.

Helen: Yeah. The role of a data analyst can be anything from jut preparing slide decks and Excel spreadsheets, all the way through to data science, where it's more of a statistical focus. I think a lot of statisticians have come into the data science world, probably around 2015. So that kind of makes sense.

I think that the role of technology and product development in the software industry has certainly changed a lot of that. And so we're no longer just looking at Google Analytics. We can pull terabytes of data from product databases - and we have the tools to slice and dice through all those data sets and make conclusions quicker, because we've now got the tooling to support that.

Len: And data analysis involves actually a lot of manual work, doesn't it? Like tagging things, and things like that.

Helen: Yes. There is this misconception that you can just click a button and the data's there, all the answers are there. But a lot of work a data analyst does is cleaning data. So, the data may not be in the database at all. It may not be tracked. And then if it is tracked, people put weird things into free text fields, and it might not be structured the same way that you need it for data analysis projects.

A lot of what I do is in software development. So, the database is designed around how people use the product, which is not necessarily what I need to do my side of things. There is a lot of - yeah, ETL work - and making sure that it's in the right format for what you need, and data cleansing is a huge part of that.

Len: It's really interesting. Once upon a time, I worked on a product that involved the idea of visualization - visualizing business information to executives, and things like that. And I've also pitched start-ups and worked in banking and stuff like that, so that involved lots of charts and numbers and things like that. And one thing I've noticed is that people believe what they see, if you know what I mean? Like, if a dashboard is shiny and well-designed, or if a chart shows - sometimes absurdly, data to like two decimal places in numbers, people trust it more.

Helen: Yeah.

Len: How do people in the profession talk about that? Because you want to make it look good, but you also don't want to give a misleading impression that your data is better than it is, and you also don't want to say, "We've got crap data."

Helen: Yeah, that is a challenge. If you don't have confidence in your data, it's important to let your stakeholder know. Because it's better for them to have the right expectation of what they're going to get out the other end. A lot of the time it isn't perfect. If you've got free text fields in your app, then it's a bit of a Wild West. So you can't necessarily guarantee that everything's going to be absolutely perfect.

I think - coming back to your point about the shiny thing, people will trust something that looks beautiful - even if it's wrong. And that's an ongoing challenge as well. The shiny thing with low confidence may get more interest than the black and white boring table which is more right.

So yeah, I think that's a challenge as well. And that's why there are more data visualization experts in the industry now. It's not enough to just have a go at it and throw it on a page and cross your fingers. You really need to have something a bit more sophisticated if you're going to grab people's attention.

Len: One feature of your career is that you've chosen to help people - currently you work with junior data analysts, getting them on board and things like that. But you've expanded that to help just the wider AWS community. And in particular, I just wanted to ask - how did you get started doing that? Did you just find yourself, "I've got some information that I'd really love to get out there?" Or was it more deliberate. Like, "I want to get into blogging?"

Helen: It's a little bit of both, I think? I guess I had come from Microsoft to Access and Excel kind of analysis world. And when I joined Xero, I had to graduate real quick into SQL and databases. A bit more of a tech focus. And so now that I see other junior analysts who have been brought on, and don't necessarily have 100% of those skills, but have the enthusiasm to learn - that's kind of where my interest lies. Because I know how overwhelming and scary it can be when you're faced with a big, scary database and no clues on how to drive it. And so, yeah - that's kind of my day job.

But I started blogging on dev.to about a year ago now. I just kind of assumed naively that it was a general tech blog. But it's more web dev focused. So I guess that's why I stand out a little bit more. There aren't too many data analysts or data professionals on the site. But it's been great, because web developers are really interested in how they can make the database work on the back end of their apps, and things to better support the data science side of the universe, so yeah, it's been great.

The dev.to community is really supportive. And one of the devs over there, James Hickey, suggested that Leanpub would be a good platform to sort of bring the message wider. So over the last few months, I've been compiling all the nuggets of best practice that I've been putting out there in the blogging world, and then putting that on the Leanpub platform as well.

Len: I've got a couple of questions about your books and getting up on our platform as well. But before going there, another way you try to help people is - and I know this is on like the front page of your website right now - an article about helping women in tech. Helping lift them up. I was wondering if you'd like to say maybe one or two words about - if you were talking to your historical self, your past self - knowing what you know now - was there any sort of like advice specific to being a woman going into tech that you would give her?

Helen: I think traditionally it has been a very male-dominated industry. Which is - there's nothing wrong with it, it's just how it is. A lot of the people on my team are males that have huge amounts of experience. And I guess it's just a traditional thing that's kind of happened. But now there is such a push for women in STEM and women in tech. And yeah, I think just not being afraid to get in there and give it a try. If you are thinking about changing careers, it's not as scary as you think and not as intimidating, given that there's so many more resources for a woman getting into the industry now. And there's meetup groups and Slack channels and Twitter support. Yeah, I think it's just - it's a lot easier now than it was maybe 10 or 20 years ago. That's not to say that it's not without it challenges, coming into something new. But I think that's true of all career changes.

Len: One question actually that comes up on almost every episode of this podcast, just because of the nature of - so many of our authors are in tech, and I like to ask people who studied Computer Science formally at university, if you were starting out now, would you do that again? And I also like to ask people who are in tech, who didn't - but who didn't study Computer Science formally - if you were starting out now and you knew you were going to go into tech, do you think you'd study Computer Science formally in university?

Helen: Oh, I don't know.

Len: I know it's a hard question.

Helen: Back in my day, if you wanted to study tech it was - well, for my particular university, it was electrical engineering, food technology. Or the traditional IT track with hardware, and we built - I think we built HTML web pages for my first year. So I don't think I would've, given that there's so many other general kind of courses that you could enjoy at university. So things like History and English. And for me, I liked Accounting. Which seems a bit unusual, I guess. I think there's so many things that you can enjoy - and when you come out the other side, you can decide what you want to do career wise after that.

Having a degree is more about showing that you can commit to something for three years or four years. And I don't think at 18 or 19 anyone's ready to make that kind of serious career decision. So, being able to make a pivot later in life with the resources that we've got now - like boot camps and online resources. I think that - yeah, if you follow your passion at university, do something you enjoy, and then make the change later - depending on what comes along. I think that's perfectly reasonable without a Computer Science background.

Len: I couldn't agree with you more. I have worked in fields that have nothing formally to do with the things I studied in university, and I've switched careers a couple of times, and I think people often underestimate what it means to commit to something that's going to take four years, that you're constantly evaluated on in a way that you can't - you more or less can't redo and finish in the right timeline. And you're mostly self-directed. It's up to you. You don't have a boss. And you're paying to do it usually as well.

Helen: Yeah.

Len: So that's actually like a really - most adults would enter into something like that with trepidation. And for someone in their formative young adulthood years - to achieve something like that is actually a bigger thing than just becoming a like well-educated person in a particular field.

Moving onto your books and to probably more of your day-to-day work that you do now. Your book AWS From A to Z, which I love - because Amazon's - I know it's just a coincidence, but Amazon's logo has that little smile arrow going from the A to Z.

Helen: Oh yeah.

Len: So, it covers a lot of AWS services that people may not be aware of. I was wondering if you could first maybe just mention - to begin with, what AWS is, now - and maybe one or two services about it that you really like and you work with. That you know about, but maybe people don't know about it. I guess I just frame it like - I'm assuming most people listening have heard of Amazon Web Services, and know that basically they built this like computer structure - like you mentioned before, so you can subscribe to it and attach yourself to it, so you don't have to have servers in the closet of your own.

Helen: Yeah, that's a good way to explain it.

Len: But it's so much more than that now.

Helen: Yes, absolutely. So, traditionally, if you are looking to set up servers to support some kind of infrastructure, it would be on premises and you would have servers in the closet or the cupboard. That beautiful image. And yeah, AWS has changed the game, and so now we can use their services in the cloud. They have data centres all over the world - depending on where you are, there will be one close to you somewhere. And that allows us to scale out capabilities.

So if you have a database that suddenly has an influx of traffic and an influx of data, it allows you to click a button and you'll be able to increase the resource that it takes to support that. Whereas traditionally, you'd have to go down and buy the capability, plug it in, figure it out, put the infrastructure in. And now you can just click a button from a console and it's all done. It's changed the game as far as infrastructure and development goes.

Which is fantastic. But it does mean that they have a lot of services that you need to get your head around. They've made it really easy to support niche infrastructure. Which is great, but at the same time, that's a lot of things to get your head around. So what I was trying to do with my AWS book, was break that down into little bite-sized chunks, where you can learn about things like Athena or Aurora or Serverless and sort of get your head around the jargon, and find the best result for what you need.

As far as my favorite services go, at the moment I'm currently working in data - so we use a product that AWS offers called Aurora, which is a Postgres database that allows us to scale out whenever we need. It's much quicker than your traditional Postgres database, which makes it good for users. And it talks directly to our data warehouse. Which is good for us - keepers of the data. I've written a little bit about those two services in the book. But then there's other things like server management and streaming data services - and just a little bit of everything.

Len: Thanks for that great explanation. It's really interesting, the diversity of services that AWS offers, and the ways they can be useful. So for people listening - get the book AWS: From A to Z, because it really is useful to understand these things.

Just to give a very practical example - some of the new features you may be paying for in some of the apps you use, are actually AWS services that people are kind of selling onto you.

So an example might be something like text to speech. All of a sudden one day, AWS comes out with a text to speech service, and then the app you use is "Like hey wait a minute, we can just pay for that AWS service and then sell it as a product of our own." It's not underhanded, but it's important to understand that behind some of the services that you're using, is actually AWS and that when they come out with a new product, that's actually one use that you - if you're like running your own company or you've got a start-up or something like that, that actually you can make of these services as well. So it's not just people in big companies or something like that, it's very practical.

And just specifically on that note, I wanted to ask - one way you can become an expert in AWS services is by reading dev.to and your posts on there, and all the other people in the community. But I believe you also got something called an AWS developer associate certification?

Helen: Yeah.

Len: Is that what it's called?

Helen: Yeah. AWS offers certifications for developers, solutions architects who have more of a high level view. SysOps administrators, and then big data and security and serverless specialities. So it proves that you've worked behind the scenes, and you've understood all the services from A to Z. And yeah, those are offered through AWS. They are - multi-choice or multi-response exams, and then you get a nice little badge to put on your LinkedIn profile. Recruiters love them. And it gives you more of a holistic overview of everything that's going on, if you've got one little piece of the puzzle in your day job. I'm working towards it right now. I've been putting a lot of time into the blog posts for the last year. So now I've finally got to the end of that year, and now I'll be working on getting my certification towards the end of this year.

Len: In your book *Big Data: From A to Z, I'm just going to quote you back at yourself. Sorry.

Helen: Oh no.

Len: "The book hopes to demystify the tools data scientists and data engineers use to build platforms and models and clear up any confusion on how machine learning, artificial intelligence and deep learning fit together." I just wanted to ask you if you could talk a little bit about how machine learning, artificial intelligence and deep learning fit together? I think to a lot of people, they've maybe read like - I mean, this kind of dates me, but they've read the odd Economist article about big data or something like that. How do these things all fit together in the big data space?

Helen: I think it's a bit of a misconception that these are all new things. They've been going on in academia in particular for a very long time. I have a new team member who's been doing this for ages. So it's not quite as new and buzz word-y as people think. It's just coming to the forefront because of tech companies who are in the media, and - yeah, it's kind of becoming more and more - more common to hear these things.

So, big data is traditionally defined by the volume of data. And there's no one measure of how big big has to be. But the velocity also - how fast it can be produced? That means it's considered big. So there's no real dictionary definition of big data, it's always changing. But the mechanisms that we use to work with it - machine learning, AI, NLP and deep learning - they're all variations on a theme, and yeah, I think reading into more of what machine learning is, is probably important for any developer. It's a good thing to get into if you are in development and you're looking for a change. Big data is a good way to go if you're looking for something new.

Len: So moving onto the last part of the interview - you've become a writer. And one thing I'd like to ask you - you've mentioned, you've got a day job and you've got this blogging job, and you're studying for this accreditation. When do you write? Do you have a specific time of day when you write, or do you save it up for the weekends? Do you have a sort of x number of words per day that you try to hit?

Helen: Ooh, that's a bit more scientific. I try and do a little bit after work and a little bit on my weekends. I love the Leanpub model of "publish often." It allows me to get something out there, and not wait for this big bang which could be months and months away.

I make a point of doing a bit of writing during the weekends. But I also think it's important not to get bogged down. Because your side hustle might become your side job, and you lose enthusiasm quickly. So I try and keep it to as long as my laptop battery lasts. I'll go sit on the sofa - and when my battery is dead, then writing time is over. And the great thing about Leanpub is that it saves on the fly. So even if my laptop dies at an inopportune moment, I've still got everything ready to go.

Len: Normally at this point I ask, "Why did you choose Leanpub?" And it sounds like it's because someone recommended it and you like the in-progress publishing.

Helen: Yes.

Len: So that's fantastic. And so, this is where we get a bit in the weeds - are you writing in our in-browser writing mode?

Helen: Yes. Dev.to also uses the same, similar Markdown syntax. So I've been able to lift and shift a lot of what I was doing on the blog and just pop it into the magic browser, and just make a few tweaks here and there and then add content as I go. So, I prefer to use the browser. I know there are options for Dropbox and GitHub, and other options, which is really nice to have that flexibility - but the browser works for me.

Len: That's fantastic to hear. Years ago when it first launched, it was gross. So I'm glad to hear that it's matured. I'm not trying to downplay - we work hard to improve things and we partly do that by talking to authors who are using things. So that's why I'm asking and why I'm trying to be funny about it.

One question I had, is - and I guess this would apply both to your books and to your blog - I know that your books come out of your blog posting. Is interacting with readers important to you? Either on social media or in comments sections, or through email? Is that something that you engage in?

Helen: Yeah, for sure. Once upon a time I had a WordPress blog and - shockingly enough - that wasn't really found by anyone or looked at - anyone, except for me. And I think dev.to really changed the game, and set me on this path. From my very first post, there was interactions with the community and comments and likes. And James, who I mentioned before - reached out, really early on actually - floating the idea of Leanpub and taking this further.

I think the community on dev.to is really important for developers that contribute there. Everyone's really friendly, it's not full of trolls - which is always nice. And Leanpub's the same. I've had lots of nice messages through the "Email the Author" function - words of encouragement, and tweaks that I can make, and it's all been friendly. So yeah, I think having a voice in the community is just as important as putting the content out there.

Len: The last question I always like to ask authors on this podcast is, if there was one feature we could build for you, something you've found missing in working with us. Or if there was one bug or a crappy thing we could fix for you - can you think of anything you would ask for?

Helen: I think the only thing that I find a little frustrating is the preview box. It doesn't quite render everything exactly how it would be in a preview. So I'm happy to go over to the preview function and generate a new preview of a PDF - but it's not quite exactly what's on the screen in the magic browser.

Len: Thank you very much for sharing that. Just for anyone listening - so the way it works is, if you're writing in our in-browser writing mode, you can have a preview window open on the right which shows you a preview. But - and this is one of those things where I kind of hang my head in shame - it actually doesn't exactly match what you're going to see in the book - well, in the PDF of the book. It's a bit tricky. Because if you're writing in that mode, your book will be output in PDF, EPUB, MOBI and our app format. So what's it going to look like? And people on EPUB and MOBI, they can set their own text size settings, and stuff like that. So it's important to understand that what it's really going to look like, has more than one answer.

But we actually do currently sometimes show you like, I think - if you've done a blurb, we'll actually show a little gray bar on the left, to the left of the blurb in the preview. And we had one author we were interacting with recently, who was like, "I really like that gray bar, but it's not in the preview." And then people get confused. "Is this a bug? Am I missing something?" And so having a clear explanation of what the preview window on the right is for, and having it at least match one thing exactly - is something that we know is an issue that we need to deal with.

So thanks for sharing that. Because we know how frustrating it can be for things not to quite match.

Well, Helen, thank you very much for taking the time to do this interview, and for being on the podcast, and for being a Leanpub author.

Helen: You're welcome, thank you for having me. It's been fun.

Len: Thank.

And thanks as always to all of you for listening to this episode of the Leanpub Frontmatter podcast. If you like what you heard, please rate and review us wherever you found us. And if you'd like to be a Leanpub author yourself, please go to our website at leanpub.com. Thanks.