Ten Habits of Great Data Analysts

PREFACE

1. Why I Love Being a Data Analyst
2. Discovering Data Analysis
3. A Career Built on Curiosity
4. When Data Became Personal
5. The Power of Real World Data

WHY I WROTE THIS BOOK

WHO SHOULD READ THIS BOOK

HOW TO USE THIS BOOK

TOOLS CHANGE. HABITS ARE FOREVER.

THE ELEPHANT IN THE ROOM

1. What About Artificial Intelligence?
2. Responsible Use of AI
3. How I Use AI in Practice
4. The Future of the Data Analyst

THE TEN HABITS OF GREAT DATA ANALYSTS

CHAPTER 1

1. Understand the Science and the People
2. Learning the Disease
3. Learning From the Literature
4. Case Study: When a Few Minutes of Reading Saves the Project
5. Listening to People with Lived Experience
6. Case Study: Listening to Lived Experience Averts a Catastrophe
7. The Importance of Team Science: The Analyst as a Collaborator
8. Curiosity Is the Foundation

CHAPTER 2

1. Start With a Clear Statistical Analysis Plan
2. What Is a Statistical Analysis Plan?
3. Translating Questions into Analysis
4. Using Table Shells
5. Operationalizing Variables
6. Clarifying the Deliverable
7. Understanding Deadlines and Timelines
8. Including Exploratory Analysis and Data Cleaning in the Data Analysis Plan
9. Plans Can Change
10. Case Study: The Registry as a Cornucopia Gone Wrong

CHAPTER 3

1. Explore the Data Before Modeling
2. Why Exploration Is Important
3. Start With the Structure
4. Examine Missing Data
5. Identify Outliers and Inconsistencies
6. Use Visual Exploration
7. Tools That Support Exploration
8. Becoming the Data Expert
9. Curiosity Leads to Insight
10. Case Study: What Exploratory Data Analysis Reveals

CHAPTER 4

1. Prioritize Data Quality
2. What Is Data Quality?
3. Common Data Quality Problems
4. Data Quality as an Analytical Skill
5. Case Study: When Missing Data Become Part of the Research Question
6. The Missing Data
7. Rethinking the Question
8. What the Missing Data Revealed
9. A Different Kind of Contribution
10. Lessons for Data Analysts
11. Communicating Data Quality Issues
12. Improving Data Quality Over Time
13. Data Quality as a Mindset

CHAPTER 5

1. Protect Data Security and Privacy
2. Understanding Sensitive Data
3. Regulatory Frameworks
4. Secure Data Access
5. Minimizing Data Exposure
6. Responsible Use of Analytical Tools
7. Transparency and Accountability
8. Ethical Responsibility
9. Case Study: Data Security Lessons from the Field
10. When “De-Identified” Data Are Not Truly De-Identified
11. Sharing Data Without Proper Review
12. Data Use Agreements and Secondary Data
13. Building a Relationship With the IRB
14. Protecting Sensitive Information in Documentation
15. Case Study: Proprietary Data Models
16. Protecting Sensitive Information in Code
17. Data Security Is a Shared Responsibility
18. Security and Scientific Integrity

CHAPTER 6

1. Communicate Clearly and Document Analytical Decisions
2. Why Communication Is Important
3. Communicating Throughout the Project
4. A Practical Communication Tool
5. Asynchronous Communication
6. The Importance of Documentation
7. Data Dictionaries
8. Case Study: The Cost of Missing Documentation
9. Building Better Documentation
10. A More Structured Approach
11. Lessons for Data Analysts
12. Communication and Documentation Builds Collaboration

CHAPTER 7

1. Systematically Tackle Complex Data
2. Thinking in Domains
3. Understanding Data Relationships
4. Identifying Unique Identifiers
5. Organizing Variables into Domains
6. Derived Variables Across Domains
7. Identifying Stratifying Variables
8. Working Efficiently Across Similar Variables
9. Protecting the Original Data
10. Understanding the Story Behind the Data
11. Helping Investigators Understand Their Data
12. Case Study: A Registry with Thousands of Variables
13. Looking for Clues in the Data
14. Reconstructing the Study Design
15. Lessons for Data Analysts

CHAPTER 8

1. Respect Qualitative Methods and Unstructured Data
2. Qualitative Methods Are Real Methods
3. Open-Ended Responses Often Provide the Missing Context
4. Coding, Recoding, and Categorization
5. Unstructured Data Are a Major Part of Modern Data Science
6. Clinical Notes Are Especially Important
7. Natural Language Processing and Computational Approaches
8. The Role of Manual Review
9. Case Study: Finding a Needle in a Haystack
10. Large Language Models (LLM) and New Possibilities
11. Use the Right Tool for the Job
12. Case Study: What Open-Ended Responses Revealed
13. Lessons for Data Analysts

CHAPTER 9

1. Know When to Do It the Old-Fashioned Way
2. The Most Tedious Solution Is Sometimes the Most Efficient Solution
3. Automation Has Costs Too
4. This Is Not an Argument Against Technology
5. Case Study: The Handwritten Logs
6. Why the Manual Approach Won
7. Other Old-Fashioned Tools That Still Work Just Fine
8. Some Heroes Don’t Wear Capes
9. Lessons for Data Analysts

CHAPTER 10

1. Keep Learning and Contributing to the Community
2. Learning Is Part of the Job
3. Learning From Real Projects
4. Learning From the Community
5. Sharing What You Learn
6. From Observation to Discovery
7. Case Study: It’s Never Too Late
8. Discovering a New Community
9. Learning in Small Pieces
10. Becoming Part of the Community
11. The Power of Curiosity
12. Lessons for Data Analysts

KEY TAKEAWAYS

1. The Ten Habits
2. What Data Analysis Really Is
3. The Responsibility
4. An Often Twisty Career Path
5. The Only Tools You Need

REFERENCES

DATA ANALYST’S TOOLKIT

1. Customizable Tools That Build Confidence

ACKNOWLEDGEMENTS

ABOUT THE AUTHOR