Data Analysis and Visualisation in Climate Science
Data Analysis and Visualisation in Climate Science
A Programmer's Guide
About the Book
Master the tools well so you can focus on the science
This book is being published as an in-progress book until about mid 2021. During this time feedback will be gathered, bugs will be fixed, further sections will be added and more code examples will be created. Until the final version is published, the book can be purchase for as little as $4.99. Anyone who buys the book during that time will have full access to all subsequent versions of the book.
Version 0.2 includes the following updates:
- Hovmoeller plots updated and additional example added
- Unnecessary np.meshgrid() commands removed from code examples
- Improved code for contour plot time axis formatting
- Working with Dates/Times section added (datetime module)
- Wind vector plot section added including examples
- Skew-T plot section added including examples
- Section on multiple line plots updated
- Plotting variables added to Table 8.1.2.1
- ax.quiver() command added to Table 8.1.2.1
- ax.subplots() commands updated (nrows and ncols)
- Typos fixed
- Cross-references and table/code/figure numbering fixed
About the Author
Table of Contents
-
- Preface
- Acknowledgements
-
1 Introduction
- 1.1 Overview and Objective
- 1.2 Concept of Local and Remote Machines
- 1.3 Software
-
2 Climate Data
- 2.1 Climate Data Overview
- 2.2 Data Use Licences
- 2.3 Data Quality
- 2.4 Accessing Climate Data
-
2.5 Types of Climate Data
- 2.5.1 Analyses and Reanalyses Products
- 2.5.2 Climate and NWP Model Output
- 2.5.3 Point observations
-
2.6 Data File Formats
- 2.6.1 Plain Text and ASCII
- 2.6.2 Binary
- 2.6.3 GRIB
- 2.6.4 netCDF
- 2.6.5 PP
-
3 Unix
-
3.1 Introduction to Unix
- 3.1.1 Linux Distributions
- 3.1.2 Desktop versus Server
- 3.1.3 High Performance Computing on a Server
-
3.2 Accessing a Remote Server
- 3.2.1 Remote Server Login Details
- 3.2.2 Virtual Private Network (VPN)
- 3.2.3 X Window System (X11 forwarding)
- 3.2.4 Connecting to a Remote Server
-
3.3 First Steps on the Unix server
- 3.3.1 The Terminal Window
- 3.3.2 The Shell
- 3.3.3 Linux Directory Structure and Home Directory
- 3.3.4 Quota
- 3.3.5 File Transfer to and from the Server
- 3.3.6 Mapping the Linux Home Directory as a Remote Network Drive
-
3.4 Some More Unix Server Basics
- 3.4.1 Unix Command Syntax}
- 3.4.2 Manual Pages
- 3.4.3 Editing Text Files
- 3.4.4 Full versus Relative Paths
- 3.4.5 Special Characters
-
3.5 Working with Files and Directories
- 3.5.1 Creating Text Files and Directories
- 3.5.2 Listing Files and Directories
- 3.5.3 Moving Around in the Directory Tree
- 3.5.4 Copying, Moving, Renaming and Deleting Files and Directories
-
3.6 Advanced Unix Commands
- 3.6.1 Examining Text Files
- 3.6.2 File and Directory Properties
- 3.6.3 File Permissions
- 3.6.4 Changing File Permissions and Ownership
- 3.6.5 Changing the Unix Account Password
- 3.6.6 Redirecting Command Output
- 3.6.7 Finding Files
- 3.6.8 File Compression and Archives
- 3.6.9 Download Files from the Command Line
-
3.7 Long-running Jobs
- 3.7.1 GNU Screen (recommended)
-
3.1 Introduction to Unix
-
4 Multi-dimensional Gridded Datasets
- 4.1 The Earth’s Coordinate System and Realms
- 4.2 The Model Grid
- 4.3 Grid Indexing and Geographical Referencing of Data Points
- 4.4 The Time Dimension
-
4.5 Horizontal Resolutions and Grid Types
- 4.5.1 Spectral Resolution
- 4.5.2 Full and Reduced Gaussian Grid
- 4.5.3 Regular latitude-longitude grid
-
4.6 Vertical Level Types
- 4.6.1 Pressure, Potential Temperature and Potential Vorticity Levels
- 4.6.2 Sigma (Model) Levels
- 4.6.3 Sigma-Hybrid Levels
-
5 The netCDF File Format
- 5.1 Introduction to the netCDF File Format
-
5.2 netCDF File Headers
- 5.2.1 Exploring netCDF File Headers with ncdump
- 5.2.2 Exploring netCDF File Headers with CDO
- 5.2.3 Exploring netCDF File Headers with ncview
- 5.3 Packed netCDF Files
- 5.4 netCDF File Format Conventions
-
6 Python - Concepts and Work Environment
- 6.1 Python Overview
-
6.2 Python Concepts
- 6.2.1 Python Modules and Packages
- 6.2.2 Package Dependencies
- 6.2.3 Package Managers, Repositories and Channels
- 6.2.4 Python Virtual Environments
-
6.3 Conda
- 6.3.1 Creating a Conda Environment
- 6.3.2 Activating and Deactivating Conda Environments
- 6.3.3 Installing Python Packages
- 6.3.4 Listing and Deleting Conda Environments
-
6.4 Python Code Development
- 6.4.1 Python Code Editors
- 6.4.2 Python IDEs
- 6.4.3 Browser-based Python Code Editing
-
7 Python - Programming Basics
-
7.1 Basic Python Programming Building Blocks
- 7.1.1 Declaring Variables
- 7.1.2 Variable Types and Conversion Between them
- 7.1.3 Functions
- 7.1.4 Methods and Attributes
- 7.1.5 Controlling the Code Flow
-
7.2 Applying Python in Climate Data Analysis
- 7.2.1 Error Messages when Running Code
- 7.2.2 Looping Through Input Files
- 7.2.3 Reading Data Files Into NumPy Variables
- 7.2.4 Executing Unix System Commands from Within Python
-
7.3 Introduction to Numpy
- 7.3.1 Creating Numpy Arrays
- 7.3.2 Indexing NumPy Arrays
- 7.3.3 Saving and Loading NumPy Variables
-
7.4 Working with Dates and Times
- 7.4.1 Creating Date Objects
-
7.4.2 Converting Dates to Strings (
strftime)
-
7.5 Tips and Solutions
- 7.5.1 String Formatting of Numbers
- 7.5.2 Zero-padding Integer Values in Filenames
- 7.5.3 Calculate Height from Geopotential with MetPy
-
7.1 Basic Python Programming Building Blocks
-
8 Python - Creating Plots
-
8.1 Matplotlib
- 8.1.1 Setting up Plotting Page (Figure and Axes)
- 8.1.2 Main Plotting Commands
- 8.1.3 Colour Names and Colour Maps
-
8.2 Line Plots
- 8.2.1 Line Plot with Labels
- 8.2.2 Line Plot with Arrows
- 8.2.3 Multiple Lines Plot with Markers and Legend
- 8.2.4 Multiple Lines Plot with two Scales
- 8.2.5 Multiple Lines Plot with Standard Deviation
-
8.3 Scatter Plots
- 8.3.1 Scatter Plot with a Legend
- 8.3.2 Scatter Plot with Divergent Colour Bar
- 8.3.3 Scatter Plot on a Map with Colour Bar and Legend
-
8.4 Map Plots
- 8.4.1 Cartopy Map Projections and Data Transformation
- 8.4.2 Simple Map of SST Anomalies
- 8.4.3 Map with Stipples for Statistical Significance
-
8.5 Bar Graphs
- 8.5.1 Anomalies Bar Graph
-
8.6 Hovmöller Plots
- 8.6.1 Hovmöller Plot with Time as a Function of Latitude
- 8.6.2 Hovmöller Plot with Time as a Function of Pressure
-
8.7 Vertical Cross-Section Plots
- 8.7.1 Meridional Cross-Section
- 8.7.2 Vertical Cross-Section Between two Points
-
8.8 Skew-T Plots
- 8.8.1 Brief Introduction to Skew-T Plots
- 8.8.2 Simple Skew-T Plot
- 8.8.3 Multiple Skew-T Plots
-
8.9 Vector and Streamline Plots
- 8.9.1 Black Wind Vectors on Filled Colour Contours
- 8.9.2 Coloured Wind Vectors
-
8.10 Looping Through Multiple Panels
-
8.10.1 Multiple Line Plots (
axes.flatmethod) -
8.10.2 Multiple Line Plots (
pop()function) -
8.10.3 Multiple Map Plots (
axes.flatmethod)
-
8.10.1 Multiple Line Plots (
-
8.1 Matplotlib
-
9 Data Analysis with CDO
- 9.1 What is CDO?
- 9.2 Useful CDO Resources
-
9.3 Basic Syntax of CDO Commands
- 9.3.1 CDO Options
- 9.3.2 CDO Operator Categories
- 9.3.3 Using Multiple CDO Operators
- 9.3.4 CDO Operator Parameters
- 9.3.5 CDO Command Input and Output Files
- 9.4 Merging Files
-
9.5 Selections
- 9.5.1 Selecting Variables
- 9.5.2 Selecting Spatial Subsets (Geographical Regions)
- 9.5.3 Selecting Vertical Levels
- 9.5.4 Selecting Time Subsets
-
9.6 Basic Statistics
- 9.6.1 Statistics over the Time Domain
- 9.6.2 Statistics over the Spatial Domain
- 9.6.3 Statistics over the Vertical Domain
- 9.6.4 Statistics over the Zonal Domain
- 9.6.5 Statistics over the Meridional Domain
- 9.6.6 Statistics over Ensembles
-
9.7 Interpolations
- 9.7.1 Interpolation to a new horizontal grid (remapping)
- 9.7.2 Interpolation in the Vertical Domain
- 9.7.3 Interpolation in the Time Domain
-
9.8 Basic Arithmetic
- 9.8.1 Artithmetic Between Two Files
- 9.8.2 Arithmetic Using a Constant Value
-
9.9 Applying CDO in Climate Computations
- 9.9.1 Indian Ocean Dipole Example
- 9.9.2 Sahel Rainfall Variability Example
- 9.9.3 Creating a Land-Sea Mask File
- 9.10 Using CDO with Python
-
Appendix
- References
- List of Acronyms
- Notes
Authors have earned$9,549,183writing, publishing and selling on Leanpub, earning 80% royalties while saving up to 25 million pounds of CO2 and up to 46,000 trees.
Learn more about writing on Leanpub
The Leanpub 45-day 100% Happiness Guarantee
Within 45 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
See full terms
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers), EPUB (for phones and tablets) and MOBI (for Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
Top Books
- #1
Algebra-Driven Design
Sandy MaguireA how-to field guide on building leak-free abstractions and algebraically designing real-world applications.
- #2
Production Haskell
Matt ParsonsAre you excited about Haskell, but don't know where to begin? Are you thrilled by the technical advantages, but worried about the unknown pitfalls? This book has you covered.
- #3
Machine Learning Engineering
Andriy Burkov"If you intend to use machine learning to solve business problems at scale, I'm delighted you got your hands on this book."
—Cassie Kozyrkov, Chief Decision Scientist at Google
"Foundational work about the reality of building machine learning models in production."
—Karolis Urbonas, Head of Machine Learning and Science at Amazon
- #4
CCIE Service Provider Version 4 Written and Lab Exam Comprehensive Guide
Nicholas RussoThe service provider landscape has changed rapidly over the past several years. Networking vendors are continuing to propose new standards, techniques, and procedures for overcoming new challenges while concurrently reducing costs and delivering new services. Cisco has recently updated the CCIE Service Provider track to reflect these changes; this book represents the author's personal journey in achieving that certification.
- #5
CCIE SP v4.1 - Workbook
Łukasz Bromirski, Piotr Jablonski, and Nicholas RussoAre you striving to prepare to and pass CCIE SP lab exam? Take the opportunity and get this workbook! With the attached initial cfg files you will prepare yourself for the CCIE SP exam as well as learn SP technologies applicable to all kinds of today modern networks! This workbook covers blueprint topics and provides challenging examples.
- #6
Ansible for DevOps
Jeff GeerlingAnsible is a simple, but powerful, server and configuration management tool. Learn to use Ansible effectively, whether you manage one server—or thousands.
- #7
The Hundred-Page Machine Learning Book
Andriy BurkovEverything you really need to know in Machine Learning in a hundred pages.
- #8
Cloud Strategy
Gregor Hohpe“Strategy is the difference between making a wish and making it come true.” A successful migration to the cloud shouldn’t be driven by wishes, but guided by a sound strategy, frameworks, and decision models. This book tells you how—without becoming superficial nor getting lost in technology and product details.
- #9
C++ Best Practices
Jason TurnerLevel up your C++, get the tools working for you, eliminate common problems, and move on to more exciting things!
- #10
Thinking with Types
Sandy MaguireThis book aims to be the comprehensive manual for type-level programming. It's about getting you from here to there---from a competent Haskell programmer to one who convinces the compiler to do their work for them.
Top Bundles
- #1
Quality Software
11 Books
The Quality Software Bundle is for managers, would-be managers, and any of us who find themselves being managed and confused. This comprehensive bundle covers the entire span of software development approaches, from hacking through waterfall, cascade, prototyping, Iterative enhancement, reusable code, off-the-shelf, to Agile teams. The bundle... - #2
The Node.js Bundle
3 Books
This bundle combines three bestselling Leanpub Node.js books into a package that gives you everything you need to get started with developing Node.js applications at an unbeatable price. - #3
The Tester's Library
8 Books
The Tester's Library consists of eight five-star books that every software tester should read and re-read. As bound books, this collection would cost over $200. Even as e-books, their price would exceed $80, but in this bundle, their cost is only $49.99. Here are the books, and why they should be in your library: Perfect Software and Other... - #4
agile
11 Books
In this bundle, you will find 10 different agile books. They are about different aspects of being agile. - finding a job - doing coding dojo's - Retrospectives - Personal kanban - a non-typical coaching book and even a book that gives you an insight in the lives of some agile people. - #5
WTFlop 6M + HU - Beta Bundle
6 Books
- #6
Fifty Quick Ideas
3 Books
Get all three books for the price of two! Fifty Quick Ideas books are full of practical, real-world techniques that you can use to improve teamwork, build better products and build them in a better way. - #7
Growing Agile: Coach's Guide Series
4 Books
This bundle provides a collection of training and workshop plans for a variety of agile topics. The series is aimed at agile coaches, trainers and ScrumMasters who often find themselves needing to help teams understand agile concepts. Each book in the series provides the plans, slides, handouts and activity instructions to run a number of... - #8
Marionette.js A to Z
3 Books
Marionette.js has become a popular Backbone-based javascript library, because it helps you avoid boiler-plate code and focus on what makes your app special. But how do you get started?Whether you want to finally get started with dynamic client-side programming (like one reader did), or are an experienced front-end dev who has inherited a... - #9
Build A Better Backbone App
3 Books
The best way to learn new development skills is through experience, but that takes time you don't have.Get the best of both worlds with this bundle: you'll learn how to produce modern web applications by learning from experienced developers like Derick Bailey and David Sulc. BackboneJS is one of the favorite tools on the web today, but it... - #10
People Skills—Soft but Difficult
7 Books
Perhaps you've been told that "lack of people skills" has been holding you back. No wonder: you may have had hundreds of hours of technical training, but little or no "people skills" guidance.You've heard it said that people skills are "soft," whereas technical skills are "hard." For you, though, technical skills are "easy," but people skills...
