Data Analysis and Visualisation in Climate Science
Data Analysis and Visualisation in Climate Science
A Programmer's Guide
About the Book
Master the tools well so you can focus on the science
The book is in a good shape now and a final version is planned to be released by the end of 2022. The next update will included a wind rose example, list comprehensions, introduction to pathlib and sections on xarray and pandas.
Version 0.3 includes the following updates:
- code updated to Python 3.8
- code example for trend line plot using linear regression
- section added on glob.glob()
- introduction to f-strings and associated code updates
- minor code updates
- hi-resolution Cartopy map features
- fixed skew-t plot monotonicity issue
- improved solutions for creating sequences of dates
- some more NumPy solutions
- fixed some cross-references
Version 0.2 includes the following updates:
- Hovmoeller plots updated and additional example added
- Unnecessary np.meshgrid() commands removed from code examples
- Improved code for contour plot time axis formatting
- Working with Dates/Times section added (datetime module)
- Wind vector plot section added including examples
- Skew-T plot section added including examples
- Section on multiple line plots updated
- Plotting variables added to Table 8.1.2.1
- ax.quiver() command added to Table 8.1.2.1
- ax.subplots() commands updated (nrows and ncols)
- Typos fixed
- Cross-references and table/code/figure numbering fixed
About the Author
Table of Contents
-
- Preface
- Acknowledgements
-
1 Introduction
- 1.1 Overview and Objective
- 1.2 Concept of Local and Remote Machines
- 1.3 Software
-
2 Climate Data
- 2.1 Climate Data Overview
- 2.2 Data Use Licences
- 2.3 Data Quality
- 2.4 Accessing Climate Data
-
2.5 Types of Climate Data
- 2.5.1 Analyses and Reanalyses Products
- 2.5.2 Climate and NWP Model Output
- 2.5.3 Point observations
-
2.6 Data File Formats
- 2.6.1 Plain Text and ASCII
- 2.6.2 Binary
- 2.6.3 GRIB
- 2.6.4 netCDF
- 2.6.5 PP
-
3 Unix
-
3.1 Introduction to Unix
- 3.1.1 Linux Distributions
- 3.1.2 Desktop versus Server
- 3.1.3 High Performance Computing on a Server
-
3.2 Accessing a Remote Server
- 3.2.1 Remote Server Login Details
- 3.2.2 Virtual Private Network (VPN)
- 3.2.3 X Window System (X11 forwarding)
- 3.2.4 Connecting to a Remote Server
-
3.3 First Steps on the Unix server
- 3.3.1 The Terminal Window
- 3.3.2 The Shell
- 3.3.3 Linux Directory Structure and Home Directory
- 3.3.4 Quota
- 3.3.5 File Transfer to and from the Server
- 3.3.6 Mapping the Linux Home Directory as a Remote Network Drive
-
3.4 Some More Unix Server Basics
- 3.4.1 Unix Command Syntax}
- 3.4.2 Manual Pages
- 3.4.3 Editing Text Files
- 3.4.4 Full versus Relative Paths
- 3.4.5 Special Characters
-
3.5 Working with Files and Directories
- 3.5.1 Creating Text Files and Directories
- 3.5.2 Listing Files and Directories
- 3.5.3 Moving Around in the Directory Tree
- 3.5.4 Copying, Moving, Renaming and Deleting Files and Directories
-
3.6 Advanced Unix Commands
- 3.6.1 Examining Text Files
- 3.6.2 File and Directory Properties
- 3.6.3 File Permissions
- 3.6.4 Changing File Permissions and Ownership
- 3.6.5 Changing the Unix Account Password
- 3.6.6 Redirecting Command Output
- 3.6.7 Finding Files
- 3.6.8 File Compression and Archives
- 3.6.9 Download Files from the Command Line
-
3.7 Long-running Jobs
- 3.7.1 GNU Screen (recommended)
-
3.1 Introduction to Unix
-
4 Multi-dimensional Gridded Datasets
- 4.1 The Earth’s Coordinate System and Realms
- 4.2 The Model Grid
- 4.3 Grid Indexing and Geographical Referencing of Data Points
- 4.4 The Time Dimension
-
4.5 Horizontal Resolutions and Grid Types
- 4.5.1 Spectral Resolution
- 4.5.2 Full and Reduced Gaussian Grid
- 4.5.3 Regular latitude-longitude grid
-
4.6 Vertical Level Types
- 4.6.1 Pressure, Potential Temperature and Potential Vorticity Levels
- 4.6.2 Sigma (Model) Levels
- 4.6.3 Sigma-Hybrid Levels
-
5 The netCDF File Format
- 5.1 Introduction to the netCDF File Format
-
5.2 netCDF File Headers
- 5.2.1 Exploring netCDF File Headers with ncdump
- 5.2.2 Exploring netCDF File Headers with CDO
- 5.2.3 Exploring netCDF File Headers with ncview
- 5.3 Packed netCDF Files
- 5.4 netCDF File Format Conventions
-
6 Python - Concepts and Work Environment
- 6.1 Python Overview
-
6.2 Python Concepts
- 6.2.1 Python Modules and Packages
- 6.2.2 Package Dependencies
- 6.2.3 Package Managers, Repositories and Channels
- 6.2.4 Python Virtual Environments
-
6.3 Conda
- 6.3.1 Creating a Conda Environment
- 6.3.2 Activating and Deactivating Conda Environments
- 6.3.3 Installing Python Packages
- 6.3.4 Listing and Deleting Conda Environments
-
6.4 Python Code Development
- 6.4.1 Python Code Editors
- 6.4.2 Python IDEs
- 6.4.3 Browser-based Python Code Editing
-
7 Python - Programming Basics
-
7.1 Basic Python Programming Building Blocks
- 7.1.1 Declaring Variables
- 7.1.2 Variable Types and Conversion Between them
- 7.1.3 Functions
- 7.1.4 Methods and Attributes
- 7.1.5 Controlling the Code Flow
-
7.2 Applying Python in Climate Data Analysis
- 7.2.1 Error Messages when Running Code
- 7.2.2 Looping Through Input Files
- 7.2.3 Reading Data Files Into NumPy Variables
- 7.2.4 Executing Unix System Commands from Within Python
-
7.3 Introduction to Numpy
- 7.3.1 Creating Numpy Arrays
- 7.3.2 Indexing NumPy Arrays
- 7.3.3 Saving and Loading NumPy Variables
- 7.3.4 Some NumPy Solutions
-
7.4 Working with Dates and Times
- 7.4.1 Creating Date Objects
-
7.4.2 Converting Datetime Objects to Strings (
strftime) - 7.4.3 Creating a NumPy Array with Datetime Objects
-
7.5 Tips and Solutions
- 7.5.1 String Formatting of Numbers
- 7.5.2 Zero-padding Integer Values in Filenames
- 7.5.3 Calculate Height from Geopotential with MetPy
-
7.1 Basic Python Programming Building Blocks
-
8 Python - Creating Plots
-
8.1 Matplotlib
- 8.1.1 Setting up Plotting Page (Figure and Axes)
- 8.1.2 Main Plotting Commands
- 8.1.3 Colour Names and Colour Maps
-
8.2 Line Plots
- 8.2.1 Line Plot with Labels
- 8.2.2 Line Plot with Arrows
- 8.2.3 Multiple Lines Plot with Markers and Legend
- 8.2.4 Multiple Lines Plot with two Scales
- 8.2.5 Multiple Lines Plot with Standard Deviation
-
8.3 Scatter Plots
- 8.3.1 Scatter Plot with a Legend
- 8.3.2 Scatter Plot with Divergent Colour Bar
- 8.3.3 Scatter Plot on a Map with Colour Bar and Legend
- 8.3.4 Adding Trend Line Based on Linear Regression
-
8.4 Map Plots
- 8.4.1 Cartopy Map Projections and Data Transformation
- 8.4.2 High-Resolution Map Features
- 8.4.3 Simple Map of SST Anomalies
- 8.4.4 Map with Stipples for Statistical Significance
-
8.5 Bar Graphs
- 8.5.1 Anomalies Bar Graph
-
8.6 Hovmöller Plots
- 8.6.1 Hovmöller Plot with Time as a Function of Latitude
- 8.6.2 Hovmöller Plot with Time as a Function of Pressure
-
8.7 Vertical Cross-Section Plots
- 8.7.1 Meridional Cross-Section
- 8.7.2 Vertical Cross-Section Between two Points
-
8.8 Skew-T Plots
- 8.8.1 Brief Introduction to Skew-T Plots
- 8.8.2 Simple Skew-T Plot
- 8.8.3 Multiple Skew-T Plots
-
8.9 Vector and Streamline Plots
- 8.9.1 Black Wind Vectors on Filled Colour Contours
- 8.9.2 Coloured Wind Vectors
-
8.10 Looping Through Multiple Panels
-
8.10.1 Multiple Line Plots (
axes.flatmethod) -
8.10.2 Multiple Line Plots (
pop()function) -
8.10.3 Multiple Map Plots (
axes.flatmethod)
-
8.10.1 Multiple Line Plots (
-
8.1 Matplotlib
-
9 Data Analysis with CDO
- 9.1 What is CDO?
- 9.2 Useful CDO Resources
-
9.3 Basic Syntax of CDO Commands
- 9.3.1 CDO Options
- 9.3.2 CDO Operator Categories
- 9.3.3 Using Multiple CDO Operators
- 9.3.4 CDO Operator Parameters
- 9.3.5 CDO Command Input and Output Files
- 9.4 Merging Files
-
9.5 Selections
- 9.5.1 Selecting Variables
- 9.5.2 Selecting Spatial Subsets (Geographical Regions)
- 9.5.3 Selecting Vertical Levels
- 9.5.4 Selecting Time Subsets
-
9.6 Basic Statistics
- 9.6.1 Statistics over the Time Domain
- 9.6.2 Statistics over the Spatial Domain
- 9.6.3 Statistics over the Vertical Domain
- 9.6.4 Statistics over the Zonal Domain
- 9.6.5 Statistics over the Meridional Domain
- 9.6.6 Statistics over Ensembles
-
9.7 Interpolations
- 9.7.1 Interpolation to a new horizontal grid (remapping)
- 9.7.2 Interpolation in the Vertical Domain
- 9.7.3 Interpolation in the Time Domain
-
9.8 Basic Arithmetic
- 9.8.1 Artithmetic Between Two Files
- 9.8.2 Arithmetic Using a Constant Value
-
9.9 Applying CDO in Climate Computations
- 9.9.1 Indian Ocean Dipole Example
- 9.9.2 Sahel Rainfall Variability Example
- 9.9.3 Creating a Land-Sea Mask File
- 9.10 Using CDO with Python
-
Appendix
- References
- List of Acronyms
- Notes
The Leanpub 60-day 100% Happiness Guarantee
Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
See full terms
Do Well. Do Good.
Authors have earned$11,812,115writing, publishing and selling on Leanpub, earning 80% royalties while saving up to 25 million pounds of CO2 and up to 46,000 trees.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
Top Books
- #1
Stratospheric
Tom Hombergs, Björn Wilmsmann, and Philip RiecksFrom Zero to Production with Spring Boot and AWS. All you need to know to get a Spring Boot application into production with AWS. No previous AWS knowledge required.
Go to stratospheric.dev for a tour of the contents.
- #2
C++20 - The Complete Guide
Nicolai M. JosuttisAll new language and library features of C++20 (for those who know previous C++ versions).
The book presents all new language and library features of C++20. Learn how this impacts day-to-day programming, to benefit in practice, to combine new features, and to avoid all new traps.
Buy early, pay less, free updates.
Other books:
- #3
OpenIntro Statistics
David Diez, Christopher Barr, Mine Cetinkaya-Rundel, and OpenIntroA complete foundation for Statistics, also serving as a foundation for Data Science.
Leanpub revenue supports OpenIntro (US-based nonprofit) so we can provide free desk copies to teachers interested in using OpenIntro Statistics in the classroom and expand the project to support free textbooks in other subjects.
More resources: openintro.org.
- #4
Jetpack Compose internals
Jorge CastilloJetpack Compose is the future of Android UI. Master how it works internally and become a more efficient developer with it. You'll also find it valuable if you are not an Android dev. This book provides all the details to understand how the Compose compiler & runtime work, and how to create a client library using them.
- #5
Advanced Web Application Architecture
Matthias NobackThe missing manual for making your web applications future-proof
- #6
Ansible for DevOps
Jeff GeerlingAnsible is a simple, but powerful, server and configuration management tool. Learn to use Ansible effectively, whether you manage one server—or thousands.
- #7
Maîtriser Apache JMeter
Philippe Mouawad, Bruno Demion (Milamber), and Antonio Gomes RodriguesToute la puissance d'Apache JMeter expliquée par ses commiteurs et utilisateurs experts. De l'intégration continue en passant par le Cloud, vous découvrirez comment intégrer JMeter à vos processus "Agile" et Devops.
If you're looking for the newer english version of this book, go to Master JMeter : From load testing to DevOps
- #8
Aprendiendo Git
Miguel Angel Durán GarcíaGit no es complicado... ¡Si lo entiendes! 😜
¿Sientes que sabes usarlo porque has memorizado todos los comandos que necesitas? ¡Pero no entiendes qué hace cada cosa y por qué! Así es normal que, cuando exista un problema, te cueste resolverlo.
¡Con este libro vas a entender de una vez por todas todo lo que es Git y cómo sacarle provecho!
- #9
D3 Start to Finish
Peter CookD3 Start to Finish shows you how to build a custom, interactive and beautiful data visualisation using the JavaScript library D3.js (versions 6 & 7).
The book covers D3.js concepts such as selections, joins, requests, scale functions, events & transitions. You'll put these concepts into practice by building a custom, interactive data visualisation.
- #10
R Programming for Data Science
Roger D. PengThis book brings the fundamentals of R programming to you, using the same material developed as part of the industry-leading Johns Hopkins Data Science Specialization. The skills taught in this book will lay the foundation for you to begin your journey learning data science. Printed copies of this book are available through Lulu.
Top Bundles
- #1
Software Architecture
2 Books
"Software Architecture for Developers" is a practical and pragmatic guide to modern, lightweight software architecture, specifically aimed at developers. You'll learn:The essence of software architecture.Why the software architecture role should include coding, coaching and collaboration.The things that you really need to think about before... - #2
CCIE Service Provider Ultimate Study Bundle
2 Books
Piotr Jablonski, Lukasz Bromirski, and Nick Russo have joined forces to deliver the only CCIE Service Provider training resource you'll ever need. This bundle contains a detailed and challenging collection of workbook labs, plus an extensively detailed technical reference guide. All of us have earned the CCIE Service Provider certification... - #4
Pattern-Oriented Memory Forensics and Malware Detection
2 Books
This training bundle for security engineers and researchers, malware and memory forensics analysts includes two accelerated training courses for Windows memory dump analysis using WinDbg. It is also useful for technical support and escalation engineers who analyze memory dumps from complex software environments and need to check for possible... - #6
Retromat eBook Bundle for Agile Retrospectives
2 Books
If you facilitate retrospectives this bundle is for you: "Plans for Retrospectives" helps beginners learn the lay of the land with tried-and-true plans. Once you know your way around, "Run great agile retrospectives" contains all 135+ activities in Retromat for you to mix and match. - #7
Static Analysis and Automated Refactoring
2 Books
As PHP developers we are living in the "Age of Static Analysis". We can use a tool like PHPStan to learn about potential bugs before we ship our code to production, and we can enforce our team's programming standards using custom PHPStan rules. Recipes for Decoupling by Matthias Noback teaches you in great detail how to do this, while also... - #9
Practical FP in Scala + Functional event-driven architecture
2 Books
Practical FP in Scala (A hands-on approach) & Functional event-driven architecture, aka FEDA, (Powered by Scala 3), together as a bundle! The content of PFP in Scala is a requirement to understand FEDA so why not take advantage of this bundle!?
