Acknowledgements

Mike

First and foremost I would like to express my thanks to Mike Bostock, the driving force behind d3.js. His efforts are tireless and his altruism in making his work open and available to the masses is inspiring.

Partners, Supporters and Contributors.

Mike has worked with a crew of like-minded individuals in bringing D3 to the World. Vadim Ogievetsky and Jeffrey Heer share honours for the work on D3: Data-Driven Documents and while there has been a cast of over 40 people contributing to the D3 code base, Jason Davies stands out as the man who has provided a generous portion especially in the area of mapping.

Nick Zhu has created a fantastic resource in dc.js (which is built on top of d3.js and crossfilter) and has been kind enough to provide good advice and permission to include some of his work in the dc.js section.

Advice given by Christophe Viau has been a great help in getting me settled into the on-line world and his energy in managing and directing the D3 community is amazing.

Proof Reading

I am particularly grateful for the assistance given by Filiep Spyckerelle and Robin Bennett who selflessly donated their time and expertise in proofreading above and beyond the call of duty (where this document contains any errors, they are most certainly mine).

The d3.js Community

Big thanks go out to the D3 community. Whether providing advice on Google Groups or Stack Overflow, contributing examples on bl.ocks.org or just giving back in the form of time and effort to similar work. Well done all.

Cover art

Out of the blue and in yet another example of the friendly and giving nature of people involved in this community I was contacted by Jose (‘Tactician Jenro’) who offered to use his skills to design a cover for the book. I think he did a great job and was super helpful. If you think that he could help you out with a project, you can get in touch with him at jenrothetactician@gmail.com or @tacticianjenro.

Leanpub

Lastly, I want to pay homage to Leanpub who have made the publishing of this document possible. They offer an outstanding service for self-publishing and have made the task of providing and distributing content achievable.

Make sure you get the most up to date copy of D3 Tips and Tricks

If you’ve received a copy of this book from any location other than Leanpub then it’s possible that you haven’t got the latest version. Go to https://leanpub.com/D3-Tips-and-Tricks and download the most recent version. After all, it won’t cost you anything :-). If you find some value in the work, please consider contributing when you download it so that Leanpub get something for hosting the book (and I’ll think of you fondly). Please also be aware that this book is for version 3 of d3.js. If you are considering writing code for version 4 you will want the new edition here. Version 5 of the book is here.

What is d3.js?

d3.js (hereafter abridged as D3) is “a JavaScript library for manipulating documents based on data”.

But that description doesn’t do it justice.

D3 is all about helping you to take information and make it more accessible to others via a web browser.

It’s a JavaScript library. That means that it’s a tool that can be used in conjunction with other tools to get a job done. Those other tools are mainly HTML and CSS (amongst others) but you don’t need to know too much about either to use D3 (although it will help :-)).

It’s an open framework, which means that there are no hidden mysteries about how it does its magic and it allows others to contribute to a constant cycle of improvement.

It’s built to leverage web standards which means that modern browsers don’t have to do anything special to use D3, they just have to support the framework that the Internet has adopted for ease of use.

The beauty of D3 is that it allows you to associate data and what appears on the screen in a way that directly links the two. Change the data and you change the object on the screen. D3’s trick is to let you set what appears on the screen. A circle, a line, a point on a map, a graph, a bouncing ball, a gradient (and way, way more). Once the data and the object are linked the possibilities are endless.

It won`t do everything for you in your quest to create the perfect visualization, but it does give you the ability to achieve that goal.

It bridges the gap between the static display of data and the desire of people to mess about with it. That applies equally to the developer who wants to show something cool and to the end user who wants to be able to explore information interactively.

It was (and still is being) developed by Mike Bostock who has not just spent time writing the code, but writing the documentation for D3 as well. There is an extensive community of supporters who also contribute to the code, provide technical support online and generally have fun creating amazing visualizations. Their contributions are extraordinary (you only have to look at the work of Jason Davies to be amazed).

Introduction

I never set out to write treatise on D3…

I am a simple user of this extraordinary framework and when I say simple, I really mean I had no idea how to get it to do anything when I started; I needed to do a lot of searching and learned by trial-and-error (emphasis on the errors which were entirely mine). The one thing that I did know was that the example graphics shown by Mike Bostock and others were the sort of graphical goodness that I wanted to play with.

So to get from the point of having no skills whatsoever to the point where I could begin to code up something to display data in a way I wanted, I had to capture the information as I went. The really cool thing about this sort of process is that it doesn’t need to occur all at once. You can start with no knowledge whatsoever (or pretty close) and by standing on the shoulders of other’s work, you can add building blocks to improve what you’re seeing and then change the blocks to adapt and improve.

For example (and this is pretty much how it started). I wanted to draw a line graph, so I imported an example and then got it running locally on my computer. Then I worked out how to change the example data for my data. Then I worked out how to move the Y axis from the right to the left. Then how to make the axis labels larger, change the tick size, make the lines fatter, change the colour, add a label, fill the area under the graph, put the graph in the centre of the page, add a glow to the text to help it stand out, put it in a framework (bootstrap), add buttons to change data sets, animate the transitions between data sets, update the data automatically when it changed, add a pan and zoom feature, turn parts of the graph into hyperlinks to move to other graphs… And then I started on bar graphs :-).

The point to take away from all of this is that any one graph is just a collection of lots of blocks of code, each block designed to carry out a specific function. Pick the blocks you want and implement them.

I found it was much simpler to work on one thing (block) at a time, and this helped greatly to reduce the uncertainty factor when things didn’t work as anticipated. I’m not going to pretend that everything I’ve done while trying to build graphs employs the most elegant or efficient mechanism, but in the end, if it all works on the screen, I walk away happy :-). That’s not to say I have deliberately ignored any best practices – I just never knew what they were. Likewise, wherever possible, I have tried to make things as extensible as possible.

You will find that I have typically eschewed a simple “Do this approach” for more of a story telling exercise. This means that some explanations are longer and more flowery than might be to everyone’s liking, but there you go, try to be brave :-)

I’m sure most authors try to be as accessible as possible. I’d like to do the same, but be warned… There’s a good chance that if you ask me a technical question I may not know the answer. So please be gentle with your emails :-).

Email: d3noobmail+contact@gmail.com

What do you need to get started?

Let’s be frank. Not everyone will be inclined to develop a graphic using D3.

However, that doesn’t mean that it’s beyond those with a little computer savy and a willingness to have a play. Remember failure is your friend (I am fairly sure that I am also related by blood). Just learn from your mistakes and it’ll all work out.

So, here in no particular order is a list of good things to know. None of which are essential, but any one (or more) of which will make your life slightly easier.

• HyperText Markup Language (HTML)
• JavaScript
• Cascading Style Sheets (CSS)
• Web Servers
• PHP

HTML

This stands for HyperText Markup Language and is the stuff that web pages are made of. Check out the definition and other information on Wikipedia for a great overview. Just remember that all you’re going to use HTML for is to hold the code that you will use to present your information. This will be as a .html (or .htm) file and they can be pretty simple (we’ll look at some in a moment).

JavaScript

JavaScript is what’s called a ‘scripting language’. It is the code that will be contained inside the HTML file that will make D3 do all its fanciness. In fact, D3 is a JavaScript Library, it’s the native language for using D3.

Knowing a little bit about this would be really good, but to be perfectly honest, I didn’t know anything about it before I started. I read a book along the way (JavaScript: The Missing Manual from O’Reilly) and that helped with context, but the examples that are available for D3 graphics are understandable, and with a bit of trial and error, you can figure out what’s going on.

In fact, most of what this collection of information’s about is providing examples and explanations for the JavaScript components of D3.

Cascading Style Sheets (CSS)

Cascading Style Sheets (everyone appears to call them ‘Style Sheets’ or ‘CSS’) is a language used to describe the formatting (or “look and feel”) of a document written in a markup language. The job of CSS is to make the presentation of the components you will draw with D3 simpler by assigning specific styles to specific objects. One of the cool things about CSS is that it is an enormously flexible and efficient method for making everything on the screen look more consistent and when you want to change the format of something you can just change the CSS component and the whole look and feel of your graphics will change.

Web Servers

Ok, this can go one of two ways. If you have access to a web server and know where to put the files so that you can access them with your browser, you’re on fire. If you’re not quite sure, read on…

A web server will allow you to access your HTML files and will provide the structure that allows it to be displayed on a web browser. There are some simple instructions on the main D3 wiki page for setting up a local server. Or you might have access to a remote one and be able to upload your files. However, for a little more functionality and a whole lot of ease of use, I can thoroughly recommend WampServer as a free and simple way to set up a local web server that includes PHP and a MySQL database (more on those later). Go to the WampServer web page (http://www.wampserver.com/en/) and see if it suits you.

Throughout this document I will be describing the files and how they’re laid out in a way that has suited my efforts while using WAMP, but they will work equally well on a remote server. I will explain a little more about how I arrange the files later in the ‘Getting D3’ section.

There are other options of course. You could host code on GitHub and present the resulting graphics on bl.ocks.org. This is a great way to make sure that your code is available for peer review and sharing with the wider community.

One such alternative option that I have recently started playing with is Plunker (http://plnkr.co/) This is a lightweight collaborative online editing tool. It’s so cool I wrote a special section for it which you can find later in this document. This is definitely worth trying if you want to use something simple without a great deal of overhead. If you like what you see, perhaps consider an alternative that provides a greater degree of capability if you go on to greater d3.js things.

PHP

PHP is a scripting language for the web. That is to say that it is a programming language which is executed when you load web pages and it helps web pages do dynamic things.

You might think that this sounds familiar and that JavaScript does the same thing. But not quite.

JavaScript is designed so that it travels with the web page when it is downloaded by a browser (the client). However, PHP is executed remotely on the server that supplies the web page. This might sound a bit redundant, but it’s a big deal. This means that the PHP which is executed doesn’t form part of the web page, but it can form the web page. The implication here is that the web page you are viewing can be altered by the PHP code that runs on a remote server. This is the dynamic aspect of it.

In practice, PHP could be analogous to the glue that binds web pages together. Allowing different portions of the web page to respond to directions from the end user.

It is widely recognised not only as a relatively simple language to learn, but also as a fairly powerful one. At the same time it comes into criticism for being somewhat fragmented and sometimes contradictory or confusing. But in spite of any perceived shortcomings, it is a very widely used and implemented language and one for which there is no obvious better option.

Other Useful Stuff

Text Editor

A good text editor for writing up your code will be a real boost. Don’t make the fatal mistake of using an office word processor or similar. THEY WILL DOOM YOU TO A LIFE OF MISERY. They add in crazy stuff that you can’t even see and never save the files in a way that can be used properly.

Preferably, you should get an editor that will provide some assistance in the form of syntax highlighting which is where the editor knows what language you are writing in (JavaScript for example) and highlights the text in a way that helps you read it. For example, it will change text that might appear as this;

// Get the data
d3.tsv("data/data.tsv", function(error, data) {
data.forEach(function(d) {
    d.date = parseDate(d.date);
    d.close = +d.close;
});

Into something like this;

// Get the data
d3.tsv("data/data.tsv", function(error, data) {
data.forEach(function(d) {
    d.date = parseDate(d.date);
    d.close = +d.close;
});

Infinitely easier to use. Trust me.

There are plenty of editors that will do the trick. I have a preference for Geany, mainly because it’s what I started with and it grew on me :-).

Getting D3

Luckily this is pretty easy.

Go to the D3 repository on github and download the entire repository by clicking on the ‘ZIP’ button.

What you do with it from here depends on how you’re hosting your graphs. If you’re working on them on your local PC, then you will want to have the d3.js file in the path that can be seen by the browser. Again, I would recommend WAMP (a local web server) to access your files locally. If you’re using WAMP, then you just have to make sure that it knows to use a directory that will contain the d3 directory and you will be away.

The following image is intended to provide a very crude overview of how you can set up the directories.

• webserver: Use this as your ‘base’ directory where you put your files that you create. That way when you open your browser you point to this directory and it allows you to access the files like a normal web site.
• d3: This would be your unzipped d3 directory. It contains all the examples and more importantly the d3.v3.js file that you need to get things going. You will notice in the code examples that follow there is a line like the following;
  <script type="text/javascript" src="d3/d3.v3.js"></script>.
  This tells your browser that from the file it is running (one of the graph html files) if it goes into the ‘d3’ folder it will find the d3.v3.js file that it can load.
• data: I use this directory to hold any data files that I would use for processing. For example, you will see the following line in the code examples that follow d3.tsv("data/data.tsv", function(error, data) {. Again, that’s telling the browser to go into the ‘data’ directory and to load the ‘data.tsv’ file.
• js: Often you will find that you will want to include other JavaScript libraries to load. This is a good place to put them.

Where to get information on d3.js

D3 has made huge advances in providing an extensible and practical framework for manipulating data as web objects. At the same time there has been significant increase in information available for people to use it. The following is a far from exhaustive list of sources, but from my own experience it represents a useful subset of knowledge.

d3js.org

d3js.org would be the first port of call for people wanting to know something about d3.js.

From the overview on the main page you can access a dizzying array of examples that have been provided by the founder of d3 (Mike Bostock) and a host of additional developers, artists, coders and anyone who has something to add to the sum knowledge of cool things that can be done with d3.

There is a link to a documentation page that serves as a portal to the ever important API reference, contributed tutorials and other valuable links (some of which I will mention in paragraphs ahead).

The last major link is to the Github repository where you can download d3.js itself.

It is difficult to overstate the volume of available information that can be accessed from d3js.org. It stands alone as the one location that anyone interested in D3 should visit.

Google Groups

There is a Google Group dedicated to discussions on d3.js.

In theory this forum is for discussions on topics including visualization design, API design, requesting new features, etc. With a specific direction made in the main header that “If you want help using D3, please use the d3.js tag on Stack Overflow!”.

In practice however, it would appear that a sizeable proportion of the posts there are technical assistance requests of one type or another. Having said that this means that if you’re having a problem, there could already be a solution posted there. However, if at all possible the intention is certainly that people use Stack Overflow, so this should be the first port of call for those types of inquiry.

So, by all means add this group as a favourite and this will provide you with the opportunity to receive emailed summaries of postings or just an opportunity to easily browse recent goings-on.

Stack Overflow

Stack Overflow is a question and answer site whose stated desire is “to build a library of detailed answers to every question about programming”. Ambitious. So how are they doing? Actually really well. Stack overflow is a fantastic place to get help and information. It’s also a great place to help people out if you have some knowledge on a topic.

They have a funny scheme for rewarding users that encourages providing good answers based on readers voting. It’s a great example of gamification working well. If you want to know a little more about how it works, check out this page; http://stackoverflow.com/about.

They have a d3.js tag (http://stackoverflow.com/questions/tagged/d3.js) and like Google Groups there is a running list of different topics that are an excellent source of information.

Github

Github is predominantly a code repository and version control site. It is highly regarded for its technical acumen and provides a fantastic service that is broadly used for many purposes. Not the least of which is hosting the code (and the wiki) for d3.js.

Whilst not strictly a site that specialises in providing a Q & A function, there is a significant number of repositories (825 at last count) which mention d3.js. With the help from an astute search phrase, there is potentially a solution to be found there.

The other associated feature of Github is Gist. Gist is a pastebin service (a place where you can copy and past code) that can provide a ‘wiki like’ feature for individual repositories and web pages that can be edited through a Git repository. Gist plays a role in providing the hub for the bl.ocks.org example hosting service set up by Mike Bostock.

For a new user, Github / Gist can be slightly daunting. It’s an area where you almost need to know what’s going on to know before you dive in. This is certainly true if you want to make use of its incredible features that are available for hosting code. However, if you want to browse other peoples code it’s an easier introduction. Have a look through what’s available and if you feel so inclined, I recommend that you learn enough to use their service. It’s time well spent.

bl.ocks.org

bl.ocks.org is a viewer for code examples which are hosted on Gist. You are able to load your code into Gist, and then from bl.ocks.org you can view them.

This is a really great way for people to provide examples of their work and there are many who do. However, it’s slightly tricky to know what is there. There is a current project being championed by Christophe Viau and others to provide better access to a range of D3 documentation. The early indications are that it will provide a fantastic method of accessing examples and information. Watch that space.

I would describe the process of getting your own code hosted and displaying as something that will be slightly challenging for people who are not familiar with Github / Gist, but again, in terms of visibility of the code and providing an external hosting solution, it is excellent and well worth the time to get to grips with.

Twitter

Twitter provides a great alerting service to inform a large disparate group of people about stuff.

It’s certainly a great way to keep in touch on an hour by hour basis with people who are involved with d3.js and this can be accomplished in a couple of ways. First, find as many people from the various D3 sites around the web who you consider to be influential in areas you want to follow (different aspects such as development, practical output, educational etc) and follow them. Even better, I found it useful to find a small subset who I considered to be influential people and I noted who they followed. It’s a bit ‘stalky’ if you’re unfamiliar with it, but the end result should be a useful collection of people with something useful to say.

Books

There are only a couple of books that have been released so far on d3.js.

There is “Getting Started with D3” by Mike Dewar (O’Reilly Media, June 2012). This will take you through a good set of exercises to develop your D3 skills and is accompanied by downloadable examples.

There is “Interactive Data Visualization for the Web” by Scott Murray, (O’Reilly Media, November 2012). Currently this has only been released as an ebook, but is scheduled to be released in print form in 2013. The book is based on his great set of on-line tutorials (http://alignedleft.com/tutorials/).

Of course, there is the original paper that launched D3 “D3: Data-Driven Documents” by Michael Bostock, Vadim Ogievetsky and Jeffrey Heer (IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis), 2011)

Starting with a basic graph

I’ll start by providing the full code for a simple graph and then we can go through it piece by piece (The full code for this example is also in the appendices as ‘Simple Graph’.).

Here’s what the basic graph looks like;

And here’s the code that makes it happen;

<!DOCTYPE html>
<meta charset="utf-8">
<style> /* set the CSS */

body { font: 12px Arial;}

path { 
    stroke: steelblue;
    stroke-width: 2;
    fill: none;
}

.axis path,
.axis line {
    fill: none;
    stroke: grey;
    stroke-width: 1;
    shape-rendering: crispEdges;
}

</style>
<body>

<!-- load the d3.js library -->    
<script src="http://d3js.org/d3.v3.min.js"></script>

<script>

// Set the dimensions of the canvas / graph
var margin = {top: 30, right: 20, bottom: 30, left: 50},
    width = 600 - margin.left - margin.right,
    height = 270 - margin.top - margin.bottom;

// Parse the date / time
var parseDate = d3.time.format("%d-%b-%y").parse;

// Set the ranges
var x = d3.time.scale().range([0, width]);
var y = d3.scale.linear().range([height, 0]);

// Define the axes
var xAxis = d3.svg.axis().scale(x)
    .orient("bottom").ticks(5);

var yAxis = d3.svg.axis().scale(y)
    .orient("left").ticks(5);

// Define the line
var valueline = d3.svg.line()
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y(d.close); });
    
// Adds the svg canvas
var svg = d3.select("body")
    .append("svg")
        .attr("width", width + margin.left + margin.right)
        .attr("height", height + margin.top + margin.bottom)
    .append("g")
        .attr("transform", 
              "translate(" + margin.left + "," + margin.top + ")");

// Get the data
d3.csv("data/data.csv", function(error, data) {
    data.forEach(function(d) {
        d.date = parseDate(d.date);
        d.close = +d.close;
    });

    // Scale the range of the data
    x.domain(d3.extent(data, function(d) { return d.date; }));
    y.domain([0, d3.max(data, function(d) { return d.close; })]);

    // Add the valueline path.
    svg.append("path")
        .attr("class", "line")
        .attr("d", valueline(data));

    // Add the X Axis
    svg.append("g")
        .attr("class", "x axis")
        .attr("transform", "translate(0," + height + ")")
        .call(xAxis);

    // Add the Y Axis
    svg.append("g")
        .attr("class", "y axis")
        .call(yAxis);

});

</script>
</body>

The full code for this example can be found on github, in the appendices of this book or in the code samples bundled with this book (simple-graph.html and data.csv). A live example can be found on bl.ocks.org. Please note that the <head></head> tags are omitted which is a common thing for d3 examples (I don’t know why). This can cause problems for some browsers in certain conditions.

Once we’ve finished explaining these parts, we’ll start looking at what we need to add in and adjust so that we can incorporate other useful functions that are completely reusable in other diagrams as well.

The end point being something hideous like the following;

I say hideous since the graph is not intended to win any beauty prizes, but there are several components to it which some people may find useful (gridlines, area fill, axis label, drop shadow for text, title, text formatting).

So, we can break the file down into component parts. I’m going to play kind of fast and loose here, but never fear, it’ll all make sense.

HTML

Here’s the HTML portions of the code;

<!DOCTYPE html>
<meta charset="utf-8">
<style>

    The CSS is in here

</style>
<body>
<script src="http://d3js.org/d3.v3.min.js"></script>

<script>

    The D3 JavaScript code is here

</script>
</body>

Compare it with the full code. It kind of looks like a wrapping for the CSS and JavaScript. You can see that it really doesn’t boil down to much at all (that doesn’t mean it’s not important).

There are plenty of good options for adding additional HTML stuff into this very basic part for the file, but for what we’re going to be doing, we really don’t need to bother too much.

One thing probably worth mentioning is the line;

<script src="http://d3js.org/d3.v3.min.js"></script>

That’s the line that identifies the file that needs to be loaded to get D3 up and running. In this case the file is sourced from the official d3.js repository on the internet (that way we are using the most up to date version). The D3 file is actually called d3.v3.min.js which may come as a bit of a surprise. That tells us that this is version 3 of the d3.js file (the v3 part) which is an indication that it is separate from the v2 release, which was superseded in late 2012. The other point to note is that this version of d3.js is the minimised version (hence min). This means that any extraneous information has been removed from the file to make it quicker to load.

Later when doing things like implementing integration with bootstrap (a pretty layout framework) we will be doing a great deal more, but for now, that’s the basics done.

The two parts that we left out are the CSS and the D3 JavaScript.

CSS

The CSS is as follows;

body { font: 12px Arial;}

path { 
    stroke: steelblue;
    stroke-width: 2;
    fill: none;
}

.axis path,
.axis line {
    fill: none;
    stroke: grey;
    stroke-width: 1;
    shape-rendering: crispEdges;
}

Cascading Style Sheets give you control over the look / feel / presentation of web content. The idea is to define a set of properties to objects in the web page.

They are made up of ‘rules’. Each rule has a ‘selector’ and a ‘declaration’ and each declaration has a property and a value (or a group of properties and values).

For instance in the example code for this web page we have the following rule;

body { font: 12px Arial;} 

body is the selector. This tells you that on the web page, this rule will apply to the ‘body’ of the page. This actually applies to all the portions of the web page that are contained in the ‘body’ portion of the HTML code (everything between <body> and </body> in the HTML bit). { font: 12px Arial;} is the declaration portion of the rule. It only has the one declaration which is the bit that is in between the curly braces. So font: 12px Arial; is the declaration. The property is font: and the value is 12px Arial;. This tells the web page that the font that appears in the body of the web page will be in 12 px Arial.

Sure enough if we look at the axes of the graph…

We see that the font might actually be 12px Arial!

Let’s try a test. I will change the Rule to the following;

body { font: 16px Arial;}

and the result is…

Ahh…. 16px of goodness!

And now we change it to…

body { font: 16px times;}

and we get…

Hmm… Times font…. I think we can safely say that this has had the desired effect.

So what else is there?

What about the bit that’s like;

path { 
    stroke: steelblue;
    stroke-width: 2;
    fill: none;
}

Well, the whole thing is one rule, ‘path’ is the selector. In this case, ‘path’ is referring to a line in the D3 drawing nomenclature.

For that selector there are three declarations. They give values for the properties of ‘stroke’ (in this case colour), ‘stroke-width’ (the width of the line) and ‘fill’ (we can fill a path with a block of colour).

So let’s change things :-)

path { 
    stroke: red;
    stroke-width: 5;
    fill: yes;
}

Wow! The line is now red, it looks about 5 pixels wide and it’s tried to fill the area (roughly defined by the curve) with a black colour.

It ain’t pretty, but it certainly did change. In fact if we go;

    fill: blue;

We’ll get…

So the ‘fill’ property looks pretty flexible. And so does CSS.

D3 JavaScript

The D3 JavaScript part of the code is as follows;

var margin = {top: 30, right: 20, bottom: 30, left: 50},
    width = 600 - margin.left - margin.right,
    height = 270 - margin.top - margin.bottom;

var parseDate = d3.time.format("%d-%b-%y").parse;

var x = d3.time.scale().range([0, width]);
var y = d3.scale.linear().range([height, 0]);

var xAxis = d3.svg.axis().scale(x)
    .orient("bottom").ticks(5);

var yAxis = d3.svg.axis().scale(y)
    .orient("left").ticks(5);

var valueline = d3.svg.line()
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y(d.close); });
    
var svg = d3.select("body")
    .append("svg")
        .attr("width", width + margin.left + margin.right)
        .attr("height", height + margin.top + margin.bottom)
    .append("g")
        .attr("transform", 
              "translate(" + margin.left + "," + margin.top + ")");

// Get the data
d3.csv("data.csv", function(error, data) {
    data.forEach(function(d) {
        d.date = parseDate(d.date);
        d.close = +d.close;
    });

    // Scale the range of the data
    x.domain(d3.extent(data, function(d) { return d.date; }));
    y.domain([0, d3.max(data, function(d) { return d.close; })]);

    svg.append("path")		// Add the valueline path.
        .attr("class", "line")
        .attr("d", valueline(data));

    svg.append("g")			// Add the X Axis
        .attr("class", "x axis")
        .attr("transform", "translate(0," + height + ")")
        .call(xAxis);

    svg.append("g")			// Add the Y Axis
        .attr("class", "y axis")
        .call(yAxis);

});

Again there’s quite a bit of detail in the code, but it’s not so long that you can’t work out what’s doing what.

Let’s examine the blocks bit by bit to get a feel for it.

Setting up the margins and the graph area.

The part of the code responsible for defining the canvas (or the area where the graph and associated bits and pieces is placed ) is this part.

var margin = {top: 30, right: 20, bottom: 30, left: 50},
    width = 600 - margin.left - margin.right,
    height = 270 - margin.top - margin.bottom;

This is really (really) well explained on Mike Bostock’s page on margin conventions here http://bl.ocks.org/3019563, but at the risk of confusing you here’s my crude take on it.

The first line defines the four margins which surround the block where the graph (as an object) is positioned.

var margin = {top: 30, right: 20, bottom: 30, left: 50},

So there will be a border of 30 pixels at the top, 20 at the right and 30 and 50 at the bottom and left respectively. Now the cool thing about how these are set up is that they use an object to define everything. That means if you want to do calculations in the JavaScript later, you don’t need to put the numbers in, you just use the variable that has been set up. In this case margin.right = 20!

So when we go to the next line;

    width = 600 - margin.left - margin.right,

the width of the inner block of the canvas where the graph will be drawn is 600 pixels – margin.left – margin.right or 600-50-20 or 530 pixels wide. Of course now you have another variable ‘width’ that we can use later in the code.

Obviously the same treatment is given to height.

Another cool thing about all of this is that just because you appear to have defined separate areas for the graph and the margins, the whole area in there is available for use. It just makes it really useful to have areas designated for the axis labels and graph labels without having to juggle them and the graph proper at the same time.

So, let’s have a play and change some values.

var margin = {top: 80, right: 20, bottom: 80, left: 50},
    width = 400 - margin.left - margin.right,
    height = 270 - margin.top – margin.bottom;

Here we’ve made the graph narrower (400 pixels) but retained the left / right margins and increased the top bottom margins while maintaining the overall height of the canvas. The really cool thing that you can tell from this is that while we shrank the dimensions of the area that we had to draw the graph in, it was still able to dynamically adapt the axes and line to fit properly. That is the really cool part of this whole business. D3 is running in the background looking after the drawing of the objects, while you get to concentrate on how the data looks without too much maths!

Getting the Data

We’re going to jump forward a little bit here to the bit of the JavaScript code that loads the data for the graph.

I’m going to go out of the sequence of the code here, because if you know what the data is that you’re using, it will make explaining some of the other functions that are coming up much easier.

The section that grabs the data is this bit.

d3.csv("data.csv", function(error, data) {
    data.forEach(function(d) {
        d.date = parseDate(d.date);
        d.close = +d.close;
    });

In fact it’s a combination of a few bits and another piece that isn’t shown!, But let’s take it one step at a time :-)

There’s lots of different ways that we can get data into our web page to turn into graphics. And the method that you’ll want to use will probably depend more on the format that the data is in than the mechanism you want to use for importing.

For instance, if it’s only a few points of data we could include the information directly in the JavaScript.

That would make it look something like;

var data = [
    {date:"1-May-12",close:"58.13"},
    {date:"30-Apr-12",close:"53.98"},
    {date:"27-Apr-12",close:"67.00"},
    {date:"26-Apr-12",close:"89.70"},
    {date:"25-Apr-12",close:"99.00"}
];

The format of the data shown above is called JSON (JavaScript Object Notation) and it’s a great way to include data since it’s easy for humans to read what’s in there and it’s easy for computers to parse the data out. For a brief overview of JSON there is a separate section in the “Assorted Tips and Tricks Chapter” that may assist.

But if you’ve got a fair bit of data or if the data you want to include is dynamic and could be changing from one moment to the next, you’ll want to load it from an external source. That’s when we call on D3’s ‘Request’ functions.

The different types of data that can be requested by D3 are;

• text: A plain old piece of text that has options to be encoded in a particular way (see the D3 API).
• json: This is the afore mentioned JavaScript Object Notation.
• xml: Extensible Markup Language is a language that is widely used for encoding documents in a human readable forrm.
• html: HyperText Markup Language is the language used for displaying web pages.
• csv: Comma Separated Values is a widely used format for storing data where plain text information is separated by (wait for it) commas.
• tsv: Tab Separated Values is a widely used format for storing data where plain text information is separated by a tab-stop character.

Details on these ingestion methods and the formats for the requests are well explained on the D3 Wiki page. In this particular script we will look at the csv request method.

Back to our request…

d3.csv("data.csv", function(error, data) {
    data.forEach(function(d) {
        d.date = parseDate(d.date);
        d.close = +d.close;
    });

The first line of that piece of code invokes the d3.csv request (d3.csv) and then the function is pointed to the data file that should be loaded (data.csv). This is referred to as the ‘url’ (unique resource locator) of the file. In this case the file is stored locally (in the same directory as the simple-graph.html file), but the url could just as easily point to a file somewhere on the Internet.

The format of the data in the data.csv file looks a bit like this;

date,close
1-May-12,58.13
30-Apr-12,53.98
27-Apr-12,67.00
26-Apr-12,89.70
25-Apr-12,99.00

(although the file is longer (about 26 data points)). The ‘date’ and the ‘close’ heading labels are separated by a comma as are each subsequent date and number. Hence the ‘comma separated values’ :-).

The next part is part of the coolness of JavaScript. With the request made and the file requested, the script is told to carry out a function on the data (which will now be called ‘data’).

function(error, data) {

There are actually more things that get acted on as part of the function call, but the one we will consider here is contained in the following lines;

    data.forEach(function(d) {
        d.date = parseDate(d.date);
        d.close = +d.close;
    });

This block of code simply ensures that all the numeric values that are pulled out of the csv file are set and formatted correctly. The first line sets the data variable that is being dealt with (called slightly confusingly ‘data’) and tells the block of code that, for each group within the ‘data’ array it should carry out a function on it. That function is designated ‘d’.

    data.forEach(function(d) { 

The information in the array can be considered as being stored in rows. Each row consists of two values: one value for ‘date’ and another value for ‘close’.

The function is pulling out values of ‘date’ and ‘close’ one row at a time.

Each time it gets a value of ‘date’ and ‘close’ it carries out the following operations;

        d.date = parseDate(d.date);

For this specific value of date being looked at (d.date), d3.js changes it into a date format that is processed via a separate function ‘parseDate’. (The ‘parseDate’ function is defined in a separate part of the script, and we will examine that later.) For the moment, be satisfied that it takes the raw date information from the csv file in a specific row and converts it into a format that D3 can then process. That value is then re-saved in the same variable space.

The next line then sets the ‘close’ variable to a numeric value (if it isn’t already) using the ‘+’ operator.

        d.close = +d.close;

So, at the end of that section of code, we have gone out and picked up a file with data in it of a particular type (comma separated values) and ensured that it is formatted in a way that the rest of the script can use correctly.

Now, the astute amongst you will have noticed that in the first line of that block of code (d3.csv("data.csv", function(error, data) {) we opened a normal bracket ( ( ) and a curly bracket ( { ), but we never closed them. That’s because they stay open until the very end of the file. That means that all those blocks that occur after the d3.csv bit are referenced to the ‘data’ array. Or put another way, it uses ‘data’ to draw stuff!

But anyway, let’s get back to figuring what the code is doing by jumping back to the end of the margins block.

Formatting the Date / Time.

One of the glorious things about the World is that we all do things a bit differently. One of those things is how we refer to dates and time.

In my neck of the woods, it’s customary to write the date as day - month – year. E.g 23-12-2012. But in the United States the more common format would be 12-23-2012. Likewise, the data may be in formats that name the months or weekdays (E.g. January, Tuesday) or combine dates and time together (E.g. 2012-12-23 15:45:32). So, if we were to attempt to try to load in some data and to try and get D3 to recognise it as date / time information, we really need to tell it what format the date / time is in.

Time for a little demonstration (see what I did there).

We will change our data.csv file so that it only includes two points. The first one and the last one with a separation of a month and a bit.

date,close
1-May-12,58.13
26-Mar-12,606.98

The graph now looks like this;

Nothing too surprising here, a very simple graph (note the time scale on the x axis).

Now we will change the later date in the data.csv file so that it is a lot closer to the starting date;

date,close
29-Mar-12,58.13
26-Mar-12,606.98

So, just a three day difference. Let’s see what happens.

Ahh…. Not only did we not have to make any changes to our JavaScript code, but it was able to recognise the dates were closer and fill in the intervening gaps with appropriate time / day values. Now, one more time for giggles.

This time we’ll stretch the interval out by a few years.

date,close
29-Mar-21,58.13
26-Mar-12,606.98

and the result is…

Hopefully that’s enough encouragement to impress upon you that formatting the time is a REALLY good thing to get right. Trust me, it will never fail to impress :-).

Back to formatting.

The line in the JavaScript that parses the time is the following;

var parseDate = d3.time.format("%d-%b-%y").parse;

This line is used when the data.forEach(function(d) portion of the code (that we looked at a couple of pages back) used d.date = parseDate(d.date) as a way to take a date in a specific format and to get it recognised by D3. In effect it said “take this value that is supposedly a date and make it into a value I can work with”.

The function used is the d3.time.format(specifier) function where the specifier in this case is the mysterious combination of characters %d-%b-%y. The good news is that these are just a combination of directives specific for the type of date we are presenting.

The % signs are used as prefixes to each separate format type and the ‘-’ (minus) signs are literals for the actual ‘-’ (minus) signs that appear in the date to be parsed.

The d refers to a zero-padded day of the month as a decimal number [01,31].

The b refers to an abbreviated month name.

And the y refers to the year (without the centuries) as a decimal number.

If we look at a subset of the data from the data.csv file we see that indeed, the dates therein are formatted in this way.

1-May-12,58.13
30-Apr-12,53.98
27-Apr-12,67.00
26-Apr-12,89.70
25-Apr-12,99.00

That’s all well and good, but what if your data isn’t formatted exactly like that?

Good news. There are multiple different formatters for different ways of telling time and you get to pick and choose which one you want. Check out the Time Formatting page on the D3 Wiki for a the authoritative list and some great detail, but the following is the list of currently available formatters (from the d3 wiki);

• %a - abbreviated weekday name.
• %A - full weekday name.
• %b - abbreviated month name.
• %B - full month name.
• %c - date and time, as “%a %b %e %H:%M:%S %Y”.
• %d - zero-padded day of the month as a decimal number [01,31].
• %e - space-padded day of the month as a decimal number [ 1,31].
• %H - hour (24-hour clock) as a decimal number [00,23].
• %I - hour (12-hour clock) as a decimal number [01,12].
• %j - day of the year as a decimal number [001,366].
• %m - month as a decimal number [01,12].
• %M - minute as a decimal number [00,59].
• %p - either AM or PM.
• %S - second as a decimal number [00,61].
• %U - week number of the year (Sunday as the first day of the week) as a decimal number [00,53].
• %w - weekday as a decimal number [0(Sunday),6].
• %W - week number of the year (Monday as the first day of the week) as a decimal number [00,53].
• %x - date, as “%m/%d/%y”.
• %X - time, as “%H:%M:%S”.
• %y - year without century as a decimal number [00,99].
• %Y - year with century as a decimal number.
• %Z - time zone offset, such as “-0700”.
• There is also a a literal “%” character that can be presented by using double % signs.

As an example, if you wanted to input date / time formatted as a generic MySQL ‘YYYY-MM-DD HH:MM:SS’ TIMESTAMP format the D3 parse script would look like;

parseDate = d3.time.format("%Y-%m-%d %H:%M:%S").parse;

Setting Scales Domains and Ranges

This is another example where, if you set it up right, D3 will look after you forever.

From our basic web page we have now moved to the section that includes the following lines;

var x = d3.time.scale().range([0, width]);
var y = d3.scale.linear().range([height, 0]);

The purpose of these portions of the script is to ensure that the data we ingest fits onto our graph correctly. Since we have two different types of data (date/time and numeric values) they need to be treated separately (but they do essentially the same job). To examine this whole concept of scales, domains and ranges properly, we will also move slightly out of sequence and (in conjunction with the earlier scale statements) take a look at the lines of script that occur later and set the domain. They are as follows;

    x.domain(d3.extent(data, function(d) { return d.date; }));
    y.domain([0, d3.max(data, function(d) { return d.close; })]);

The idea of scaling is to take the values of data that we have and to fit them into the space we have available.

If we have data that goes from 53.98 to 636.23 (as the data we have for ‘close’ in our csv file does), but we have a graph that is 210 pixels high (height = 270 - margin.top – margin.bottom;) we clearly need to make an adjustment.

Not only that. Even though our data goes from 53.98 to 636.23, that would look slightly misleading on the graph and it should really go from 0 to a bit over 636.23. It sound’s really complicated, but let’s simple it up a bit.

First we make sure that any quantity we specify on the x axis fits onto our graph.

var x = d3.time.scale().range([0, width]);

Here we set our variable that will tell D3 where to draw something on the x axis. By using the d3.time.scale() function we make sure that D3 knows to treat the values as date / time entities (with all their ingrained peculiarities). Then we specify the range that those values will cover (.range) and we specify the range as being from 0 to the width of our graphing area (See! Setting those variables for margins and widths are starting to pay off now!).

Then we do the same for the Y axis.

var y = d3.scale.linear().range([height, 0]);

There’s a different function call (d3.scale.linear()) but the .range setting is still there. In the interests of drawing a (semi) pretty picture to try and explain, hopefully this will assist;

I know, I know, it’s a little misleading because nowhere have we actually said to D3 this is our data from 53.98 to 636.23. All we’ve said is when we get the data, we’ll be scaling it into this space.

Now hang on, what’s going on with the [height, 0] part in y axis scale statement? The astute amongst you will note that for the time scale we set the range as [0, width] but for this one ([height, 0]) the values look backwards.

Well spotted.

This is all to do with how the screen is laid out and referenced. Take a look at the following diagram showing how the coordinates for drawing on your screen work;

The top left hand of the screen is the origin or 0,0 point and as we go left or down the corresponding x and y values increase to the full values defined by height and width.

That’s good enough for the time values on the x axis that will start at lower values and increase, but for the values on the y axis we’re trying to go against the flow. We want the low values to be at the bottom and the high values to be at the top.

No problem. We just tell D3 via the statement y = d3.scale.linear().range([height, 0]); that the larger values (height) are at the low end of the screen (at the top) and the low values are at the bottom (as you most probably will have guessed by this stage, the .range statement uses the format .range([closer_to_the_origin, further_from_the_origin]). So when we put the height variable first, that is now associated at the top of the screen.

We’ve scaled our data to the graph size and ensured that the range of values is set appropriately. What’s with the domain part that was in this section’s title?

Come on, you remember this little piece of script don’t you?

    x.domain(d3.extent(data, function(d) { return d.date; }));
    y.domain([0, d3.max(data, function(d) { return d.close; })]);

While it exists in a separate part of the file from the scale / range part, it is certainly linked.

That’s because there’s something missing from what we have been describing so far with the set up of the data ranges for the graphs. We haven’t actually told D3 what the range of the data is. That’s also the reason this part of the script occurs where it does. It is within the portion where the data.csv file has been loaded as ‘data’ and it’s therefore ready to use it.

So, the .domain function is designed to let D3 know what the scope of the data will be. This is what is then passed to the scale function.

Looking at the first part that is setting up the x axis values, it is saying that the domain for the x axis values will be determined by the d3.extent function which in turn is acting on a separate function which looks through all the ‘date’ values that occur in the ‘data’ array. In this case the .extent function returns the minimum and maximum value in the given array.

• function(d) { return d.date; } returns all the ‘date’ values in ‘data’. This is then passed to…
• The .extent function that finds the maximum and minimum values in the array and then…
• The .domain function which returns those maximum and minimum values to D3 as the range for the x axis.

Pretty neat really. At first you might think it was overly complex, but breaking the function down into these components allows additional functionality with differing scales, values and quantities. In short, don’t sweat it. It’s a good thing.

The x axis values are dates; so the domain for them is basically from the 26th of March 2012 till 1st of May 2012. The y axis is done slightly differently

    y.domain([0, d3.max(data, function(d) { return d.close; })]);

Because the range of values desired on the y axis goes from 0 to the maximum in the data range, that’s exactly what we tell D3. The ‘0’ in the .domain function is the starting point and the finishing point is found by employing a separate function that sorts through all the ‘close’ values in the ‘data’ array and returns the largest one. Therefore the domain is from 0 to 636.23.

Let’s try a small experiment. Let’s change the y axis domain to use the .extent function (the same way the x axis does) to see what it produces.

The JavaScript for the y domain will be;

    y.domain(d3.extent(data, function(d) { return d.close; }));

You can see apart from a quick copy paste of the internals, all I had to change was the reference to ‘close’ rather than ‘date’.

And the result is…

Look at that! The starting point for the y axis looks like it’s pretty much on the 53.98 mark and the graph itself certainly touches the x axis where the data would indicate it should.

Now, I’m not really advocating making a graph like this since I think it looks a bit nasty (and a casual observer might be fooled into thinking that the x axis was at 0). However, this would be a useful thing to do if the data was concentrated in a narrow range of values that are quite distant from zero.

For instance, if I change the data.csv file so that the values are represented like the following;

Then it kind of loses the ability to distinguish between values around the median of the data.

But, if I put in our magic .extent function for the y axis and redraw the graph…

How about that?

The same data as the previous graph, but with one simple piece of the script changed and D3 takes care of the details.

Setting up the Axes

Now we come to our next piece of code;

var xAxis = d3.svg.axis().scale(x)
    .orient("bottom").ticks(5);

var yAxis = d3.svg.axis().scale(y)
    .orient("left").ticks(5);

I’ve included both the x and y axes because they carry out the formatting in very similar ways. It’s worth noting that this is not the point where the axes get drawn. That occurs later in the piece where the data.csv file has been loaded as ‘data’.

D3 has it’s own axis component that aims to take the fuss out of setting up and displaying the axes. So it includes a number of configurable options.

Looking first at the x axis;

var xAxis = d3.svg.axis().scale(x)
    .orient("bottom").ticks(5);

The axis function is called with d3.svg.axis(). Then the scale is set using the x values that we set up in the scales, ranges and domains section using .scale(x). Then a curious thing happens, we tell the graph to orientate itself to the bottom of the graph .orient("bottom"). If I tell you that “bottom” is the default setting, then you could be forgiven for thinking that technically, we don’t need to specify this since it will go there anyway, but it does give us an opportunity to change it to "top" to see what happens;

Well, I hope you didn’t see that coming, because I didn’t. It transpires that what we’re talking about is the orientation of the values and ticks about the axis line itself. Ahh… Ok. Useful if your x axis is at the top of your graph, but for this one? Not so useful.

The next part (.ticks(5)) sets the number of ticks on the axis. Hopefully you just did a quick count across the bottom of the previous graph and went “Yep, five ticks. Spot on”. Well done if you did, but there’s a little bit of a sneaky trick up D3’s sleeve with the number of ticks on a graph axis.

For instance, here’s what the graph looks like when the .ticks(5) value is changed to .ticks(4).

Eh? Hang on. Isn’t that some kind of mistake? There are still five ticks. Yep, sure is! But wait… we can keep dropping the ticks value till we get to two and it will still be the same. At .ticks(2) though, we finally see a change.

How about that? At first glance that just doesn’t seem right, then you have a bit of a think about it and you go “Hmm… When there were 5 ticks, they were separated by a week each, and that stayed that way till we got to a point where it could show a separation of a month.”.

D3 is making a command decision for you as to how your ticks should be best displayed. This is great for simple graphs and indeed for the vast majority of graphs. Like all things related to D3, if you really need to do something bespoke, it will let you if you understand enough code.

The following is the list of time intervals that D3 will consider when setting automatic ticks on a time based axis;

• 1-, 5-, 15and 30-second.
• 1-, 5-, 15and 30-minute.
• 1-, 3-, 6and 12-hour.
• 1 and 2-day.
• 1-week.
• 1 and 3-month.
• 1-year.

Just for giggles have a think about what value of ticks you will need to increase to until you get D3 to show more than five ticks.

Hopefully you won’t sneak a glance at the following graph before you come up with the right answer.

Yikes! The answer is 10! And then when it does, the number of ticks is so great that they jumble all over each other. Not looking to good there. However, you could rotate the text (or perhaps slant it) and it could still fit in (that must be the topic of a future how-to). You could also make the graph longer if you wanted, but of course that is probably going to create other layout problems. Try to think about your data and presentation as a single entity.

The code that formats the y axis is pretty similar;

var yAxis = d3.svg.axis().scale(y)
    .orient("left").ticks(5);

We can change the orientation to "right" if we want, but it won’t be winning any beauty prizes.

Nope. Not a pretty sight.

What about the number of ticks? Well this scale is quite different to the x axis. Formatting the dates using logical separators (weeks, months) was tricky, but with standard numbers, it should be a little easier. In fact, there’s a fair chance that you’ve already had a look at the y axis and seen that there are 6 ticks there when the script is asking for 5 :-)

We can lower the tick number to 4 and we get a logical result.

We need to raise the count to 10 before we get more than 6.

Adding data to the line function

We’re getting towards the end of our journey through the script now. The next step is to get the information from the array ‘data’ and to place it in a new array that consists of a set of coordinates that we are going to plot.

var valueline = d3.svg.line()
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y(d.close); });

I’m aware that the statement above may be somewhat ambiguous. You would be justified in thinking that we already had the data stored and ready to go. But that’s not strictly correct.

What we have is data in a raw format, we have added pieces of code that will allow the data to be adjusted for scale and range to fit in the area that we want to draw, but we haven’t actually taken our raw data and adjusted it for our desired coordinates. That’s what the code above does.

The main function that gets used here is the d3.svg.line() function. This function uses accessor functions to store the appropriate information in the right area and in the case above they use the x and y accessors (that would be the bits that are .x and .y). The d3.svg.line() function is called a ‘path generator’ and this is an indication that it can carry out some pretty clever things on its own accord. But in essence its job is to assign a set of coordinates in a form that can be used to draw a line.

Each time this line function is called on, it will go through the data and will assign coordinates to ‘date’ and ‘close’ pairs using the ‘x’ and ‘y’ functions that we set up earlier (which of course are responsible for scaling and setting the correct range / domain).

Of course, it doesn’t get the data all by itself, we still need to actually call the valueline function with ‘data’ as the source to act on. But never fear, that’s coming up soon.

Adding the SVG Canvas.

As the title states, the next piece of script forms and adds the canvas that D3 will then use to draw on.

var svg = d3.select("body")
    .append("svg")
        .attr("width", width + margin.left + margin.right)
        .attr("height", height + margin.top + margin.bottom)
    .append("g")
        .attr("transform", 
              "translate(" + margin.left + "," + margin.top + ")");

So what exactly does that all mean?

Well D3 needs to be able to have a space defined for it to draw things. When you define the space it’s going to use, you can also give the space you’re going to use an identifying name and attributes.

In the example we’re using here, we are ‘appending’ an SVG element (a canvas that we are going to draw things on) to the <body> element of the HTML page.

We also add an element ‘g’ that is referenced to the top left corner of the actual graph area on the canvas. ‘g’ is actually a grouping element in the sense that it is normally used for grouping together several related elements. So in this case those grouped elements will have a common reference.

(the image above is definitely not to scale, but I hope you get the general idea)

Interesting things to note about the code. The .attr(“stuff in here”) parts are attributes of the appended elements they are part of.

For instance;

    .append("svg")
        .attr("width", width + margin.left + margin.right)
        .attr("height", height + margin.top + margin.bottom)

tells us that the ‘svg’ element has a “width” of width + margin.left + margin.right and the “height” of height + margin.top + margin.bottom.

Likewise…

    .append("g")
        .attr("transform", 
              "translate(" + margin.left + "," + margin.top + ")");

tells us that the element “g” has been transformed by moving(translating) to the point margin.left, margin.top. Or to the top left of the graph space proper. This way when we tell something to be drawn on our canvas, we can use the reference point “g” to make sure everything is in the right place.

Actually Drawing Something!

Up until now we have spent a lot of time defining, loading and setting up. Good news! We’re about to finally draw something!

We jump lightly over some of the code that we have already explained and land on the part that draws the line.

    svg.append("path")		     // Add the valueline path.
        .attr("d", valueline(data));

This area occurs in the part of the code that has the data loaded and ready for action.

The svg.append("path") portion adds a new path element . A path element represents a shape that can be manipulated in lots of different ways (see more here: http://www.w3.org/TR/SVG/paths.html). In this case it inherits the ‘path’ styles from the CSS section and on the following line (.attr("d", valueline(data));) we add the attribute “d”.

This is an attributer that stands for ‘path data’ and sure enough the valueline(data) portion of the script passes the ‘valueline’ array (with its x and y coordinates) to the path element. This then creates a svg element which is a path going from one set of ‘valueline’ coordinates to another.

Then we get to draw in the axes;

    svg.append("g")                // Add the X Axis
        .attr("class", "x axis")
        .attr("transform", "translate(0," + height + ")")
        .call(xAxis);

    svg.append("g")               // Add the Y Axis
        .attr("class", "y axis")
        .call(yAxis);

We have covered the formatting of the axis components earlier. So this part is actually just about getting those components drawn onto our canvas.

So both axes start by being appended to the “g” group. Then each has its own classes applied for styling via CSS. If you recall from earlier, they look a little like this;

.axis path,
.axis line {
    fill: none;
    stroke: grey;
    stroke-width: 1;
    shape-rendering: crispEdges;
}

Feel free to mess about with these to change the appearance of your axes.

On the x axis, we have a transform statement (.attr("transform", "translate(0," + height + ")")). If you recall, our point of origin for drawing is in the top left hand corner. Therefore if we want our x axis to be on the bottom of the graph, we need to move (transform) it to the bottom by a set amount. The set amount in this case is the height of the graph proper (height). So, for the point of demonstration we will remove the transform line and see what happens;

Yep, pretty much as anticipated.

The last part of the two sections of script ( .call(xAxis); and .call(yAxis); ) call the x and y axis functions and initiate the drawing action.

Wrap Up

Well that’s it. In theory, you should now be a complete D3 ninja.

OK, perhaps a slight exaggeration. In fact there is a strong possibility that the information I have laid out here is at best borderline useful and at worst laden with evil practices and gross inaccuracies.

But look on the bright side. Irrespective of the nastiness of the way that any of it was accomplished or the inelegance of the code, if the picture drawn on the screen is pretty, you can walk away with a smile. :-)

This section concludes a very basic description of one type of a graphic that can be built with D3. We will look at adding value to it in subsequent chapters.

I’ve said it before and I’ll say it again. This is not a how-to for learning D3. This is how I have managed to muddle through in a bumbling way to try and achieve what I wanted to do. If some small part of it helps you. All good. Those with a smattering of knowledge of any of the topics I have butchered above (or below) are fully justified in feeling a large degree of righteous indignation. To those I say, please feel free to amend where practical and possible, but please bear in mind this was written from the point of view of someone with no experience in the topic and therefore try to keep any instructions at a level where a new entrant can step in.

Things you can do with the basic graph

The following headings in this section are intended to be a list of relatively simple ‘block’ type improvements that you can do to your graph to add functionality. The idea is to be able to use the simple graph that was used for the explanation of how D3 worked and just slot in code to add functionality (let’s hope it works for you :-)).

I have included the full code for a graph that includes rotated axis label, title, grid lines and filled area as an appendix (Graph with Many Features) for those who would prefer to see the code as a block.

The full code for this example can also be found on github or in the code samples bundled with this book (graph-with-many-features.html and data.csv). A live example can be found on bl.ocks.org

Adding Axis Labels

What’s the first thing you get told at school when drawing a graph?

“Always label your axes!”

So, time to add a couple of labels!

First things first (because they’re done slightly differently), the x axis. If we begin by describing what we want to achieve, it may make the process of implementing a solution a little more logical.

What we want to do is to add a simple piece of text under the x axis and in the centre of the total span. Wow, that does sound easy.

And it is, but there are different ways of accomplishing it, and I think I should take an opportunity to demonstrate them. Especially since one of those ways is a BAD idea. Lets start with the bad idea first :-).

This is the code we’re going to add to the simple line graph script;

    svg.append("text")             // text label for the x axis
        .attr("x", 265 )
        .attr("y",  240 )
        .style("text-anchor", "middle")
        .text("Date");

We will put it in between the blocks of script that add the x axis and the y axis.

    svg.append("g")              // Add the X Axis
        .attr("class", "x axis")
        .attr("transform", "translate(0," + height + ")")
        .call(xAxis);

    //	PUT THE NEW CODE HERE!

    svg.append("g")             // Add the Y Axis
        .attr("class", "y axis")
        .call(yAxis);

Before we describe what’s happening, let’s take a look at the result;

Well, it certainly did what it was asked to do. There’s a ‘Date’ label as advertised! (Yes, I know it’s not pretty.) Let’s describe the code and then work out why there’s a better way to do it.

    svg.append("text")                // text label for the x axis
        .attr("x", 265 )
        .attr("y", 240 )
        .style("text-anchor", "middle")
        .text("Date");

The first line appends a “text” element to our canvas. There is a lot more to learn about “text” elements at the home of the World Wide Web Consortium (W3C). The next two lines ( .attr("x", 265 ) and .attr("y", 240 ) ) set the attributes for the x and y coordinates to position the text on the canvas.

The second last line (.style("text-anchor", "middle")) ensures that the text ‘style’ is such that the text is centre aligned and therefore remains nicely centred on the x,y coordinates that we send it to.

The final line (.text("Date");) adds the actual text that we are going to place.

That seems really simple and effective and it is. However, the bad part about it is that we have hard coded the location for the date into the code. This means if we change any of the physical aspects of the graph, we will end up having to re-calculate and edit our code. And we don’t want to do that.

Here’s an example. If I decide that I would prefer to increase the height of the graph by editing the line here;

    height = 270 - margin.top - margin.bottom;

and making the height 350 pixels;

    height = 350 - margin.top - margin.bottom;

The result is as follows;

EVERYTHING about the graph has adjusted itself, except our nasty, hard coded ‘Date’ label. This is far from ideal and can be easily fixed by using the variables that we set up ever so carefully earlier.

So, instead of;

        .attr("x", 265 )
        .attr("y", 240 )

lets let our variables do the walking and use;

        .attr("x", width / 2 )
        .attr("y",  height + margin.bottom)

So with this code we tell the script that the ‘Date’ label will always be halfway across the width of the graph (no matter how wide it is) and at the bottom of the graph with respect to it’s height and the bottom margin (remember it uses a coordinates system that increases from the top down).

The end result of using variables is that if I go to an extreme of changing the height and width of my graph to;

    width = 400 - margin.left - margin.right,
    height = 200 - margin.top - margin.bottom;

We still finish up with an acceptable result;

Well, for the label position at least :-).

So the changes to using variables is just a useful lesson that variables rock and mean that you don’t have to worry about your graph staying in relative shape while you change the dimensions. The astute readers amongst you will have learned this lesson very early on in your programming careers, but it’s never a bad idea to make sure that users that are unfamiliar with the concept have an indicator of why it’s a good idea.

Now the third method that I mentioned at the start of our x axis odyssey. This is not mentioned because it’s any better or worse way to implement your script (The reason that I say this is because I’m not sure if it’s better or worse.) but because it’s sufficiently different to make it look confusing if you didn’t think of it in the first place.

So, we’ll take our marvellous coordinates code;

    .attr("x", width / 2 )
    .attr("y",  height + margin.bottom)

And replace it with a single (longer) line;

    .attr("transform",
          "translate(" + (width/2) + " ," + 
                         (height+margin.bottom) + ")")

This uses the "transform" attribute to move (translate) the point to place the ‘Date’ label to exactly the same spot that we’ve been using for the other two examples (using variables of course).

So, that’s the x axis label. Time to do the y axis. The code we’re going to use looks like this;

    svg.append("text")
        .attr("transform", "rotate(-90)")
        .attr("y", 0 – margin.left)
        .attr("x",0 - (height / 2))
        .attr("dy", "1em")
        .style("text-anchor", "middle")
        .text("Value");

For the sake of neatness we will put the piece of code in a nice logical spot and this would be following the block of code that added the y axis (but before the closing curly bracket)

    svg.append("g")			// Add the Y Axis
        .attr("class", "y axis")
        .call(yAxis);

    //	PUT THE NEW CODE HERE!

});

And the result looks like this;

There we go, a label for the y axis that is nicely centred and (gasp!) rotated by 90 degrees! Woah, does the leetness never end! (No. No it does not.)

So, how do we get to this incredible result?

The first thing we do is the same as for the x axis and append a text element to our canvas (svg.append("text")).

Then things get interesting.

        .attr("transform", "rotate(-90)")

Because that line rotates everything by -90 degrees. While it’s obvious that the text label ‘Value’ has been rotated by -90 degrees (from the picture), the following lines of code show that we also rotated our reference point (which can be a little confusing).

        .attr("y", 0 – margin.left)
        .attr("x",0 - (height / 2))

Let’s get graphical to illustrate how this works;

Here’s our starting position, with x,y in the 0,0 coordinate of the graph drawing area surrounded by the margins.

When we apply a -90 degrees transform we get the equivalent of this;

Here the 0,0 coordinate has been shifted by -90 degrees and the x,y designations are flipped so that we now need to tell the script that we’re moving a ‘y’ coordinate when we would have otherwise been moving ‘x’.

Hence, when the script runs…

        .attr("y", 0 – margin.left)

… we can see that this is moving the x position to the left from the new 0 coordinate by the margin.left value.

Likewise when the script runs…

        .attr("x",0 - (height / 2))

… this is actually moving the y position from the new 0 coordinate halfway up the height of the graph area.

Right, we’re not quite done yet. The following line has the effect of shifting the text slightly to the right.

        .attr("dy", "1em")

Firstly the reason we do this is that our previous translation of coordinates means that when we place our text label it sits exactly on the line of 0 – margin.left. But in this case that takes the text to the other side of the line, so it actually sits just outside the boundary of the overall canvas.

The "dy" attribute is another coordinate adjustment move, but this time a relative adjustment and the “1em” is a unit of measure that equals exactly one unit of the currently specified text point size. So what ends up happening is that the ‘Value’ label gets shifted to the right by exactly the height of the text, which neatly places it exactly on the edge of the canvas.

The two final lines of this part of the script are the same as for the x axis. They make sure the reference point is aligned to the centre of the text (.style("text-anchor", "middle")) and then it prints the text (.text("Value");). There, that wasn’t too painful.

The astute amongst you (and I’m picking that if you’re reading this far into the book, you will definitely qualify as astute or at least suckers for punishment) will notice that the example that I have included in the appendices and online does not have the ‘Value’ label. Instead it has the text ‘Price ($)’and it appears on the area of the graph itself! Well spotted. The technique used above is identical to the one used in the description here, but in the example code we have taken an additional step of demonstrating how to place a white shadowy background under the text (more of that to come in a couple of sections!).

How to add a title to your graph

If you’ve read through the adding the axis labels section most of this will come as no surprise.

What we want to do to add a title to the graph is to add a text element (just a few words) that will appear above the graph and centred left to right.

The code block we will use will looks like this;

    svg.append("text")
        .attr("x", (width / 2))				
        .attr("y", 0 - (margin.top / 2))
        .attr("text-anchor", "middle")	
        .style("font-size", "16px") 
        .style("text-decoration", "underline") 	
        .text("Value vs Date Graph");

And the end result will look like this;

A nice logical place to put the block of code would be towards the end of the JavaScript. In fact I would put it as the last element we add. So here;

    svg.append("g")              // Add the Y Axis
        .attr("class", "y axis")
        .call(yAxis);

    // PUT THE NEW CODE HERE!

});

Now since the vast majority of the code for this block is a regurgitation of the axis labels code, I don’t want to revisit that and bloat up this document even more, so I will direct you back to that section if you need to refresh yourself on any particular line. But….. There are a couple of new ones in there which could benefit from a little explanation.

Both of them are style descriptors and as such their job is to apply a very specific style to this element.

        .style("font-size", "16px") 
        .style("text-decoration", "underline") 	

What they do is pretty self explanatory. Make the text a specific size and underline it. But what is perhaps slightly more interesting is that we have this declaration in the JavaScript code and not in the CSS portion of the file.

Smoothing out graph lines

When you draw a line graph, what you’re doing is taking two (or more) sets of coordinates and connecting them with a line (or lines). I know that sounds simplistic, but bear with me. When you connect these points, you’re telling the viewer of the graph that in between the individual points, you expect the value to vary in keeping with the points that the line passes through. So in a way, you’re trying to interpret the change in values that are not shown.

Now this is not strictly true for all graph types, but it does hold for a lot of line graphs.

So… when connecting these known coordinates together, you want to make the best estimate of how the values would be represented. In this respect, sometimes a straight line between points is not the best representation.

For instance. Earlier, when demonstrating the extent function for graphing we showed a graph of the varying values with the y axis showing a narrow range.

The resulting variation of the graph shows a fair amount of extremes and you could be forgiven for thinking that if this represented a smoothly flowing analog system of some kind then some of those sharp peaks and troughs would not be a true representation of how the system or figures varied.

So how should it look? Ahh… The $64,000 question. I don’t know :-). You will have a better idea since you are the person who will know your data best. However, what I do know is that D3 has some tricks up its sleeve to help.

We can easily change what we see above into;

How about that? And the massive amount of code required to carry out what must be a ridiculously difficult set of calculations?

    .interpolate("basis")	

So where does this neat piece of code go? Here;

var valueline = d3.svg.line()
    .interpolate("basis")	 		// <=== THERE IT IS!
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y(d.close); });

So is that it? Nooooo…….. There’s more! This is one form of interpolation effect that can be applied to your data, but there is a range and depending on your data you can select the one that is appropriate.

Here’s the list of available options and for more about them head on over to the D3 wiki and look for ‘line.interpolate’.

• linear – Normal line (jagged).
• step-before – a stepping graph alternating between vertical and horizontal segments.
• step-after - a stepping graph alternating between horizontal and vertical segments.
• basis - a B-spline, with control point duplication on the ends (that’s the one above).
• basis-open - an open B-spline; may not intersect the start or end.
• basis-closed - a closed B-spline, with the start and the end closed in a loop.
• bundle - equivalent to basis, except a separate tension parameter is used to straighten the spline. This could be really cool with varying tension.
• cardinal - a Cardinal spline, with control point duplication on the ends. It looks slightly more ‘jagged’ than basis.
• cardinal-open - an open Cardinal spline; may not intersect the start or end, but will intersect other control points. So kind of shorter than ‘cardinal’.
• cardinal-closed - a closed Cardinal spline, looped back on itself.
• monotone - cubic interpolation that makes the graph only slightly smoother.

Because in the course of writing this I took an opportunity to play with each of them, I was pleasantly surprised to see some of the effects and it seems like a shame to deprive the reader of the same joy :-). So at the risk of deforesting the planet (so I hope you are reading this in electronic format) here is each of the above interpolation types applied to the same data.

This is also an opportunity to add some reader feedback awesomeness. Many thanks to ‘enjalot’ for the great suggestion to plot the points of the data as separate circles on the graphs. Since the process of interpolation has the effect of ‘interpreting’ the trends of the data to the extent that in some cases, the lines don’t intersect the actual data much at all.

Each of the following shows the smoothing curve and the data that is used to plot the graph.

Just in case you’re in the mood for another example, here are voronoi tessellations drawn with various d3 line interpolators (the original interactive version by ‘shawnbot’ can be found here).

First a version using the linear interpolation when each of the points is joined faithfully with a straight line.

Now a version where the polygons are formed with the ‘basis-closed’ interpolator (note how the lines don’t go through the points that describe the bounds of the polygons/blobs).

And lastly, using the ‘cardinal-closed’ interpolator, while the line travels through each point in the polygon, they overshoot in an effort to maintain a nice curve and the resulting polygon/blobs overlap.

So, over to you to decide which format of interpolation is going to suit your data best:-).

Adding grid lines to a graph

Grid lines are an important feature for some graphs as they allow the eye to associate three analogue scales (the x and y axis and the displayed line).

There is currently a tendency to use graphs without grid lines online as it gives the appearance of a ‘cleaner’ interface, but they are still widely used and a necessary component for graphing.

This is what we’re going to draw;

How to build grid lines?

We’re going to use the axis function to generate two more axis elements (one for x and one for y) but for these ones instead of drawing the main lines and the labels, we’re just going to draw the tick lines. Really long ticklines (I’m considering calling them long cat lines).

To create them we have to add in 3 separate blocks of code.

1. One in the CSS section to define what style the grid lines will have.
2. One to define the functions that generate the grid lines. And…
3. One to draw the lines.

The grid line CSS

This is the total styling that we need to add for the tick lines;

.grid .tick {
    stroke: lightgrey;
    stroke-opacity: 0.7;
    shape-rendering: crispEdges;
}
.grid path {
          stroke-width: 0;
}

Just add this block of code at the end of the current CSS that is in the simple graph template (just before the </style> tag).

The CSS here is done in two parts.

The first portion sets the line colour (stroke), the opacity (transparency) of the lines and make sure that the lines are narrow (crispEdges).

    stroke: lightgrey;
    stroke-opacity: 0.7;
    shape-rendering: crispEdges;

The colour is pretty standard, but in using the opacity style we give ourselves the opportunity to use a good shade of colour (if grey actually is a colour) and to juggle the degree to which it stands out a little better.

The second part is the stroke width.

    stroke-width: 0;

Now it might seem a little weird to be setting the stroke width to zero, but if you don’t (and we remove the style) this is what happens;

If you look closely (compare with the previous picture if necessary) the main lines for the axis have turned thicker. The stroke width style is obviously adding in new (thicker) axis lines and we’re not interested in them at the moment. Therefore, if we set the stroke width to zero, we get rid of the problem.

Define the grid line functions

We will need to define two functions to generate the grid lines and they look a little like this;

function make_x_axis() {		
    return d3.svg.axis()
        .scale(x)
        .orient("bottom")
        .ticks(5)
}

function make_y_axis() {		
    return d3.svg.axis()
        .scale(y)
        .orient("left")
        .ticks(5)
}

Each function will carry out it’s configuration when called from the later part of the script (the drawing part).

A good spot to place the code is just before we load the data with the d3.csv

        //   <== Put the functions here!
// Get the data
d3.csv("data.csv", function(error, data) {
    data.forEach(function(d) {
        d.date = parseDate(d.date);
        d.close = +d.close;
    });
}

Both functions are almost identical. They give the function a name (make_x_axis and make_y_axis) which will be used later when the piece of code that draws the lines calls out to them.

Both functions also show which parameters will be fed back to the drawing process when called. Both make sure they use the d3.svg.axis function and then they set individual attributes which make sense.

They make sure they’ve got the right axis (.scale(x) and .scale(y)). They set the orientation of the axes to match the incumbent axes (.orient("bottom") and .orient("left")). And they set the number of ticks to match the number of ticks in the main axis (.ticks(5) and .ticks(5)). You have the opportunity here to do something slightly different if you want. For instance, think back to when we were setting up the axis for the basic graph and we messed about, seeing how many ticks we could get to appear. If we increase the number of ticks that appear in the grid (lets say to .ticks(30) and .ticks(10))) we get the following;

So the grid lines can now show divisions of 50 on the y axis and per day on the x axis :-)

Draw the lines

The final block of code we need is the bit that draws the lines.

    svg.append("g")			
        .attr("class", "grid")
        .attr("transform", "translate(0," + height + ")")
        .call(make_x_axis()
            .tickSize(-height, 0, 0)
            .tickFormat("")
        )

    svg.append("g")			
        .attr("class", "grid")
        .call(make_y_axis()
            .tickSize(-width, 0, 0)
            .tickFormat("")
        )

The first two lines of both the x and y axis grid lines code above should be pretty familiar by now. The first one appends the element to be drawn to the group “g”. the second line (.attr("class", "grid")) makes sure that the style information set out in the CSS is applied.

The x axis grid lines portion makes a slight deviation from conformity here to adjust its positioning to take into account the coordinates system .attr("transform", "translate(0," + height + ")").

Then both portions call their respective make axis functions (.call(make_x_axis() and .call(make_y_axis()).

Now comes the really interesting bit.

What you will see if you go to the D3 API wiki is that for the .tickSize function, the following is the format.

axis.tickSize([major[​[, minor], end]])

That tells us that you get to specify the size of the ticks on the axes, by the major ticks, the minor ticks and the end ticks (that is to say the lines on the very end of the graph which, in the case of the example we are looking at, aren’t there!).

So in our example we are setting our major ticks to a length that corresponds to the full height or width of the graph. Which of course means that they extend across the graph and have the appearance of grid lines! What a neat trick.

Something I haven’t done before is to see what would happen if I included the tick lines for the minor and end ticks. So here we go :-)

Darn! Disappointment. We can see a minor tick line for the y axis, but nothing for the x axis and nothing on the ends. Clearly I will have to run some experiments to see what’s going on there (later).

The last thing that is included in the code to draw the grid lines is the instruction to suppress printing any label for the ticks;

        .tickFormat("")

After all, that would become a bit confusing to have two sets of labels. Even if one was on top of the other. They do tend to become obvious if that occurs (they kind of bulk out a bit like bold text).

And that’s it. Grid lines!

Make a dashed line

Dashed lines totally rock!

One of the best parts about it is that they’re so simple to do!

Literally one line!!!!

So lets imagine that we want to make the line on our simple graph dashed. All we have to do is insert the following line in our JavaScript code here;

    svg.append("path")
        .attr("class", "line")
        .style("stroke-dasharray", ("3, 3"))  // <== This line here!!
        .attr("d", valueline(data));

And our graph ends up like this;

Hey! It’s dashtastic!

So how does it work?

Well, obviously "stroke-dasharray" is a style for the path element, but the magic is in the numbers.

Essentially they describe the on length and off length of the line. So "3, 3" translates to 3 pixels (or whatever they are) on and 3 pixels off. Then it repeats. Simple eh?

So, experiment time :-)

What would the following represent?

“5, 5, 5, 5, 5, 5, 10, 5, 10, 5, 10, 5”

Try not to cheat…

Ahh yes, Mr. Morse would be proud.

And you can put them anywhere. Here’s our axes perverted with dashes;

    svg.append("g")
        .attr("class", "x axis")
        .attr("transform", "translate(0," + height + ")")
        .style("stroke-dasharray", ("3, 3"))
        .call(xAxis);

    svg.append("g")
        .attr("class", "y axis")
        .style("stroke-dasharray", ("3, 3"))
        .call(yAxis);

Well… I suppose you can have too much of a good thing. With great power comes great responsibility. Use your dash skills wisely and only for good.

Filling an area under the graph

Lines are all very well and good, but that’s not the whole story for graphs. Sometimes you’ve just got to go with a fill.

Filling an area with a solid colour isn’t too hard. I mean we did it by mistake back a few pages when we were trying to draw a line.

But to do it in a nice coherent way is fairly straight forward.

It takes three sections of code in much the same way that we drew our grid lines earlier;

1. One in the CSS section to define what style the area will have.
2. One to define the functions that generate the area. And…
3. One to draw the area.

The end result will looks a bit like this;

CSS for an area fill

This is pretty straight forward and only consists of two rules;

.area {
    fill: lightsteelblue;
    stroke-width: 0;
}

Put them at the bottom of your <style> section.

The first one (fill: lightsteelblue;) sets the colour of our fill (and in this case we have chosen a lighter shade of the same colour as our line to match it) and the second one (stroke-width: 0;) sets the width of the line that surrounds the area to zero. This last rule is kind of important in making a filled area work well. The whole idea is that the graph is made up of separate elements that will compliment each other. There’s the axes, the line and the fill. If we don’t tell the code that there is no line surrounding the filled area, it will assume that there is one and add it in like this.

So what has happened here is that the area element has inherited the line property from the path element and surrounding the area is a 2px wide steelblue line. Not too pretty. Let’s not go there.

Define the area function

We need a function that will tell the area what space to fill. This is accessed from the d3.svg.area function

The code that we will use is as follows;

var area = d3.svg.area()
    .x(function(d) { return x(d.date); })
    .y0(height)
    .y1(function(d) { return y(d.close); });

I have placed it in between the axis variable definitions and the line definitions here;

var yAxis = d3.svg.axis().scale(y)
    .orient("left").ticks(5);
                         <==== Put the new code here!
var valueline = d3.svg.line()
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y(d.close); });

So the only changes to the code are the addition of the y0 line and the renaming of the y line y1.

Here’s a picture that might help explain;

As should be apparent, the top line (y1) follows the valueline line and the bottom line is at the constant ‘height’ value. Everything in between these lines is what gets filled. The function in this section describes the area.

Draw the area

Now to the money maker.

The final section of code in the area filling odyssey is as follows;

    svg.append("path")
        .datum(data)
        .attr("class", "area")
        .attr("d", area);

We should place this block directly after the domain functions but before the drawing of the valueline path;

    x.domain(d3.extent(data, function(d) { return d.date; }));
    y.domain([0, d3.max(data, function(d) { return d.close; })]);
                                //   <== Area drawing code here!
    svg.append("path")		
        .attr("class", "line")
        .attr("d", valueline(data));

This is actually a pretty good idea to put it there since the various bits and pieces that are drawn in the graph are done so one after the other. This means that the filled area comes first, then the valueline is layered on top and then the axes come last. This is a pretty good sequence since if there are areas where two or more elements overlap, it might cause the graph to look ‘wrong’.

For instance, here is the graph drawn with the area added last.

You should be able to notice that part of the valueline line has been obscured and the line for the y axis where it coincides with the area is obscured also.

Looking at the code we are adding here, the first line appends a path element (svg.append("path")) much like the script that draws the line.

The second line (.datum(data)) declares the data we will be utilising for describing the area and the third line (.attr("class", "area")) makes sure that the style we apply to it is as defined in the CSS section (under ‘area’).

The final line (.attr("d", area);) declares “d” as the attributer for path data and calls the ‘area’ function to do the drawing.

And that’s it!

Filling an area above the line

Pop Quiz:

How would you go about filling the area ABOVE the graph?

In this instance, you could fill the lower area as has been demonstrated here, and with a small change you can fill another area with a solid colour above another line.

How is this incredible feat achieved?

Well, remember the code that defined the area?

var area = d3.svg.area()
    .x(function(d) { return x(d.date); })
    .y0(height)
    .y1(function(d) { return y(d.close); });

All we have to do is tell it that instead of setting the y0 constant value to the height of the graph (remember, this is the bottom of the graph) we will set it to the constant value that is at the top of the graph. In other words zero (0).

    .y0(0)

That’s it.

Now, I’m not going to go over the process of drawing two lines and filling each in different directions to demonstrate the example I described, but this provides a germ of an idea that you might be able to flesh out :-)

Adding a drop shadow to allow text to stand out on graphics.

I’ve deliberately positioned this particular tip to follow the ‘filling an area’ description because it provides an opportunity to demonstrate the principle to slightly better effect.

There have been several opportunities where I have wanted to place text overlaid on graphs for convenience sake only to have it look overly messy as the text interferes with the graph.

Anyway, what we’ll do is leave the fill in place and place the title back on the graph, but position the title so that it lays on top of the fill like so;

The additional code for the title is the following and appears just after the drawing of the axes.

    svg.append("text")
        .attr("x", (width / 2))				
        .attr("y", 25 )
        .attr("text-anchor", "middle")	
        .style("font-size", "16px") 
        .style("text-decoration", "underline") 	
        .text("Value vs Date Graph");

(the only change from the previous title example is the ‘y’ attribute which has been hard coded to 25 to place it inconveniently on the filled area.)

So, what we want to end up with is something like the following…

In my humble opinion, it’s just enough to make the text acceptable :-).

The method that I’ll describe to carry this out is designed so that the drop shadow effect can be applied to any text elements in the graph, not the isolated example that we will use here. In order to implement this marvel of utility we will need to make changes in two areas. One in the CSS where we will define a style for white shadowy backgrounds and the second to draw it.

CSS for white shadowy background

The code to add to the CSS section is as follows;

text.shadow {
    stroke: white;
    stroke-width: 2.5px;
    opacity: 0.9;
}

The first line designates that the style applies to text with a ‘shadow’ label. The stroke is set to white. the width of the line is set to 2.5px and it is made to be slightly see-through. So by setting the line that surrounds the text to be thick, white and see-through gives it a slightly ‘cloudy’ effect. If we remove the black text from over the top we get a slightly better look;

Of course if you want to have a play with any of these settings, you should have a go and see what works best for your graph.

Drawing the white shadowy background.

Now that we’ve set the style for our background, we need to draw it in.

The code for this should be extremely familiar;

    svg.append("text")
        .attr("x", (width / 2))				
        .attr("y", 25 )
        .attr("text-anchor", "middle")	
        .style("font-size", "16px") 
        .style("text-decoration", "underline") 	
        .attr("class", "shadow")		// <=== Here's the different line
        .text("Value vs Date Graph");

That’s because it’s identical to the piece of code that was used to draw the title except for the one line that is indicated above. The reason that it’s identical is that what we are doing is placing a white shadow on the graph and then the text on top of it, if it deviated by a significant amount it will just look silly. Of course a slight amount could look effective, in which case adjust the ‘x’ or ‘y’ attributes.

One of the things I pointed out in the previous paragraph was extremely important. That’s the bit that tells you that we needed to place the shadow before we placed the black text. For the same reason that we placed the area fill on first in the area fill example, If black text goes on before the shadow, it will look pretty silly. So place this block of code just before the block that draws the title.

So the line that has been added in is the one that tells D3 that the text that is being drawn will have the white cloudy effect. And at the risk of repeating myself, if you have several text elements that could benefit from this effect, once you have the CSS code in place, all you need to do is duplicate the block that adds the text and add in that single line and voila!

Again the astute amongst you will note that in the example code in the appendices and online, the shadow effect is applied to the Y axis label. Never fear, it’s exactly the same principle :-).

Adding more than one line to a graph

All right, we’re starting to get serious now. Two lines on a graph is a bit of a step into a different world in one respect. I mean that in the sense that there’s more than one way to carry out the task, and I tend to do it one way and not the other mainly because I don’t fully understand the other way :-(.

So, how are we going to do this? I think that the best way will be to make the executive decision that we have suddenly come across more data and that it is also in our data.csv file (which we’ll rename data2.csv just to highlight the difference between the two data sets). In fact it looks a little like this (apologies in advance for the big ugly block of data);

date,close,open
1-May-12,68.13,34.12
30-Apr-12,63.98,45.56
27-Apr-12,67.00,67.89
26-Apr-12,89.70,78.54
25-Apr-12,99.00,89.23
24-Apr-12,130.28,99.23
23-Apr-12,166.70,101.34
20-Apr-12,234.98,122.34
19-Apr-12,345.44,134.56
18-Apr-12,443.34,160.45
17-Apr-12,543.70,180.34
16-Apr-12,580.13,210.23
13-Apr-12,605.23,223.45
12-Apr-12,622.77,201.56
11-Apr-12,626.20,212.67
10-Apr-12,628.44,310.45
9-Apr-12,636.23,350.45
5-Apr-12,633.68,410.23
4-Apr-12,624.31,430.56
3-Apr-12,629.32,460.34
2-Apr-12,618.63,510.34
30-Mar-12,599.55,534.23
29-Mar-12,609.86,578.23
28-Mar-12,617.62,590.12
27-Mar-12,614.48,560.34
26-Mar-12,606.98,580.12

Three columns, date open and close. The first two are exactly what we have been dealing with all along and the last (open) is our new made up data. Each column is separated by a comma (hence .csv (comma separated values)), which is the format we’re currently using to import data.

We should save this as a new file so we don’t mess up our previous data, so (as mentioned earlier) let’s call it data2.csv.

There is a copy of this file and the sample code at github and in the code samples bundled with this book (dual-line-with-labels.html and data2.csv). A live example can be found on bl.ocks.org. The dual-line-with-labels.html file includes the double lines and also a labelling scheme which we will be describing in a later section.

We will build our new code using our simple graph template to start with, so the immediate consequence of this is that we need to edit the line that was looking for ‘data.csv’ to reflect the new name.

d3.csv("data2.csv", function(error, data) {

So when you browse to our new graph’s html file, we don’t see any changes. It still happily loads the new data, but because it hasn’t been told to do anything with it, nothing new happens.

What we need to do now it to essentially duplicate the code blocks that drew the first line for the second line.

The good news is that in the simplest way possible that’s just two code blocks. The first sets up the function that defines the new line;

var valueline2 = d3.svg.line()
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y(d.open); });

You should notice that this block is identical to the block that sets up the function for the first line, except this one is called (imaginatively) valueline2. We should put it directly after the block that sets up the function for valueline.

The second block draws our new line;

    svg.append("path")          // Add the valueline2 path.
        .attr("d", valueline2(data));

Again, this is identical to the block that draws the first line, except this one is called valueline2. We should put it directly after the block that draws valueline.

After those three small changes, check out your new graph;

Hey! Two lines! Hmm…. Both being the same colour is a bit confusing. Good news. We can change the colour of the second line by inserting a line that adjusts it’s stroke (colour) very simply.

So here’s what our new drawing block looks like;

    svg.append("path")		// Add the valueline2 path.
        .style("stroke", "red")
        .attr("d", valueline2(data));

And as if by magic, here’s our new graph;

Wow. Right about now, we’re thinking ourselves pretty clever. But there’s two places where we’re not doing things right. We took a simple way, but we took some short cuts that might bite us in the posterior.

The first mistake we made was not ensuring that our variable "d.open" is being treated as a number or a string. We’re fortunate in this case that it is, but this can’t always be assumed. So, this is an easy fix and we just need to put the following (indicated line) in our code;

// Get the data
d3.csv("data.csv", function(error, data) {
    data.forEach(function(d) {
        d.date = parseDate(d.date);
        d.close = +d.close;
        d.open = +d.open;        //  <=== Add this line in!
    });

The second and potentially more fatal flaw is that nowhere in our code do we make allowance for our second set of data (the second line’s values) exceeding our first lines values.

That might not sound too normal straight away, but consider this. What if when we made up our data earlier, some of the new data exceeded our maximum value in our original data? As a means of demonstration, here’s what happens when our second line of data has values higher than the first lines;

Ahh…. We’re not too clever now.

Good news though, we can fix it!

The problem comes about because when we set the domain for the y axis this is what we put in the code;

y.domain([0,d3.max(data, function(d) {return d.close;})]);

So that only considers d.close when establishing the domain. With d.open exceeding our domain, it just keeps drawing off the graph!

The good news is that ‘Bill’ has provided a solution for just this problem here;

All you need to replace the y.domain line with is this;

y.domain([0, d3.max(data, function(d) { 
    return Math.max(d.close, d.open); })]);

It does much the same thing, but this time it returns the maximum of d.close and d.open (whichever is largest). Good work Bill.

If we put that code into the graph with the higher values for our second line we are now presented with this;

And it doesn’t matter which of the two sets of data is largest, the graph will always adjust :-)

You will also have noticed that our y axis has auto adjusted again to cope. Clever eh?

Labelling multiple lines on a graph

Our previous example of a graph with multiple lines is a thing of rare beauty, but which line relates to which set of data? We have data that defines values for open and close, but we don’t know which line is which.

In this section we will add labels to our lines so that we know what it what.

This section was inspired by a question from a reader (Arun b.s) of the d3noob.org blog where the question was asked “How can we put text at the end of each line on the graph?”.

The question was so good I realised that it had to be part of the book, so here you go :-).

It’s actually not too difficult. What we are trying to achieve is to find the position of the end of each line and to add a text label at that position so that the association of proximity denotes the linkage. Of course we’re going to go a little further and colour the text so that it’s really clear which label belongs with which line, but you get the idea.

Each line requires a single block of script to add the text. The block that adds the open label is as follows;

svg.append("text")
  .attr("transform", "translate("+(width+3)+","+y(data[0].open)+")")
  .attr("dy", ".35em")
  .attr("text-anchor", "start")
  .style("fill", "red")
  .text("Open");

So firstly it appends a textual element to the svg object;

svg.append("text")

Then it finds the position of the end of the line;

  .attr("transform", "translate("+(width+3)+","+y(data[0].open)+")")

To do this we use the transform and translate attribute and find the x position that equates to the end of the graph plus 3 pixels ((width+3)) (we add in the three pixels to create a small separation between the end of the line and the label). The y position is far more interesting. We need to find the position of the last point in our line for the open data. Because the data is in the form of an indexed array and because the data has the latest date at the start of the array, we only need to find the point at the 0 position of the array. This is data[0].open. But of course, we also need to adjust our data for our scale and range, so we transform it using the y function (in the same way that we do it for the valueline and valueline2 points. So the script to find the point on the screen in the y direction is y(data[0].open).

If our data was arranged with the last date at the end of our data we would have to find the final index point and we would use y(data[data.length-1].open)).

Then it’s just a matter of aligning and justifying our text correctly;

  .attr("dy", ".35em")
  .attr("text-anchor", "start")

Then colouring it the correct colour;

  .style("fill", "red")

And adding out text;

  .text("Open");

We put this block of code after the blocks that add in the axes so that they make sure they’re on top of anything else we draw.

The only other small change we want to make is to change the right margin for the graph that we set at the start of our script from 20 to 40 so that there is enough room to add our label without cutting it off.

After that you have a marvellously labelled multi-line graph!

The full code for this example can be found on github or in the code samples bundled with this book (dual-line-graph-with-labels.html and data2.csv). A working example can be found on bl.ocks.org.

Now, I’d like to pretend that this is perfection, but it isn’t. If our lines end too close together, the labels will interfere with each other, so in the ideal world I would include a bit of fanciness to prevent that, but for the purposes of this exercise we can consider ourselves happy.

Multiple axes for a graph

Alrighty… Let’s imagine that we want to show our wonderful graph with two lines, much like we already have, but that the data that the lines are made from is significantly different in magnitude from the original data (in the example below, the data for the second line has been reduced by approximately a factor of 10 from our original data).

date,close,open
1-May-12,58.13,3.41
30-Apr-12,53.98,4.55
27-Apr-12,67.00,6.78
26-Apr-12,89.70,7.85
25-Apr-12,99.00,8.92
24-Apr-12,130.28,9.92
23-Apr-12,166.70,10.13
20-Apr-12,234.98,12.23
19-Apr-12,345.44,13.45
18-Apr-12,443.34,16.04
17-Apr-12,543.70,18.03
16-Apr-12,580.13,21.02
13-Apr-12,605.23,22.34
12-Apr-12,622.77,20.15
11-Apr-12,626.20,21.26
10-Apr-12,628.44,31.04
9-Apr-12,636.23,35.04
5-Apr-12,633.68,41.02
4-Apr-12,624.31,43.05
3-Apr-12,629.32,46.03
2-Apr-12,618.63,51.03
30-Mar-12,599.55,53.42
29-Mar-12,609.86,57.82
28-Mar-12,617.62,59.01
27-Mar-12,614.48,56.03
26-Mar-12,606.98,58.01

Now this isn’t a problem in itself. D3 will still make a reasonable graph of the data, but because of the difference in range, the detail of the second line will be lost.

What I’m proposing is that we have a second y axis on the right hand side of the graph that relates to the red line.

The mechanism used is based on the great examples put forward by Ben Christensen here.

The full code for this example can be found on github or in the code samples bundled with this book (dual-line-dual-axes.html and data2a.csv). A working example can be found on bl.ocks.org.

First things first, there won’t be space on the right hand side of our graph to show the extra axis, so we should make our right hand margin a little larger.

var margin = {top: 30, right: 40, bottom: 30, left: 50},

I went for 40 and it seems to fit pretty well.

Then (and here’s where the main point of difference for this graph comes in) you want to amend the code to separate out the two scales for the two lines in the graph. This is actually a lot easier than it sounds, since it consists mainly of finding anywhere that mentions y and replacing it with y0 and then adding in a reciprocal piece of code for y1.

Let’s get started.

Firstly, change the variable declaration for y to y0 and add in y1.

var x = d3.time.scale().range([0, width]);
var y0 = d3.scale.linear().range([height, 0]);
var y1 = d3.scale.linear().range([height, 0]);

Then change our yAxis declaration to be specific for y0 and specifically left. And add in a declaration for the right hand axis;

var yAxisLeft = d3.svg.axis().scale(y0)    //  <==  Add in 'Left' and 'y0'
    .orient("left").ticks(5);

var yAxisRight = d3.svg.axis().scale(y1)   // new declaration for 'Right', 'y1'
    .orient("right").ticks(5);             // and includes orientation .

Note the orientation change for the right hand axis.

Now change our valueline declarations so that they refer to the y0 and y1 scales.

var valueline = d3.svg.line()
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y0(d.close); });    // <== y0
var valueline2 = d3.svg.line()
    .x(function(d) { return x(d.date); })
    .y(function(d) { return y1(d.open); });     // <== y1

There are a few different ways for the scaling to work, but we’ll stick with the fancy max method we used in the dual line example (although technically it’s not required).

   y0.domain([0, d3.max(data, function(d) { return Math.max(d.close); })]); 
   y1.domain([0, d3.max(data, function(d) { return Math.max(d.open); })]); 

Again, here’s the y0 and y1 changed and added and the maximums for d.close and d.open are separated out). The final piece of the puzzle is to draw the new axis, but we also want to make a slight change to the original y axis. Since we have two lines and two axes, we need to know which belongs to which, so we can colour code the text in the axes to match the lines;

    svg.append("g")
        .attr("class", "y axis")
        .style("fill", "steelblue")
        .call(yAxisLeft);	

    svg.append("g")				
        .attr("class", "y axis")	
        .attr("transform", "translate(" + width + " ,0)")	
        .style("fill", "red")		
        .call(yAxisRight);	

In the above code you can see where we have added in a ‘style’ change for the yAxisLeft to make it ‘steelblue’ and a complementary change in the new section for yAxisRight to make that text red.

The yAxisRight section obviously needs to be added in, but the only significant difference is the transform / translate attribute that moves the axis to the right hand side of the graph.

And after all that, here’s the result…

Now, let’s not kid ourselves that it’s a thing of beauty, but we should console our aesthetic concerns with the warm glow of understanding how the function works :-).

How to rotate the text labels for the x Axis.

The observant reader will recall the problem we had observed earlier when increasing the number of ticks on our x axis to 10. The effect had been to produce a large number of x axis ticks (actually 19) but they had run together and become unreadable.

We postulated at the time that an answer to the problem might be to rotate the text to provide more space. Well, it’s about time we solved that problem.

The answer I found most usable was provided by Aaron Ward on Google Groups.

The full code for this example can be found on github or in the code samples bundled with this book (simple-graph-rotated-axis-text.html and data.csv). A working example can be found on bl.ocks.org. The example code also includes the formatting of the x axis ticks in a specific format as described in the next section.

Starting out with our simple graph example, we should increase the number of ticks on the x axis to 10 to highlight the problem in the previous image.

The first substantive change would be a little housekeeping. Because we are going to be rotating the text at the bottom of the graph, we are going to need some extra space to fit in our labels. So we should change our bottom margin appropriately.

var margin = {top: 30, right: 40, bottom: 50, left: 50},	

I found that 50 pixels was sufficient.

The remainder of our changes occur in the block that draws the x axis.

    svg.append("g")
        .attr("class", "x axis")
        .attr("transform", "translate(0," + height + ")")
        .call(xAxis)
        .selectAll("text")	
            .style("text-anchor", "end")
            .attr("dx", "-.8em")
            .attr("dy", ".15em")
            .attr("transform", "rotate(-65)");	

It’s pretty standard until the .call(xAxis) portion of the code. Here we remove the semicolon that was there so that the block continues with its function.

Then we select all the text elements that comprise the x axis with the .selectAll("text"). From this point onwards, we are operating on the text elements associated with the x axis. In effect; the following 4 ‘actions’ are applied to the text labels.

The .style("text-anchor", "end") line ensures that the text label has the end of the label ‘attached’ to the axis tick. This has the effect of making sure that the text rotates about the end of the date. This makes sure that the text all ends up at a uniform distance from the axis ticks.

The dx and dy attribute lines move the end of the text just far enough away from the axis tick so that they don’t crowd it and not too far away so that it appears disassociated. This took a little bit of fiddling to ‘look’ right and you will notice that I’ve used the ‘em’ units to get an adjustment if the size of the font differs.

The final action is kind of the money shot.

The transform attribute applies itself to each text label and rotates each line by -65 degrees. I selected -65 degrees just because it looked OK. There was no deeper reason.

The end result then looks like the following;

This was a surprisingly difficult problem to find a solution to that I could easily understand (well done Aaron). That makes me think that there are some far deeper mysteries to it that I don’t fully appreciate that could trip this solution up. But in lieu of that, enjoy!

Format a date / time axis with specified values

OK then. We’ve been very clever in rotating our text, but you will notice that D3 has used it’s own good judgement as to what format the days / date will be represented as.

Not that there’s anything wrong with it, but what if we want to put a specific format of date / time nomenclature as axis labels?

No problem. D3 has your back.

This is actually a pretty easy thing to do, but there are plenty of options for the formatting, so the only really tricky part is deciding what to put where.

But, before we start doing anything we are going to have to expand our bottom margin even more than we did with the rotate the axis labels feature.

var margin = {top: 30, right: 40, bottom: 70, left: 50},

That should see us right.

Right, now the simple part :-). Changing the format of the label is as simple as inserting the tickFormat command into the xAxis declaration a little like this;

var xAxis = d3.svg.axis().scale(x)
    .orient("bottom").ticks(10)
    .tickFormat(d3.time.format("%Y-%m-%d")); // insert the tickFormat function

An example using this code can be found on github or in the code samples bundled with this book (simple-graph-rotated-axis-text.html and data.csv). A working example can be found on bl.ocks.org. The example code also includes the rotating of the x axis text as described in the previous section.

What the tickFormat allows is the setting of formatting for the tick labels. The d3.time.format portion of the code is specifying the exact format of those ticks. This formatting is described using the same arguments that were explained in the earlier section on formatting date time values. That means that the examples we see here (%Y-%m-%d) should display the year as a four digit number then a hyphen then the month as a two digit number, then another hyphen, then a two digit number corresponding to the day.

Let’s take a look at the result;

There we go! You should be able to see this file in the downloads section on d3noob.org with the general examples as formatted-date-time-axis-labels.html.

So how about we try something a little out of the ordinary (extreme)?

How about the full weekday name (%A), the day (%d), the full month name (%B) and the year (%Y) as a four digit number?

    .tickFormat(d3.time.format("%A %d %B %Y"));

We will also need some extra space for the bottom margin, so how about 140?

var margin = {top: 30, right: 40, bottom: 140, left: 50},

and….

Oh yeah… When axis ticks go bad…

But seriously, that does work as a pretty good example of the flexibility available.

Update data dynamically - On Click

OK, you’re going to enjoy this section. Mainly because it takes the traditional graph that we know, love and have been familiar with since childhood and adds in an aspect that that has been missing for most of your life.

Animation!

Graphs are cool. Seeing information represented in a graphical way allows leaps of understanding that are difficult or impossible to achieve from raw data. But in this crazy ever-changing world, a static image is only as effective as the last update. The ability to being able to have the most recent data represented in your graph and to have it occur automatically provides a new dimension to traditional visualizations.

So what are we going to do?

First we’ll spend a bit of time setting the scene. We’ll add a button to our basic graph file so that we can control when our animation occurs, we’ll generate a new data set so that we can see how the data changes easily, then we’ll shuffle the code about a bit to make it do its magic. While we’re shuffling the code we’ll take a little bit of time to explain what’s going on with various parts of it that are different to what we might have seen thus far. Then we’ll change the graph to update automatically (on a schedule) when the data changes.

The code for this example can be found on github or in the code samples bundled with this book (data-load-button.html, data.csv and data-alt.csv). A working example can be found on bl.ocks.org.

Adding a Button

It’s all well and good animating your data, but if you don’t know when it’s supposed to happen or what should happen, it’s a little difficult to evaluate how successful you’ve been.

To make life easy, we’re going to take some of the mystery out of the equation (don’t worry, we’ll put it back later) and add a button to our graph that will give you control over when your graph should update it’s data. When complete it should look like this;

To add a button, we will take our simple-graph.html example and just after the <body> tag we add the following code;

<div id="option">
    <input name="updateButton" 
           type="button" 
           value="Update" 
           onclick="updateData()" 
    />
</div>

The HTML <div> element (or HTML Document Division Element) is used to assign a division or section in an HTML document. We use it here as it’s good practice to keep sections of your HTML document distinct so that it’s easier to perform operations them at a later date.

In this case we have given the div the identifier “option” so that we can refer to it later if we need to (embarrassingly, we won’t be referring to it at all, but it’s good practice none the less).

The following line adds our button using the HTML <input> tag. The <input> tag has a wide range of attributes (options) for allowing user input. Check out the links to w3schools and Mozilla for a whole lot of reading.

In our <input> line we have four different attributes;

• name
• type
• value
• onclick

Each of these attributes modifies the <input> function in some way so that our button does what we want it to do.

name:
This is the name of the control (in this case a button) so that we can reference it in other parts of our HTML script.

type:
Probably the most important attribute for a button, this declares that our type of input will be a button! There are heaps of other options for type which would form a significant section in itself.

value:
For a button input type, this is the starting value for our button and forms the label that our button will have.

onclick:
This is not an attribute that is specific to the <input> function, but it allows the browser to capture a mouse clicking event when it occurs and in our case we tell it to run the updateData() function (which we’ll be seeing more of soon).

Updating the data

To make our first start at demonstrating changing the data, we’ll add another data file to our collection. We’ll name it data-alt.csv (you can find it in the sample code collection that can be downloaded with the book on Leanpub). This file changes our normal data (only the values, not the structure) just enough to see a movement of the time period of the graph and the range of values on the y axis (this will become really obvious in the transition).

Changes to the d3.js code layout

While going through the process of working out how to do this, the iterations of my code were mostly horrifying to behold. However, I think my understanding has improved sufficiently to allow only a slight amendment to our simple-graph.html JavaScript code to get this going.

What we should do is add the following block of code to our script towards the end of the file just before the </script> tag;

function updateData() {

    // Get the data again
    d3.csv("data-alt.csv", function(error, data) {
       	data.forEach(function(d) {
	    	d.date = parseDate(d.date);
	    	d.close = +d.close;
	    });

    	// Scale the range of the data again 
    	x.domain(d3.extent(data, function(d) { return d.date; }));
            y.domain([0, d3.max(data, function(d) { return d.close; })]);

    // Select the section we want to apply our changes to
    var svg = d3.select("body").transition();

    // Make the changes
        svg.select(".line")   // change the line
            .duration(750)
            .attr("d", valueline(data));
        svg.select(".x.axis") // change the x axis
            .duration(750)
            .call(xAxis);
        svg.select(".y.axis") // change the y axis
            .duration(750)
            .call(yAxis);

    });
}

What’s happening in the code?

There are several new concepts and techniques in this block of code for us to go through but we’ll start with the overall wrapper for the block which is a function call.

The entirety of our JavaScript code that we’re adding is a function called updateData. This is the subject of the first line in the code above (and the last closing curly bracket). It is called from the only other piece of code we’ve added to the file which is the button in the HTML section. So when that button is clicked, the updateData function is carried out.

Then we get our new data with the block that starts with d3.csv("data-alt.csv". This is a replica of the block in the main part of the code with one glaring exception. It is getting the data from our new file called data-alt.csv. However, one thing it’s doing that bears explanation is that it’s loading data into an array that we’ve already used to generate our line. At a point not too far from here (probably the next page) we’re going to replace the data that made up our line on the page with the new data that’s just been loaded.

We then set the scale and the range again using the x.domain and y.domain lines. We do this because it’s more than possible that our data has exceeded or shrunk with respect to our original domains so we recalculate them using our new data. The consequence of not doing this would be a graph that could exceed it’s available space or be cramped up.

Then we assign the variable svg to be our selection of the "body" div (which means the following actions will only be carried out on objects within the "body" div.

The other part of that line is the transition command (.transition()). This command goes to the heart of animating dynamic data and visualizations and is a real treasure.

As the name suggests, a transition is a method for moving from one state to another. In its simplest form for a d3.js visualisation, it could mean moving an object from one place to another, or changing an object’s properties such as opacity or colour. In our case, we will take our data which is in the form of a line, and change some of that data. And when we change the data we will get d3 to manage the change via a transition. At the same time (because we’re immensely clever) we will also make sure we change the axes if they need it.

So in short, we’re going to change this…

… into this…

Obviously the line values have changed, and both axes have changed as well. And using a properly managed transition, it will all occur in a smooth ballet :-).

So, looking at the short block that manages the line transition;

        svg.select(".line")   // change the line
            .duration(750)
            .attr("d", valueline(data));

We select the ".line" object and since we’ve already told the script that svg is all about the transition (var svg = d3.select("body").transition();) the attributes that follow specify how the transition for the line will proceed. In this case, the code describes the length of time that the transition will take as 750 milliseconds (.duration(750)) and uses the new data as transcribed by the valueline variable from the original part of the script (.attr("d", valueline(data));).

The same is true for both of the subsequent portions of the code that change the x and y axes. We’ve set both to a transition time of 750 milliseconds, although feel free to change those values (make each one different for an interesting effect).

Other attributes for the transition that we could have introduced would be a delay (.delay(500), perhaps to stagger the movements) and more interestingly an easing attribute (.ease(type[, arguments…])) which will have the effect of changing how the movement of a transition appears (kind of like a fast-slow-fast vs linear, but with lots of variations).

But for us we’ll survive with the defaults.

In theory, you’ve added in your new data file (data-alt.csv) and made the two changes to the simple graph file (the HTML block for the button and the JavaScript one for the updateData function). The result has been a new beginning in your wonderful d3 journey!

Update data dynamically – Automatically

I have no doubt that the excitement of updating your data and graph with the magic of buttons is quite a thrill. But believe it or not, there’s more to come.

In the example we’re going to demonstrate now, there are no buttons to click, the graph will simply update itself when the data changes.

I know, I know. It’s like magic!

So the sort of usage scenario that you would be putting this to is when you had a dashboard type display or a separate window just for the purposes of showing a changing value like a stock ticker or number of widgets sold (where the number was changing frequently).

So, how to create the magic?

Starting with the data-load-button.html file, firstly we should remove the button, so go ahead and delete the button block that we had in the HTML section (the bit that looked like this…).

<div id="option">
    <input name="updateButton" 
                 type="button" 
                value="Update" 
                onclick="updateData()" />
</div>

Now, we have two references in our JavaScript where we load our data. One loads data.csv initially, then when the button was pushed, we loaded data-alt.csv. We’re going to retain that arrangement for the moment, because we want to make sure we can see something happening, but ultimately, we would just have them referencing a single file.

So, the magic piece of script that will do your updating is as follows;

var inter = setInterval(function() {
                updateData();
        }, 5000); 

And we should put that just above the function updateData() { line in our code.

The key to this piece of code is the setInterval function which will execute specified code (in this case it’s updateData(); which will go and read in our new information) over and over again at a set interval (in this case 5000 milliseconds (}, 5000);)).

I honestly wish it was harder, but sadly it’s that simple. You now have in your possession the ability to make your visualizations do stuff on a regular basis, all by themselves!

There is a copy of this sample code and the data files at github and in the code samples bundled with this book (data-load-automatic.html, data.csv and data-alt.csv). A live example can be found on bl.ocks.org.

How to test?

Well, just load up your new file in a browsers. After an interval of 5 seconds, you should see the graph change all by itself. How cool is that?

You know it gets better though…

If you open your data-alt.csv file and change a value (increase one of the close values by a factor of 10 or something equally noticeable). Then save the file. Keep an eye on your graph. Before 5 seconds is up it should have changed to reflect your new data.

Elements, Attributes and Styles

This chapter is intended to provide an overview of some of the simpler things that d3.js can do, but in a way that may help some understand a little more about how images can be added to a web page and how they can be manipulated.

Loosely speaking we will look at how objects (elements (like circles, rectangles, lines and even text) can be declared and added to a page, how their attributes in relation to the page (position, size, shape, actions) can be changed and how their style (colour, width, transparency) can be applied.

As we go through the explanation of different changes that can be applied to different elements there will be a small amount of repetition where there is cross-over with related drawing features. Please be patient :-). The aim is to have each section as complete in its own right as practical.

The Framework

To be able to demonstrate how these three related aspects of drawing objects work we will have to use a small, simple script to draw them in your web browser.

We will just take a moment to explain the script that draws a circle.

Here’s the contents of the file in it’s entirety. I have imaginatively called it circle.html.

<!DOCTYPE html>
<meta charset="utf-8">

<body>

<!-- load the d3.js library -->	
<script type="text/javascript" src="d3/d3.v3.js"></script>

<script>
 
var holder = d3.select("body") // select the 'body' element
      .append("svg")           // append an SVG element to the body
      .attr("width", 449)      // make the SVG element 449 pixels wide
      .attr("height", 249);    // make the SVG element 249 pixels high

// draw a circle
holder.append("circle")        // attach a circle
    .attr("cx", 200)           // position the x-center
    .attr("cy", 100)           // position the y-center
    .attr("r", 50);            // set the radius

</script>

</body>

Please feel free to jump ahead slightly if you understand how a HTML file with JavaScript goes together :-).

The HTML part of the file can be thought of as a wrapper for the JavaScript that will draw our circle. These are the HTML parts here…

<!DOCTYPE html>
<meta charset="utf-8">

<body>

<!-- load the d3.js library -->	
<script type="text/javascript" src="d3/d3.v3.js"></script>

<script>
 
</script>

</body>

This portion of the file is built using HTML ‘tags’. These will set up the environment for the Javascript.

The tags tell the web browser what sort of language is being used and the type of characters used to write the code…

<!DOCTYPE html>
<meta charset="utf-8">

Areas of the code are labelled.

Like the body…

<body>

In this area we can put the stuff that will be 
displayed on our web page.

</body>

And the place where we put the JavaScript…

<script>

Our d3.js code will go here.

</script>

We even load an external file that contains JavaScript that will help run our code.

<!-- load the d3.js library -->	
<script type="text/javascript" src="d3/d3.v3.js"></script>

Yes, that’s the line that loads d3.js. Once it’s loaded we can use the instructions that it makes available to make other JavaScript code (in this case ours) work.

Then we have the JavaScript code that allows us to use the functions made possible by d3.js.

var holder = d3.select("body") // select the 'body' element
      .append("svg")           // append an SVG element to the body
      .attr("width", 449)      // make the SVG element 449 pixels wide
      .attr("height", 249);    // make the SVG element 249 pixels high

// draw a circle
holder.append("circle")        // attach a circle
    .attr("cx", 200)           // position the x-center
    .attr("cy", 100)           // position the y-center
    .attr("r", 50);            // set the radius

I’ve broken the code into two separate portions to provide some clarity to their function. We could make it one block, but that wouldn’t necessarily make it easier to understand.

Firstly we add a ‘holder’ for our graphics on the web page. I’ve named it holder but we could just as easily named it anything we wanted.

var holder = d3.select("body") // select the 'body' element
      .append("svg")           // append an SVG element to the body
      .attr("width", 449)      // make the SVG element 449 pixels wide
      .attr("height", 249);    // make the SVG element 249 pixels high

The first thing we do when declaring our holder is to select the body element of our web page (Remember those <body> tags in the HTML part earlier?).

Then we append a Scalable Vector Graphic (SVG) object to the body and we make it 449 pixels wide and 249 pixels high.

The width and height are ‘attributes’ of the SVG object. That is to say they describe a property of the object.

The second block of our JavaScript finally draws our circle.

holder.append("circle")        // attach a circle
    .attr("cx", 200)           // position the x-center
    .attr("cy", 100)           // position the y-center
    .attr("r", 50);            // set the radius

The first line appends a new element (a circle) to our SVG ‘holder’.

The second and third lines declare the attribute of our circle that specify where the centre of the circle is. In this case it’s at the x/y position 200/100 (cx/cy).

The last line adds the radius attribute r. Here it is set to 50 pixels.

The three attributes cx, cy and r are all required when drawing a circle. There are other attributes we can put in there (and when we look at some of the upcoming elements, you should get a feel for them), but these are the minimum.

The purpose of describing this block of code that draws a circle isn’t to show you how to draw a circle. This has only been a way of showing you how the code in the following sections is laid out and how it works. The elements we are going to generate can be drawn with exactly the same file but with just the section that adds the circle altered.

For example if you were to change this block of code;

holder.append("circle")        // attach a circle
    .attr("cx", 200)           // position the x-center
    .attr("cy", 100)           // position the y-center
    .attr("r", 50);            // set the radius

For this block of code;

holder.append("rect")          // attach a circle
    .attr("x", 150)            // x position of the top-left corner
    .attr("y", 50)             // y position of the top-left corner
    .attr("width", 100)        // set the rectangle width
    .attr("height", 100);      // set the rectangle height

Instead of drawing a circle we would be drawing a rectangle.

So this is what our circle will look like;

Because it will help a great deal to have a common frame of reference, I’m going to display the elements on a grid that looks a little like this;

With the grid in place it’s far easier to see that the centre of our circle is indeed at the coordinates x = 200, y = 100 and that the radius is 50.

The circle is still somewhat plain, but bear with me because as we start to explore what we can do with styles and attributes we can add some variation to our elements.

With that explanation behind us we should begin our odyssey into the world of d3 elements.

Elements

We will begin by describing what we mean when we talk about an ‘element’.

There is considerable scope for confusion when talking about elements on a web page. Are we talking about HTML elements, SVG elements or something different?

In fact we are going to be describing a subset of SVG elements. Specifically those that are described in the d3.js API reference (since that’s why we’re here right?). These are a collection of common shapes and objects which include circles, ellipses, rectangles, lines, polylines, polygons, text and paths.

“Text?” I hear you say. “Doesn’t sound like a shape.” I suppose it depends on how you think of it. We can use text in different ways in d3, but for this particular exercise we can regard text as an SVG element.

Circle

A circle is a simple SVG shape that is described by three required attributes.

• cx: The position of the centre of the circle in the x direction (left / right) measured from the left side of the screen.
• cy: The position of the centre of the circle in the y direction (up / down) measured from the top of the screen.
• r: The radius of the circle from the cx, cy position to the perimeter of the circle.

The following is an example of the code section required to draw a circle in conjunction with the HTML file outlined at the start of this chapter;

holder.append("circle")        // attach a circle
    .attr("cx", 200)           // position the x-center
    .attr("cy", 100)           // position the y-center
    .attr("r", 50);            // set the radius

This will produce a circle as follows;

The centre of the circle is at x = 200 and y = 100 and the radius is 50 pixels.

Ellipse

An ellipse is described by four required attributes;

• cx: The position of the centre of the ellipse in the x direction (left / right) measured from the left side of the screen.
• cy: The position of the centre of the ellipse in the y direction (up / down) measured from the top of the screen.
• rx: The radius of the ellipse in the x dimension from the cx, cy position to the perimeter of the ellipse.
• ry: The radius of the ellipse in the y dimension from the cx, cy position to the perimeter of the ellipse.

The following is an example of the code section required to draw an ellipse in conjunction with the HTML file outlined at the start of this chapter;

holder.append("ellipse")       // attach an ellipse
    .attr("cx", 200)           // position the x-centre
    .attr("cy", 100)           // position the y-centre
    .attr("rx", 100)           // set the x radius
    .attr("ry", 50);           // set the y radius

This will produce an ellipse as follows;

The centre of the ellipse is at x = 200 and y = 100 and the radius is 50 pixels vertically and 100 pixels horizontally.

Rectangle

A rectangle is described by four required attributes and two optional ones;

• x: The position on the x axis of the left hand side of the rectangle (required).
• y: The position on the y axis of the top of the rectangle (required).
• width: the width (in pixels) of the rectangle (required).
• height: the height (in pixels) of the rectangle (required).
• rx: The radius curve of the corner of the rectangle in the x dimension (optional).
• ry: The radius curve of the corner of the rectangle in the y dimension (optional).

The following is an example of the code section required to draw a rectangle (using only the required attributes) in conjunction with the HTML file outlined at the start of this chapter;

holder.append("rect")       // attach a rectangle
    .attr("x", 100)         // position the left of the rectangle
    .attr("y", 50)          // position the top of the rectangle
    .attr("height", 100)    // set the height
    .attr("width", 200);    // set the width

This will produce a rectangle as follows;

The top left corner of the rectangle is at 100, 50 and the rectangle is 200 pixels wide and 100 pixels high.

The following code section includes the optional attributes for the curved corners;

holder.append("rect")       // attach a rectangle
    .attr("x", 100)         // position the left of the rectangle
    .attr("y", 50)          // position the top of the rectangle
    .attr("height", 100)    // set the height
    .attr("width", 200)     // set the width
    .attr("rx", 10)         // set the x corner curve radius
    .attr("ry", 10);        // set the y corner curve radius

This will produce a rectangle (with curved corners) as follows;

The corners are curved with radii in the x and y direction of 10 pixels.

Line

A line is a simple line between two points and is described by four required attributes.

• x1: The x position of the first end of the line as measured from the left of the screen.
• y1: The y position of the first end of the line as measured from the top of the screen.
• x2: The x position of the second end of the line as measured from the left of the screen.
• y2: The y position of the second end of the line as measured from the top of the screen.

The following is an example of the code section required to draw a line in conjunction with the HTML file outlined at the start of this chapter. A notable addition to this code is the style declaration. In this case the line has no colour and this can be added with the stroke style which applies a colour to a line;

holder.append("line")          // attach a line
    .style("stroke", "black")  // colour the line
    .attr("x1", 100)     // x position of the first end of the line
    .attr("y1", 50)      // y position of the first end of the line
    .attr("x2", 300)     // x position of the second end of the line
    .attr("y2", 150);    // y position of the second end of the line

This will produce a line as follows;

The line extends from the point 100,50 to 300,150.

Polyline

A polyline is a sequence of connected lines described with a single attribute. The d3.js wiki rightly makes the point that “it is typically more convenient and flexible to use the d3.svg.line path generator in conjunction with a path element”. So while drawing a polyline using this method may be possible, bear in mind that depending on your application, there may be a better way.

• points: The points attribute is a list of x,y coordinates that are the locations of the connecting points of the polyline.

The following is an example of the code section required to draw a polyline in conjunction with the HTML file outlined at the start of this chapter. A notable addition to this code are the style declarations. In this case the line of the polyline has no colour and this can be added with the stroke style which applies the colour black to a line. Likewise the area that is bounded by the polyline will be automatically filled with black unless we explicitly tell the object not to. This is achieved in this example by addition of the fill style to none.

holder.append("polyline")      // attach a polyline
    .style("stroke", "black")  // colour the line
    .style("fill", "none")     // remove any fill colour
    .attr("points", "100,50, 200,150, 300,50");  // x,y points

This will produce a polyline as follows;

The polyline extends from the point 100,50 to 200,150 to 300,50.

Polygon

A polygon is a sequence of connected lines which form a closed shape described with a single attribute. The d3.js wiki rightly makes the point that “it is typically more convenient and flexible to use the d3.svg.line path generator in conjunction with a path element”. So while drawing a polygon using this method may be possible, bear in mind that depending on your application, there may be a better way.

• points: The points attribute is a list of x,y coordinates that are the locations of the connecting points of the polygon. The last point is in turn connected to the first point.

The following is an example of the code section required to draw a polygon in conjunction with the HTML file outlined at the start of this chapter. A notable addition to this code are the style declarations. In this case the line of the polygon has no colour and this can be added with the stroke style which applies the colour black to a line. Likewise the area that is bounded by the polygon will be automatically filled with black unless we explicitly tell the object not to. This is achieved in this example by addition of the fill style to none.

holder.append("polygon")       // attach a polygon
    .style("stroke", "black")  // colour the line
    .style("fill", "none")     // remove any fill colour
    .attr("points", "100,50, 200,150, 300,50");  // x,y points 

This will produce a polygon as follows;

The polygon extends from the point 100,50 to 200,150 to 300,50 and then back to 100,50.

Path

A path is an outline of an SVG shape which is described with a ‘mini-language’ inside a single attribute.

• d: This attribute is a list of instructions that allow a shape to be drawn in a complex way using a ‘mini-language’ of commands. These commands are written in a shorthand of single letters such as M-moveto, Z-closepath, L-lineto, C-curveto. These commands can be absolute (normally designated by capital letters) or relative (lower case).

The following is an example of the code section required to draw a triangle in conjunction with the HTML file outlined at the start of this chapter. A notable addition to this code are the style declarations. In this case the line of the path has no colour and this can be added with the stroke style which applies the colour black to a line. Likewise the area that is bounded by the path will be automatically filled with black unless we explicitly tell the object not to. This is achieved in this example by addition of the fill style to none.

holder.append("path")          // attach a path
    .style("stroke", "black")  // colour the line
    .style("fill", "none")     // remove any fill colour
    .attr("d", "M 100,50, L 200,150, L 300,50 Z");  // path commands 

This will produce a path as follows;

The path mini-language first moves (M) to 100,50 then draws a line (L) to 200,150 then draws another line (L) to 300,50 then closes the path (Z).

Clipped Path (AKA clipPath)

A clipPath is the path of a SVG shape that can be used in combination with another shape to remove any parts of the combined shape that doesn’t fall within the clipPath. That sounds slightly confusing, so we will break it down a bit to hopefully clarify the explanation.

Let’s imagine that we want to display the intersection of two shapes. What we will do is define our clipPath which will act as a ‘cookie cutter’ which can cut out the shape we want (we will choose an ellipse). Then we will draw our base shape (which is analogous to the dough) that we will use our cookie cutter on (our dough will be shaped as a rectangle). The intersection of the cookie cutter and the dough is our clipped path.