Visualizations of Habit and Routine

Lately I've become interested in the design of visualizations that draw out patterns in habit & routine. To explain what I mean, here are a bunch of nice examples...

Let's start with a visualization of a twitter user's posting habits from xefer.com:


This simple diagram of a baby's sleep times comes from Trixie Tracker:
Simple but effective! Thanks to Nathan's flowingdata for these two examples. (See also a wonderful visualization of the stabilization of a baby's sleep patterns in Winfree "The Timing of Biological Clocks" Page 31, also shown in Card et al "Information Visualization..." Page 5/6).

It seems that some form of heatmap is the most common means of representing habitual behaviours - see e.g., Andrienko et al for a visualization of traffic densities around Milan (red is lots of traffic):
This picture of hotel visitation patterns (Weaver et al) shows the number of visitors over a weekly timescale:
I like the summary at the bottom and right of the main area showing aggregated trends.

Nathan Eagle & Alex Pentland's paper on "Eigenbehaviours" differentiates various routine patterns from a dataset & presents them clearly:
This reminds me of Wijk & Selow's classic paper too.

Does anyone have any suggestions on other visualizations of habits and routines?

Summer internship at i2, Cambridge UK

Time for a shameless plug! If there are any students of information visualization out there looking for an interesting internship this summer - my company is offering one. Looking forward to hearing from you...

The Search Box

I've been a fan of Google Suggest since I first saw it - and I think that the news that it will now become the default experience for all Google searches is very significant because it will change everyone's expectations of what should happen when they start typing into a search box everywhere:



Of course several search boxes already do this kind of thing (the search box & 'awesome bar' in Firefox 3, the 'omnibox' in Google Chrome) but I believe that its use will become widespread not just for web based searching but also within desktop applications.

In 2005 just after Google Suggest came out I implemented my own version of it - it doesn't take much to get the basics right - you just need a reverse index. A bit more work and some stats can help eliminate common 'stop words' & give spelling suggestions; some further work and you can index two or three word phrases. Some of this stuff ended up in a desktop product in my day job.

Of course in a disconnected desktop product one doesn't have the huge amount of statistical information on what is searched for & clicked on, but to get a simple version of 'suggest' up and running can be quite quick and gives the user real tangible benefits.

The search box _should_ never be the same again...

Visualizaton Goals & Features

What are the goals of visualization? And what are the features that support those goals?

My 10 cents worth:

The basic goal is to facilitate reasoning and thought about what is being visualized. That reasoning could revolve around causality, hypothesis, predictions, inferences, habits, modus operandi, contradictions, uncertainty, and a whole host of problems the user is trying to solve. Often the reasoning revolves around external data and/or knowledge too. Visualization should expose structure in the data such as patterns, clusters, gaps, bursts of activity, outliers & trends, etc. And at the end of the reasoning process the great thing about visualization is that one should end up with a picture that can be used to disseminate one's insights to other people.

So what key features enable these goals to be achieved?
* A Summarization/Overview to give the big picture
* Zoomability
* Drill-down on data for the detail
* Easy navigation around the visual
* Filtering information by category or query
* Different types of visualization expose different patterns (geographic, timeline, textual, lists, link diagrams, etc.)
* Brushing & linking visualizations together can help the filtering & exploration

Other basic things which must be in place in order to succeed:
* Ease of import and export - and adhering to any standards
* Some basic searching of the data
* One must be able to read the data - in particular any text
* Scaling well as the data size gets very large
* Links out to other systems for further information is key
* Links back in to the visualization from other systems can also be powerful
* Interoperability with other visualization tools and other applications in general
* Commentary, scribbling and drawing on the visualizations is a great way to add understanding - a picture alone is rarely enough

And don't forget the more esoteric things too:
* It needs a positive emotional response so it must look good and not conflict with user's expectations
* It can use standard visual symbolism, conventions & metaphors
* It must use the basic visual variables well (shape, colour, position, etc.)
* Transitions between visualizations must be smooth to allow the user to keep their context
* It should use design techniques like 'information scent' & obvious affordances
* It should facilitate playfulness where ever possible - don't punish 'mistakes'!

Phew - glad I got that off my chest - back to the day job :-)

Intellipedia

I can't remember when I first read about Intellipedia, but I know it was a long time ago. The other day I watched a really inspirational video from the people inside the CIA who successfully overcame typical middle-management obstacles in order to produce it. I particularly like their comments on the evils of email and shared drives for collaboration. The original proposal document is also well worth a read.

Interesting too that Intellipedia's iVideo sharing component reportedly uses Adobe Flash, but there is no public mention of using visualization tools yet, despite visualization techniques being widely used in the intelligence analysis community.

Details Matter

Recently I've noticed a few examples where smallish visual design details really matter.

What Colin Ware in "Visual Thinking for Design" refers to as "multiscale structure" in shown-off very effectively in Vizster and SocialAction.



Another example is the way that Thinkbase constructs links of similar types:

Notice how the actors & roles aren't linked directly to the film but go via intermediate nodes. And also see how well the space is used and how the links are similar lengths & short - both of these are great for aesthetics and link-following tasks.

Both these approaches build on the position visual variable to effectively to 'clump' the like-nodes together. But these designs also add other visual variables (connecting line, enclosing shape) which assist visual pattern finding, and importantly can offer affordances for interactivity too.

My Day Job

Here's what I've been doing these last few months -



As you can see, I've been working on a collaboration system for intelligence analysts. (Hence the guns & crimes!)

It is all written using Adobe tools - Flex and Flash - so people just need to click a link to launch the application in their browser, which is seriously cool. We've designed it to work with touch screens so you just need one press to drill down on the data.

So far it has gone really well & now that we've done the basics I'm hoping to get some innovative visualizations in there too. Watch this space...!

Update 1: "The better part of valour is discretion..." - I've decided to remove the video for now while I sort out whether my employers are happy for me to show it. With a bit of luck it will be back soon! Joe

Update 2: It seems that it is fine to air this after all.

Design & Reach

Following on from my last post, Mike Danziger and I chatted on email & he wrote up some impressions of the InfoVis conference. Stephen Few responded to some of the points, and a couple (1, 2) of subsequent postings, and some other comments (3, 4) have shown that people are interested. Sorry for being so late in responding myself - the day job sometimes gets in the way!

For me, the key contribution has been Pat Hanrahan's. I feel the same way & I'm grateful to him for providing some academic respectability to what would otherwise just be my own opinions. From my own pragmatic software industry perspective, I'd like to say something about how his suggestions could be taken forward.

Delivery mechanisms are key: to appeal to the masses one needs reach. Interactive visualizations must be delivered to people's eyes & to their fingertips. Static images in papers aren't enough: people don't have much time or patience & won't enjoy having to read lots of text in order to learn how the interaction works.

One approach is to put good visualization capability into commonly used tools such as Excel (1). That way people can manage their data themselves. Because the user has the ability to load and edit the data behind the visualizations this means a high degree of skill is needed when crafting the software so it has the necessary flexibility. Each tool has different extension points & platforms. In practical terms this means a software company is forced to choose a very small list of supported environments & work flows.

The more obvious route is to exploit the immediacy and universality of web delivery mechanisms. Thanks to Flash, Silverlight & Java there is a huge audience out there with suitable runtimes. It is good to see more and more experimental visualizations using these. (Though problems with data management are still there of course...)

Reach isn't enough: in order to bring something compelling to people one must embrace designers. Graphic designers, user experience designers, interaction designers, the works! The right kind of designers can keep a visualization clean, useful & informative but also imbue it with style, panache & memorability. There is a design revolution happening now in the software industry & it will sweep up information visualization tools along the way.

The combination of the need for reach and good design is the main reason why I'm so interested in the Adobe platform. Because they already have designers using their tools, they don't need to woo them to new platform. Add a massive install base (flash) and increasingly workable languages (mxml, as3) and it is hard to dismiss. Nice to see I'm not alone in thinking this.