"Picture This"

Computer Business Review, December 2002

Data visualisation tools offer an intuitive way to shorten decision-making cycles and involve more people in the process. Madan Sheina investigates.

Cast your mind back to a scene from Steven Spielberg's 1993 blockbuster Jurassic Park. Dr Grant's niece, a young Ariana Richards, exclaims: "This is a Unix system, I know this!", as she navigates through a virtual 3D representation of the park's computer security system in a desperate bid to stave off a velociraptor attack. Technically it was not a Unix system, but a Silicon Graphics 3D Virtual File System that runs on Unix. But dinosaurs aside, this is just one example of the limitless applications of data visualisation technology.

Data visualisation provides graphic metaphors for understanding large amounts of integrated data. Its benefits vary by application and data type. For example, visualisation techniques can enable users to quickly compare the relative scale of data, detect patterns within it and provide a richer context for understanding that data.

The technology provides an effective way to communicate complex data to a broad user audience. While some data visualisations can be highly advanced, users do not need to be experts on complex 3D thinking to reap the benefits. "Visualisation captures the imagination and thus becomes more readily adopted by people," says Don Campbell, VP of product innovation and technology at Cognos.

Beyond static displays

In a decision-support context, pictures are potentially worth millions of dollars for the insights they generate. Data visualisation is closely related to business intelligence (BI). The star-schemas and related models widely used in BI applications naturally lend themselves to visualisation techniques because their n-dimension structures provide an easily understood basis for comparisons.

Data visualisation in a BI context follows a similar distinction to that between reporting, which presents data in static (view only) charts and graphs, and analysis, which enables users to interact with data in search of insights. "Visualisation is good in cases where you don't know what you don't know, when analysis begins with a broad question like, 'How's business?" says Campbell.

According to Ramana Rao, co-founder and CTO of Inxight Software: "Classic BI reporting typically occurs late in the decision-making process in order to reinforce points of persuasion, when you already know the answers. Data visualisation sits at the beginning of the process by supporting discovery and exploration, when you are looking for questions not answers."

Certain types of BI reporting, such as financial performance or sales activity, are are often only supported by low-end visualisations - bar and pie charts - which rely on user experience and intuition to uncover insights in data. "Many organisations have sunk millions of dollars into BI tools, but users have only got 80% of the value of analysis," says Steve Sukman, president of business software at Advanced Visual Systems (AVS).

Sukman points to a new breed of interactive data visualisation product that takes the analysis to a higher level. These tools help business users to analyse exceptional events and uncover complex relationships in large data sets in a more automated fashion than would be possible using traditional report-based BI solutions.

Visual interaction

Interaction is the main ingredient of visualisation technology that distinguishes it from classic BI graphics. "Humans are smart because they interact," says Rao. Similarly, all analytic users want to interact with a visualisation to explore and study aspects of its data. Users should not mistake simply rotating a 3D bar chart as interaction. True interaction is achieved by allowing users to make dynamic, data-driven changes to the display by refreshing an online visualisation with recent data.

Most interactive visualisations present graphics at the summary level first, allowing users to grasp an overall picture of the data. Users can then move graphics up or down, change visual properties, and zoom in for increased detail. Other tools allow for more sophisticated interaction, such as data filtering and animation.

Knowing what data to analyse and how to interpret the results are the biggest challenges faced in visualisation, according to Peter Dorrington, principal initiative manager at SAS Institute UK. "It was lack of clarity in this area that gave rise to numerous failed data warehousing projects as organisations attempted to collect and store masses of data without really understanding how it would be used."

Determining how much knowledge and training users require to interpret results and how organisations can share the insights are also key. "If data visualisation is to avoid the pitfalls that data warehousing fell into, it must be seen as part of a larger intelligence framework that includes the skill and ability of an organisation to act as a result of what it discovers," says Dorrington.

The best data visualisation tools easily integrate with a wide range of existing applications and data sources. Questions of where the data will come from and where it is stored - dynamically joined or stored in a data mart/warehouse - are of paramount importance in enabling easy visualisation. "Users shouldn't have to jump through hoops to get data into a data visualisation tool," says Mike McDonald, CEO of Visual Mining.

Myriad of applications

The potential for creating new visualisations for solving business problems is limited only by the imagination. Starting from applications in life sciences, research and engineering, data visualisation can be used in applications such as designing sails for boats and analysing world ozone depletion.

Interactive data visualisation opens up new types of analysis that are not possible with traditional multi-dimensional (online analytical processing - OLAP) analysis tools. Richard Brath, director of business development at Visual Insights, highlights certain classes of analysis that require more specialised visualisations. "It's hard to use OLAP techniques for path analysis or fraud detection, which typically focus on individual, rather than consolidated, records."

Brath points to an increasing trend to use visualisation scenes to analyse clickstream and traffic through an ecommerce web site and to identify purchasing patterns and trends. "Modelling movement through a web site using a map of pages and links is a better visual metaphor than presenting a bar-chart displaying a number of page views," says Brath.

Additionally, analysing online customer data often involves plotting thousands of data points in a single view, which requires a specialised visualisation (such as a constellation graph) that can scale to any number of data points. These specialised chart types tend to be so interactive that the visualisation transcends its visual origins to become a self-contained interface for navigating and analysing large data sets.

Visual pure-plays

The scope of available applications has led to a proliferation of vendors touting visualisation software. Specialised, standalone tools allow users to develop complex visual depictions of structured and unstructured data; while integrated BI tools offer visualisation functionality as an adjunct to their query and analysis capabilities.

There are many standalone vendors that offer very sophisticated 3D data visualisation products, which can often be integrated with query, reporting and analysis tools. These include players such as Visual Insights, AVS and Visual Mining. These vendors' tools support sophisticated visual data representations - up to 7D charts, graphs, scatter diagrams, carpet plots and map-based visualisations - that can be embedded into runtime applications and viewed in web pages. Most have a strong focus on the visualisation of structured information - sourcing data from relational data warehouses/data marts, transaction processing systems, and even TCP/IP streaming sources. Some specialised vendors offer developer toolkits for building custom, one-off visualisations, while others provide a more complete visualisation solution targeted at specific markets.

Many toolkit providers offer visual interfaces to drag and drop components into place to form a data-to-visualisation processing stream. For example, AVS and Visual Mining use component suites to build 3D visualisation capabilities into existing business applications. AVS' OpenViz product integrates a graphics display system, visual charts, plots, visual data representations, interactive data selection, and data integration tools. Similarly, Visual Mining uses a set of Java applets to dynamically 'chart-enable' a variety of web applications without programming.

Visual Insights, a Lucent Technologies Bell Laboratories spin-off, develops eBizinsights suite, an ebusiness performance solution designed to understand web site performance, promotional effectiveness and visitor demographics. "Our Visual Path Analysis module collects clickstream data from logs or directly from a web server and converts it into information-rich content, which makes it easier to spot high drop-off pages and
identify optimal revenue generating paths through a web site," says Visual Insights' Brath. The software clearly differentiates eBizinsights from traditional web analytics tools such as Accrue and WebTrends.

Integrated BI tools

As the market shifts towards end-user self-service, BI vendors are starting to offer ad hoc data visualisation techniques to their traditional reporting and analysis tools to enhance the usability of their product suites.

One of the first vendors to develop its own visualisation module was Cognos. Its Visualizer product is now an integrated component of the Cognos Platform for Enterprise Business Intelligence (EBI), which also includes Impromptu and PowerPlay tools for query and multidimensional analysis.

Computer Associates (CA) inherited Forest & Trees, a web-enabled data visualisation development tool, when it acquired Platinum Software.

Visualisation capabilities now appear in a number of CA's product portfolios, including management and control products, such as Unicenter TND, and its CleverPath portal. Oracle, meanwhile, does not have a separate product, but integrates visualisation functionality into Oracle Discoverer, an ad hoc query and analysis tool that is part of the company's BI solution that is integrated into its Oracle9i database system.

Some BI vendors opt for an OEM approach for supporting visualisation. Comshare and SAS Institute both offer visualisation technology from Inxight Software; SAS uses it in conjunction with its Text Miner product, but also offers SAS/Insight to provide interactive visualisations for statistical data analysis. Other vendors such as Business Objects and Hyperion also include third-party visualisation components in their BI suites.

Softer visualisation

Unstructured data visualisation products differ significantly in their functionality, aiming to enhance document search and analysis by creating a visual map of key concepts extracted from text-based content. The value of this type of visualisation is that it highlights the inherent relationships between large volumes of disparate unstructured content spread across multiple systems.

Users commonly run into problems finding and contextualising information on the Web because of low precision and recall keyword searches. Corporate taxonomies help, but often fail to show the context of individual documents. Autonomy, an enterprise search and portal provider, tightly incorporates a spectral map visualisation component in its search engine that allows users to quickly move around a map to display information about related content.

Hypertext links on a web site also lend themselves to visual navigation aids. Inxight Software, which was spun-off by Xerox PARC to commercialise its visualisation and language analysis tools, provides visualisation software in the form of 'Star Trees'. The technology uses a technique called 'focus and context' to highlight a particular area of a hyperlinked map (the focus) while displaying a large amount of closely linked items (the context) as a tree structure. The benefit of this approach is twofold. It quickly provides a sense of where a hypertext document fits into a larger collection; and it helps users navigate through interconnected content. Inxight's own research found that users could browse over 60% faster with a Star Tree than a traditional 2D tree layout such as Microsoft Explorer.

Future vision

Ultimately, data visualisation will not take off as a standalone technology; it will need to become an integral part of mainstream business applications that people already use. "We live in a 3D world, therefore we should work in a 3D world as well," says Cognos' Campbell. "The goal is to remove the distinction of explicitly working with a specialised tool, and making high-level visualisations pop-up naturally in an application."

The market is clearly moving in this direction. Specialised data visualisation functions are already being embedded across a range of enterprise applications. AVS' visualisation components can be found in PeopleSoft 8 and Blue Martini's customer interaction system. Visual Insights' software is used in i2 Technologies' supply chain suite and Avaya's call centre applications.

The reality of today's business environment is becoming more complex. At the same time, people are starting to take an analytic perspective of their business. "Humans struggle to deal with complexity if it is presented in rows and columns," says Rao. "Yet we are still intelligent enough to know that we don't have the time to waste trying to do so." Data visualisation is clearly emerging as the 'intelligent' way to move forward.

CBR Opinion

Data visualisation technology is being driven by growing volumes of ebusiness data and an increasing involvement of non-technical users in the decision-making process. Data visualisation tools can be used to reveal key business issues and relationships in business data. The potential applications for visualisation technology may extend as far as our imagination dictates. But for data visualisation to move into the mainstream, it will need to be implemented as an integral part of typical business applications. Visualisation models, therefore, continue to gain a foothold in traditional numeric applications as well as in hypertext, relational data and unstructured data applications.