Description

Evaluating visualizations is complex. We have developed a value-driven methodology (called ICE-T) to help researchers, designers, and practitioners determine the value of visualizations.

This site contains links to the research papers describing the methodology, supplemental material about the study, and materials available for download so that others can use the methodology.

Materials

To conduct an evaluation of a visualization using our ICE-T methodology, we provide the following material.

First, you will need one or more visualizations to evaluate. Ideally, these visualization exhibit a reasonable amount of complexity and interactivity (i.e., not a simple, static visual chart).

Next, recruit participants to conduct the study. Each participant should be given a description of the visualization so that they are knowledgeable about how the visualization works, and what data is being shown. How many participants are needed depends on the particular experimental design.

Each participant should fill out the ICE-T survey form for every visualization being evaluated.

Once each participant has completed the survey, the scores from the heuristics can be analyzed, and the value of the visualization determined.

Research Papers

  1. John Stasko, “Value-Driven Evaluation of Visualizations”, Proceedings of BELIV 2014, Paris, France, November 2014, pp. 46-53. [PDF]
  2. Emily Wall, Meeshu Agnihotri, Laura Matzen, Kristin Divis, Michael Haass, Alex Endert, and John Stasko, “A Heuristic Approach to Value-Driven Evaluation of Visualizations”, IEEE Transactions on Visualization and Computer Graphics, (Paper presented at InfoVis ‘18) Vol. 25, No. 1, January 2018, pp. 491-500. [PDF]

Team Members

Project Funding and Support

This work was partially supported by the Laboratory Directed Research and Development program at Sandia National Laboratories. Sandia is a multi-mission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

GT Vis Lab

Sandia