Heuristic Evaluation Plan

1 Introduction

The Parallel Coordinates Explorable Explainer (PCEE) is a web application, which provides an interactive tutorial about the parallel coordinates visualisation technique for multidimensional data. It is designed for newcomers to the topic to learn the basics about parallel coordinates and become familiar with this kind of graphical chart. PCEE is an open-source project; the source code is available at [And2025a]. A live version of the tool is hosted at [And2025b]. Four experts in data visualisation will review and evaluate the application, using the method of Heuristic Evaluation.

2 Evaluation Methodology

The heuristic evaluation (HE) method is used to evaluate a user interface (UI) with a set of common guiding principles, known as the heuristics. The method was first described by Jakob Nielsen and Rolf Molich in 1990 [Nie1990]. A small number (typically three or four) of specialised evaluators look through the UI and use their judgement and experience to assess where potential issues may arise. A small set of typically ten to fifteen heuristics is provided to suggest what to look out for. For this project, the Andrews General Usability Heuristics (see Section A.1) will be used. They are adapted from the updated set of 10 usability heuristics published by Nielsen in 1993 [Nie1993]. In 2024, Nielsen revised the descriptions of the original heuristics and provided new examples [Nie2024]. Redacción Aguayo describes some strategies and best practices for heuristic evaluation [Agu2025]. For further reading, Chauncey Wilson provides a good description of heuristic evaluation [Wil2013, Chapter 1].

An important aspect of a HE is that the evaluation is done by each evaluator separately, working alone. Sometimes, each evaluator makes their own notes and takes screenshots and/or video clips to illustrate their findings as they go along. For this project, since the evaluators' time is precious, notes will be taken by the facilitator, and screenshots and/or video clips will be created or extracted by the facilitator afterwards from each session recording.

After the four individual evaluations, the findings will be aggregated into a long list using a spreadsheet. Positives and negative findings will be compiled into separate lists. Where multiple evaluators find similar issues, these will be merged into one. The aggregated list of negative findings (problems) will then be distributed to three project team members (not the evaluators), to assign a severity rating to each problem. Finally, the problems will be sorted in decreasing order of mean severity, so the most serious problems are at the top of the list.

3 User Profiles

Two user profiles were identified for PCEE:

• Students in an InfoVis class: PCEE can be used to support lecture material in an Information Visualisation course. The lecturer might demonstrate the concept of parallel coordinates in class and students can revisit and work through the tutorial at home.

• Independent Study: PCEE can be accessed online, so can enable a broad variety of people to learn about the topic of parallel coordinates without the need to attend a course.

4 Extent of the Evaluation

The evaluation will cover the entire PCEE tool, with a live version that is hosted at https://tugraz-isds.github.io/pcee/. A specific branch will be set up to archive the exact version that will be used for the heuristic evaluation.

4 Evaluators and Evaluation Environments

The evaluators will use the evaluation environments shown in Table1.

LosslessCut 3.55.1 will be used on Debian 13 for video editing.

References

[Agu2025]

Redacción Aguayo; Heuristic Evaluation: Strategies and Best Practices; visited 30 Sep 2025. https://aguayo.co/en/blog-aguayo-user-experience/heuristic-evaluation-strategies-best-practices/

[And2025a]

Keith Andrews; Parallel Coordinates Explorable Explainer (PCEE); visited 30 Sep 2025. https://github.com/tugraz-isds/pcee

[And2025b]

Keith Andrews; Parallel Coordinates: An Explorable Explainer; visited 30 Sep 2025. https://tugraz-isds.github.io/pcee/

[Nie1990]

Jakob Nielsen and Rolf Molich; Heuristic Evaluation of User Interfaces; Proc. Conference on Human Factors in Computing Systems (CHI’90). Seattle, Washington, USA, 01 Apr 1990, pages 249–256. doi:10.1145/97243.97281

[Nie1993]

Jakob Nielsen; Usability Engineering; Academic Press, 01 Jan 1993. ISBN 0125184069. https://dl.acm.org/doi/book/10.5555/2821575

[Nie1994]

Jakob Nielsen; The Theory Behind Heuristic Evaluations; Nielsen Norman Group, 01 Nov 1994. https://nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/theory-heuristic-evaluations/

[Nie2024]

Jakob Nielsen; 10 Usability Heuristics for User Interface Design; Nielsen Norman Group, 30 Jan 2024. https://nngroup.com/articles/ten-usability-heuristics/

[Wil2013]

Chauncey Wilson; User Interface Inspection Methods; Morgan Kaufmann, 15 Nov 2013. ISBN 012410391X.

A Materials

The following materials will be used by the evaluation team.