2026 | Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

Evaluation-First Design for Data Visualization Interfaces

The EvalOps overlay applied to the Data-First stages. Three aspects are illustrated: cadence (nested loops that carry evidence forward), roles (structured participation shifting across phases), and metrics (evolving goals-to-metrics grounding that links phases). The diagram also preserves the Precondition, Core, and Analysis bands from the Design Study Methodology (DSM). EvalOps makes these evaluation activities more explicit and continuous across phases.

Abstract

Existing frameworks in visualization and HCI emphasize iteration, data grounding, and stakeholder needs; however, they have not fully explored how evaluation might persist across phases, adapt to compressed timelines, and aid stakeholder engagement and elicitation. Building on prior frameworks, we introduce Evaluation-first design EvalOps that centers evaluation as a material component in the design process, emphasizing tighter feedback loops, co-evaluation with stakeholders, malleable forms of evaluation, and goals-to-metrics grounding. We illustrate how EvalOps shapes design outcomes through two case studies of data-visualization and LLM-enabled reasoning tools, demonstrating how evaluation-driven design facilitates alignment and trust, uncovers opportunities earlier, and supports cohesiveness under rapidly changing constraints. We contrast EvalOps with current visualization design methodologies and discuss opportunities for expanding evaluation-centered framings to other active areas of design research.

BibTeX

@inproceedings{10.1145/3772318.3791057,
author = {Shrestha, Bijesh and Shrestha, Hilson and Bonilla, Karen and Crouser, R. Jordan and Harrison, Lane},
title = {Evaluation-First Design for Data Visualization Interfaces},
year = {2026},
isbn = {9798400722783},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3772318.3791057},
doi = {10.1145/3772318.3791057},
abstract = {Existing frameworks in visualization and HCI emphasize iteration, data grounding, and stakeholder needs; however, they have not fully explored how evaluation might persist across phases, adapt to compressed timelines, and aid stakeholder engagement and elicitation. Building on prior frameworks, we introduce Evaluation-first design EvalOps that centers evaluation as a material component in the design process, emphasizing tighter feedback loops, co-evaluation with stakeholders, malleable forms of evaluation, and goals-to-metrics grounding. We illustrate how EvalOps shapes design outcomes through two case studies of data-visualization and LLM-enabled reasoning tools, demonstrating how evaluation-driven design facilitates alignment and trust, uncovers opportunities earlier, and supports cohesiveness under rapidly changing constraints. We contrast EvalOps with current visualization design methodologies and discuss opportunities for expanding evaluation-centered framings to other active areas of design research.},
booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems},
articleno = {443},
numpages = {8},
keywords = {Visualization, Design Study, Evaluation},
location = {Barcelona, Spain},
series = {CHI '26}
}