2025 | IEEE Visualization and Visual Analytics (VIS)

ReVISit 2: A Full Experiment Life Cycle User Study Framework

Zach Cutler, Jack Wilburn, Hilson Shrestha, Yiren Ding, Brian Bollen, Khandaker Abrar Nadib, Tingying He, Andrew McNutt, Alexander Lex

Paper

Overview of reVISit. The top row shows five stages of the experiment lifecycle and the technical features that support each one. Experiment Design: a DSL, Vega instrumentation, Python bindings and data tracking. Debug & Pilot: a study browser, participant simulator, data preview and participant replay. Data Collection: progress viewing and participant rejection. Analysis: data export and participant replay. Dissemination: deep-linked results, navigable stimuli, shareable data and forkable studies. A reVISit bar runs across the middle. Below it are two sociotechnical layers: Learning Materials (documentation, libraries, examples, reference studies) and Community Building (help forum, tutorials, replications). — ReVISit 2 supports each stage of the online user study life cycle. In addition to this linear path, the experiment life cycle ispopulated by internal loops between stages. For instance, issues revealed in the piloting of an experiment can lead back to the designphase to address those issues, and the process of disseminating an experiment can beget ideas for new experiments or replications.

Abstract

Online user studies of visualizations, visual encodings, and interaction techniques are ubiquitous in visualization research.Yet, designing, conducting, and analyzing studies effectively is still a major burden. Although various packages support such userstudies, most solutions address only facets of the experiment life cycle, make reproducibility difficult, or do not cater to nuanced studydesigns or interactions. We introduce reVISit 2, a software framework that supports visualization researchers at all stages of designingand conducting browser-based user studies. ReVISit supports researchers in the design, debug & pilot, data collection, analysis, anddissemination experiment phases by providing both technical affordances (such as replay of participant interactions) and sociotechnicalaids (such as a mindfully maintained community of support). It is a proven system that can be (and has been) used in publication-qualitystudies—which we demonstrate through a series of experimental replications. We reflect on the design of the system via interviewsand an analysis of its technical dimensions. Through this work, we seek to elevate the ease with which studies are conducted, improvethe reproducibility of studies within our community, and support the construction of advanced interactive studies.

BibTeX

@inproceedings{cutler2026revisit2,
author={Cutler, Zach and Wilburn, Jack and Shrestha, Hilson and Ding, Yiren and Bollen, Brian and Nadib, Khandaker Abrar and He, Tingying and McNutt, Andrew and Harrison, Lane and Lex, Alexander},
  journal={IEEE Transactions on Visualization and Computer Graphics}, 
  title={ReVISit 2: A Full Experiment Life Cycle User Study Framework}, 
  year={2026},
  volume={32},
  number={1},
  pages={13-23},
  keywords={Surveys;Reproducibility of results;Data visualization;Visualization;Libraries;Browsers;Videos;Software;Instruments;Graphical user interfaces;User studies;crowdsourcing;visualization experiments},
  doi={10.1109/TVCG.2025.3633896}
}