October 2023 | IEEE Visualization and Visual Analytics (VIS)

reVISit: Supporting Scalable Evaluation of Interactive Visualizations

Yiren Ding, Jack Wilburn, Hilson Shrestha, Akim Ndlovu, Kiran Gadhave, Carolina Nobre, Alexander Lex, Lane T. Harrison
Paper
The workflow of generating a study with reVISit. First, an experimenter generates a (1) study specification describing the study using the reVISit DSL. They also create (2) study stimuli and components such as consent forms and trainings. Stimuli and components can be specified as HTML pages, markdown, static images, React components, etc. The specification and the study components are then compiled to an (3) interactive web app, which can be deployed to the web. The web app includes an admin interface for quickly browsing and debugging a study. Data from the study can either be (4) downloaded at the end of a trial, or stored on a server. Collected data can be (5) analyzed with external software such as R and SPSS, or examined and analyzed using the reVISit analytics interface

Abstract

reVISit is an open-source software toolkit and framework for creating, deploying, and monitoring empirical visualization studies. Running a quality empirical study in visualization can be demanding and resource-intensive, requiring substantial time, cost, and technical expertise from the research team. These challenges are amplified as research norms trend towards more complex and rigorous study methodologies, alongside a growing need to evaluate more complex interactive visualizations. reVISit aims to ameliorate these challenges by introducing a domain-specific language for study set-up, and a series of software components, such as UI elements, behavior provenance, and an experiment monitoring and management interface. Together with interactive or static stimuli provided by the experimenter, these are compiled to a ready-to-deploy web-based experiment. We demonstrate reVISit’s functionality by re-implementing two studies — a graphical perception task and a more complex, interactive study. reVISit is an open-source community project, available at https://revisit.dev/.

BibTeX

@article{ding2023revisit,
  title={reVISit: Supporting Scalable Evaluation of Interactive Visualizations},
  author={Ding, Yiren and Wilburn, Jack and Shrestha, Hilson and Ndlovu, Akim and Gadhave, Kiran and Nobre, Carolina and Lex, Alexander and Harrison, Lane},
  year={2023},
  publisher={IEEE Visualization and Visual Analytics (VIS)}
}
Preview Video
Presentation Video