Traceability for Transparent and Scrutinizable Visualization Research

Although visualization researchers have acknowledged and embraced requirements for making their work reproducible, not all visualization research is inherently so.

In particular the design of visualization tools and systems requires some amount of subjectivity, making these design practices ill-suited to reproducibility and replication requirements.

Visualization researchers have instead proposed that these research studies be transparent and scrutinizable, but leave a gap in characterizing how to do so.Our proposed work fills this gap by establishing empirically based theories, pragmatic processes, and experimental tools for making visualization research transparent and scrutinizable.

We will center this work on the concept of traceability that is a step beyond transparency - it allows others to trace how artifacts, insights, contributions, and conclusions of a study came to be.

We will ground our research in the practices of visualization researchers in Sweden and the US, and build our tools on top of OSF and GitHub, standard research repositories used by scientists and engineers around the world.

Our proposed work has the potential to bring a more nuanced and human-centered approach to standards for making research more accessible, scrutinizable, and trustworthy.