EvaluatAR: A Cross-Device Evaluation Framework for Rapid Prototyping of Bystander PETs in AR
Authors: Syed Ibrahim Mustafa Shah Bukhari (Virginia Tech), Matthew Corbett (Army Cyber Institute at West Point), Bo Ji (Virginia Tech), Brendan David-John (Virginia Tech)
Volume: 2026
Issue: 4
Pages: 947–963
DOI: https://doi.org/10.56553/popets-2026-0153
Artifact: Available, Functional
Abstract: Augmented Reality (AR) headsets continuously sense their surroundings, capturing nearby bystanders and raising privacy risks. Visual bystander privacy-enhancing technologies (PETs) mitigate this risk by detecting bystanders in egocentric scene views and applying privacy transformations (e.g., obfuscation). However, traditional PET evaluation is human-dependent, high-overhead, and device-specific, making it difficult to reproduce across devices. We present EvaluatAR, a cross-device evaluation framework for rapid prototyping at the early stage of PET evaluation. Our framework enables controlled replication of experimental conditions by standardizing PET inputs (sensor data and visual stimuli) and outputs through a record-replay workflow. We validate EvaluatAR through three case studies on HoloLens 2, Magic Leap 2, and Meta Quest 3 across implicit (continuous, context-driven) and explicit (intent-driven) PETs: (1) cross-device replay of inputs to a PET to reveal device-specific privacy-performance trade-offs; (2) generalizability of the same framework workflow across implicit and explicit PET design categories; and (3) replay of privacy-relevant edge cases to diagnose failures and validate PET modifications, yielding an improvement over the state-of-the-art baseline. These results demonstrate EvaluatAR's support for rapid, iterative PET development to advance reproducible cross-device evaluation of bystander PETs at a critical moment in the emergence of ubiquitous AR.
Keywords: Privacy-enhancing technologies (PETs), bystander privacy, evaluation framework
Copyright in PoPETs articles are held by their authors. This article is published under a Creative Commons Attribution 4.0 license.
