Speculative Designs for Emergent Personal Data Trails: Signs, Signals and Signifiers

Towards tools for Understanding, Literacy and Informed Consent

26th April 2020

CHI 2020 Hawaii

Road signs.png

UPDATE: CHI 2020 is no longer running due to COVID-19. See "Accepted Papers" for an incredible collection of papers exploring specific emergent personal data trails.

Call for participation


With the rise of smart cities, the IoT, smart meters/grids and autonomous vehicles, people’s ordinary everyday actions leave increasingly intimate portraits of their habits and preferences. These personal data trails urgently necessitate the attention of designers who are tasked with acquainting users with how, where and when their personal data is collected, and the inferences that may be drawn from these traces.

This one-day workshop will bring together researchers and practitioners to develop strategies for designing around emergent personal data trails. The purpose is to share knowledge and engage participants in the design of conventions, tools and mechanisms for safely and informedly managing and navigating emergent and near-future personal data trails.

We invite submissions from a wide range of researchers who are: (a) Working broadly in the space(s) of smart cities, smart homes, present/future energy systems, conservation, IoT, computer vision, autonomous vehicles or related research areas who are keen to engage (or are already engaged) in the data trails and privacy/consent/surveillance aspects of these topics -OR - (b) Working broadly on informed consent, privacy, surveillance, personal data management of existing data trails in an unrelated field who are keen to engage (or are already engaged) smart city/smart home/smart energy futures.

We invite visual responses to the call. Prospective workshop participants are encouraged to submit a 2-8 page pictorial that outlines a position around a present or future data trail(s) broadly relevant to smart cities. Pictorials must be submitted in SIGCHI Extended Abstracts format, as a PDF less than 15MB. Submit your paper by email to Steve Snow- s.snow@uq.edu.au by 11:59pm PST 18/02/2020.

Stock Market Down


Stephen Snow
Research Fellow,
University of Queensland

Steve's research involves mixed methods research into privacy and consent aspects of future energy systems, the user-centred design of visualisations for improving energy awareness and consumer/network relations. 

Awais Hameed Khan
Design Researcher,
University of Queensland 

Awais specialises in design futures (Design Fiction, Speculative Design, Critical Design) and has worked in Brand Management for Unilever before moving to the University of Queensland. 

Stephen Viller
Associate Professor,
University of Queensland

Stephen Viller is a researcher and educator in people centred design methods, particularly applied to the design of social, domestic and mobile computing technologies and understanding people in their everyday settings. He has over 20 years’ experience in the fields of Computer Supported Cooperative Work (CSCW), Interaction Design, and Human-Computer Interaction (HCI).

Ben Matthews
Senior Lecturer,
University of Queensland

Ben Matthews researches the human and social aspects of designing technologies. He has worked in a range of design domains with various industry partners: audiology (Oticon), diabetes care (Novo Nordisk), domestic Internet of Things devices, industrial components (Danfoss), passport processing (DFAT), remote mental health services, toys and play (Lego Group, Kompan).

Ewa Luger
Chancellor's Fellow in Digital Arts and Humanities,
University of Edinburgh

Ewa is a Chancellor’s Fellow in Digital Arts and Humanities, a Fellow of the Alan Turing Institute (Ethical AI by Design: Formalising an HCI agenda), a consulting researcher at Microsoft Research UK (AI and Ethics), and the Research Excellence Framework (REF) co-ordinator for Design. Her work explores applied ethical issues within the sphere of machine intelligence and data-driven systems including ethical AI and consent.

James Pierce
Assistant Prof. at California College of the Arts,
Researcher at UC Berkeley

Pierce has longstanding research interests in speculative design, design theory, sustainable design, and everyday social practices. His current research investigates issues of privacy, cybersecurity, trust, and fairness with emerging interactive, connected, and artificially intelligent technologies. He has previously worked at the Palo Alto Research Centre investigating smart home and sustainable energy.

Richard Gomer
Product Director, Consentua. Researcher,
University of Southampton.

Richard leads the vision and development of consent management platform, Consentua, helping clients build trustworthy relationships with their customers and comply with their data protection obligations. At the University of Southampton, Richard's research focuses on Privacy, Data Protection, Consent, Values-by-Design and the implications of bounded rationality on design, and building links with policy makers, politicians and government.

Dorota Filipczuk
PhD Candidate, Informed Consent
University of Southampton

A former Google software intern, Dorota researches consent mechanisms for mobile applications, including consent mechaisms and game design to improve cyber security literacy within organisation.

Scott Heiner
PhD Candidate,
University of Queensland

Scott's research explores provocations on how users could regain control of their personal data, as well as formal methods applied to concurrent hybrid software.


09:00 - 09:15

Opening and Introduction

Welcome to participants, introduction of organisers, introduce purpose of workshop, and justify clustering of presentation topics.
09:15 - 10:45


Authors present their work in topic clustered groups, with five minutes per participant. Other participants have post-it notes and butchers paper to annotate.
10:45 - 11:15

Coffee Break

11:15 - 12:15

Discussion and Organisation

Key findings with regard to management, navigation and informed consent of personal data trails are distilled, and three working groups are synergised.
12:15 - 13:15


(Participants encouraged to attend informal lunch at Convention Centre's rooftop.)
13:15 - 14:15

Journey Mapping

Participants map out context-specific journeys, and possible attack surfaces of data leakage, risk or consequence. This will include a visual mapping outlining these journeys. These will be presented to the group.
14:15 - 15:45

Signs, Signals and Signifiers

Based on the above journeys, attack surfaces, and user information, signs, signals and signifiers are designed with the intention to acquaint users with these emergent data trails.
15:45 - 16:05

Coffee Break

16:05 - 17:15

Signs, Signals and Signifiers (Part 2)

Comparing and translating individual group signs, signals and signifier mock-ups, participants will discuss overlaps, redundancies and converging ideas.
17:15 - 17:30


After facilitators wrap up, further work and the development of a community of practice will be discussed. Opportunities for further collaboration will be highlighted.

For more detailed information about the schedule of events, click here.

Accepted papers

The list of currently accepted papers is below- full papers are hyperlinked to each title.

Visualizing Data Trails: Metaphors and a Symbolic Language for Interfaces

Omar Sosa-Tzec, University of Michigan,



This paper starts by presenting four mobile interface design concepts to make personal data trails visible.
Frosted screen, rainbow heatmap, hungry zombie, and data current are the labels given to these concepts.
After reflecting on these concepts, the paper focuses on some elements of personal data trails, and explores a visual system of icons to indicate the user the possible use and abuse of the data they produce during the UX. The elements proposed by this paper are surveillance, commodification, data aggregation, data input, affect and arousal, preferences, and community. 

Investigating Pathogen Trails As A Design Strategy to Combat Invisible Health Dangers in Everyday Environments

Ayanna Seals New York University
Odded Nov, New York University
Jennifer Otiono, Wellesley College, jotiono@wellesley.edu
Orit Shaer, Wellesley College
Mad Ball, Open Humans Foundation


An individual’s living environment has the ability to serve as both an aid and a hindrance to their health management. Addressing the environment as a strategy of care can become increasingly difficult when concentrating on the "invisible living environment", a space inhabited by microbiota such as harmful bacteria, archaea, protists, fungi and viruses. In the case of infectious diseases, designing solutions for this may be assisted in the study of the transmission paths of the pathogens involved in the process, while incorporating location points of risk mitigation involving human health behavior change. Our work notes existing data on transmission and proposes a solution to explore in future work. Implications of the use of the bacterial transmission paths are suggested to be of interest to researchers and designers involved in the development of related health solutions.

On How People Navigate Through Their Personal Web of Things

Dhivya Eswaran, Carnegie Mellon, deswaran@cs.cmu.edu

Adam Fourney, Microsoft Research

Paul Bennett, Microsoft Research

Shamsi Iqbal, Microsoft Research

Shane Williams, Microsoft Research

In a digital setting, people accomplish their goals by interacting with multiple files, web pages, emails, and applications – each of which often relates to multiple projects and people. This leads to a rich interconnected personal web of things, comprising heterogeneous entities and information interaction trails. In this work, we seek to understand a single dimension of the personal web in work-related settings, namely, how people transition between activities. To this end, we develop an instrumentation platform to log all activities that a person performs across all desktop applications (e.g. web browsers, editors and email clients). With the goal of understanding how people’s transition across activities relate to their productivity, we conduct a study involving ten participants to collect logs of activities performed by them on their primary work devices over a 4-5 week duration. Our analysis of this data provides an initial insight into people’s behavior in this space, which envelops personal corpora (e.g. emails, documents), public resources accessed (e.g. webpages), and activities undertaken across multiple applications. We conclude by outlining how our observations can be leveraged to provide better productivity support, and more broadly, the rich set of technical questions and challenges the personal web setting poses for exciting future research.


Personal health data trails in the context of stigmatizing long-term conditions 

CarolineClaisse, University of Northumbria, caroline.claisse@northumbria.ac.uk

Abigail C. Durrant, Northumbria University

Ewa Luger, The University of Edinburgh

Jon Bird, Bristol University

Spencer Warren, Bristol University

Connor O'Rourke, Bristol University

Lynne Coventry, Northumbria University

Elizabeth Sillence, Northumbria University

Emma Simpson, Northumbria University

Karen Lloyd, University College of London

Jo Gibbs, University College of London

Shema Tariq, University College of London

Bakita Kasadha, Terrence Higgins Trust

Simone Stumpf, City, University of London


In this position paper, we consider personal data trails in the context of critically exploring technology design
for supporting the self-management of stigmatizing long-term conditions like Human Immunodeficiency
Virus (HIV). Drawing on developing discourses in Critical Digital Health and Personal Health Informatics, we frame pertinent research questions for critically exploring the design of trusted systems and mechanisms for securely managing and navigating emergent and near-future personal data trails in health and beyond.

Human-Data Entanglements in the Home: Five Speculative Sketches

Audrey Desjardins, University of Washington, adesjard@uw.edu


In this position pictorial, I present 5 speculative concepts for engaging with IoT data on a personal, situated, and imaginative level. These concepts purposefully use alternative modes of representing home IoT data as an approach to open up new possibilities when thinking about human-data entanglements. The five speculative sketches are the result of a 9-month long design anthropology inquiry which examines the
data byproducts of Internet of Things technologies in the home.

superreflective data: speculative imaginings

MaxVan-Kleek, University of Oxford, max.van.kleek@cs.ox.ac.uk


It's the year 2020, and every space and place on- and off-line has been augmented with digital things that observe, record, transmit, and compute, for the purposes of recording endless data traces of what is happening in the world. Individually, these things (and the invisible services the power them) have reached considerable sophistication in their ability to analyse and dissect such observations, turning streams of audio and video into informative parcels–data fragments containing spoken utterances, activity data, and
moment-to-moment minutiae about the world and its inhabitants. For this workshop, we propose two hypothetical mini scenarios different from our current digital world. In the former, instead of hoarding it, data controllers turn captured data over to those who need it as quickly as possible, working together to combine, validate, and refine it for maximum usefulness. This simultaneously addresses the data fragmentation and privacy problem, by handing over long-term data governance to those that value it the most. In the latter, we discuss ethical dilemmas using the long-term use of such rich data and its tendency to cause people to relentlessly optimise.

The Politics of Emergent Data and the Implications for Identity: Position Paper

Anh-Ton Tran, Georgia Institute of Technology, anhton@gatech.edu


This paper describes the author’s position on personal data. The author describes two research areas they are engaged with, their study of the political in data representation and their aesthetic inquiry into identity and tool transparency. This paper then converges these two research themes into how emergent data trails from the smart home and IoT may mediate how identity is understood when self-generated data is interpreted and transformed into an artifact.

Conscious data trails about my – and my family – relationship with energy​

Inmaculada Rodriguez, University of Barcelona, inmarodriguez@ub.edu

A. Puig, University of Barcelona

D. Tellols, University of Barcelona

K. Samsó, University of Barcelona


Energy is needed for almost everyday task, from waking up and having breakfast in the morning, to working at the office, and to having fun. This vast use of energy encourages the development of applications for sustainable behaviour. In this research we bet for digital Cultural Probes (CP) – a well-known HCI technique used to inform and inspire applications design – to get (conscious) data trails from users and so design smart cities facilities and tools for energy conservation and sustainable consumption. Participants of the cultural probes are children who develop tasks to provide designers with information about families’ habits of energy consumption and their knowledge and interest in Smart Grid related issues. While performing the CP we engaged the children in performing CP tasks with the mission of saving people wasting energy crazily, under the influence of a devil wizard.

Building BROOK: A Multi-modal and Facial Video Database for Human-Vehicle Interaction Research

Xiangjun Peng, University of Nottingham (China), zy22056@nottingham.edu.cn

Zhentao Huang, University of Nottingham (China)   

Xu Sun, University of Nottingham (China)    



With the growing popularity of Autonomous Vehicles, more opportunities have bloomed in the context of Human-Vehicle Interactions. However, the lack of comprehensive and concrete database support for such specific use case limits relevant studies in the whole design spaces. In this paper, we present our work-in-progress BROOK, a public multi-modal database with facial video records, which could be used to  haracterise drivers’ affective states and driving styles. We first explain how we over-engineer such database in details, and what we have gained through a ten-month study. Then we showcase a Neural Network-
based predictor, leveraging BROOK, which supports multi-modal prediction (including physiological data of heart rate and skin conductance and driving status data of speed) through facial videos. Finally we discuss related issues when building such a database and our future directions in the context of BROOK.

The Venue

1801 Kalakaua Ave, Honolulu, HI 96815, United States

From April 25th to 30th, CHI will take place in beautiful Honolulu, on the island of Oahu, Hawaiʻi, USA.