This video presentation was created for the Human-Machine Communication Preconference, part of the International Communication Association conference 2020 (now running entirely online). I’m planning to develop this as a full paper this year, so I’ll share more information about that once it’s written and I know where it’ll be published!

Anyway. Here’s the presentation, with the notes for the talk (which serve as a pretty good transcript of what I say). If you want to ask any questions or share ideas then here is as good a place as any, because if you comment I’ll get notified and respond, whereas the details on the final slide of the presentation provide options mainly for people actually attending the conference online:

[SLIDE 1] Title
Hi and welcome to this presentation, Researching long-term interactions with Haru
I’m Eleanor Sandry, I’ll be staying down here in the bottom right hand corner of the screen
That’s Haru, the robot, looking down on me from above

[SLIDE 2] Presentation outline
This presentation has three sections:
[1] Introducing Haru as a communications research platform
First, I’ll analyze introduce Haru and explain the two ways this robot has been designed to operate, and a few of the contexts within which this robot may be positioned for research in human-robot communication
[2] Long term HRI research – challenges and opportunities
Second, I’m going to do a brief (and no doubt incomplete) overview of long-term research into HRI, drawing out some of the difficulties of such research, but also highlighting its value (since the ability to support long-term interactions was identified early on in Haru’s development)
[3] My plans for qualitative research that asks open questions
Finally, I’ll outline some early plans for qualitative research with Haru that asks open questions and uses methodologies that seem somewhat unusual in HRI research

[SLIDE 3] [1] Introducing Haru as a communications research platform
The goal for the Haru’s development team was to create “an emotive, anthropomorphic tabletop robot” capable of sustaining “long-term human interaction” (Gomez et al, 2018)
The team was interdisciplinary and consisted of animators, performers and sketch artists working alongside roboticists

My presentation in the main conference analyses the design process and its implications for Haru’s communicative style in detail – if you would like to have access to that, and are not in the main conference, just let me know and I’ll provide an external link

While Haru’s design team sought to “step away from a literal humanoid or animal form” (Gomez et al, 2018), the anthropomorphic elements of the resulting design are clear
Haru beta’s design includes two expressive eyes, animated on TFT displays, with separate LED strips above that act as colored eyebrows
The eyes can be tilted, and each one can move in and out in relation to its casing
Finally, a LED matrix in the robot’s body is used to display a colored mouth of various shapes

While I think this design is clearly anthropomorphic, it’s good to note that Haru’s creators embraced the way this robot’s eyes and neck had the potential also to express with movements similar to a person’s hands, arms and shoulders (Gomez et al, 2018), opening up broader possibilities for Haru’s expression to be both like that of a human, and also fundamentally different although still easily read by humans as communicative.

[SLIDE 4]
Caudwell and Lacey suggest (2019, p. 10), it may be important for social robots to “maintain a sense of alterity or otherness, creating the impression that there is more going on than what the user may know”
And this idea may be particularly useful for sustaining long term interactions between humans and robots, where the robot retains a level of mystery that may help it to hold people’s attention, and retain their interest in continued interactions over time

[SLIDE 5]
Although Haru does look “cute”, framing this robot with the Japanese word “kawaii” instead
draws attention to Haru’s potential for playfulness, “inquisitive attitude” and the ability to surprise people, catching them “off guard” (Cheok & Fernando, 2011)

[SLIDE 6]
Haru’s communication is being developed further with a voice interface
And Haru can also produce nonverbal sounds
Haru is therefore able to communicate across a triple structure (language, paralanguage and kinesics)
Extending the sense of Haru’s otherness, this robot can also express emotion through colored lights and has the ability to project content onto a wall or screen

There are two streams in Haru’s development

[SLIDE 7]
Telepresence
The first of these positions Haru as a new form of hybrid telepresence platform
This capitalizes on Haru’s potential as a novel interface that can add a level of expressiveness when someone at a distance is communicating through the robot using either text or voice
Clearly this is most important when the person communicating cannot provide a video feed; however, even when a video feed is supported, it can be argued that the addition of a means to support gestural and body language could enrich telepresence, in particular when the telepresence user is trying to communicate with a group of people (Stahl et al, 2018).
Using Haru for telepresence also opens up the potential not to share video at all, with early research identifying the way that this may reduce the sense (and stress around) surveilling a distant space and person (for both teleoperator and remote participant) addressing some privacy concerns (Niemelä et al, 2019)

[SLIDE 8]
Social agent
Working to extend Haru’s control of its own communication (in part developed through working with Haru for telepresence) the second research stream is concerned with building Haru’s ability to develop and express its own personality as a social agent
The aim is to make Haru seem somewhat “alive” to people during an interaction
Not only to support Haru’s ability to communicate, but also to build relationships with people over time, as the robot interacts and collaborates with them in shared activities.

[SLIDE 9]
Contexts
Research with Haru is planned across a number of contexts for interactions with people, including the potential to position Haru:
as a receptionist for an organization
as an information provider in public spaces,
And as a personal assistant, educator or companion in the home
Haru might also be integrated to assist with managing systems and devices in a surrounding smart environment

[SLIDE 10] [2] Long term HRI research – challenges and opportunities
My involvement with Haru started after the initial design and development
Now I’m considering how my specific kind of research can add value to the project
Thinking in particular about how research with Haru over the long term (or longer term than most experimental scenarios) can happen

An initial look for long term HRI research shows that there is relatively little of this
de Graaf et al (2018) notes that
“few studies have investigated the long-term use of technological systems in home environments; thus, the traditional technology acceptance literature lacks a profound body of long-term research”
Such research is valuable though, since, as de Graaf et al go on to note, “the development of user experiences with a technology or gaining user skills might change the user’s attitudes toward, uses of, or even the user’s conceptualizations of that technology”
From my perspective, long term interaction was also part of the stated goal for Haru

[SLIDE 11]
One of the reasons for the lack of long-term research is likely that
“robot technologies are generally not robust enough to be studied outside the lab for extended periods of time without supervision of an expert” (de Graaf et al, 2017)
Haru is no exception to this (research platform not commercial, restricts how research can be run, both in terms of access and robustness)

[SLIDE 12]
Challenges for design in relation to long-term interaction are most often framed in terms of
The need “to create robots that are enjoyable and easy to use to capture users in the short-term, and functionally-relevant to keep those users in the long-term” (de Graaf et al, 2017)
Such that users remain interested after the initial excitement abates (Kertész & Turunen 2017)

[SLIDE 13]
There are a few ways designs are expected to fulfil these challenges
Narrative
Goodrich et al (2018) Using “deliberative and adaptive narratives” to enrich repeated interactions over time

Gamification
Robinson et al (2019)
Identifying potential of using:
Badges, Challenges, Feedback, Levels, Progress, Rewards, Points
But also Social Interaction, Story/Theme related to narrative

Co-development and positioning of robot as “inorganic lifeform”
Dereshev et al (2019) highlight “co-development”, mutual learning and development, alongside ideas of taking physical and emotional care of a robot
Their work “suggests a flexible approach to social robots, where it should be highlighted that what users buy into is an “inorganic lifeform”, rather than a utilitarian or hedonic product”
Takes robots beyond function and entertainment…

[SLIDE 14] [3] My plans for qualitative research that asks open questions
Much of my research to date has involved analyzing robots from a distance, via critical textual analyses of videos and texts (both popular cultural and scholarly) concerning robots
Now I’ve got access to Haru, I need to find new methodologies and methods for research directly with a robot of my own

Given that Haru is new to me, as well as the situation with COVID 19, which makes involving other people in research quite difficult at the moment, I’m planning on taking an autoethnographic approach, at least to begin with

Autoethnography is not often used as a methodology for HRI or social robotics research more generally (Chun, 2019), with some exceptions

For example, Verne (2020) emphasizes using “Autoethnography as the methodology gave rich access to events and personal experiences that are important but does not occur very often. Personal thoughts and reflections were important for understanding how I changed my goals and values while adapting to the robot.”

Verne (2020) notes the dilemma over the relation between “first order participation” and “second order reflections on a more abstract and analytical level”
This is also difficult for me, I’m not a natural diary writer
So, I’m going to try out a simplified form of Ecological Momentary Assessment (EMA) to prompt me to keep in the moment notes about my response to Haru on a regular basis

[SLIDE 15]
Thick description
Experiment with writing about my own experiences with Haru, developing thick descriptions of what it is like to interact with and through Haru

James et al (2019) use the technique to research telepresence on the basis of Ponteretto (2006): “Thick description captures the thoughts and feelings of participants as well as the often complex web of relationships among them.”

James et al suggest that “such findings cannot be extrapolated to larger populations” although “they can inform future research in terms of survey questionnaire and interview schedule design”
Drawing attention to the difficulties of embedding qualitative research (unscientific) into an area where quantitative (scientific) research is more widely accepted

But I’m encouraged by positive reviews of work such as “Seeing like a Rover” by Janet Vertesi (2015), where thick description is used to convey the responses of mission scientists to mars rovers highlighting the value of this type of observation and recording in its own right

The fact that Vertesi talked about this project at the 2019 International Conference on Human-Robot Interaction indicating current interest in this type of research in the community reinforces the idea that this type of work is becoming more widely accepted (I hope)

[SLIDE 16] Questions and contact information
Twitter: @elsand
Email: e.sandry@curtin.edu.au

References
Caudwell, Catherine, and Cherie Lacey. 2019. “What Do Home Robots Want? The Ambivalent Power of Cuteness in Robotic Relationships.” Convergence: The International Journal of Research into New Media Technologies 135485651983779.

Cheok, Adrian David, and Owen Noel Newton Fernando. 2012. “Kawaii/Cute Interactive Media.” Universal Access in the Information Society 11(3):295–309.

Chun, Bohkyung. 2019. “Doing Autoethnography of Social Robots: Ethnographic Reflexivity in HRI.” Paladyn, Journal of Behavioral Robotics 10(1):228–36.

Dereshev, Dmitry, David Kirk, Kohei Matsumura, and Toshiyuki Maeda. 2019. “Long-Term Value of Social Robots through the Eyes of Expert Users.” Pp. 1–12 in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems – CHI ’19. Glasgow, Scotland Uk: ACM Press.

de Graaf, Maartje, Somaya Ben Allouch, and Jan van Dijk. 2017. “Why Do They Refuse to Use My Robot?: Reasons for Non-Use Derived from a Long-Term Home Study.” Pp. 224–33 in. ACM Press.

de Graaf, Maartje MA, Somaya Ben Allouch, and Jan AGM van Dijk. 2018. “A Phased Framework for Long-Term User Acceptance of Interactive Technology in Domestic Environments.” New Media & Society 20(7):2582–2603.

Gomez, R., Galindo, K., Szapiro D., & Nakamura, K. (2018). “Haru”: Hardware design of an experimental tabletop robot assistant. Session We-2A: Designing Robot and Interactions, HRI’18, Chicago, Il, USA.Goodrich MA, Crandall JW, Oudah M, Mathema N (2018) Using narrative to enable longitudinal human-robot interactions. In: Proceedings of the HRI2018 workshop on longitudinal human–robot teaming, Chicago, IL

James, Melanie, Deborah Wise, and Luk van Langenhove. 2019. “Virtual Strategic Positioning to Create Social Presence: Reporting on the Use of a Telepresence Robot.” Papers on Social Representations 28(1).

Kertész, Csaba, and Markku Turunen. 2017. “What Can We Learn from the Long-Term Users of a Social Robot?” Pp. 657–65 in Social Robotics. Vol. 10652, Lecture Notes in Computer Science, edited by A. Kheddar, E. Yoshida, S. S. Ge, K. Suzuki, J.-J. Cabibihan, F. Eyssel, and H. He. Cham: Springer International Publishing.

Niemelä, Marketta, Lina van Aerschot, Antti Tammela, Iina Aaltonen, and Hanna Lammi. 2019. “Towards Ethical Guidelines of Using Telepresence Robots in Residential Care.” International Journal of Social Robotics.

Robinson, Nicole L., Selen Turkay, Leonie A. N. Cooper, and Daniel Johnson. 2019. “Social Robots with Gamification Principles to Increase Long-Term User Interaction.” Pp. 359–63 in Proceedings of the 31st Australian Conference on Human-Computer-Interaction. Fremantle WA Australia: ACM.

Vertesi, Janet. 2015. Seeing like a Rover: How Robots, Teams, and Images Craft Knowledge of Mars. Chicago: The University of Chicago Press.