This video presentation was created for the International Communication Association conference 2020 (now running entirely online). It formed part of a panel organised by the Human-Machine Communication Interest Group, Social Robots in Interpersonal Relationships and Education. I’m planning to develop this as a full paper this year, so I’ll share more information about that once it’s written and I know where it’ll be published!

There’s a full paper that sits behind this presentation (which, because it is only around 12 minutes long, couldn’t cover all the material in that paper). The paper is being developed as an open access article in Frontiers in Robotics and AI – Human-Robot Interaction. I’ll post about that once it’s written, reviewed and published!

Here’s the presentation, with the notes for the talk (which serve as a pretty good transcript of what I say). If you want to ask any questions or share ideas then here is as good a place as any, because if you comment I’ll get notified and respond, whereas the details on the final slide of the presentation provide options mainly for people actually attending the conference online:

[SLIDE 1] Title
Hi and welcome to this presentation for the paper, Communicating with Haru
I’m Eleanor Sandry, down in the bottom right hand corner of the screen
And that’s Haru, the robot, looking down on me from above
I wasn’t involved in the initial design of Haru, but became a member of the Socially Intelligent Robotics Consortium around a year ago
My coauthors are from Honda Research Institute Japan, where Haru was designed and built

[SLIDE 2] Presentation outline
This presentation has three sections:
[1] Haru’s communication as shaped by a design process
First, I’ll analyze the development of Haru beta, highlighting how the design process privileged particular ideas about communication
[2] Alternative ways to understand Haru as a communicator
Second, I’ll explore alternative understandings of Haru as a communicator
[3] Continuing development and research with Haru
Finally, I’ll outline how perspectives on communication suggest ways to continue Haru’s development and could also frame future research with this robot

[SLIDE 3] [1] Haru’s communication as shaped by a design process
Haru’s development team followed a customized Design Thinking process to coordinate contributions from an interdisciplinary team of animators, performers and sketch artists working alongside roboticists
I say customised, because typically Design Thinking processes begin with an Empathize stage and then moves on to Define the problem (or problems) a design seeks to address (or needs to take into account)

Define
However, for Haru, destined to be a flexible communications research platform, it made sense to pre-define the overarching aim of creating “an emotive, anthropomorphic tabletop robot” capable of sustaining “long-term human interaction” (Gomez et al, 2018)
Animators in the team then used their skills in making inanimate objects come “alive” to produce sketches of various ways that Haru’s design might achieve this goal
For Haru, as for animated characters from a number of popular films, these sketches emphasize how giving objects faces and making them bend and twist in ways impossible for those objects in the physical world, creates animated characterizations that can be emotionally expressive in very humanlike ways

[SLIDE 4] Adding sociopsychological perspective
As well as supporting an anthropomorphic design path, the process of considering particular animation styles and techniques for normally inanimate objects in films and cartoons also shapes this robot’s communication in strongly sociopsychological terms
The sociopsychological tradition of communication theory regards communication as a form of information transfer, within which the aim of the sender of a message is to persuade the receiver of something (Craig, 1999)
In terms of robot design, this can be linked with the development of robots that can express emotions and are therefore likely to draw people into interactions often by being “cute” as seen with Kismet (Turkle et al, 2004) and Jibo (Caudwell & Lacey, 2019)
It’s easy to see how Haru could also convey a cute personality

Having come up with an overall concept for Haru, supported by a high-level goal and set of sketches showing Haru as an animated character, the team moved on to consider how Haru’s nonverbal communication of emotion would work in more detail

Empathize
In the Empathize stage of the design process, Haru’s design team worked with a set of volunteers
The volunteers were shown the sketches of Haru from the Define stage and were then asked to use a combination of body language and facial expression to act out particular emotions as they would themselves, and also as they imagined Haru would (Gomez et al, 2018).
Unlike the Empathize phase of most Design Thinking processes, where designers empathize with users, for Haru it was more a case of asking users to try to empathize with the robot, something seen as important for design, but also key to Haru’s ongoing success.

This process can be linked with François Delsarte’s method for acting, which relies on a performer’s ability to code emotions into readily recognizable nonverbal facial and bodily expressions that precisely communicate specific emotional responses

[SLIDE 5] Adding cybernetic-semiotic perspective
In terms of communication theory, such an approach not only fits well against the persuasive, sociopsychological perspective discussed above, but also supports the way this style of emotional expression can be part of cybernetic-semiotic exchanges where meaning is coded in intersubjective (here across human and robot) language or other signs (theoretical structure from Craig (1999), developed in Sandry (2015)).

[SLIDE 6] Resulting Haru beta design
While Haru’s design team sought to “step away from a literal humanoid or animal form” (Gomez et al, 2018), the anthropomorphic elements of the resulting design are clear
Haru beta’s design includes two expressive eyes, animated on TFT displays, with separate LED strips above that act as colored eyebrows
The eyes can be tilted, and each one can move in and out in relation to its casing
Finally, a LED matrix in the robot’s body is used to display a colored mouth of various shapes

While it does make a great deal of practical sense to design a robot with sociopsychological and cybernetic-semiotic communication success in mind, with the aim of creating a robot that is compellingly cute, familiar and easy for humans to interact and communicate with…

[SLIDE 7] [2] Alternative ways to understand Haru as a communicator
…the phenomenological perspective on communication provides a reminder that any attempt to know, or to understand the other fully, is fraught with difficulty: the other’s difference from the self is a chasm that cannot be bridged by an empathetic stance (Levinas, 1969; Craig, 1999; Pinchevski, 2005)
This raises questions not only in relation to understanding the communication of a robot such as Haru, but also for the design process discussed above, which relies on empathizing with users as well as relying on their ability to empathize with Haru and act out how this robot might express a particular emotion
It is therefore good to note that Haru’s creators embraced the way this robot’s eyes and neck had the potential also to express with movements similar to a person’s hands, arms and shoulders (Gomez et al, 2018)
This opens up broader possibilities for Haru’s expression to be both like that of a human, and also fundamentally different (given its very different form and potential to express in non-humanlike ways that can still be easily read by humans as communicative).

[SLIDE 8]
The phenomenological perspective not only highlights the difficulties of making a robot with non-humanlike form express in humanlike ways, but also introduces the idea that a robot other’s difference is an integral part of why one might communicate with it as opposed to a problem that must be overcome (Sandry, 2015)

[SLIDE 9]
Caudwell and Lacey suggest (2019, p. 10), it may well be important for social robots to “maintain a sense of alterity or otherness, creating the impression that there is more going on than what the user may know”
Keeping a sense of mystery

[SLIDE 10]
“This sense of alterity” is one way to break down “the strict power differential that is initially established by their cute aesthetic”

It makes a robot potentially more interesting to communicate with (in particular in the long term), more than a compliant communicator always focused on responding to humans; instead, such a robot can be recognized as having the potential to act on its own, provide information or call for human attention and response as required

[SLIDE 11]
Kawaii
As part of the recognition of otherness, rather than framing Haru as “cute” it may be more productive to adopt the Japanese term, “kawaii”
While this term is often translated as, or at least closely associated with, the English word “cute” and its meaning (Cheok & Fernando, 2011), more correctly describing something as kawaii draws attention to a potential for playfulness, “an inquisitive attitude” and the ability to surprise people to catch them “off guard” (Cheok & Fernando, 2011)
The idea of Haru’s ability to surprise people resonates with the importance of “interruption” in phenomenological perspectives on encounters between selves and others where difference plays a key role (Pinchevski, 2005; Sandry, 2015)
As opposed to being non-threateningly familiar, Haru thus has the potential to be quirky and unusual, drawing people’s attention and inviting their participation in continued communication

[SLIDE 12]
Sociocultural perspective
Defining Haru as kawaii is complicated by the fact that different people, and even the same person across changing circumstances, may or may not choose to appraise the same object as kawaii (Nittono, 2016)
The shifting attribution of the term kawaii depending on the preference of individual people and changes in context raises the importance of considering a sociocultural perspective on communication in human interactions with Haru
This perspective analyses communication as a means of producing, reproducing, and negotiating shared understandings of the world (Carey, 1992; Craig, 1999), heavily reliant on the overarching cultural setting as well as the detailed context of an interaction between particular individuals

[SLIDE 13] [3] Continuing development and research with Haru
A consideration of sociopsychological, cybernetic-semiotic, phenomenological and sociocultural perspectives on interactions with Haru suggests that it is useful to adopt a more dynamic understanding of communication overall, which could be important across design and prototyping contexts, as well as in planning user studies

From a dynamic systems perspective, communication is not about the transmission or exchange of fixed pieces of information, because, as Alan Fogel argues, “information is created in the process of communication”, such that “meaning making is the outcome of a finite process of engagement” (2006, p. 14)
This shifts a cybernetic-semiotic focus from a preoccupation with clear and precise messages, to considering the value of iterative exchanges of feedback and response through which meaning emerges
It also reinforces the idea that sociopsychological persuasion may rely not so much on any fixed perception of “cuteness” or even “kawaii”, but rather on a personality that develops and changes within and between interactions, dependent on context and individuals involved

In particular, although Delstarte’s idea of coding emotions for performance is a practical part of the design process discussed in above, considering how emotional communication can emerge through dynamic interchanges highlights the potential for an alternative acting paradigm to play a part
This alternative view is typified by the Stanislavski technique, within which performers are expected to coordinate with one another in the moment of interaction, behaving in ways shaped as reactions or responses to other performers (Moore, 1960; Hoffman, 2007)
When human-robot interactions are considered from this perspective, the precise coding of emotion or of information becomes less important than the ability of the robot to respond to changes in its environment as well as to the particular person with which it is currently engaged in interaction

[SLIDE 14]
A focus on dynamic communication, draws attention to the potential for nonverbal, embodied communication to support exchanges that are not restricted by turn-taking, but rather become continuous processes “within which signs can overlap even as they are produced by the participants” (Sandry, 2015, p. 69)
Not seen in this interaction with Haru playing rock, paper scissors – heavily reliant on turn-taking – but to be a focus in the future

[SLIDE 15]
Alongside this, while development of Haru beta concentrated on designing the robot’s body to allow expressive emotional communication, this robot will also need to communicate flexibly in a range of other ways if it is to fulfil the goal of being a platform to support human-robot communication research more fully
The continuing development of Haru adopts a broad understanding of what constitutes communication as “a triple audiovisual reality” (Poyatos, 1997) using verbal language (speech itself), paralanguage (tone of voice, nonverbal voice modifiers, and sounds), and kinesics (eye, face, and body movements) from human communication, extended in Haru using nonhuman communication channels, such as harumoji (on smart devices) and the projection of content onto nearby surfaces

[SLIDE 16]
The full paper goes into the various contexts within which Haru is likely to be situated for research, from my perspective I have been thinking what work I can contribute, in particular towards the idea of investigating responses to Haru in the long term
Autoethnographic/autohermeneutic research
Then moving into user experience studies extending this

[SLIDE 17] Questions and contact information
Twitter: @elsand
Email: e.sandry@curtin.edu.au

References
Carey, J. (1992). Communication as culture: Essays on media and society. New York: Routledge.

Caudwell, C., & Lacey, C. (2019). What do home robots want? The ambivalent power of cuteness in robotic relationships. Convergence: The International Journal of Research into New Media Technologies, 135485651983779. https://doi.org/10.1177/1354856519837792Craig, R. T. (1999). Communication theory as a field. Communication Theory, 9(2), 119–161.

Cheok, A. D., & Fernando, O. N. N. (2012). Kawaii/Cute interactive media. Universal Access in the Information Society, 11(3), 295–309. https://doi.org/10.1007/s10209-011-0249-5

Craig, R. T. 1999. “Communication Theory as a Field.” Communication Theory 9(2):119–61.

Fogel, A. (2006). Dynamic systems research on interindividual communication: The transformation of meaning-making. The Journal of Developmental Processes, 1, 7–30.

Gomez, R., Galindo, K., Szapiro D., & Nakamura, K. (2018). “Haru”: Hardware design of an experimental tabletop robot assistant. Session We-2A: Designing Robot and Interactions, HRI’18, Chicago, Il, USA.

Hoffman, G. (2007). Ensemble: Fluency and embodiment for robots acting with humans (Ph.D.). Massachusetts Institute of Technology.

Levinas, E. (1969). Totality and infinity. Pittsburgh: Duquesne University Press.

Maltby, R. (2003). Hollywood cinema (2nd ed.). Malden MA: Blackwell Pub.

Moore, S. (1960). An actor’s training: The Stanislavski method. London: Victor Gollanz Ltd.

Nittono, H. 2016. The two-layer model of ‘kawaii’: a behavioural science framework for understanding kawaii and cuteness. East Asian Journal of Popular Culture 2(1), 79-95.

Pinchevski, A. (2005). By way of interruption: Levinas and the ethics of communication. Pittsburgh, Pennsylvania: Dusquene University Press.

Poyatos, F. (1983). New perspectives in nonverbal communication studies in cultural anthropology, social psychology, linguistics, literature, and semiotics. Oxford; New York: Pergamon Press.

Poyatos, F. (1997). The reality of multichannel verbal-nonverbal communication in simultaneous and consecutive interpretation. In F. Poyatos (Ed.), Nonverbal communication and translation: New perspectives and challenges in literature, interpretation and the media (pp. 249–282). Amsterdam; Philadelphia: J. Benjamins.

Sandry, E. (2015). Robots and communication. New York: Palgrave Macmillan.

Simmons, R., Makatchev, M., Kirby, R., Lee, M. K., Fanaswala, I., Browning, B., … Sakr, M. (2011). Believable Robot Characters. AI Magazine, 32(4), 39. https://doi.org/10.1609/aimag.v32i4.2383

Stahl, C., Anastasiou, D., & Latour, T. (2018). Social Telepresence Robots: The role of gesture for collaboration over a distance. Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference on – PETRA ’18, 409–414. https://doi.org/10.1145/3197768.3203180

Sutherland, C. J., Ahn, B. K., Brown, B., Lim, J., Johanson, D. L., Broadbent, E., … Ahn, H. S. (2019). The Doctor will See You Now: Could a Robot Be a medical Receptionist? 2019 International Conference on Robotics and Automation (ICRA), 4310–4316. https://doi.org/10.1109/ICRA.2019.8794439

Turkle, S., Breazeal, C. L., Dasté, O., & Scassellati, B. (2004). Encounters with Kismet and Cog: Children respond to relational artifacts. IEEE-RAS/RSJ International Conference on Humanoid Robots. Presented at the Los Angeles. Los Angeles.

Zachiotis, G., Andrikopoulos, G., Gomez, R., Nakamura K., & Nikilakopoulos G. (2018). A survey on the application trends of home service robotics. Proceedings of the 2018 IEEE International Conference on Robotics and Biometrics, Kuala Lumpur, Malaysia.