When the Interface Has a Body
Beyond the Screen — Week 1 of Jae’s Exploration of Human–Robot Interaction
My starting point in robotics is interaction design. With a background in XR, games, and interaction design prototyping, I’m interested in how people make sense of systems through space, movement, and timing.
Human–robot interaction brings those questions into a shared physical environment. A robot occupies space alongside us. Its movement can draw our attention, interrupt an activity, or change where we feel comfortable placing our hands.
This series documents my exploration of that relationship through research, design questions, and small prototypes.
For the first week, I started with one idea:
A robot’s behavior is part of its interface.
Imagine a small robot on a desk. It has a non-urgent notification to share while someone is working. It could turn toward the person and immediately speak. It could also orient toward them, pause, and wait for a response before continuing.
Both behaviors might deliver the same information. But they establish different expectations about attention, interruption, and control. What happens if the person chooses not to engage? Even before the robot speaks, an interaction has begun.
What carries over from XR—and what changes
XR gives me a useful starting point. Position, orientation, scale, and timing can all affect how an interaction is understood. A cue can be visible yet poorly placed. A motion can be smooth yet difficult to interpret.
With a physical robot, those decisions also shape the shared environment. A turn toward an object may communicate interest, but it may also bring part of the robot closer to someone’s hand. A movement that looks appealing in an animation still needs to work within the constraints of a physical mechanism.
A robot communicates not only through the messages it displays, but also through where it positions its body and how it moves. The questions do not disappear when an interface gains a body; they expand.
Design question | Typical screen-based UX | Physical robot interaction |
|---|---|---|
How does the system communicate its state? | Text, icons, color, and animation | Direction, posture, speed, and distance, alongside screens and sound |
What counts as user input? | Clicks, taps, and typed input | Speech and gestures, but also approaching, moving away, or not responding |
What changes when the system acts? | Primarily the interface and its digital state | The position of physical objects and the space available for people to move |
Who is the user? | Primarily the person operating the service | The operator, but also people nearby or passing through |
How can the system recover from an error? | Undo, retry, or restore a previous state | It may also need to stop, move away, or return to a safe state |
What should we evaluate? | Task success, comprehension, and completion time | These measures, plus how people interpret its behavior, whether they feel interrupted, and whether they can intervene |
That makes me want to describe behavior at two levels: what the robot does, and what a person might understand or do in response.
“Rotate toward the user” describes an action. It leaves several interaction questions open. Does the movement mean “I heard you,” “I need your attention,” or “I’m about to move closer”? How can someone decline? What happens when their response is ambiguous?
These questions belong in the behavior specification.
A research reference: ELEGNT
One reference for this exploration is ELEGNT, a 2025 paper by Yuhan Hu, Peide Huang, Mouli Sivapurapu, and Jian Zhang. The researchers developed a lamp-like robot and explored movement that combines functional objectives with expressive qualities.
Their study compared expression-driven and function-driven movements across six scenarios. They reported improvements in engagement and perceived robot qualities with expression-driven movement, particularly in social tasks.
What interests me is the design space this opens. A lamp-like body can convey cues through its posture and motion, without relying on a human face.
My design interpretation is that expressive movement should have a purpose. In a particular moment, it might help a person understand where the robot is directing its attention or what it is about to do. Adding more movement would not automatically make that moment better. A long expressive sequence before a simple task could become an interruption of its own.
I wanted to explore that relationship with something I could make immediately.
Three small motion sketches
I used a small paper model beside a laptop and moved it by hand, recording three short clips.
The broader scenario was a desk companion with a non-urgent notification. I wanted it to request attention while allowing the person to leave that request unanswered.
For this first study, I explored how different movements might be interpreted. These were rough, hand-performed sketches rather than programmed robot behaviors or a controlled comparison of a single variable.
I showed the clips to one adult and collected their feedback. My notes captured the following interpretations, paraphrased and translated here:
What I asked about | What the participant described |
|---|---|
What the movements seemed to mean |
|
What might happen if they did nothing |
|
What might follow the “no” gesture | It might pause, then repeat “no” after some time |
The movements prompted recognizable interpretations. But the response that caught my attention was about what would happen after the movement stopped.
The participant expected repetition.
What does a pause promise?
I had been focusing on what a movement communicates in the moment. The feedback added another layer: what future behavior does it lead someone to expect?
In this session, stopping did not necessarily signal that the interaction was over. The participant’s predictions suggested that the pause could be a temporary wait before another attempt.
That distinction matters for the desk companion I want to explore. If a non-urgent notification appears likely to keep repeating, the person may expect that leaving it unanswered will only postpone the next request.
I did not test whether the participant felt pressured or interrupted, so I cannot claim that effect. What I did learn is that the ending of a movement deserves its own design attention.
A gesture can communicate “I have something for you” while leaving “What happens if I ignore you?” unresolved.
This was one informal feedback session. The model was moved by hand, and the clips could not demonstrate a robot responding to someone in real time. The interpretations are useful clues for another iteration, rather than evidence that these movements will be understood consistently.
The next iteration
For the next version, I want to keep the attention-requesting movement consistent and change how it ends:
Version A: The model finishes the movement and holds its final pose.
Version B: The model finishes the same movement, then visibly returns to a resting posture.
I would show the two versions without explaining the intended difference and ask what the viewer expects to happen next.
My hypothesis is that returning to rest may make the request feel more clearly finished. It could also be read as preparation for another movement. That is something to test.
The question has become more specific:
How can a robot make a request—and make it clear that the request can go unanswered?
For future motion studies, I want to record both the meaning someone reads in a gesture and the behavior they expect to follow it. This first experiment gave me a reason to pay as much attention to how a robot stops as to how it starts.