clarify what they are seeing or referring to.
This PhD project will investigate how AI-enhanced AR glasses can support communication between a person carrying out a task and a remote human helper. The central research question is: How can AI-mediated AR glasses support communication between a wearer and a remote human helper in workplace or technical training contexts?
The project will focus on Human-Computer Interaction, Human-AI Interaction, and Computer-Supported Cooperative Work. Rather than aiming to develop new computer vision models from scratch, the project will explore how existing AI and computer vision technologies can be meaningfully integrated into interaction design. For example, AI could help recognise objects or scenes in the wearer’s environment, translate a remote helper’s verbal instructions into visual AR cues, or summarise what the helper sees and says so that the wearer can more easily act on the guidance.
A likely application scenario is a trainee or worker completing a technical task while being supported remotely by an expert. This could include maintenance, repair, equipment setup, laboratory work, or workplace training. The project may use one of these contexts as a primary case study while allowing flexibility to adapt the specific application domain according to the student’s interests, background, and available collaborators.
The student will design, prototype, and evaluate interaction concepts for AI-mediated remote assistance through AR glasses. This may include low-fidelity design concepts, Wizard-of-Oz studies, interactive prototypes, or lightweight AR prototypes using existing tools and AI services. The emphasis will be on understanding how AI outputs should be presented, how they affect communication between the wearer and the remote helper, and how users interpret, trust, correct, or rely on AI-mediated information.
Possible research activities may include:
- reviewing existing work on AR-based remote collaboration, AI-mediated communication, and workplace training;
- conducting interviews or observational studies to understand the needs of trainees, workers, and remote experts;
- designing interaction techniques that combine human guidance with AI-generated visual or textual support;
- developing and evaluating prototypes that use existing computer vision or vision-language models;
- studying how AI support affects communication, coordination, workload, trust, and task performance.
The project would suit applicants with interests in HCI, UX research, mixed-methods research, AR/VR interaction design, or prototyping. Strong programming or AR development experience would be beneficial, but the main contribution is expected to be in the design and evaluation of human-centred AI-mediated interaction, rather than in the development of new AI algorithms.
There is flexibility in the final direction of the project. Depending on the student’s expertise and interests, the research may place greater emphasis on interaction design, user studies, AR prototyping, communication between workers and experts, or the evaluation of AI-generated support in realistic training scenarios.
Funding Notes
There is no funding for this project
References
Billinghurst, M., Clark, A., and Lee, G. (2015). A survey of augmented reality. Foundations and Trends in Human-Computer Interaction, 8(2-3), 73-272.
Ens, B., Lanir, J., Tang, A., Bateman, S., Lee, G., Piumsomboon, T., and Billinghurst, M. (2019). Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of Human-Computer Studies, 131, 81-98.
Gauglitz, S., Nuernberger, B., Turk, M., and Hollerer, T. (2014). World-stabilized annotations and virtual scene navigation for remote collaboration. Proceedings of the ACM Symposium on User Interface Software and Technology.
Wang, P. et al. (2021). AR/MR remote collaboration on physical tasks: A review. Robotics and Computer-Integrated Manufacturing, 68, 102071.
Wang, D. et al. (2020). From human-human collaboration to human-AI collaboration: Designing AI systems that can work together with people. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems.
