• Context-Aware Mixed Reality: A Learning-based Framework for Semantic-level Interaction

      Chen, Long; Tang, Wen; Zhang, Jian Jun; John, Nigel W.; Bournemouth University; University of Chester; University of Bradford (Wiley Online Library, 2019-11-14)
      Mixed Reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic-based interactive MR framework that is beyond current geometry-based approaches, offering a step change in generating high-level context-aware interactions. Our key insight is that by building semantic understanding in MR, we can develop a system that not only greatly enhances user experience through object-specific behaviors, but also it paves the way for solving complex interaction design challenges. In this paper, our proposed framework generates semantic properties of the real-world environment through a dense scene reconstruction and deep image understanding scheme. We demonstrate our approach by developing a material-aware prototype system for context-aware physical interactions between the real and virtual objects. Quantitative and qualitative evaluation results show that the framework delivers accurate and consistent semantic information in an interactive MR environment, providing effective real-time semantic level interactions.
    • Recent Developments and Future Challenges in Medical Mixed Reality

      Chen, Long; Day, Thomas W.; Tang, Wen; John, Nigel W.; Bournemouth University and University of Chester (2017-11-23)
      Mixed Reality (MR) is of increasing interest within technology driven modern medicine but is not yet used in everyday practice. This situation is changing rapidly, however, and this paper explores the emergence of MR technology and the importance of its utility within medical applications. A classification of medical MR has been obtained by applying an unbiased text mining method to a database of 1,403 relevant research papers published over the last two decades. The classification results reveal a taxonomy for the development of medical MR research during this period as well as suggesting future trends. We then use the classification to analyse the technology and applications developed in the last five years. Our objective is to aid researchers to focus on the areas where technology advancements in medical MR are most needed, as well as providing medical practitioners with a useful source of reference.
    • Training Powered Wheelchair Manoeuvres in Mixed Reality

      Day, Thomas W.; John, Nigel W.; University of Chester (IEEE Xplore, 2019-09)
      We describe a mixed reality environment that has been designed as an aid for training driving skills for a powered wheelchair. Our motivation is to provide an improvement on a previous virtual reality wheelchair driving simulator, with a particular aim to remove any cybersickness effects. The results of a validation test are presented that involved 35 able bodied volunteers divided into three groups: mixed reality trained, virtual reality trained, and a control group. No significant differences in improvement was found between the groups but there is a notable trend that both the mixed reality and virtual reality groups improved more than the control group. Whereas the virtual reality group experienced discomfort (as measured using a simulator sickness questionnaire), the mixed reality group experienced no side effects.
    • Visualization beyond the Desktop--the Next Big Thing

      Roberts, Jonathan C.; Ritsos, Panagiotis D.; Badam, Sriram Karthik; Brodbeck, Dominique; Kennedy, Jessie; Elmqvist, Niklas; University of Chester (IEEE, 2014-08-15)
      Visualization researchers need to develop and adapt to today’s new devices and tomorrow’s technology. Today, people interact with visual depictions through a mouse. Tomorrow, they’ll be touching, swiping, grasping, feeling, hearing, smelling, and even tasting data.
    • Wheelchair-MR: A Mixed Reality Wheelchair Training Environment

      Day, Thomas W.; University of Chester (IEEE, 2017-09-20)
      In previous work we have demonstrated that Virtual Reality can be used to help train driving skills for users of a powered wheelchair. However, cybersickness was a particular problem. This work-in-progress paper presents a Mixed Reality alternative to our wheelchair training software, which overcomes this problem. The design and implementation of this application is discussed. Early results shows some promise and overcomes the cybersickness issue. More work is needed before a larger scale study can be undertaken.