With subtitling glasses, hearing-impaired students should be able to follow lectures without barriers. Practical tests at Mainz University of Applied Sciences have revealed advantages, but also new challenges. In this guest article, Alessandra Ress and Daniel Bayer explain the extent to which the use of subtitling glasses can actually contribute to barrier-free teaching.

Videos without subtitles, missing transcripts or classroom teaching in rooms that are poorly insulated against background noise: there are many barriers for students with hearing impairments in everyday university life. Even if there are now some rays of hope – the Best3 study shows, that this group of students with severe disabilities is comparatively satisfied at around 56% – there is still a lot of room for improvement in terms of reducing barriers.

At the Teaching Competence Center, we test technological and didactic developments for their practical suitability at Mainz University of Applied Sciences. During our research, we became aware of XRAI, the provider of the augmented reality app XRAI Glasswhich explicitly promotes its use in teaching/learning settings. Specifically, live subtitling via augmented reality glasses is to be used to display subtitles directly below the person speaking, thereby reducing barriers for people with hearing impairments.

The app is installed on a standard smartphone connected to AR glasses. When we contacted the app developer at the end of 2022, we were initially given access to the beta version of the app, which could only be used on Android. XRAI Glass is now also available for iOS.

The setup in a live context includes the following components:

• A smartphone with the “Nebula” and “XRAI Glass” apps installed on it
• The glasses incl. Cable (in our case the “XREAL Air” model)
• A Bluetooth microphone

We are aware that such tools are not uncontroversial. Raul Krauthausen for example, has developed AR glasses in connection with Disability dongles a term introduced by Liz Jackson to describe designs and technologies developed by non-disabled people as a supposed solution to real or perceived problems of disabled people.

Embedded in other measures, we still wanted to try out XRAI Glass in practice, especially as we saw potential in various groups of students: On the one hand, we tested the glasses with Students with hearing impairmentson the other hand with international students without German as their mother tonguewho have paid particular attention to the live translation of subtitles, which is also possible with XRAI Glass.

How did the tests and feedback go? An overview in three thematic blocks:

Before we got down to the actual practice, we introduced the students to the hardware and software mentioned above, which we provided. Although not all participants had worked with AR before, the familiarization went largely smoothly: after an initial demonstration, the students were able to use the apps and glasses independently. Only the XRAI app itself turned out to be a little sluggish, sometimes requiring several attempts to start it or bring the AR view to the desired height in the field of vision.

A bigger hurdle than the technical component is the major issue of data protection. Theoretically, there are several providers for subtitles and translations available via the app, but we only used Vosk to avoid storing them in the cloud.

The subtitles generated in this way are only saved locally, but if personal data is disclosed in the recorded conversation, it will appear in the transcript. An example: A student uses the XRAI app in a seminar where a discussion is taking place. The participants mention the names of their fellow students. This information would then be saved locally in the transcript as part of the conversation.

Our solution: For courses in which the glasses are to be used, we offer to explain the benefits and functionality to all participants in advance, i.e. also to “passive users”. The person using the XRAI undertakes in writing to delete the stored data promptly. This also represents a compromise in the face of individual concerns that the transcriptions would offer an advantage in examination situations – an aspect that once again made it clear to us that the principle of compensating for disadvantages is not present for everyone.

Once the bureaucratic and technical hurdles have been overcome, the actual use can begin. Anyone who has ever used automatic subtitling knows that it is not perfect and the choice of words is sometimes quite creative – for example, “digital teaching” sometimes becomes “guitar teaching”. Nevertheless, the assessment of the students with hearing impairments was optimistic, and the context made it clear what was meant, even in the case of errors. In cases where students have previously had an automatic transcription running on the screen in seminars, the AR aspect was also positively emphasized: This means you no longer have to constantly look back and forth between slides on the wall and a screen with subtitles.

The international students were significantly more critical, which we attribute to various reasons: For example, they are less likely to use automatically generated subtitles and may therefore have a lower tolerance for errors. Secondly, reliability varies depending on the language.

In a survey, international students predominantly stated that they would not use the XRAI app. Their main concern was that the app looked too conspicuous.

Among students with hearing impairments, the conclusion is more tolerable. “At the beginning, [die Brille] may be a little awkward, but it becomes more manageable over time. However, you have to bear in mind that it is even more complicated without aids,” concludes Nick Massing, a digital media student who took part in one of the tests. Another student praises the subtitling, but criticizes the fact that the frame is not particularly flexible and therefore uncomfortable to wear over a longer period of time – all the more so for people who also wear normal glasses.

Subtitling glasses like the XRAI are not a sure-fire success, as we are well aware after the tests we have conducted over the past few months. In terms of foreign languages, we now see them primarily as a means of giving students a practical understanding of the basic possibilities and functions of AR.

In the context of accessibility, we see glasses as a solid addition for short-term assignments such as lectures. However, it is also clear that it does not relieve teachers of their responsibility to break down barriers in other ways, e.g. by providing transcripts or notes. In addition, the Group of students with hearing impairments is too heterogeneousto make a general judgment about their benefit.

The tests and feedback from people who are not affected have also shown us that there is still room for improvement in terms of awareness of accessibility. In particular, teachers and other event organizers must be aware that Accessibility is not a “nice to add” , but legal obligations exist and assistive technologies compensate for disadvantages. Where data protection and accessibility or audit rules and accessibility stand in the way of each other, willingness is required, to find practical solutions.We will continue to keep our eyes open for helpful technologies – but above all, we will remain in contact with the students concerned in order to recognize needs even where we are not aware of them at first, and to remove barriers wherever possible.

Daniel Bayer

Daniel Bayer is head of the Teaching Competence Center, which is the central point of contact at Mainz University of Applied Sciences for teaching design and development. Together with his team, he deals with issues such as how new technologies can be integrated into teaching and learning processes in a didactically meaningful way.

Alessandra Reß

Alessandra Reß has been working in e-learning support at various universities in Rhineland-Palatinate since 2018, most recently as project manager for digital teaching at Mainz University of Applied Sciences. Her focus is on the areas of OER and digital accessibility. She also works as an author and editor.