Module Three – Multisensory inclusion

The important of multisensory inclusion in a Digital Future

Photo by Geetanjal Khanna on Unsplash

The importance of touch has been highlighted as a major factor in human connection during the Covid pandemic and the rising issues mental health from social disconnection. We are social beings and it is natural to want to reach out to others for companionship and comfort. Isolation can be damaging to well being. Are learners missing out on the lack of physical presence in the online classroom? Can technology help alleviate the psychological distress that is caused by the lack of physical presence? Can additional sensory technology in the future replace the physical presence in online education? This is where mixed reality technology development may bridge a gap, even better if the touch sensation was included.

For Multisensory-Enabling Technologies (METs) to be developed effectively, the concept of ‘Digi-grasping’ should be applied as an approach to coding and the manipulation of the digital world. The concept of digi-grasping is active, embodied sense-making and existing in the interface between the digital and physical worlds. In being aware of the modes of being and ways of acting in the digital world, ‘it is possible to create a stronger ethical and aesthetic connection between the digital technologies and society’. (Dufva & Dufva, 2019) (p. 18). In developing multisensory interfaces this is good advice.

Activity 1

15 minutes listening

The Touch Test: The Results on BBC Radio 4 reveals the results of joint venture with Wellcome Collection in January 2020. Close to 40,000 people from 112 countries took part to study attitudes to touch. The premise is that ‘Touch is said to be the first sense to develop but is it also the most underrated?’ (BBC, 2020). If this is the case the use of haptic devices, such as Ultraleap’s Stratos systems (Ultraleap, 2020) will have a place in future multisensory interfaces for learning. Mid-air experiences can be useful for those with visual impairment or some learning disabilities.

The Touch Test results are available on the public domain on the BBC website. They cover 5 areas in 15 minute blocks. Recordings cover Health and Touch, Touch Culture, Don’t Touch, Touch Hunger and the most relevant to this OER – Digital Touch.

Listen to the recording in the public domain – Anatomy of Touch: Digital Touch

Note you will need to register for a free account

Activity 2

15 minutes considering the questions below

  1. In this recording, Carey Jewitt considers the ethics of digital touch. She asks what actually are you wanting to replicate? She highlights people are incredibly good at creating meaning through very reduced forms digital, especially in established relationships (BBC, 2020). What would be the implications of a digitised touch if the participant was uncomfortable with the sensation or the context?
  2. Neuroscientist David Eagleman, describes a wristband developed for deaf people to translates touch into sound. It allows the brain to make patterns and after a while can create a direct perception of hearing from touch (BBC, 2020). For example just seeing a dog can trigger the perception of barking. The upper limits are not known from what can be learnt about the environment from this method. If a touch can generate the perception sound in this way, what may be the implications for the wearer of the wristband if they enter a new environment?
  3. Huggy pyjamas that sends a digital hug are designed for parents who can not get back to spend time with their children before they go to sleep. Who would own the hug if the parent gave the task to say an assistant? Would this be a betrayal of trust between the parent and child? Is this a form of abuse?

Reflection – some of the observations from the Anatomy of Touch: Digital Touch echo some of Dufva and Dufva’s research. They reference Merleau-Ponty’s use of ‘the term grasping to point to an activity that is intentional but not necessarily conscious’ (Merleau-Ponty, 1945).  METs of the future will inevitability be designed with the intention that ‘the digital is invisible’: users will be totally unaware of the interface, why it is are designed so, or how it maybe shaping the user’s behaviour (Dufva & Dufva, 2019). Future developers should be aware that ‘not only are we transformed by the way we use our tools; we are not aware of how we are being transformed, so we need all the more to try to make explicit what the Net is doing for us and what it is doing to us in the process.’ (Dreyfus, 2008). In the development of METs ‘Digi-grasping is not a measure of the rational knowledge of digitality but rather a concept for thinking about and analysing the embodied experience of digitality’ (Dufva & Dufva, 2019) (p. 23). This reinforces the author of this OER concerns to what is presented in a future MET experience and developers must consider their responsibility to users.

You have now completed Module Three – The importance of multisensory inclusion