“Tapping into Memory” – Swinburne University of Technology profiles our research into designing virtual worlds for people with dementia

By | September 15, 2014

The latest edition of Swinburne University of Technology’s  VENTURE magazine has a story about my doctoral research into virtual worlds and how  they can support well-being in people with dementia, reproduced below.


Issue Two 2014 – Issue #21

Print this article

Tapping into memory

Story by By Vicki Steggall

View articles in related topics: Ageing, Neuroscience, Computing


Tapping into memory

The virtual world is a place of endless possibilities and it’s creating fascinating new opportunities for people
with dementia.

Swinburne PhD student Mandy Salomon, a senior researcher at the Smart Services Co-operative Research Centre (Smart Services CRC), wondered if the key elements of virtual‑reality worlds could help people suffering mid‑to‑later stage dementia. What if someone feeling isolated and anxious in a residential care facility could have the experience of walking into a kitchen and making a cup of tea, just like they did at home? And what if they could decorate a bedroom themselves? Or visit a garden to hear birdsong and watch flowers bloom?

“There are many real‑world applications coming from the threads of virtual reality,” says Salomon. “I’ve tried to draw those threads together and apply them to dementia, where surprisingly little had been done.”

Virtual‑home environment

Her project, AVED (Applying Virtual Environments for Dementia Care), is a pilot prototype of an interactive, tablet based, 3D environment, especially created for people with dementia. It was designed with the help of residents with the condition living in aged-care facilities. “We chose familiar places for our 3D environment; a sitting room, kitchen and garden, to offer people meaningful tasks drawn from their past,” she explains. “Home is universal and we know people miss it. “They can decorate their virtual rooms using colours, fabric swatches and paintings or drag their favourite photos into wall frames. Or [they can] just touch and explore. It’s a bit like a game but there’s no prescribed narrative; it’s whatever they feel comfortable with.”

Users are helped by a friendly on-screen ‘companion’, with whom they can interact and dress appropriately, such as giving her a hat to wear in the virtual garden.

Salomon is doing her PhD under the aegis of the Smart Services CRC, which is linked to Swinburne. Her extensive field research informed her design for the virtual world, and a team of former and current Swinburne students led by PhD students James Bonner and Norman Wang have been commissioned to build it, using game technology.

Having dementia patients use technology may seem counter-intuitive, but the tablet opens a new world. “One of the lovely things,” says Salomon, “is that people with this level of dementia don’t bring fear to the tablet. It’s visual, without confusing keypads, so their approach is, ‘what is this thing showing me?’”

To assess how AVED is working, Salomon looks at several factors. Firstly, how did they ‘experience’ the activity; what kinds of interactions did they undertake? Did it provide ‘proximal’ value, involving interaction and engagement with others? Did it provoke reflection about themselves and the world? Finally, she looks at its effect on all-important patient wellbeing. “These are all complex questions, both to pose and to answer, so they form part of our exploration,” she says.

“We closely observed nursing‑home residents using it,” says Bonner, “and found that when we revisit, they remember us and the application, which, given their condition, is quite profound. It makes us feel fantastic.” One of the saddest effects of early dementia is the loss of personhood – the things that made that person who they are. AVED, drawing on reminiscence theory, investigates whether taking people back to something familiar also increases their motivation. An example is a man who selected a professional looking avatar, then talked about how to work with employees. It was a glimpse into who he was, provoked by using the program and reflecting about it. Another woman, anxious and unresponsive, enjoyed opening a box of chocolates and touching each chocolate, which then burst into stars. Remembering her love of chocolate drew her out of her isolation.

“It will be interesting at the neuro-scientific level,” says Salomon. “What is happening here and what parts of the brain are we tapping into? If a person can’t understand shapes on a page but they can decorate a room, what is going on?”

Building familiar content

AVED’s musical activities build on the reminiscence base, allowing people to choose their favourite record, watch it slip out of the cover and start playing. Or see a keyboard and play it. Or just plonk the keys. It means someone who feels uncomfortable in the nursing‑home sing-along, or doesn’t know the songs, no longer ‘opts out’, with the attendant risk of increased isolation and disengagement, but can enjoy music from their own past, provided by family or carers.

This ability to build content – ranging from music to photographs and old videos – benefits not just the patient, but also family members who often want to help their loved one but feel sidelined by the dementia process. “In time, we want people to be able to do the things they loved virtually, like being with pets, or the things they wish they’d done, like ride a horse,” says Salomon. “We are developing a fully functional prototype and the PhD gives me the chance to see how far it can go.”

Salomon’s work has support from Alzheimer’s Australia Vic, under the aegis of the Smart Services CRC, and the Barbara Dicker Foundation, an Alumni philanthropic trust. James Bonner is completing a PhD on Artificial Intelligence, and Norman Wang a PhD on Immersion and Psychology. More projects for people living with dementia are in the planning stages.

Share and Enjoy

One thought on ““Tapping into Memory” – Swinburne University of Technology profiles our research into designing virtual worlds for people with dementia

  1. audrey

    i think its fantastic that people are now looking at what can work to increase the quality of life for those who suffer this debilitating illness.


Leave a Reply

Your email address will not be published. Required fields are marked *