The Full Story
STORY++
as part of "Project the Future of Wellbeing Hackathon 2022" at the MIT Media Lab (in collaboration with SAMSUNG), My team and I created a story-telling device that utilizes speech-to-text and AI in order to augment the environment around us and help us bridge an age gap between multi-generational households. We have won 1st place for this project.
STORY++ Live Demo
Older adults experience loneliness and social isolation due to living alone,
the loss of family or friends,
chronic illness.
(CDC.ORG)
People that engage more in social activities with their family have been found less likely to develop depression.
(Yang, X. and Yin, D., 2022. The protective effect of caring for grandchildren on the mental health of the elderly: a structural equation modeling analysis. International journal of environmental research and public health.)
Story++ turns your narrative into dynamic audio-visual experiences
Speech2Text
Google cloud Natural Language Processing API
Object detection
OpenCV contour detection
and semantic segmentation
Generative AI
CLIP and diffusion text to image generation
Physical + digital recombination
Mask around object and image and create background projection
Winning team from left to right: Aastha Shah, Wazeer Zulfikar, Yuval Gur, Cathy Fang, Valdmar Danry, Jason Hou