How do we leverage interactive, generative art in a meaningful way such that it isn't just a fleeting novelty i.e why should people care?
What control can users have that is unique to each of them?
This led us to using speech and voice modality as the basis for generation.
How do we introduce an emotional connection?
We decided to use sentiment analysis to analyze the users' speech in addition to the physical attributes of their voice to allow for their underlying emotions to shine through.
Patience is a rare commodity and in today's world, keeping a user's attention can be tricky. Thus, we wanted the final product to be processed in real-time to have a takeaway ready for users by the end of the experience.
The result of extensive brainstorming was the next gen of
PERSONALIZED POSTCARDS.
Using voice input to let users generate unique, personalized artwork to be paired with a word cloud derived using sentiment analysis of their speech and producing a personalized postcard in real time.
STROKE WIDTH
For a personalized touch, the amplitude of the input sound (user's voice) was mapped to stroke width.
The louder the voice., the thicker the stroke and vice versa.
THRESHOLD
To avoid background noise impacting the art piece, a decibel threshold was set. If the input was less than the threshold, the art would stop generating.
This feature could also be utilized by users to add selective emphasis.
Using nltk, Python and Google's speech-to-text-api to run sentiment analysis on the speech input, we were able to generate word clouds in real-time that captured the user's sentiment.
Word cloud generation
Color and Emotion
Positive
Neutral
Negative
The design was inspired by common postcards.
The front had the generated art and the back was divided into the word cloud on the left and a space for a written note on the right.
We decided to add three broad prompts to help users start talking / relaying a message for the word cloud generation.
The three prompts and the associated postcard messages were:
1) Recount a memory
"Remember when…"
2) Sing a song
"A song for you..."
3) Message to a loved one
"Psst did you know…"
The postcards were generated dynamically by replacing the generated art and word clouds after each run of the code.
