Impact of cultural-shift on multimodal sentiment analysis
Presented at The Sixth International Symposium on Intelligent Systems Technologies and Applications (ISTA'20)
Published: 17 November 2021, Journal of Intelligent & Fuzzy Systems
Presented at The Sixth International Symposium on Intelligent Systems Technologies and Applications (ISTA'20)
Published: 17 November 2021, Journal of Intelligent & Fuzzy Systems
Presented at The Sixth International Symposium on Intelligent Systems Technologies and Applications (ISTA'20)
Published: 17 November 2021, Journal of Intelligent & Fuzzy Systems
Jump To
Abstract . Final Presentation . Contributions . Challenges and Takeaways
MY ROLE
Literature Review: 30+ hours of participant observation on the Subreddit. This involved participating in discussions, reading and posting questions and comments on the sub, answering questions and messaging participants.
Experiment Design: Designed and conducted 3 semi-structured interviews to understand participants' experiences on the subreddit and as part of the community.
Conducting Experiments: I transcribed, reviewed and coded the interviews to analyze emerging categories about participant behaviors.
Data analysis: I transcribed, reviewed and coded the interviews to analyze emerging categories about participant behaviors.
RESEARCH SCOPE
Jan - Apr '23 (4 months)
Tulika Banerjee
Ramyani Ghosh
Upasana Bhattacharjee
Vaibhav Kumar
TOOLS/SKILLS
Experiment design, data analysis, cognitive science experimentation, surveys, fitness tracking
KEYWORDS
stress, creativity, cognitive science, convergent thinking, divergent thinking, alternative uses test, remote associates test, problem solving
MY ROLE
Literature Review: 30+ hours of participant observation on the Subreddit. This involved participating in discussions, reading and posting questions and comments on the sub, answering questions and messaging participants.
Experiment Design: Designed and conducted 3 semi-structured interviews to understand participants' experiences on the subreddit and as part of the community.
Conducting Experiments: I transcribed, reviewed and coded the interviews to analyze emerging categories about participant behaviors.
Data analysis: I transcribed, reviewed and coded the interviews to analyze emerging categories about participant behaviors.
RESEARCH SCOPE
Jan - Apr '23 (4 months)
Tulika Banerjee
Ramyani Ghosh
Upasana Bhattacharjee
Vaibhav Kumar
TOOLS/SKILLS
Experiment design, data analysis, cognitive science experimentation, surveys, fitness tracking
Jump To
Abstract . Final Presentation . Contributions . Challenges and Takeaways
KEYWORDS
stress, creativity, cognitive science, convergent thinking, divergent thinking, alternative uses test, remote associates test, problem solving
MY ROLE
Data cleaning: 30+ hours of participant observation on the Subreddit. This involved participating in discussions, reading and posting questions and comments on the sub, answering questions and messaging participants.
Data processing: Designed and conducted 3 semi-structured interviews to understand participants' experiences on the subreddit and as part of the community.
Data analysis: I transcribed, reviewed and coded the interviews to analyze emerging categories about participant behaviors.
Data visualization: I transcribed, reviewed and coded the interviews to analyze emerging categories about participant behaviors.
RESEARCH SCOPE
Tulika Banerjee
Niraj Yagnik
Advisor: Anusha Hegde, Manipal Institute of Technology
TOOLS/SKILLS
data cleaning, vectorization, TF-DF, jupyter notebook, python, data pipelining, data analysis, NLP
KEYWORDS
Multimodal analysis, natural language processing, non-verbal cues, classification algorithms, cultural-shift
Jump To
Abstract . Final Presentation . Contributions . Challenges and Takeaways
Abstract
Abstract
Human communication is not limited to verbal speech but is infinitely more complex, involving many non-verbal cues such as facial emotions and body language.
This paper aims to quantitatively show the impact of non-verbal cues, with primary focus on facial emotions, on the results of multi-modal sentiment analysis.
The paper works with a dataset of Spanish video reviews. The audio is available as Spanish text and is translated to English while visual features are extracted from the videos. Multiple classification models are made to analyze the sentiments at each modal stage i.e. for the Spanish and English textual datasets as well as the datasets obtained upon coalescing the English and Spanish textual data with the corresponding visual cues.
The results show that the analysis of Spanish textual features combined with the visual features outperforms its English counterpart with the highest accuracy difference, thereby indicating an inherent correlation between the Spanish visual cues and Spanish text which is lost upon translation to English text.
Human communication is not limited to verbal speech but is infinitely more complex, involving many non-verbal cues such as facial emotions and body language.
This paper aims to quantitatively show the impact of non-verbal cues, with primary focus on facial emotions, on the results of multi-modal sentiment analysis.
The paper works with a dataset of Spanish video reviews. The audio is available as Spanish text and is translated to English while visual features are extracted from the videos. Multiple classification models are made to analyze the sentiments at each modal stage i.e. for the Spanish and English textual datasets as well as the datasets obtained upon coalescing the English and Spanish textual data with the corresponding visual cues.
The results show that the analysis of Spanish textual features combined with the visual features outperforms its English counterpart with the highest accuracy difference, thereby indicating an inherent correlation between the Spanish visual cues and Spanish text which is lost upon translation to English text.
Contributions
Contributions
01
01
01
Students have limited time available to them. With so many events happening around campus, it is impossible for them to keep track of them all and hence, students miss out on events of importance to them and lose out on a holistic campus experience.
Students have limited time available to them. With so many events happening around campus, it is impossible for them to keep track of them all and hence, students miss out on events of importance to them and lose out on a holistic campus experience.
02
02
02
It is crucial that alerts and campus-wide notifications reach students on time, especially in emergencies. Other times, receiving notifications on time can save valuable time and energy.
It is crucial that alerts and campus-wide notifications reach students on time, especially in emergencies. Other times, receiving notifications on time can save valuable time and energy.
Challenges & Takeaways
Challenges & Takeaways
Wearing many hats
Unlike most hackathons, we did not have the ability to choose our own teams and were assigned to a team of four based on a business chemistry test. We were effectively strangers with a common goal.
Given the short timeframe of the challenge, this implied instantly assessing our diverse skillsets and interests for efficient work distribution and tuning ourselves to be on the same wavelength. Moreover, our schedules made it difficult for us to meet in person, reinforcing strong active communication.
Wearing many hats
Unlike most hackathons, we did not have the ability to choose our own teams and were assigned to a team of four based on a business chemistry test. We were effectively strangers with a common goal.
Given the short timeframe of the challenge, this implied instantly assessing our diverse skillsets and interests for efficient work distribution and tuning ourselves to be on the same wavelength. Moreover, our schedules made it difficult for us to meet in person, reinforcing strong active communication.
Wearing many hats
Unlike most hackathons, we did not have the ability to choose our own teams and were assigned to a team of four based on a business chemistry test. We were effectively strangers with a common goal.
Given the short timeframe of the challenge, this implied instantly assessing our diverse skillsets and interests for efficient work distribution and tuning ourselves to be on the same wavelength. Moreover, our schedules made it difficult for us to meet in person, reinforcing strong active communication.
Stakeholders, stakeholders, stakeholders
The Innovation challenge took place in the middle of finals week, and we found ourselves juggling this along with classes and exams. As such, one of our teammates dropped out of the challenge, leaving us with a team of three.
I took the initiative to coordinate our schedules and fill in for work if one of my teammates was held up. This truly challenged and honed my time management skills.
Stakeholders, stakeholders, stakeholders
The Innovation challenge took place in the middle of finals week, and we found ourselves juggling this along with classes and exams. As such, one of our teammates dropped out of the challenge, leaving us with a team of three.
I took the initiative to coordinate our schedules and fill in for work if one of my teammates was held up. This truly challenged and honed my time management skills.
Stakeholders, stakeholders, stakeholders
The Innovation challenge took place in the middle of finals week, and we found ourselves juggling this along with classes and exams. As such, one of our teammates dropped out of the challenge, leaving us with a team of three.
I took the initiative to coordinate our schedules and fill in for work if one of my teammates was held up. This truly challenged and honed my time management skills.
©2024 Tulika Banerjee
©2024 Tulika Banerjee
©2024 Tulika Banerjee