The motivation behind this project stems from a deep desire to democratize the interview process and make it more accessible, personalized, and fair. Job interviews can be incredibly stressful, and traditional methods often fail to account for individual differences in language proficiency, communication style, and comfort levels. By harnessing the power of AI and advanced communication technologies, we can create an interview experience that not only assesses a candidate's qualifications but also provides real-time support and feedback. This platform aims to empower job seekers, helping them present their best selves and gain valuable insights to improve their performance. In a world where talent is everywhere but opportunities are not, this project seeks to bridge that gap and bring us closer to a more equitable job market.
So with this motivation, I have created an innovative AI-powered platform that personalizes the interview experience for job candidates. Users can fill out a survey detailing their job position, experience, skills, goals, and preferred language for the interview. Based on their responses, they can choose to conduct the interview via web browser or WhatsApp Business.
If a user opts for WhatsApp, they will receive interview questions generated by the Gemini API directly on WhatsApp, with the ability to respond and receive follow-up questions in their chosen language. Upon completion, they receive a voice call and a WhatsApp message with AI-generated feedback.
For those who prefer a web browser interview, the platform provides a video interface that opens the user's camera and microphone. Questions appear onscreen, and the AI analyzes body language using TensorFlow and PoseNet. Users receive real-time feedback on their posture and eye contact, can replay their responses, and view live transcripts of their speech. After the interview, they exit the video room and receive detailed feedback.
Next-Gen AI Interview: Multilingual and Real-Time Analysis
This repository contains my submission for the Twilio Challenge.
What is this project about?
The motivation behind this project is to democratize the interview process, making it more accessible, personalized, and fair. Traditional interviews often fail to accommodate individual differences in language proficiency, communication styles, and comfort levels. By leveraging AI and advanced communication technologies, this platform aims to provide an interview experience that not only assesses a candidate's qualifications but also offers real-time support and feedback.
Key Features
Personalized Interview Setup: Users fill out a survey detailing their job position, experience, skills, goals, and preferred language for the interview.
Multilingual Support: Interviews can be conducted in the user's preferred language via WhatsApp Business or a web browser.
AI-Generated Questions and Feedback: Gemini API generates context-specific interview questions and AI-generated feedback.
Real-Time Body Language Analysis: TensorFlow and PoseNet analyze user posture and eye…
Twilio APIs form the backbone of my project, enabling seamless communication across different channels. Twilio WhatsApp Business API ensures smooth delivery of interview questions and responses, while Twilio Voice API handles voice feedback calls. Twilio Video API powers the browser-based video interviews, integrating with TensorFlow and PoseNet for real-time body language analysis. The Gemini API generates intelligent, context-specific questions and feedback, while Microsoft Text Translator ensures language accessibility. Together, these technologies create a cohesive and intuitive interview experience that is accessible, multilingual, and highly interactive.
Features
Personalized Interview Setup
Multilingual Support
Dual Interview Modes (WhatsApp Business Mode, Web Browser Mode)
AI-Generated Questions and Feedback
Real-Time Body Language Analysis
Replay and Transcript Features
Users can replay their spoken answers to review performance.
Live transcription of spoken responses displayed on screen.
Seamless Integration with Twilio APIs
Comprehensive Feedback
Tech Stack
Frontend: React JS
Backend: Express JS
Styling: Tailwind CSS
AI and ML: TensorFlow, PoseNet
APIs:
Twilio APIs:
WhatsApp Business API
Voice API
Video API
Gemini API (for intelligent question generation and feedback)
Microsoft Text Translator API (for multilingual support)
Additional Prize Categories
Twilio Times Two: The project uses Twilio Whatsapp Business API, Twilio Voice API and Twilio Video API.
Impactful Innovators: This project is beneficial for job seekers that can use AI-driven, multilingual support and real-time feedback, empowers them to shine and bridging the gap between global talent and opportunities.
Entertaining Endeavors: The platform enhances the interview experience by incorporating interactive features like real-time body language analysis and replayable responses, making the process both informative and engaging for candidates.