Swara Ibu
Voice-based emotional analysis for postpartum depression support — 1st Place
Overview
SwaraIbu is an AI-powered mobile application designed to support postpartum mothers by analyzing voice input to detect emotional distress and provide early mental health support. The app was built as part of the Slashcom Android Hackathon organized by UPN Veteran Jakarta, where it won 1st place. Mothers can express their feelings via voice recordings, which are then processed through a multi-stage AI pipeline — speech-to-text via OpenAI Whisper, audio feature extraction via Librosa, emotion and stress classification, and semantic crisis detection via Sentence-BERT.
Problems
Postpartum depression often goes undetected
Postpartum depression affects many new mothers but frequently goes undiagnosed due to limited emotional support, low awareness of early symptoms, and the absence of accessible mental health screening tools. Many mothers suffer in silence without timely intervention.
No accessible voice-based mental health tool for mothers
Existing mental health tools require typing, questionnaires, or clinical visits — barriers for exhausted new mothers. There was no tool allowing mothers to simply speak their feelings and receive AI-powered emotional analysis.
Emergency detection and family notification gap
When a mother experiences a mental health crisis, family members are often unaware until it's too late. The app needed to detect crisis indicators from voice and automatically alert designated companions.
Building AI-powered mobile app within hackathon timeline
Integrating multiple AI models (Whisper, Librosa, Sentence-BERT, emotion classification) into a mobile application and deploying backend APIs within a hackathon's tight timeline presented significant technical and coordination challenges.
Solutions
Voice-based emotional analysis pipeline
Built a comprehensive AI pipeline: voice recording → OpenAI Whisper for speech-to-text → Librosa for audio feature extraction (MFCC, spectrogram) → emotion and stress classification model → semantic crisis detection via Sentence-BERT. Each stage processes the data and passes results to the next for holistic analysis.
Emergency alert system with companion notification
When the crisis detection model identifies high-risk emotional states from voice analysis, the app automatically triggers an emergency alert. The alert is sent to the mother's designated companion (family member) with relevant context about the detected condition.
Mood tracking and companion access mode
Implemented mood tracking that records emotional states over time, helping mothers and their companions understand mental health patterns. Companion Access Mode allows trusted family members to view summarized emotional insights while maintaining privacy controls.
Modular AI architecture with FastAPI backend
Each AI component (speech-to-text, emotion classification, crisis detection) is deployed as separate Python-based services communicating via FastAPI. This modular approach allowed parallel development during the hackathon and enables independent model updates.
Contributions
Developed the entire Android frontend application using Kotlin and Jetpack Compose with Clean Architecture principles
Implemented voice recording UI with real-time audio visualization and recording controls for user-friendly interaction
Built the authentication flow with Firebase Authentication, including login, registration, and session persistence
Integrated the FastAPI backend endpoints for voice upload, emotion analysis, and crisis detection within the mobile app
Designed and implemented the companion access mode UI — allowing family members to view summarized emotional conditions after verification
Created the mood tracking dashboard with graphical visualization of emotional patterns over time using Jetpack Compose charts
Implemented push notification integration for emergency alerts triggered by crisis detection
Collaborated with the backend team to coordinate API contracts, data flow, and real-time communication between app and AI services
Features
Voice Recognition & Analysis
Analyzes voice input using Whisper STT and Librosa audio extraction to detect emotional stress and mental health risks in postpartum mothers
Emergency Alert
Triggers automatic alerts and critical warnings to designated companions when high-risk emotional states are detected from voice analysis
Mood Tracking
Tracks emotional changes over time with visual charts to help users and companions understand mental health patterns and trends
Companion Access Mode
Allows trusted family members to view summarized emotional conditions with secure verification and privacy controls
AI-Powered Analysis
Multi-stage AI pipeline combining Whisper, Librosa, Sentence-BERT, and emotion classification for comprehensive voice-based mental health assessment
Tech Stack
- Kotlin
- Jetpack Compose
- Firebase
- Gemini API
- FastAPI
- OpenAI Whisper
- Librosa
Results
1st Place at SLASHCOM Hackathon
Awarded first place at the Slashcom Android Hackathon organized by UPN Veteran Jakarta for innovation in mental health technology
Functional Voice-Based AI Prototype
Delivered a working mobile application integrating speech recognition, emotion analysis, and crisis detection within hackathon timeline
Multi-Model AI Integration
Successfully integrated OpenAI Whisper, Librosa, Sentence-BERT, and custom emotion classification models into a single mobile application
End-to-End Mobile + AI Pipeline
Demonstrated complete flow from voice recording on Android → API upload → AI processing → result display and emergency alerting