letras sa senyas
a cross-platform mobile application built with nuxt, vue, and capacitor. includes a dedicated administrator dashboard and local machine learning inference for real-time filipino sign language gesture recognition.
the design
letras sa senyas' branding
web-based administrator dashboard
letras sa senyas' ui
the architecture
- nuxt.js
- vue
- capacitor
- kotlin
- firebase
- python, opencv, mediapipe
- tensorflow
- figma
utilizes a hybrid neural network architecture consisting of:
- a fully-connected neural network for static gesture recognition
- a long short-term memory network for dynamic gesture recognition
this system was built in collaboration with 2 other people, as per our thesis requirements. my role as the project manager and lead developer of the project was to oversee and help with the designing of the overall system, though my main focus was on the training, development, and integration of the hybrid neural network architecture.
entity relationship diagram
data flow diagram
administrator dashboard dfd
me testing the static gesture recognition model
running the data collection script