Implementasi Computer Vision dalam Klasifikasi Kalimat BISINDO Berbasis WebCam dengan Menggunakan Model YOLOv8 dan MediaPipe

The limited public understanding of Indonesian Sign Language (BISINDO) remains a communication barrier for deaf individuals. This research develops a Computer Vision-based BISINDO translation system that recognises sign words in real time through a webcam and arranges them into sentences. The system is built as a staged pipeline comprising YOLOv8 as a hand Region of Interest (ROI) detector, MediaPipe Holistic as a landmark extractor, LSTM and BiLSTM as word classifiers, and a rule-based Natural Language Processing (NLP) module as a sentence composer, implemented as a website-based prototype for one-way communication with text and speech output through gTTS. The dataset comprises 5,000 videos across 40 word classes, consisting of 2,000 public Kaggle videos and 3,000 self-recorded videos, divided at a ratio of 70:15:15. Testing was conducted through four schemes distinguished by signer composition and model architecture. In the single-signer scheme, both LSTM and BiLSTM achieved an accuracy of 99.64% with a macro F1-score of 0.996, while the scheme with a more diverse signer composition yielded 85.92% (LSTM) and 88.03% (BiLSTM). The YOLOv8n model achieved an mAP@50 of 0.9947 and an mAP@50–95 of 0.8593. Sentence composition testing achieved a success rate of 40%, with a system processing speed of 14 FPS. These results indicate that performance under the single-signer scheme does not yet reflect the model's generalisation capability; therefore, the accuracy of 88.03% is taken as the reference performance of the model in this research.