BISINDO INDONESIAN SIGN LANGUAGE RECOGNITION USING MEDIAPIPE HOLISTIC AND LSTM DEEP LEARNING MODEL

AUZIQNI, ROBBY (2022) BISINDO INDONESIAN SIGN LANGUAGE RECOGNITION USING MEDIAPIPE HOLISTIC AND LSTM DEEP LEARNING MODEL. S1 thesis, Universitas Mercu Buana Jakarta.

Preview	Text (HAL COVER) 01 Cover.pdf Download (933kB) | Preview
Preview	Text (ABSTRAK) 02 Abstrak.pdf Download (45kB) | Preview
	Text (BAB I) 03 Bab 1.pdf Restricted to Registered users only Download (157kB)
	Text (BAB II) 04 Bab 2.pdf Restricted to Registered users only Download (163kB)
	Text (BAB III) 05 Bab 3.pdf Restricted to Registered users only Download (165kB)
	Text (BAB IV) 06 Bab 4.pdf Restricted to Registered users only Download (373kB)
	Text (BAB V) 07 Bab 5.pdf Restricted to Registered users only Download (126kB)
	Text (BAB VI) 08 Bab 6.pdf Restricted to Registered users only Download (424kB)
	Text (DAFTAR PUSTAKA) 09 Daftar Pustaka.pdf Restricted to Registered users only Download (77kB)
	Text (LAMPIRAN) 10 Lampiran.pdf Restricted to Registered users only Download (235kB)

Abstract

There is a gap between normal people and deaf people regarding communication. This is because not all normal people learn sign language like deaf people. Therefore, there needs to be something that can close the gap by utilizing technology that can recognize sign language. There are two sign languages used widely in Indonesia, i.e., Sistem Isyarat Bahasa Indonesia (SIBI) and Bahasa Isyarat Indonesia (BISINDO). The one that is used as the official sign language at school is SIBI, while BISINDO, on the other hand, is the one more commonly used by the deaf. In this research, the author presents their findings to recognize BISINDO Indonesian sign language from a series of gestures. The author collects a total of 780 image sequences for 26 BISINDO alphabet signs. The author utilizes MediaPipe holistic to extract landmarks of the face, hands, and body in each frame of an image sequence to then be processed with Long Short-Term Memory (LSTM) deep learning model. The results show that the best model in this research gets an accuracy of 97%, concluding that the model produces decent results in recognition of BISINDO sign language alphabet gestures. Keywords: BISINDO, Sign Language Recognition, MediaPipe holistic, LSTM, Deep Learning. Terdapat kesenjangan antara orang normal dan orang tuli dalam hal komunikasi. Hal ini dikarenakan tidak semua orang normal belajar bahasa isyarat seperti orang tuli. Oleh karena itu, perlu ada sesuatu yang dapat menutup celah tersebut dengan memanfaatkan teknologi yang dapat mengenali bahasa isyarat. Ada dua bahasa isyarat yang digunakan secara luas di Indonesia, yaitu Sistem Isyarat Bahasa Indonesia (SIBI) dan Bahasa Isyarat Indonesia (BISINDO). Bahasa isyarat resmi yang digunakan di sekolah adalah SIBI, sedangkan BISINDO lebih umum digunakan oleh penyandang tunarungu. Dalam penelitian ini, penulis memaparkan temuannya untuk mengenali bahasa isyarat BISINDO Indonesia dari rangkaian gerak tubuh. Penulis mengumpulkan total 780 rangkaian gambar untuk 26 gestur alfabet BISINDO. Penulis memanfaatkan MediaPipe holistic untuk mengekstrak landmark wajah, tangan, dan tubuh di setiap frame rangkaian gambar untuk kemudian diolah dengan model deep learning Long Short Term Memory (LSTM). Hasil penelitian menunjukkan bahwa model terbaik dari penelitian ini akurasi sebesar 97%, menyimpulkan bahwa model tersebut memproduksi hasil yang cukup bagus dalam pengenalan gerakan alfabet bahasa isyarat BISINDO. Kata Kunci: BISINDO, Sign Language Recognition, MediaPipe holistic, LSTM, Deep Learning.

Actions (login required)

View Item