RAMADHANI, NATHAN (2025) RANCANG BANGUN PERANGKAT LUNAK CHATBOT BERBASIS LLM META LAMA 3.1 : STUDI KASUS LAYANAN UMB CAN. S1 thesis, Universitas Mercu Buana Jakarta.
|
Text (HAL COVER)
01 COVER.pdf Download (461kB) | Preview |
|
![[img]](https://repository.mercubuana.ac.id/style/images/fileicons/text.png) |
Text (BAB I)
02 BAB 1.pdf Restricted to Registered users only Download (115kB) |
|
|
Text (BAB II)
03 BAB 2.pdf Restricted to Registered users only Download (234kB) |
|
|
Text (BAB III)
04 BAB 3.pdf Restricted to Registered users only Download (314kB) |
|
|
Text (BAB IV)
05 BAB 4.pdf Restricted to Registered users only Download (579kB) |
|
|
Text (BAB V)
06 BAB 5.pdf Restricted to Registered users only Download (30kB) |
|
|
Text (DAFTAR PUSTAKA)
07 DAFTAR PUSTAKA.pdf Restricted to Registered users only Download (166kB) |
|
|
Text (LAMPIRAN)
08 LAMPIRAN.pdf Restricted to Registered users only Download (970kB) |
Abstract
The implementation of Artificial Intelligence (AI) is becoming increasingly crucial in supporting service automation across various sectors, including higher education. Universitas Mercu Buana (UMB), through its UMB CAN unit (Career, Alumni, and Network), strives to provide the best services to students and alumni, particularly in career development and alumni engagement. Numerous programs such as Tracer Study, Tracer Games, Alumni Talks, and Campus Hiring present challenges in delivering quick and accurate responses to user inquiries. This study aims to explore the use of the Large Language Model (LLM) Meta LLaMA 3.1 in developing a chatbot as a solution to enhance UMB CAN services. The chatbot's data sources were derived from internal documents, including service guides, event articles, and Frequently Asked Questions (FAQ). These documents were processed using chunking techniques with the Recursive Character Text Splitter (chunk size 500, chunk overlap 20), then converted into vector form via embedding using the nomic-embed-text model. Context retrieval was carried out using vector similarity search through Pinecone, and the results were used within a Retrieval-Augmented Generation (RAG) framework to generate responses. The chatbot was developed modularly using the Flowise platform, supporting a low-code approach. The LLM was deployed locally through Ollama, making the system efficient without relying on external APIs. The findings show that the chatbot system is capable of providing relevant and contextual responses to user queries. Out of 15 tested questions, the chatbot achieved a response accuracy of 93.3% and an Exact Match Rate (EMR) of 90%, indicating that most responses were either identical or very close to the expected answers. Performance evaluation also showed fast response times, with a median response time (p50) of 30 ms, p90 of 50 ms, and an average actual response time of approximately ±100 ms from the user side. This system has the potential to reduce staff workload and improve service efficiency, although challenges remain, such as high hardware requirements and dependency on wellstructured source data. Kata kunci: Large Language Model, Meta LLaMA 3.1, Chatbot, UMB CAN, Retrieval-Augmented Generation, Pinecone Penerapan kecerdasan buatan (AI) semakin krusial dalam mendukung otomatisasi layanan di berbagai sektor, termasuk pendidikan tinggi. Universitas Mercu Buana (UMB), melalui unit kerja UMB CAN (Career, Alumni, and Network), berupaya memberikan layanan terbaik kepada mahasiswa dan alumni, khususnya dalam pengembangan karir dan relasi alumni. Banyaknya program yang dijalankan, seperti Tracer Study, Tracer Games, Alumni Talks, dan Campus Hiring, menimbulkan tantangan dalam menyediakan respons yang cepat dan akurat terhadap pertanyaan pengguna. Penelitian ini bertujuan mengeksplorasi pemanfaatan teknologi Large Language Model (LLM) Meta LLaMA 3.1 dalam pengembangan chatbot sebagai solusi peningkatan layanan UMB CAN. Sumber data chatbot diperoleh dari dokumen internal, seperti panduan layanan, artikel kegiatan, dan Frequently Asked Questions (FAQ). Data tersebut diproses menggunakan teknik chunking dengan Recursive Character Text Splitter (chunk size 500, chunk overlap 20), lalu direpresentasikan dalam bentuk vektor melalui proses embedding menggunakan model nomic-embed-text. Proses pencarian konteks dilakukan melalui vector similarity search menggunakan Pinecone, dan hasilnya digunakan dalam skema Retrieval-Augmented Generation (RAG) untuk menghasilkan jawaban. Implementasi chatbot dikembangkan secara modular menggunakan platform Flowise, yang mendukung pendekatan low-code. Penggunaan LLM secara lokal melalui Ollama terbukti efisien tanpa bergantung pada API eksternal. Hasil penelitian menunjukkan bahwa sistem chatbot mampu memberikan jawaban yang relevan dan kontekstual terhadap pertanyaan pengguna. Dari total 15 pertanyaan yang diuji, chatbot menunjukkan response accuracy sebesar 93,3%, serta Exact Match Rate (EMR) sebesar 90%, yang berarti sebagian besar respons identik atau sangat mendekati jawaban yang diharapkan. Evaluasi performa juga menunjukkan waktu respon yang sangat cepat, dengan median response time (p50) sebesar 30 ms, p90 sebesar 50 ms, dan rata-rata waktu respon aktual ±100 ms dari sisi pengguna. Sistem ini juga berpotensi mengurangi beban kerja staf serta meningkatkan efisiensi layanan. Namun, tantangan masih ditemukan, seperti kebutuhan perangkat keras yang tinggi dan ketergantungan pada struktur data sumber yang terorganisasi dengan baik. Kata kunci: Model Bahasa Besar, Meta LLaMA 3.1, Chatbot, UMB CAN, Retrieval-Augmented Generation, Pinecone
Actions (login required)
 |
View Item |
