This page was translated automatically using artificial intelligence (DeepL). The German version is binding. More information about automatic translation
Artificial intelligence (AI) is transforming education – from digital learning assistants to chatbots that answer questions round the clock. For these systems to provide genuine support, they must not only recognise human language but also understand it meaningfully [1]. The focus is on two approaches: ‘prompt engineering’ and traditional techniques of natural language processing (NLP).
The study examined the use of both methods in an educational chatbot developed specifically to convey information about blockchain technology. Five typical tasks performed by such a chatbot were analysed: named entity recognition (NER), sentiment analysis (SA), answering questions (Question-Answering, QA), classifying texts (Text Classification, TC) and summarising texts (Text Summarisation, TS). For each of these tasks, both traditional NLP techniques and prompt engineering were employed. In addition, a so-called RAG system (Retrieval-Augmented Generation) was used in the prompt engineering process; this retrieves relevant documents from a specialised database and provides the language model with more context.
Standard quantitative metrics such as accuracy and F1-score were used to evaluate the methods; these were recorded automatically using the Python programming language. In addition, qualitative metrics such as relevance and completeness were also included in the evaluation to provide a more comprehensive picture of the results.
The results (Figure 1) show clear trends: for interpretative tasks such as sentiment analysis (scores: 9.51, 10, 10) and text summarisation (6.48, 6.17, 6.41), prompt engineering proved particularly effective. In contrast, for structured tasks such as named entity recognition (9.89) and text classification (9.17), traditional NLP techniques proved superior.
These differences are also reflected in the respective application potentials. Prompt engineering is particularly well-suited to dialogue-oriented scenarios, as it supports the flow of conversation and enables context-sensitive responses – a finding that confirms earlier studies. Furthermore, it is flexible, adaptable and resource-efficient, making it ideal for dynamic learning environments. Traditional NLP techniques, on the other hand, play a central role in structured analysis, system security (e.g. by filtering harmful content) and precise information retrieval (e.g. through entity recognition).
These clear findings underline the relevance of a targeted, combined use of both methods. A hybrid approach brings together the strengths of both sides and enables the development of chatbots that are conversational, robust and adaptable – a particularly promising path in the field of education.
About the person
Emma-Luisa Richter studied IT security at Mittweida University of Applied Sciences and has been working as a research assistant on the ‘BOLT’ project at the BCCM at Mittweida University of Applied Sciences since December 2024.
Bibliography
[1] Bekkar, Hibat-Allah; Chtouki, Yousra (2024): Chatbots in Education: A Systematic Literature Review, [online] ieeexplore.ieee.org/document/10690334 [10 January 2025]
[2] Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N.; Kaiser, Łukasz; Polosukhin, Illia (2017): Attention Is All You Need, [online] arxiv.org/abs/1706.03762 [12 January 2025]
