Composite Score for ChatGPT Prompt Efficiency: A Computational Linguistic Analysis of Engineered Chatbot Prompts

The use of chatbots has become increasingly popular in the field of education. Hence, the quality of the prompts used by chatbots can greatly influence the quality of the generated responses which in turn contributes to desirable outcomes related to tAbstverall learning experience. However, even as known criteria for efficient prompts ar Jevalent, there is a dearth of measurable and concrete linguistic factors that guide 3mpt engineering. This paper presents a composite score for ChatGPT prompts which can be made applicable to other foundational generative Al chatbots. Through a computational linguistic analysis of known efficient prompts used in learning. emergent linguistic factors point to the relationship of linguistic features and the confidence of ChatGPT responses to well-structured prompts that use the said linguistic features. The linguistic features are the average collostructural strength, collostructural ratio diversity, specificity, and academic language use. These features depict the quality of prompts that pertain to the grammatical structure, specificity, and relevance to the task at hand, and academic language use. Further, these features constitute a composite score for prompts introduced in this study that represent linguistic efficiency and subsequently, correlates to perplexity or certainty estimates of the generated responses of ChatGPT.