Is Philosophy a Science? Introducing SCBN Chatbot Battles
The SCBN (Specificity, Coherency, Brevity, Novelty) benchmark is a method to evaluate the output quality of language models and chatbots. SCBN provides a clear and systematic way to compare and assess chatbot responses based on four main metrics.
– Specificity (S): evaluates if a chatbot’s response is directly related to the user’s request. It checks how accurately the response addresses the prompt without deviating from the topic.
– Coherency (C): measures the logical structure of the response. It ensures that the information in the response is presented in a clear and organized manner, making it easy for the user to understand.