{"id":4820,"date":"2024-08-15T21:32:28","date_gmt":"2024-08-15T19:32:28","guid":{"rendered":"https:\/\/talkingtochatbots.com\/?page_id=4820"},"modified":"2024-12-03T02:14:45","modified_gmt":"2024-12-03T00:14:45","slug":"predicting-chatbot-arena-votes-with-the-scbn-and-rqtl-benchmarks","status":"publish","type":"post","link":"https:\/\/talkingtochatbots.com\/es\/predicting-chatbot-arena-votes-with-the-scbn-and-rqtl-benchmarks\/","title":{"rendered":"Modelos predictivos sobre el LMSYS Chatbot Arena mediante m\u00e9tricas SCBN y RQTL"},"content":{"rendered":"\n

Below is the notebook I submitted (late) to the LMSYS – Chatbot Arena Human Preference Predictions<\/a> competition on Kaggle. This notebook applies NLP techniques for classifying text with popular Python libraries such as scikit-learn and TextBlob, and my own fine-tuned versions of Distilbert<\/a>. The notebook introduces the first standardized version of SCBN<\/a> (Specificity, Coherency, Brevity, Novelty) quantitative scores for evaluating chatbot response performance. Additionally, I introduced a new benchmark for classifying prompts named RQTL (Request vs Question, Test vs Learn), which aims to refine human choice predictions and provide context for the SCBN scores based on inferred user intent. You can check all the code, annotations, and charts in the Kaggle widget below.<\/p>\n\n\n\n

Explore and run the notebook on Kaggle<\/a><\/p>\n\n\n\n

Introduction to the SCBN benchmark<\/a><\/p>\n\n\n\n

SCBN chatbot battles on Talking to Chatbots<\/a><\/p>\n\n\n\n

Expanded GitHub repository with new notebooks, scripts, and charts with insights about the SCBN-RQTL framework<\/a><\/p>\n\n\n\n

\n\n\n\n