In this episode, we talk data-driven search optimizations with Charlie Hull.Charlie is a search expert from Open Source Connections. He has built Flax, one of the leading open source search companies in the UK, has written “Searching the Enterprise”, and is one of the main voices on data-driven search.We discuss strategies to improve search systems quantitatively and much more.Key Points:Relevance in search is subjective and context-dependent, making it challenging to measure consistently.Common mistakes in assessing search systems include overemphasizing processing speed and relying solely on user complaints.Three main methods to measure search system performance: Human evaluationUser interaction data analysisAI-assisted judgment (with caution)Importance of balancing business objectives with user needs when optimizing search results.Technical components for assessing search systems: Query logs analysisSource data quality examinationTest queries and cases setupResources mentioned:Quepid: Open-source tool for search quality testingHaystack conference: Upcoming event in Berlin (September 30 - October 1)Relevance Slack communityOpenSource ConnectionsCharlie Hull:LinkedInX (Twitter)Nicolay Gerold:⁠LinkedIn⁠⁠X (Twitter)search results, search systems, assessing, evaluation, improvement, data quality, user behavior, proactive, test dataset, search engine optimization, SEO, search quality, metadata, query classification, user intent, search results, metrics, business objectives, user objectives, experimentation, continuous improvement, data modeling, embeddings, machine learning, information retrieval00:00 Introduction01:35 Challenges in Measuring Search Relevance02:19 Common Mistakes in Search System Assessment03:22 Methods to Measure Search System Performance04:28 Human Evaluation in Search Systems05:18 Leveraging User Interaction Data06:04 Implementing AI for Search Evaluation09:14 Technical Components for Assessing Search Systems12:07 Improving Search Quality Through Data Analysis17:16 Proactive Search System Monitoring24:26 Balancing Business and User Objectives in Search25:08 Search Metrics and KPIs: A Contract Between Teams26:56 The Role of Recency and Popularity in Search Algorithms28:56 Experimentation: The Key to Optimizing Search30:57 Offline Search Labs and A/B Testing34:05 Simple Levers to Improve Search37:38 Data Modeling and Its Importance in Search43:29 Combining Keyword and Vector Search44:24 Bridging the Gap Between Machine Learning and Information Retrieval47:13 Closing Remarks and Contact Information

Ver todos los episodios