Harry Potter and the Elastic Semantic Search

How can you find text about bravery with a negative sentiment? Learn to build a semantic search engine using Elasticsearch and the world of Harry Potter.

#1about 3 minutes

The evolution of NLP from early models to modern LLMs

Tracing the rapid advancement of natural language processing from early models like Word2Vec to the powerful generative AI we see today.

#2about 5 minutes

How vector embeddings represent language as numbers

Vector embeddings turn words and sentences into numerical arrays, allowing computers to understand semantic relationships through mathematical operations.

#3about 7 minutes

Using vector similarity and LLMs for semantic operations

The distance between vectors in an embedding space represents semantic similarity, enabling operations like finding related concepts or answering questions.

#4about 4 minutes

Using Elasticsearch as a vector database for search

Elasticsearch serves as a vector database to store document embeddings and integrates with models from sources like Hugging Face for inference.

#5about 7 minutes

Demonstrating advanced keyword search with the Python client

The Elasticsearch Python client enables complex, multi-field queries with boolean logic to filter data based on precise criteria before adding semantic layers.

#6about 4 minutes

Enriching data with sentiment analysis pipelines

An inference pipeline can automatically apply a sentiment analysis model to all documents, adding a new field to enable filtering by positive or negative tone.

#7about 4 minutes

Implementing semantic search with embedding models

By converting all text into vectors using an embedding model, you can perform a k-NN search to find the most semantically relevant results for a query.

#8about 5 minutes

Refining results with hybrid search techniques

Hybrid search combines the power of semantic vector search with traditional keyword filters and exclusions to create highly relevant and precise results.

#9about 19 minutes

Audience Q&A on models and implementation

The speaker answers audience questions about ensuring relevance, handling out-of-vocabulary terms, updating data sources, and debugging model outputs.