Day 12: Embeddings and Semantic Search for AI Agents

Today's outcome: Learn how an AI agent can search notes by meaning instead of requiring the question and the answer to use identical words.

This lesson is part of AI Agents in 30 Days. Day 11 showed why agents hallucinate when they answer without enough evidence. Today we begin building the retrieval layer that can find relevant evidence.

The limitation of keyword search

Keyword search works well when the query contains the same words as the document. If you search for project deadline, a traditional search system can easily find a note containing project deadline.

But people frequently express the same idea using different words:

• Question: When is my project due?

• Note: Submit the AI assignment by Friday.

The question contains neither submit nor Friday. The note contains neither project nor due. A strict keyword match may therefore rank this note poorly even though it contains the information the student needs.

Semantic search approaches the problem differently. It looks for related meaning rather than relying only on shared words. OpenAI's retrieval documentation defines semantic search as using vector embeddings to surface semantically relevant results, including results with few or no shared keywords. See the OpenAI Retrieval guide.

What is an embedding?

An embedding is a vector: an ordered list of numbers produced from an input such as a sentence, paragraph, image, or other supported data type. For text embeddings, those numbers provide a mathematical representation that can be compared with representations of other text.

You can use a simple mental model:

An embedding gives a piece of text coordinates on a map of meaning.

On this imaginary map, texts with related meaning tend to be closer together than unrelated texts. The real vectors can have many dimensions, so this map is not something we can draw perfectly on a two-dimensional screen. The picture is still useful for understanding the retrieval process.

The OpenAI embeddings guide describes an embedding as a list of floating-point numbers and explains that the distance between vectors measures their relatedness. Smaller distances indicate greater relatedness.

An embedding does not store a human-readable definition inside each number. You normally cannot inspect one position and say, "this number means deadlines." The useful information comes from the pattern across the vector and how that vector compares with others.

From notes to semantic search

A basic semantic-search flow has four steps:

1. Create an embedding for every note you want to search.

2. Store each note together with its embedding.

3. Create an embedding for the user's question using the same embedding model.

4. Compare the question vector with the note vectors and return the closest matches.

The system is not asking the language model to answer yet. It is first selecting the notes most likely to contain useful context.

The retrieved text can then be passed to an AI agent as context. In a RAG system, the model uses that retrieved context when producing its answer. We will assemble that complete pipeline on Day 14.

Carrying the example through

Imagine that a student's notebook contains these three entries:

1. The team meeting moved to Monday.

2. Submit the AI assignment by Friday.

3. Read the chapter about vector databases.

The student asks:

When must I submit the project?

A keyword-only search might focus on the word project, which does not appear in any note. Semantic search embeds the question, compares it with all three note embeddings, and should rank the second note as the closest meaning match.

The agent can now receive the relevant note as evidence and answer that the assignment is due Friday. Retrieval reduces the need to guess, but the agent should still distinguish between information found in the notes and information it has inferred.

Semantic search and keyword search are not enemies

Semantic search is not automatically better for every query.

Keyword search can be especially valuable for:

• Exact identifiers such as order numbers, error codes, or product SKUs.

• Names and technical terms whose precise spelling matters.

• Legal phrases that should not be approximated.

• Filters such as a known date, author, or category.

Semantic search is especially valuable when:

• Users describe the same concept using different vocabulary.

• Questions are written in natural language.

• Notes contain paraphrases or related concepts.

• You want to retrieve supporting passages for an AI agent.

Many practical systems combine keyword and semantic retrieval. The important lesson is to choose the search behavior that matches the kind of information your agent needs.

Try this today

Write one question and three short notes. Do not deliberately repeat the question's wording inside the correct note.

Example question:

When must I submit the project?

Example notes:

• Team meeting moved to Monday.

• AI assignment due this Friday.

• Read the chapter about vector databases.

Now ask yourself:

1. Which note is closest in meaning?

2. Would an exact keyword match find it reliably?

3. Which words or ideas make the connection clear to a person?

This no-code exercise builds the intuition you need before implementing a vector database or retrieval pipeline.

Completed example you can copy

Question:
When is my database homework due?

Notes:
A. Revise normalization before the assessment.
B. Submit the SQL assignment by Thursday evening.
C. The project team meets on Saturday.

Best semantic match:
B. Submit the SQL assignment by Thursday evening.

Why:
"Database homework" and "SQL assignment" describe related work.
"When is it due?" and "by Thursday evening" describe the same deadline concept.
The wording differs, but the meaning matches.

Common misconceptions

1. An embedding is a summary

An embedding is not a readable summary. It is a numerical representation used for comparison and other machine-learning tasks.

2. A close match must be factually correct

Similarity measures relatedness, not truth. A highly relevant note can still be outdated, incorrect, or malicious. Retrieval quality and source quality are separate concerns.

3. Semantic search eliminates keyword search

Exact matching remains useful for IDs, names, filters, and precise terminology. Hybrid search can combine both signals.

4. The closest note is always good enough

Your system still needs thresholds, sensible result counts, evaluation examples, and a fallback for queries with no trustworthy match.

5. Every whole document should become one embedding

Very long documents usually need to be divided into useful pieces before indexing. The way those pieces are created can preserve or destroy important context. That is the focus of Day 13: Chunking and Retrieval.

Knowledge check

1. Why can keyword search miss a useful note?

Because the question and the note may express the same idea using different words.

2. What does an embedding provide?

A numerical vector that allows text to be compared by relatedness.

3. Does vector similarity prove that a retrieved passage is true?

No. It indicates relatedness, not factual correctness or trustworthiness.

4. What happens after the relevant note is retrieved in a RAG workflow?

The note is supplied to the model as context so it can produce a grounded answer.

Keep learning with Korshub

If you want to go deeper into embeddings, vector databases, RAG, or AI-agent development, use Korshub to discover relevant courses and continue to the official course platform when you are ready to enroll. Course availability and offers can change, so review the current listing before continuing.

Continue the series

• Previous: Day 11 - Why Agents Hallucinate

• Roadmap: AI Agents in 30 Days

• Next: Day 13 will explain Chunking and Retrieval. It is intentionally unlinked until the article is published.