Tutorial series | OpenSearch-based AI Use Cases
Build a RAG-based Q&A system using OpenSearch and OpenAI
Learn how to build a RAG-based Q&A system using OpenSearch and OpenAI.
Basic information
- Estimated time: 40 minutes
- Recommended OS: macOS, Ubuntu
- Prerequisites
About this scenario
This tutorial stores unstructured text data such as internal documents, FAQs, and technical guides in OpenSearch and combines it with an OpenAI LLM to build a natural-language question answering (Q&A) system.
OpenSearch quickly retrieves large-scale documents, and OpenAI generates answers to user questions based on the retrieved documents.
You will cover the following:
- Convert documents into vectors (embeddings) and store them in OpenSearch
- Convert questions into vectors and perform semantic search
- Pass search results to an LLM to generate context-based answers
- Improve accuracy and freshness compared to using an LLM alone
Before you start
This tutorial assumes that you have completed Implement vector search by integrating OpenSearch with OpenAI.
Getting started
Step 1. Prepare document data for Q&A
Assume that population data for Seoul, Busan, and Incheon is stored in the following format.
POST _bulk
{ "index": { "_index": "korea_population", "_id": "1" } }
{ "text": "This is a chart and table about the population size and growth rate of the Seoul metropolitan area from 1950 to 2023. It also includes UN population projections through 2035.\nThe current population of the Seoul metropolitan area in 2023 is 25,925,000, a 0.32% increase from 2022.\nThe population of the Seoul metropolitan area in 2022 was 25,842,000, a 0.21% increase from 2021.\nThe population of the Seoul metropolitan area in 2021 was 25,788,000, a 0.05% increase from 2020.\nThe population of the Seoul metropolitan area in 2020 was 25,775,000, a 0.1% decrease from 2019." }
{ "index": { "_index": "korea_population", "_id": "2" } }
{ "text": "This is a chart and table about the population size and growth rate of the Busan metropolitan area from 1950 to 2023.\nThe current population of the Busan metropolitan area in 2023 is 3,345,000, a 0.15% decrease from 2022.\nThe population of the Busan metropolitan area in 2022 was 3,350,000, a 0.30% decrease from 2021.\nThe population of the Busan metropolitan area in 2021 was 3,360,000, a 0.45% decrease from 2020.\nThe population of the Busan metropolitan area in 2020 was 3,375,000, a 0.50% decrease from 2019." }
{ "index": { "_index": "korea_population", "_id": "3" } }
{ "text": "This is a chart and table about the population size and growth rate of the Incheon metropolitan area from 1950 to 2023.\nThe current population of the Incheon metropolitan area in 2023 is 3,015,000, a 0.85% increase from 2022.\nThe population of the Incheon metropolitan area in 2022 was 2,990,000, a 0.90% increase from 2021.\nThe population of the Incheon metropolitan area in 2021 was 2,963,000, a 1.10% increase from 2020.\nThe population of the Incheon metropolitan area in 2020 was 2,931,000, a 1.20% increase from 2019." }
Embeddings are automatically generated for each document and stored in the vector index.
Step 2. Create an ML Connector in OpenSearch
Create an ML Connector that uses the GPT-4o model.
POST _plugins/_ml/connectors/_create
{
"name": "OpenAI GPT-4o",
"description": "Connector for OpenAI GPT-4o (Korean Demo)",
"version": "1.0",
"protocol": "http",
"parameters": {
"endpoint": "api.openai.com",
"model": "gpt-4o"
},
"credential": {
"openAI_key": "${OPENAI_API_KEY}"
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"url": "https://${parameters.endpoint}/v1/chat/completions",
"headers": {
"Authorization": "Bearer ${credential.openAI_key}"
},
"request_body": "{ \"model\": \"${parameters.model}\", \"messages\": ${parameters.messages} }"
}
]
}
| 환경변수 | 설명 |
|---|---|
| OPENAI_API_KEY🖌︎ | API key issued by OpenAI |
Step 3. Deploy the model in OpenSearch
Deploy the Connector created above as a Model.
POST /_plugins/_ml/models/_register?deploy=true
{
"name": "OpenAI GPT-4o model (Korean)",
"function_name": "remote",
"description": "OpenAI GPT-4o remote model for Korean RAG demo",
"connector_id": "${CONNECTOR_ID}"
}
| 환경변수 | 설명 |
|---|---|
| CONNECTOR_ID🖌︎ | CONNECTOR_ID created above |
Step 4. Create a Search Pipeline
Configure a Search Pipeline that passes retrieved documents to the LLM.
PUT /_search/pipeline/korean-population-search-pipeline
{
"response_processors": [
{
"retrieval_augmented_generation": {
"tag": "Population Demo Pipeline",
"description": "Population RAG Pipeline using OpenAI GPT-4o",
"model_id": "${MODEL_ID}",
"context_field_list": [
"text"
],
"system_prompt": "You are an AI that answers accurately based on Korean city population data.",
"user_instructions": "Answer concisely within 100 characters based on the given data."
}
}
]
}
| 환경변수 | 설명 |
|---|---|
| MODEL_ID🖌︎ | MODEL_ID created above |
Step 5. Run RAG search
Pass the context above to the OpenAI GPT model to generate an answer.
GET /korea_population/_search?search_pipeline=korean-population-search-pipeline
{
"query": {
"match": {
"text": "How much did Incheon's population increase from 2021 to 2023?"
}
},
"size": 1,
"_source": [
"text"
],
"ext": {
"generative_qa_parameters": {
"llm_model": "gpt-4o",
"llm_question": "How much did Incheon's population increase from 2021 to 2023?",
"context_size": 5,
"timeout": 15
}
}
}
Step 6. Final Q&A response
The LLM generates an answer like the following based on the retrieved document.
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 2.024196,
"hits": [
{
"_index": "korea_qa_demo",
"_id": "3",
"_score": 2.024196,
"_source": {
"text": "This is a chart and table about the population size and growth rate of the Incheon metropolitan area from 1950 to 2023.
The current population of the Incheon metropolitan area in 2023 is 3,015,000, a 0.85% increase from 2022.
The population of the Incheon metropolitan area in 2022 was 2,990,000, a 0.90% increase from 2021.
The population of the Incheon metropolitan area in 2021 was 2,963,000, a 1.10% increase from 2020.
The population of the Incheon metropolitan area in 2020 was 2,931,000, a 1.20% increase from 2019."
}
}
]
},
"ext": {
"retrieval_augmented_generation": {
"answer": "It increased by 52,000."
}
}
}
tip
By combining Advanced Managed Search vector search with an OpenAI LLM, you can build a semantic AI search system that goes beyond fixed keyword search.
Use this approach to expand existing search infrastructure into an AI-based knowledge platform.