RAGs to Riches: Building Smarter AI with Retrieval-Augmented Generation on AWS

Posted on by Ryan Dagley

Hey fellow AI builders! As an engineer who loves tinkering with tech and sharing what I’ve learned, I’m excited to dive into Retrieval-Augmented Generation (RAG): a technique that’s been a game-changer for making AI more useful and accurate. Whether you’re building tools for work or just experimenting, RAG is worth knowing about. In this tutorial, I’ll break down what it is, why it matters, and how to set it up on AWS.

Let’s get to it.

What is RAG?

RAG combines two ideas to level up AI:

“How do you expect to get anything done with AI hallucinating all the time!?”
  • Retrieval: It searches a pool of external data – like docs, databases, or web content – for relevant info. In this case, “external” means not built into the AI model of choice.
  • Augmented Generation: It feeds that info into a generative AI model (think xAI’s Grok or OpenAI’s GPT) to craft a precise, context-rich response.

Picture a traditional AI model as a smart but isolated brain. RAG gives it a search engine to double-check its facts before answering. It’s perfect when you need fresh data or domain-specific knowledge without retraining a model from scratch.

For example, let’s say you’re interested in Mushrooms and Mycology. While ChatGPT and Grok could probably help to build a report, providing AI additional mycology info will allow a more precise understanding of the subject matter!

Why RAG Matters for Engineers

  • Current Data: Pulls in real-time info, unlike static models stuck in the past.
  • Precision: Grounds answers in facts, cutting down on wild guesses and AI hallucinations!
  • Flexibility: Use your own datasets – company docs, codebases, you name it. You can even send it mushroom identification info from a site like mine!

Since I’ve been playing with this on AWS, I’ll show you how to leverage its ecosystem – S3, SageMaker, Lambda – to make RAG sing.

How RAG Works: The Core Flow

Here’s the high-level process:

  1. A user asks a question like “What AWS service is best for ML data storage?”
  2. The retrieval system scans a knowledge base (e.g., AWS docs) for relevant snippets.
  3. The generative model uses those snippets to build a solid answer.

Underneath, you’re juggling:

  • A vector database for fast, similarity-based searches.
  • An embedding model to convert text into searchable vectors.
  • A generative AI to stitch it into natural language.

Hands-On: Building RAG on AWS

Let’s walk through setting up a RAG system on AWS. I’ll use a generic generative model (you can plug in xAI’s Grok, OpenAI, or whatever you prefer) and focus on the engineering nuts and bolts.

Step 1: Prep Your Knowledge Base

  • What: Collect documents – AWS guides, your own PDFs, etc. These are the documents that will further refine the knowledge that the AI will use.
  • AWS Tool: Amazon S3. It’s scalable, cheap, and perfect for storage.
  • How: Upload to an S3 bucket, e.g., s3://my-rag-bucket/docs/. Think of it as your data warehouse.

Step 2: Generate Embeddings

  • What: Turn your docs into vectors for searching.
  • AWS Tool: Amazon SageMaker. Spin up a notebook and use a pre-trained embedding model.
  • How:
    1. Launch a SageMaker notebook instance.
    2. Install sentence-transformers (pip install sentence-transformers).
    3. Process your docs with code similar to the following:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
python
 
from sentence_transformers import SentenceTransformer  # For converting text into vector embeddings
import boto3  # AWS SDK for Python—used to interact with S3
 
# Initialize the embedding model—loaded once for efficiency
model = SentenceTransformer('all-MiniLM-L6-v2')  # Small, fast model optimized for text similarity
 
# Set up an S3 client to access your bucket
s3 = boto3.client('s3')  # No credentials needed if running in SageMaker with an IAM role
 
# Define your S3 bucket name—replace with your actual bucket
bucket = 'my-rag-bucket'
 
# Fetch all objects from the bucket and extract their text content
# - list_objects() gets metadata for all files in the bucket
# - get_object() retrieves each file’s content, decoded from bytes to UTF-8 strings
# - Assumes all files are text-readable (e.g., .txt, .pdf may need extra parsing)
docs = [s3.get_object(Bucket=bucket, Key=key['Key'])['Body'].read().decode('utf-8')
        for key in s3.list_objects(Bucket=bucket)['Contents']]
 
# Generate embeddings for all documents
# - model.encode() processes the list of docs into a matrix of vectors
# - Each vector represents a document’s semantic meaning for later similarity search
embeddings = model.encode(docs)

Step 3: Set Up a Vector Store / Vector Database

  • What: A database to hold and query your embeddings.
  • AWS Tool: Amazon OpenSearch Service. It’s built for vector searches.
  • How:
    1. Create an OpenSearch domain via the AWS Console.
    2. Index your embeddings – use the OpenSearch API or a script.
    3. Test it with a query embedding to grab top matches.

For a single embedding, you can run the following in a command line terminal:

1
2
bash
curl -X PUT "http://your-opensearch:9200/my_index/_doc/1" -H 'Content-Type: application/json' -d '{"text": "hello world", "embedding": [-0.04, 0.32, ...]}'

The smarter way is to use a script that can loop through your embeddings and send in bulk.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
python
 
from opensearchpy import OpenSearch
 
# Connect to OpenSearch (replace with your setup)
client = OpenSearch(hosts=["http://localhost:9200"])
 
# Your index name
index_name = "my_rag_index"
 
# Create the index (only need this once)
client.indices.create(index=index_name, body={
    "mappings": {
        "properties": {
            "text": {"type": "text"},
            "embedding": {"type": "knn_vector", "dimension": 384}  # Match your model’s vector size
        }
    }
}, ignore=400)  # Ignore if it already exists
 
# Send all embeddings at once
actions = [
    {
        "_index": index_name,
        "_source": {
            "text": docs[i],
            "embedding": embeddings[i].tolist()  # Convert numpy array to list
        }
    }
    for i in range(len(docs))
]
from opensearchpy.helpers import bulk
bulk(client, actions)

Step 4: Wire Up the AI

  • What: Connect retrieval to generation.
  • AWS Tool: AWS Lambda. Lightweight and serverless—ideal for this.
  • How:
    1. Write a Lambda function to:
      • Embed the user’s query.
      • Search OpenSearch.
      • Pass results to your AI model’s API (e.g., xAI or OpenAI).
    2. Example code for the Lambda function with heavy commenting:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
python
 
import json  # For handling JSON data (e.g., Lambda response)
import requests  # For making HTTP requests to the AI API
from sentence_transformers import SentenceTransformer  # For generating text embeddings
 
# Load the embedding model once when Lambda initializes (saves time on cold starts)
model = SentenceTransformer('all-MiniLM-L6-v2')  # Lightweight, fast model for text-to-vector conversion
api_key = 'your-api-key'  # Replace with your AI provider’s API key (e.g., xAI, OpenAI)
api_endpoint = 'your-api-endpoint'  # Replace with your AI provider’s endpoint URL
 
def lambda_handler(event, context):
    # Extract the user’s query from the Lambda event (e.g., from API Gateway or direct invocation)
    query = event['query']
     
    # Convert the query into a vector embedding for similarity search
    # [0] extracts the first (and only) embedding since we pass a single query
    query_embedding = model.encode([query])[0]
     
    # Search OpenSearch for relevant documents using the query embedding
    # Note: search_opensearch() is a placeholder—implement it to query your OpenSearch domain
    retrieved_docs = search_opensearch(query_embedding)
     
    # Build a prompt combining the query and retrieved docs for the AI model
    # \n separates sections for clarity—tweak this format based on your AI provider’s needs
    prompt = f"Query: {query}\nContext: {retrieved_docs}"
     
    # Send the prompt to the AI API and get a generated response
    response = requests.post(
        api_endpoint,
        headers={'Authorization': f'Bearer {api_key}'},  # Authenticate with the API key
        json={'prompt': prompt}  # Pass the prompt as JSON payload
    )
     
    # Return a Lambda-friendly response: HTTP 200 status and the AI’s text output
    # Assumes the API returns a JSON object with a 'text' key—adjust if the structure differs
    return {'statusCode': 200, 'body': response.json()['text']}

Step 5: Test and Iterate

  • Deploy your Lambda and query it – “How does S3 work with ML?”
  • Check the response. Tweak as needed – maybe adjust the prompt or add more docs.

For your convenience, you can test the Lambda with inputs like:

1
2
3
4
5
json
 
{
  "query": "Which mushrooms are edible and considered choice in terms of flavor?"
}

Closing Thoughts: Why RAG on AWS Rocks

For me, RAG is a practical way to bridge the gap between static AI and dynamic data and AWS makes it scalable. S3 handles storage, SageMaker powers the heavy lifting, and Lambda ties it together. It’s a solid pattern I’ve shared with teammates, and I hope it sparks ideas for your projects too.

I are getting more smarter,
– Ryan

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts