Did you know that 78% of companies still don't measure the return on investment in artificial intelligence? (Source: "State of AI in Business 2026" report, McKinsey & Company, available at: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai). This is a paradox. They spend fortunes on AI tools but don't know if they generate value.

The problem starts with developer productivity. Companies spend hours training teams on new libraries and frameworks. The result? Inconsistent code and rework.

But this has changed. With RAG (Retrieval-Augmented Generation) and free Python libraries, you can build a custom code assistant that answers questions about your internal codebase. In hours, not weeks. In this tutorial, I'll show you how to build this assistant from scratch.

We'll use tools like ChromaDB for vector storage, sentence-transformers for embeddings, and OpenAI's GPT-4o mini for response generation. Three different approaches: basic RAG with semantic search, RAG with reranking, and a complete pipeline with caching.

In the end, you'll have a system that analyzes your internal documentation, answers technical questions, and suggests code snippets. All at near-zero cost.

Why Build a Custom Code Assistant Now?

The market has changed. Developers expect fast, accurate answers. Companies that don't offer efficient internal support lose productivity.

Generic assistants like ChatGPT have three serious problems: lack of specific context, risk of data leakage, and internet dependency. Answers based on public knowledge don't capture the particularities of your system. A model trained on public data might suggest libraries you don't use.

RAG solves this. It combines search in your internal knowledge base with natural language generation. If a new code pattern emerges, you update the vector database, and the assistant responds correctly. Result: more accurate and secure answers.

Furthermore, semantic search with sentence-transformers achieves over 90% recall in Portuguese for technical queries (source: paper "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", Reimers & Gurevych, 2019, available at: https://arxiv.org/abs/1908.10084). And the best part: it's free. You only need a machine with Python installed.

But having the tool isn't enough. You need to know how to use it. Let's get to the code.

Step-by-Step Tutorial: Three Approaches to Building Your Assistant

I'll divide the tutorial into three parts. Each uses a different technique. You can choose the one that fits your technical level and budget.

Approach 1: Basic RAG with ChromaDB and sentence-transformers (low cost, no API)

This is the simplest. You use local embeddings with sentence-transformers and vector storage with ChromaDB. The cost is zero: everything runs offline.

The technique lies in the pipeline design. You don't just search by keywords. You create semantic embeddings of your documents and queries. Example:

from sentence_transformers import SentenceTransformer
import chromadb

Load Portuguese embedding model

model = SentenceTransformer('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')

Create ChromaDB client

client = chromadb.Client() collection = client.create_collection(name="codigo_interno")

Add documents (example: documentation snippets)

documentos = [ "The calcular_frete function uses the Correios API with a 5-second timeout.", "The auth module implements JWT authentication with refresh token.", "For deployment, use the deploy.sh script that runs automated tests." ] embeddings = model.encode(documentos).tolist() collection.add( embeddings=embeddings, documents=documentos, ids=["doc1", "doc2", "doc3"] )

Search for an answer to a question

pergunta = "How to calculate shipping in the system?" embedding_pergunta = model.encode([pergunta]).tolist() resultados = collection.query(query_embeddings=embedding_pergunta, n_results=1) print(resultados['documents'][0])

Output: "The calcular_frete function uses the Correios API with a 5-second timeout."

With this pipeline, you can integrate it with a chatbot or web interface.

For more elaborate answers, use a local LLM like microsoft/phi-2. Pass the retrieved context and the question.

Pros: Free, offline, no external dependency.
Cons: Requires initial setup, local generation model may be less accurate.

Approach 2: RAG with Reranking and GPT-4o mini (medium cost, high accuracy)

If you want more accurate answers, add a reranking step. Use the cross-encoder model to reorder the semantic search results. Then, pass the best ones to GPT-4o mini.

The process is simple: first, retrieve the 10 most relevant documents with embeddings. Then, use a cross-encoder to rank these documents by relevance. Finally, pass the top 3 to the LLM to generate the answer.

Here's a code example:

from sentence_transformers import SentenceTransformer, CrossEncoder
import chromadb

Models

embedder = SentenceTransformer('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2') reranker = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')

Initial search

client = chromadb.Client() collection = client.get_collection(name="codigo_interno") pergunta = "How to deploy?" embedding_pergunta = embedder.encode([pergunta]).tolist() resultados = collection.query(query_embeddings=embedding_pergunta, n_results=10)

Reranking

pares = [(pergunta, doc) for doc in resultados['documents'][0]] scores = reranker.predict(pares) melhores_indices = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[:3] melhores_docs = [resultados['documents'][0][i] for i in melhores_indices]

Generate response with GPT-4o mini (via API)

import openai contexto = "\n".join(melhores_docs) prompt = f"Based on the context below, answer the question.\nContext: {contexto}\nQuestion: {pergunta}\nAnswer:" resposta = openai.ChatCompletion.create( model="gpt-4o-mini", messages=[{"role": "user", "content": prompt}], max_tokens=200 ) print(resposta.choices[0].message.content)

The cost is about US$ 0.15 per thousand output tokens for GPT-4o mini (source: OpenAI pricing page, available at: https://openai.com/api/pricing/). For 1000 queries with 200 output tokens each, the cost is approximately US$ 0.03.

Pros: High accuracy, natural responses.
Cons: Depends on paid API, requires internet.

Approach 3: Complete Pipeline with Cache (free, ready to use)

This is my favorite for those who want quick results without recurring costs. Use a local generation model like microsoft/phi-2 and implement caching for frequent responses.

from sentence_transformers import SentenceTransformer
import chromadb
from transformers import pipeline
import hashlib

Simple cache

cache = {}

Models

embedder = SentenceTransformer('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2') gerador = pipeline('text-generation', model='microsoft/phi-2')

Search and generation

def responder(pergunta): # Check cache hash_pergunta = hashlib.md5(pergunta.encode()).hexdigest() if hash_pergunta in cache: return cache[hash_pergunta]

# Semantic search
client = chromadb.Client()
collection = client.get_collection(name="codigo_interno")
embedding_pergunta = embedder.encode([pergunta]).tolist()
resultados = collection.query(query_embeddings=embedding_pergunta, n_results=3)
contexto = "\n".join(resultados['documents'][0])

# Local generation
prompt = f"Context: {contexto}\nQuestion: {pergunta}\nAnswer:"
resposta = gerador(prompt, max_new_tokens=150)[0]['generated_text']

# Store in cache
cache[hash_pergunta] = resposta
return resposta

Usage example

print(responder("How to calculate shipping?"))

The phi-2 model runs on CPU with 8GB of RAM. The cache avoids reprocessing identical questions.

Pros: Free, offline, fast after caching.
Cons: Local generation model may be less fluent than GPT-4o mini.

Cost and Time Comparison: Which to Choose?

Each approach has a trade-off. The table below summarizes:

If you need high accuracy and have a budget, go with RAG with reranking and GPT-4o mini. If you want independence and have decent hardware, go with the pipeline with cache. If it's a quick test, basic RAG.

Generating Performance Reports with Python

After building the assistant, you need to monitor its performance. Python does this with few lines. Use pandas to organize query logs and matplotlib to generate accuracy charts.

import pandas as pd
import matplotlib.pyplot as plt

Query log (example)

dados = { 'consulta': ['calculate shipping', 'deploy', 'authentication'], 'relevante': [True, True, False], 'tempo_resposta_ms': [120, 95, 150] } df = pd.DataFrame(dados)

Calculate metrics

acuracia = df['relevante'].mean() * 100 tempo_medio = df['tempo_resposta_ms'].mean() print(f"Accuracy: {acuracia:.1f}%") print(f"Average time: {tempo_medio:.0f} ms")

Response time chart

plt.bar(df['consulta'], df['tempo_resposta_ms']) plt.xlabel('Query') plt.ylabel('Time (ms)') plt.title('Response Time per Query') plt.show()

With this data, you can identify problematic queries and adjust the pipeline.

Conclusion

Building a custom code assistant with RAG and Python is a practical, low-cost strategy to increase your development team's productivity. In this tutorial, you learned three approaches: basic RAG with ChromaDB, RAG with reranking and GPT-4o mini, and a complete pipeline with local caching.

Each approach has its pros and cons, but they all share a principle: using semantic search to retrieve relevant context before generating responses. This eliminates the problems of generic assistants and ensures accurate and secure answers.

Now it's your turn. Choose the approach that best fits your scenario, implement the pipeline, and start reaping the benefits. Your development team will thank you.

Next steps: Try integrating the assistant with a chatbot on Slack or Microsoft Teams. Use tools like Streamlit to create a web interface. And don't forget to monitor performance with logs and metrics.

Share your results in the comments. Let's build better assistants together.

Related Articles

Also check out: How to Use AI to Create High-Quality Content in 2026 Also check out: From Dataset to Ollama: Fine-Tuning LLMs with Unsloth on Your GPU in 2026 Also check out: 48% Don't Test, 40% Hallucinate: How to Evaluate LLMs in 2026 — Analytical Guide