After my building-design experiment, I wanted to try something smaller: give GPT access to one reinforced concrete (RC) design code PDF and see which passages it uses to answer a question.
I also wanted to use my own GPU to prepare and search the document. That led to this setup: a local embedding model, Qdrant to store and search the vectors, and Codex CLI to read the retrieved passages and answer.
The code is in maksonlee/structural-rag on GitHub. This is a hands-on RAG demo, not a structural design application.
Why Use RAG for a Structural Code?
If I ask GPT about reinforcement in an RC column, it can explain the topic from its existing knowledge. But I may need the requirements from a particular code and edition. A general answer does not tell me whether it matches the PDF I am using.
I could find the relevant clause myself and paste it into the conversation. For an occasional question, that may be enough. RAG automates that lookup: search the document, retrieve candidate passages, and give them to GPT with the question.
I wanted an answer accompanied by text I could check against the PDF. That is the reason for keeping the retrieved passages visible throughout this walkthrough.
How the Pieces Fit Together
RAG stands for Retrieval-Augmented Generation. This project has two flows:
BUILD THE INDEX — LOCAL
Code PDF → Text + Page Numbers → Chunks → Local Embeddings → Qdrant
ANSWER A QUESTION
Question → Local Embedding → Qdrant Search → Retrieved Excerpts
|
v
GPT Context in Codex
|
v
Answer + References
An embedding represents text as a list of numbers. Similar passages tend to produce nearby vectors, which lets us search by meaning rather than requiring an exact keyword match.
Qdrant stores the vectors alongside the text, filename, and page. Given a question vector, it searches for nearby document vectors and returns their associated text.
None of this trains or modifies GPT. We build an index, search it, and supply excerpts for the current answer. GPT’s model weights stay the same.
The vectors are not an OpenAI-specific format, so an OpenAI embedding API is not required. The document and question do need compatible vectors: this project uses the same local model and its document/query encoding rules for both.
Before You Start
I ran this on Windows 11 with Ubuntu 24.04 in WSL 2 and an NVIDIA GeForce RTX 5060 Ti with 16 GB of VRAM. The walkthrough uses GPU inference, but the project also has a CPU setup.
You need Python 3.11 or later, Git, Docker with Compose, and Codex CLI. Run the commands inside WSL Ubuntu; my earlier WSL 2 and RTX 5060 Ti post covers the GPU environment.
Indexing and retrieval run locally without an OpenAI API key. For the answering step, this walkthrough uses Codex with ChatGPT sign-in.
1. Set Up the Project
Clone the repository and create a virtual environment:
mkdir -p ~/src
cd ~/src
git clone https://github.com/maksonlee/structural-rag.git
cd structural-rag
python3 -m venv .venv
source .venv/bin/activate
For GPU inference, check that WSL can see the GPU, then install the GPU dependencies:
/usr/lib/wsl/lib/nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv
python -m pip install -r requirements-gpu.txt
For CPU inference, install requirements.txt instead. Choose one dependency file for the virtual environment; the README covers troubleshooting.
The files we will follow are:
structural-rag/
├── AGENTS.md # Instructions for Codex
├── .env.example
├── docker-compose.yml
├── data/ # Your PDF goes here
└── src/
├── config.py
├── embeddings.py # Local model inference
├── ingest.py # PDF → chunks → Qdrant
└── retrieve.py # Question → matching chunks
I kept these as a few ordinary Python files so the retrieval process would be easy to follow.
2. Configure the Local Model
Having 16 GB of VRAM made it tempting to use a larger model. I compared several sizes and model families on the same PDF before settling on Qwen3-Embedding-0.6B.
With five chunks retrieved per question, it covered the designated passages for 25 of 28 questions. The larger Qwen models did not improve that total. This was a passage-retrieval check, not a score for GPT’s answers; the model comparison report has the individual results and GPU measurements.
For this demo, that gave me a reason to keep the smaller model. To use it on the GPU, copy the configuration:
cp .env.example .env
Then set these values in .env:
EMBEDDING_MODEL=Qwen/Qwen3-Embedding-0.6B
EMBEDDING_DEVICE=cuda
EMBEDDING_BATCH_SIZE=16
QDRANT_URL=http://localhost:6333
QDRANT_COLLECTION=structural_codes_qwen
TOP_K=5
HF_HUB_OFFLINE=0
For CPU inference, keep EMBEDDING_DEVICE=cpu and EMBEDDING_BATCH_SIZE=16 from the example file.
The first run downloads the model. Once it is cached, HF_HUB_OFFLINE=1 lets you run inference without further model downloads. This version’s loader supports Qwen3-Embedding-0.6B; changing to another model also requires adapting the loader. GPT is selected separately in Codex.
3. Start Qdrant
Qdrant runs as one Docker container using the project’s Compose file. Start it from the repository root:
docker compose up -d
docker compose ps
curl --fail http://localhost:6333/readyz
The curl command checks readiness. You can also inspect the database at http://localhost:6333/dashboard. Its files stay in the local qdrant_storage/ directory when the container stops.
4. Get the Code PDF
I used Taiwan’s 建築物混凝土結構設計規範（112年版，113年勘誤） from the National Land Management Agency. The official listing identifies the edition and correction, so we can work from the same publisher’s PDF.
Download it into data/:
curl --fail --location \
--output data/tw-concrete-code-2024.pdf \
https://www.nlma.gov.tw/uploads/files/011d9249cac7d6c5547786aa348e352a.pdf
You can use another legally obtained, text-based RC code PDF. PDFs, .env, model caches, and Qdrant storage are excluded from Git by the project’s .gitignore.
5. Extract and Chunk the Text
To preserve page references, ingest.py uses PyMuPDF to read one page at a time. This excerpt shows the start of the process:
for page_number, page in enumerate(document, start=1):
text = page.get_text("text", sort=True)
page_chunks = chunk_text(text)
Each chunk keeps the filename, page number, position within the page, and original text. The page reference is the physical PDF page, starting from 1, which may differ from the number printed on it.
I used 400-character chunks with an 80-character overlap, set in config.py. The window moves forward 320 characters at a time and stays within one page. The overlap keeps some surrounding text, but a clause or its exceptions can still be split.
6. Create Embeddings and Store the Chunks
The next part of ingestion turns those chunks into vectors locally, through the project’s embedding helper:
embeddings = embed_texts(model, [chunk["text"] for chunk in batch],
kind="passage", batch_size=batch_size)
Qwen runs locally through PyTorch and Transformers, producing a 1,024-number vector for each chunk. The helper handles the encoding details, including a retrieval instruction for questions. Both ingestion and search use it.
The program creates a Qdrant collection with qdrant.create_collection(), then stores the vectors alongside the text and page information:
points = [
models.PointStruct(id=str(uuid4()), vector=vector, payload=chunk)
for chunk, vector in zip(batch, embeddings)
]
qdrant.upsert(collection_name=collection, points=points, wait=True)
These operations are all part of one command:
python -m src.ingest data/tw-concrete-code-2024.pdf
My import produced 3,362 chunks from the 591-page PDF:
Stored 3362/3362 chunks.
Ingestion complete: tw-concrete-code-2024.pdf -> structural_codes_qwen.
To try another PDF or change chunk settings, choose a new QDRANT_COLLECTION in .env before importing. The importer refuses collections that already contain chunks.
7. Retrieve Passages for a Question
I used this question about the minimum clear spacing between longitudinal bars in a column:
python -m src.retrieve "柱內縱向鋼筋的最小淨間距，必須比較哪些限制？"
The command shows the retrieved text, so we can inspect the search before involving GPT. Inside retrieve.py, it first embeds the question:
question_vector = embed_texts(model, [question], kind="query")[0]
That vector goes directly into a Qdrant search:
hits = qdrant.query_points(
collection_name=collection,
query=question_vector,
limit=config["top_k"],
with_payload=True,
with_vectors=False,
).points
TOP_K=5 requests up to five matches. Our retrieve() function is ordinary application code that returns their text, filename, page, and cosine similarity score.
In this run, the first result included clause 25.2.3 on PDF page 442. Here is a shortened view of the output:
Rank 1 | tw-concrete-code-2024.pdf | PDF page 442 | score 0.8476
...
25.2.3 柱、柱墩、壓桿及牆內邊界構件內之縱向鋼筋，
...
Open page 442 to check the clause and its conditions. The score measures similarity; it cannot establish whether the clause applies. Adding --json returns the same results in the format Codex will read.
8. Let Codex Use the Retrieved Text
With Codex CLI installed, sign in using your ChatGPT account and start it from the repository root:
codex login
codex
ChatGPT plan eligibility and Codex usage limits still apply. See the authentication documentation for the sign-in options.
Ask Codex:
請先使用本專案的本機檢索工具，再依取得的規範片段回答：
柱內縱向鋼筋的最小淨間距，必須比較哪些限制？
請引用檔名與 PDF 頁碼；資料不足就明說。
The connection turned out to be a short instruction in AGENTS.md: before answering a question about this PDF, run the local retrieval command:
.venv/bin/python -m src.retrieve --json '柱內縱向鋼筋的最小淨間距，必須比較哪些限制？'
Codex receives the JSON as tool output in the conversation. Those excerpts enter GPT’s context alongside the question, and GPT generates the answer. Only the retrieved chunks are supplied.
AGENTS.md also requires source/page references and an explicit admission when the text is insufficient. It tells Codex to treat PDF text as reference material and separate explanation from code requirements. These instructions are not an automatic hook: check that the command actually ran. The README’s Codex setup section covers local tool access if it fails.
In the tested run, Codex retrieved five chunks and cited [tw-concrete-code-2024.pdf, PDF p. 442]. It also noted missing symbol definitions: the spacing clause had arrived without everything needed to explain it.
There is no OpenAI API key in this route. The question and retrieved excerpts still go to the Codex service, so local retrieval does not make the answering step offline.
Did the Retrieved Text Help?
Getting a source reference was one part of the experiment. I also wanted to know whether I would have been better off simply asking GPT.
The direct GPT versus RAG check asked the same 28 questions with and without saved Qwen 0.6B excerpts. For a beam-width question, the direct answer reversed a minimum/maximum comparison. With the retrieved text, GPT preserved the code’s comparison.
There were also three questions that GPT answered correctly on its own but could not fully answer with RAG. Two searches missed the relevant passages. In the third, PDF extraction had damaged an equation, and GPT left that part unresolved.
Saying the evidence is insufficient avoids inventing a requirement, but it still leaves the question unanswered. These cases gave me a reason to inspect what RAG retrieved, even when the final response looked reasonable.
This was a small experiment reviewed by AI, using the same questions as the model comparison. It needs new questions and independent engineering review before supporting broader claims about accuracy.
Limitations and Next Steps
A 400-character chunk is convenient to implement, but definitions, exceptions, and clauses that continue onto another page can sit just outside it. Search can also return a similar clause for the wrong member type.
Tables and equations need particular care. This version uses plain text extraction, with no OCR or table reconstruction, and the damaged equation in the comparison showed the consequence. A page reference helps us inspect the PDF; it does not make the retrieval clause-aware or ensure GPT has interpreted the text correctly.
The code edition also needs checking for a real project. A stored PDF does not update when regulations change.
Simple vector search is not enough for a professional engineering design system. The answers still require professional structural engineering review; RAG does not make GPT a structural engineer.
For a next experiment, I would keep this setup and try new questions alongside better chunk boundaries. The command output gives us something concrete to compare: which text reached GPT, what it left out, and how that affected the answer.
Did this guide save you time?
Support this site