In Engineering, Procurement and Construction (EPC) companies, every bid starts with a tender document that runs to hundreds of pages. A single RFP package holds the scope of work, technical specifications, data requirement sheets, bills of quantities, contract conditions with penalty clauses, statutory and safety obligations, and a stack of annexures. It arrives with a deadline, and tendering, engineering, supply chain and commercial teams all have to work through it in parallel.
Today that work is manual. Proposal engineers read section by section, trying to catch conflicts between sections, spot implicit requirements, flag risks, and write Tender Study Notes (TSNs) for the go/no-go decision. Each engineering discipline reads the same documents separately. A missed requirement can turn into a significant financial loss once the project is under way.
Smart RFP is our answer: a persistent, queryable knowledge base over tender documents that every persona in the bid process can ask questions of in plain language. Instead of a separate tool for each task, one well-built knowledge base can serve requirement extraction, risk flagging, contradiction detection and content generation through a common interface. The AI surfaces and structures information. The analyst still interprets, decides and owns the outcome.
Retrieval-augmented generation (RAG) is the foundation, because tender analysis needs answers grounded in the source document and traceable back to it. This post describes three RAG experiments we ran on a real 450-page EPC tender, and what they taught us about where the real bottlenecks are.
The short version: the LLM was never the problem. Document coverage and query intelligence were.
Why tenders are hard for AI
Before the experiments, it’s worth being clear about what makes these documents difficult.
Parsing. Tenders aren’t clean, flat text:
| Content type | Examples | What it needs |
|---|---|---|
| Narrative text | Scope, qualification requirements, bidding rules | Standard PDF text extraction |
| Structured tables | Bills of quantities, data requirement sheets, equipment specs, milestone schedules | Layout-aware table extraction |
| Scanned pages | Legacy drawings, signed pages | OCR |
| Embedded diagrams | Grid maps, single-line diagrams, schematics | Image parsing, multimodal embedding |
| CAD drawings | Engineering drawings referenced in specs | Specialist CAD tooling |
| Cross-references | IEC/IS/IEEE standards, references between sections | Reference resolution and linking |
If an answer lives in a table or a diagram that was never ingested, no retrieval strategy can find it.
Scattered requirements. A requirement is often introduced in one section, qualified in another and modified again in an annexure. In our tender, the project schedule states a headline commercial operation date (COD), but force majeure clauses, third-party dependencies and extension conditions elsewhere materially change the real timeline. Retrieve only the most similar chunk and you get the headline without the nuance.
Contradictions. In EPC tenders, conflicts between sections aren’t typos. They’re genuine scope risks. A list of future provisions in the notification can differ from the corresponding list in the technical specification. A penalty clause can conflict with a milestone schedule defined elsewhere. Finding these means retrieving related clauses from different parts of the document and comparing them. Standard similarity search returns passages that are individually relevant, not ones that are structurally related.
Terminology. BoQ, DRS, COD, LILO, PLCC, GIS, STU: EPC tenders are dense with domain abbreviations that general-purpose embedding models don’t handle well.
Three layers of a RAG system
It helped to think of RAG as three functional layers, each of which can sit anywhere between simple and complex:
- Query intelligence: turning the user’s question into a good retrieval signal. Simple: embed the query. Complex: rephrase it, decompose it into sub-queries, and fuse several retrieval signals.
- Knowledge management: building the knowledge base. Simple: chunk text into a flat vector store. Complex: metadata, graph and hybrid embeddings, and structured stores that support filtered and multi-hop retrieval.
- Response intelligence: producing the answer. Simple: prompt an LLM with the retrieved context. Complex: multi-step reasoning, structured outputs and agentic orchestration.

The experiments were designed to move along that spectrum one step at a time, so that any change in the scores could be traced back to a specific change in the architecture.
And this is the knowledge base we’re working towards: tenders, past bids, TSN templates, standards, win/loss data and diagrams flowing through parsing, chunking, embedding and a vector store, with bid outputs such as TSNs, risk registers and contradiction reports produced on top.

The dataset and how we measured
We used a real 450-page RFP, issued by a power sector body, for a power transmission project: substations at two voltage levels in both air-insulated and gas-insulated designs, transmission lines, underground cables, communication systems, protection schemes, and data requirement sheets for more than 20 categories of equipment, plus qualification rules, bid formats and legal annexures. It cites more than 30 IEC/IS/IEEE standards. In other words, a typical EPC tender.
All three pipelines were evaluated on the same eight queries, two in each of four categories:
- Contradiction detection: finding differences in scope descriptions between sections
- Technical parameters: values that exist only in tables, such as a circuit breaker’s rated current or a substation’s battery capacity
- Spatial and layout: substation locations and connections, shown only in grid map diagrams
- Role-differentiated obligations: the same question about COD obligations, asked by a commercial lead and by an electrical engineer, where the right answers differ
Ground-truth answers were prepared by hand. They’re a practical baseline rather than a gold-standard benchmark, and building a fuller evaluation set is on the roadmap. Scoring used Ragas, with GPT-4.1 as the judge, on five metrics: faithfulness, context recall, answer relevancy, factual correctness and semantic similarity.
Experiment 1: a baseline RAG pipeline
Setup: text-only PDF parsing, 1,024-token chunks with 100-token overlap, all-MiniLM-L6-v2 embeddings, ChromaDB, Llama-3.1-8B served through Groq, LangChain, and a Streamlit UI on AWS EC2. The prompt had four explicit rules: answer only from the retrieved context, cite the section or clause, flag contradictions you can see in the retrieved chunks, and say “I am not sure” when the context isn’t enough.
What we learned:
- Straightforward questions answered from narrative text (submission timelines, qualification thresholds, general scope) worked reasonably well.
- Anything that depended on a table failed, because tables were never in the vector store.
- Contradiction detection was partial. The model cited one side of a conflict without the other, because the retriever returned individually relevant chunks, not related ones.
- The 8B model occasionally made up figures when the real value sat in a table it couldn’t see.
The structured prompt still mattered. With the instruction to say “I am not sure”, the model admitted gaps instead of inventing answers, and it flagged contradictions whenever both sides happened to land in the same retrieval window. In tender analysis, where an invented requirement can end up in a bid, that discipline is valuable.
But prompting can’t fix retrieval. When the conflicting clauses weren’t retrieved together, no instruction could bridge the gap.
Experiment 2: an enterprise RAG stack on Databricks
Setup: the same text-only content, stored in a Databricks volume and indexed with Databricks native embeddings in Databricks Vector Search, with Llama-3.1-70B served from a Databricks endpoint.
What we learned:
- The bigger model reasoned noticeably better on questions involving several clauses. It synthesised across retrieved passages instead of echoing the closest one, and it pointed out tension between passages more often, even without being told to.
- Faithfulness rose to 0.97, so the system was reliably grounded in what it retrieved.
- Context recall barely moved (0.52), and factual correctness stayed low. The cause was the same as in the baseline: tables and images still weren’t ingested, so a bigger model couldn’t answer questions whose answers weren’t there.
That made the next step clear. This was a parsing and ingestion problem, not a model problem.
Experiment 3: the enhanced pipeline
The third experiment went back to the small 8B model and changed everything around it instead. Ingestion of tables and images was a proof of concept, so it ran on a curated 23-page subset of the tender.
Knowledge management: multimodal ingestion. Three content streams went into a dedicated ChromaDB collection:
- Narrative text chunks, tagged with metadata for engineering discipline, section type and clause reference
- Table chunks extracted with pdfplumber from ten table-heavy pages, keeping row structure intact
- Figure chunks: GPT-4o descriptions of the grid map images, with PyMuPDF used to extract the images
Query intelligence: compound decomposition and role inference.
- An LLM breaks each question down along EPC dimensions (scope, schedule, commercial, legal and technical) and retrieves for each one independently.
- An LLM also infers the user’s role and filters retrieval to the matching discipline: electrical, commercial, civil or general.
Response intelligence: the answer is assembled from the per-dimension results, with explicit cross-section contradiction checks and role-specific framing.
Here’s what that changed, on four representative queries:
| Capability | Baseline / Databricks | Enhanced pipeline |
|---|---|---|
| Contradiction detection: “What’s included in the future provision for substation A?” | Baseline: “I am not sure.” Databricks: a partial list from one section, with no comparison. | Retrieves the notification and the technical section separately and flags the difference: the notification lists two ICTs and two ICT bays; the technical section lists five ICTs plus three bus reactors and three reactor bays that the notification omits. It recommends treating the technical section as authoritative. |
| Table extraction: “What battery capacity is available at substation B?” | Baseline: answers from the wrong table row. Databricks: “The context does not provide this.” | Answers from the substation details table with row structure preserved, citing the page. |
| Role-based retrieval: “What are the key COD obligations?” | The same generic answer for every role. | For a commercial lead: the scheduled COD, when transmission charges start, and the requirement to commission all elements together. For an electrical engineer: the reactors to be commissioned, the commissioning scope and testing requirements. Same question, structurally different answers. |
| Image parsing: “Where is substation C and how does it connect to the grid?” | Partial answers from text only, missing the routing between substations. | Describes the location and connections (line type, bay counts at each end) from the grid map. |
In total, decomposition surfaced six specific contradictions between the notification and the technical specification that neither text-only pipeline could find, because those passages were never retrieved together.
The results
| Pipeline | Faithfulness | Context recall | Answer relevancy | Factual correctness | Semantic similarity |
|---|---|---|---|---|---|
| Baseline RAG | 0.76 | 0.47 | 0.85 | 0.23 | 0.64 |
| Databricks RAG | 0.97 | 0.52 | 0.92 | 0.12 | 0.63 |
| Enhanced pipeline | 0.73 | 0.72 | 0.90 | 0.41 | 0.83 |
Three things stand out.
The text-only ceiling. Context recall stayed between 0.47 and 0.52 for both text-only pipelines, regardless of model size. Tables, bills of quantities, drawings and image-based specs simply weren’t in the vector store, and no retrieval tuning can find what was never embedded. Multimodal ingestion broke that ceiling, raising context recall to 0.72 and factual correctness to 0.41.
A small model, a better result. The enhanced pipeline used the same 8B model as the baseline. The gains came from what it was given to read and how the question was asked, not from a bigger model.
Read faithfulness with care. The enhanced pipeline’s faithfulness (0.73) is lower than the Databricks pipeline’s. Ragas checks whether each claim in an answer can be traced to a retrieved chunk. Decomposition synthesises across several chunks, so its claims often span chunk boundaries, and Ragas penalises that even when the synthesis is correct. For this architecture, context recall and answer relevancy are the more meaningful signals. Role-based retrieval is also invisible to these scores, even though it’s one of the most useful changes for real tender work.
What it can and can’t do yet
Today the pipeline can:
- Answer narrative questions, and table-based questions about bills of quantities and technical parameters
- Say “I am not sure” instead of inventing an answer
- Detect contradictions across EPC dimensions
- Give role-specific answers to the same question
- Describe spatial layouts from grid maps and diagrams
It can’t yet reliably:
- Run multimodal ingestion over the full 450 pages
- Link requirements to the external standards they cite
- Generate structured outputs such as TSNs, risk registers and technical query logs
- Follow multi-hop questions across more than two sections
- Tell deliberately ambiguous wording apart from genuine contradictions
- Answer across past bids, templates and organisational standards
Lessons learned
- Parsing completeness is the first investment. The single biggest gain came from getting tables and diagrams into the knowledge base. For a 450-page tender, commercial parsers such as unstructured.io, LlamaParse or landing.ai cost a few cents a page, which is negligible next to analysts’ time. Use them for the baseline, and build custom pieces only for domain needs: discipline metadata, reference linking and CAD drawings.
- Contradiction detection is a retrieval architecture problem. Prompts help when both sides of a conflict are retrieved together. They can’t help when they aren’t. Decomposition, and eventually multi-step agentic retrieval, can.
- Teach the system to say “I don’t know”. In a high-stakes domain, an honest gap is far better than a confident invention.
- Roles change the right answer. The same question means different things to a commercial lead and an electrical engineer. Metadata-filtered retrieval by role is cheap and effective.
- Change one layer at a time. Moving from simple to complex in steps made it possible to attribute each score change to a specific change in the architecture.
- Ground truth for long technical documents is hard. Our hand-written answers had gaps exactly where the answers lived in unparsed tables. A proper evaluation set is a project in its own right.
What’s next
The experiments establish the retrieval foundation for a production system:

The roadmap from here:
- Scale multimodal ingestion to the full document, with automatic content-type detection, a chunking strategy for each content type, and parallel processing
- Reference linking within the RFP and out to IEC/IS/IEEE standards, to support multi-hop retrieval
- A multi-document knowledge base covering past bids, TSN templates, QA/QC standards and win/loss history
- Structured outputs: prompt chains that draft TSNs, risk registers, technical query logs and BoQ summaries
- A proper evaluation dataset covering every query type across several RFPs
- Agentic orchestration: specialised agents for contradiction detection, risk analysis, BoQ generation and Q&A, coordinated by an orchestrator that plans retrieval passes, compares evidence and reconciles conflicts, so users no longer have to break questions down themselves
We’re also looking at LlamaIndex for its hierarchical and multi-document indexing, Qdrant as a more production-ready vector store, and more capable LLMs for the structured-output phase.
The takeaway
For tender intelligence, generation quality was never the bottleneck. The baseline already produced grounded, relevant answers to narrative questions. What held it back was what the system could see and how it asked. Multimodal ingestion fixed the first. Query decomposition and role-based retrieval fixed the second. The highest-value tender tasks will ultimately be handled by the agentic layer built on top of that foundation.
If you’re applying RAG to long technical documents, I’d love to compare notes. Find me on LinkedIn.