Travel Booking RAG Prototyping (LangChain, LanceDB, Chroma)
RAG Prototyping for a Travel Agency's Tour Catalogue
A one-week spike that tested whether a standard LangChain RAG stack could ground a travel assistant in a real 122-row tour catalogue, across two vector stores, three retrieval styles, and four agent shapes. Every framework trialed here (LangChain, LanceDB, Chroma, CrewAI, PandasAI) was rejected in favor of a hand-built multi-agent framework in the production system that followed the same month. The value of this spike was the comparison itself, and the persona-wrapping pattern that did carry forward.
The problem
A travel agency selling Muslim-friendly tours (Umrah, Hajj, and general international travel) needed a chat assistant that could answer customer questions about tours and visas, grounded only in the agency's own catalogue, in a mix of English and Turkish. A generic LLM answering from general knowledge would invent tour details, prices, and visa rules for a catalogue of exactly 122 real tours, which is unacceptable for anything touching a booking decision. The naive fix, dumping the whole CSV into a prompt, does not scale past a small catalogue and gives the model no way to filter by date, price band, or visa status. The team needed to know which combination of vector store, retrieval method, and agent framework could reliably answer structured, filterable questions over this catalogue before committing engineering time to one architecture.
Constraints
- ·One-week timebox to compare approaches, not to ship a production system.
- ·Small, well-structured data: 122 CSV rows with 18 named columns (tour name, itinerary, price tiers by room type and child age band, visa flag, dates, currency).
- ·Bilingual input required: Turkish and English questions against an English-labeled schema.
- ·Bedrock-only inference and embeddings (Cohere multilingual embedding model, Claude 3 and 3.5 Sonnet for generation), matching the client's existing AWS account.
- ·Local, unauthenticated prototypes only, not exposed to real customers.
- ·Sole engineer, so every framework had to be evaluated quickly rather than deeply integrated.
Architecture
No single architecture. Ten-plus standalone scripts explore the same problem in parallel: two vector stores (LanceDB, Chroma) hold the same catalogue independently, three retrieval styles sit on top (raw vector search, metadata-filtered search, and an LLM SelfQueryRetriever), and four agent shapes wrap that retrieval (a two-tool tool-calling agent, a langchain_experimental CSV agent behind Flask, a PandasAI SmartDataframe swap-in, and an untied CrewAI script). A second Bedrock call rewrites every raw answer in character as the agency's assistant before it reaches the user.
- ›Ingestion: CSV rows to LanceDB (hand-built pyarrow schema) or Chroma (per-row Document with 18-field metadata), each embedded with Bedrock Cohere multilingual embeddings.
- ›Separate visa .txt ingestion: CharacterTextSplitter (1000/200) into the same LanceDB table, a different corpus from the CSV.
- ›Retrieval: raw LanceDB vector search, metadata-filtered Chroma similarity search, or a SelfQueryRetriever built from an 18-field AttributeInfo schema.
- ›Agents: a two-tool tool-calling agent (travel_agent, visa_document), a langchain_experimental CSV agent, a PandasAI SmartDataframe tool, and a standalone CrewAI agent with CSVSearchTool.
- ›Persona pass: a second ChatBedrock call rewrites the raw tool output in character before it is returned.
- ›Two throwaway Flask endpoints, /chat_csv and /run_agent, exposed for manual local testing only.
How one request flows
- 01A question arrives at /run_agent or /chat_csv (Flask, local only, no auth).
- 02The tool-calling agent decides between travel_agent (full CSV context) and visa_document (LanceDB RAG) based on the question, or the CSV agent runs pandas code directly against the file.
- 03travel_agent either dumps the whole 122-row CSV as a string into the prompt, or (in later iterations) asks a PandasAI SmartDataframe to answer from the data directly.
- 04visa_document embeds the question, searches the LanceDB visa table, and answers from the top matches.
- 05The raw agent output is handed to a second Bedrock call carrying the fixed in-character persona prompt: answer only from the given data, ask for missing trip details, and surface a WhatsApp handoff link on request.
- 06The rewritten, in-character answer is returned as the Flask response.
Engineering decisions
Two vector stores run side by side, not chosen up front
What. The same 122-row catalogue was ingested into both LanceDB (a hand-built pyarrow schema) and Chroma (LangChain's wrapper, per-row Document with 18-field metadata), queried independently.
Why. With a catalogue this small, the real question was not raw search quality but which store made metadata filtering and the SelfQueryRetriever path easiest to build against.
Tradeoff. Running both cost extra setup time for no production benefit. The spike accepted that cost deliberately to make an informed choice instead of guessing.
SelfQueryRetriever over a hand-written 18-field schema
What. An AttributeInfo list describing every tour column (name, itinerary, price tiers, visa flag, dates) let an LLM translate a free-text question, including Turkish, into a filtered Chroma query instead of a plain similarity search.
Why. Plain vector search cannot answer 'tours under 900 EUR that do not need a visa'. A structured filter can, and the LLM was trusted to build that filter from the schema.
Tradeoff. Every added column meant one more AttributeInfo entry to maintain by hand, and the retriever's output was never evaluated against a labeled set, so its actual filter accuracy was never measured.
CSV rows embedded whole, never chunked
What. Each catalogue row became one Document, page content the raw row values, unlike the separate visa .txt corpus, which was chunked (1000/200) before embedding.
Why. A tour row is a single coherent unit, one tour, splitting it would break the itinerary and price fields apart from the tour name they belong to.
Tradeoff. This works only because rows stay short. It would not survive a catalogue with long free-text itineraries per row.
PandasAI tried as a drop-in replacement for the raw dataframe-to-string prompt
What. Later iterations (3.py, 4.py, 5.py) swapped the travel_agent tool from a raw CSV-to-string prompt to a PandasAI SmartDataframe, letting questions run as natural-language pandas queries.
Why. PandasAI promised structured, code-generated answers (counts, filters, comparisons) instead of an LLM eyeballing a dumped table.
Tradeoff. Rejected for production: PandasAI adds its own code-execution layer on top of an already dangerous CSV agent, and did not carry a clear accuracy win over the simpler prompt-based approach for a 122-row table.
CrewAI tried and left untied
What. query.py builds a standalone CrewAI Agent with a CSVSearchTool over the same catalogue, on Claude 3.5 Sonnet via Bedrock, never wired into either Flask app.
Why. Worth a quick look because CrewAI's role/goal/backstory framing suited the persona requirement on paper.
Tradeoff. Rejected early: the overhead of CrewAI's crew/task abstraction did not pay for itself against a single-agent, single-tool problem this size, so it was never integrated further.
Persona rewrite as a second, separate LLM call
What. Rather than baking the in-character tone into the retrieval or tool-calling prompt, every agent variant passes its raw answer through a second, dedicated ChatBedrock call carrying the full persona prompt.
Why. Separating 'get the right facts' from 'say it in character' kept the retrieval prompts focused on accuracy and let the persona be tuned independently without touching the data-grounding logic.
Tradeoff. Doubles the generation calls per question. Accepted for a prototype, and this exact separation is the one pattern that carried forward into the production system.
All frameworks here rejected for a hand-built multi-agent system
What. None of LangChain's agent classes, LanceDB, Chroma, CrewAI, or PandasAI shipped. The production booking platform built the same month replaced all of it with a custom multi-agent framework written from scratch.
Why. The comparison in this spike showed each framework added abstraction and failure surface without a clear win for a catalogue this size and this narrow a domain.
Tradeoff. The shipped system gave up the convenience of these libraries (built-in agent loops, retriever abstractions) in exchange for full control over cost, latency, and failure handling, a tradeoff only worth making because the comparison had already been run here.
What changed along the way
- →Started with a plain dataframe-to-string prompt (agent.py, 1.py, 2.py) and moved to PandasAI's SmartDataframe (3.py, 4.py, 5.py) partway through, looking for more structured answers.
- →Added a second, dedicated persona-rewrite call after early runs answered correctly but out of character.
- →Tried a ReAct-style agent (create_react_agent) with a second visa_agent tool in 5.py, but that tool referenced a visa_data.csv file that was never actually created, so it was never run end to end.
- →Dropped the CrewAI branch (query.py) after one script. It never got wired into either live endpoint.
- →Concluded the week by rejecting all of these frameworks in favor of a hand-built multi-agent system for production.
Security and reliability
- ·Both Flask endpoints (/chat_csv, /run_agent) run with no authentication, local prototypes only, never exposed to the internet.
- ·/chat_csv wraps langchain_experimental's create_csv_agent with allow_dangerous_code=True, which lets the agent write and execute arbitrary pandas/Python against caller input. Accepted as a prototype-only risk, never carried into production.
- ·Module-level global state (a bare x = "" written from inside a tool function) carries the LLM response between calls in most of the agent scripts, not safe under concurrent requests.
- ·No hardcoded credentials: Bedrock access comes from the environment/AWS config, confirmed by the AWS SDK's own log line picking up the shared credentials file.
- ·No evaluation harness and no reranking anywhere in the spike, retrieval quality was judged by eye, not measured.
Metrics
What shipped
- ✓Working, if disconnected, prototypes of CSV-grounded Q&A over a real tour catalogue using LanceDB, Chroma, PandasAI, and a langchain_experimental CSV agent.
- ✓A tested LLM self-query pattern (SelfQueryRetriever plus a hand-written AttributeInfo schema) for turning free-text questions into filtered lookups over the 18 tour columns.
- ✓A persona-wrapping pattern (raw tool output rewritten in-character by a second Bedrock call) that carried forward into the production the agency assistant.
- ✓This spike's comparisons fed directly into the decision to drop LangChain and LanceDB-in-the-loop for a hand-built multi-agent framework in the production system (id 29_4).
Lessons learned
- ·For a small, well-structured catalogue (122 rows, 18 columns), a metadata-filtered or self-query retriever earns its complexity only if the questions genuinely need structured filtering, not just semantic lookup.
- ·Splitting 'get the right facts' from 'say it in character' as two separate LLM calls is worth keeping even when everything else about the stack changes, it carried forward into production.
- ·Running two vector stores and four agent shapes in parallel for a week is a legitimate way to de-risk a framework choice before committing, cheaper than committing early and finding out later.
- ·General-purpose agent frameworks (LangChain's agent classes, CrewAI, PandasAI) add real abstraction cost that does not always pay off against a narrow, well-scoped retrieval problem.
- ·allow_dangerous_code=True is a reasonable prototype shortcut only when the endpoint is genuinely local and unauthenticated, it should never survive into anything reachable by a real user.