RAG Stacks

Reference for SeaGit’s RAG Stack template: what it deploys, how documents are kept in sync, how models are governed, and how the API and web UI are secured. For a walkthrough, start with Deploy a RAG Stack in Minutes.

Architecture

A RAG stack is a group of four SeaGit applications deployed together into a cluster in your AWS account and managed as one unit from the Stack tab:

vectordb — Qdrant with its index on an encrypted EBS volume.
rag-api — embeds questions with Amazon Titan Text Embeddings v2, retrieves the top-k passages from Qdrant and asks an Amazon Bedrock chat model to answer from them, with citations.
rag-web-ui (optional) — chat, model picker and drag-and-drop upload.
rag-sync — one import job, CronJob and/or event consumer per data source and for uploads.

Each component gets its own IAM role with only the permissions it needs: read-only access to the folders you chose, write access to the uploads folder, and bedrock:InvokeModel on exactly the approved chat models plus the embedding model.

With multi-cluster ingestion, every started cluster in the environment runs a complete stack and builds its own index from the same sources, so each region can answer on its own.

Documents and sync

Data sources

A data source is an S3 bucket (optionally one folder and a list of file types) that the stack reads. Each source gets its own sync job. You choose whether to import existing files now, how to keep it in sync, and whether files deleted from the source are removed from the index.

Uploads

Uploads are files added through the web UI or POST /v1/ingest. Two separate choices control them:

Where uploaded files are kept
- A new S3 bucket for this stack — created and owned by the stack; optionally deleted with it.
- Your existing S3 bucket — give the bucket and a folder (for example docs/in/). SeaGit only adds its own access rules for that folder; only that folder is read and synced. Turn on Import existing files now to index what is already there at deploy time.
Keep uploads in sync — one of the modes below. Choosing Off turns uploads off: the web UI hides the upload box and /v1/ingest answers 403.

Sync modes

Mode	When a file becomes searchable	What runs in the cluster
On change (S3 events)	A new or changed file is indexed within seconds; a deleted file can be removed from the index.	An event consumer in the cluster, fed by S3 → EventBridge (a direct S3 notification for the stack’s own bucket).
On a schedule	Files are picked up on each run of the cron schedule you set (hourly by default). Overlapping runs are skipped.	A Kubernetes CronJob that compares the folder with the index.
On change + safety sweep	Seconds for normal changes, and the scheduled sweep catches anything an event missed. The default.	Both of the above.
Off / Don’t sync	For uploads: the upload box and upload API are turned off. For a data source: only the import at deploy time.	Nothing.

Right after a stack is created, S3 can take a few minutes to start sending events for a bucket. A file uploaded in that window is picked up by the next safety sweep — another reason On change + safety sweep is the default.

Buckets in another AWS account

A bucket can live in any AWS account your organization has connected as a provider. Under Bucket access:

Automatic — SeaGit finds the bucket’s account among your organization’s AWS providers. If the bucket is in a different account from the cluster, enter its AWS account ID so SeaGit knows where to look.
Pick credentials — choose the provider that owns the bucket.
Use an existing IAM role — launch the provided CloudFormation link in the bucket’s account and paste the role ARN; SeaGit never needs credentials for that account.

SeaGit creates a role in the bucket’s account scoped to your folder, and for event sync an EventBridge rule for that bucket and folder only. Deleting the stack removes both and leaves your bucket and files untouched. If the account ID you enter doesn’t own the bucket, the components that use it (rag-api and rag-sync) fail to deploy and nothing is created in the bucket’s account; delete the stack and deploy it again with the right account.

Approved models

Which Bedrock chat models a stack may use is decided at three levels, each a subset of the one above:

Account (account settings → Approved models) — all catalog models, or a chosen list, plus the default for organizations without their own list.
Organization (Approved models in the organization sidebar) — narrows the account’s list. “Nothing approved” means no model is available.
Stack — the chat models picked in the deploy wizard, and a default.

The rules are enforced, not advisory: /v1/models lists only the stack’s models, a request naming any other model is refused with 400, and the stack’s IAM role can only invoke those models. When an admin removes a model from the account or an organization, organization lists are pruned, the Stack tab warns about stacks still using it, and the next redeploy from the Stack tab drops the model and its permission.

Anthropic, Meta and Mistral models also need model access enabled for the AWS account in the Amazon Bedrock console. If it is missing, the web UI says which model isn’t enabled instead of failing silently.

Models from an LLM Gateway

By default a RAG stack calls Amazon Bedrock directly. Under Models from, choose An LLM Gateway stack instead and its answers go through that gateway: its budgets, personal-data redaction, call log and traces apply, and any model it serves can be used, including self-hosted ones.

Which gateways: one on the same cluster (called at its in-cluster address) or one on a domain this environment covers (called over HTTPS). A gateway reachable only inside its own cluster is offered only on that cluster.
Chat models and the default are picked from the models the gateway serves. Embedding model stays Titan Text Embeddings v2 on Bedrock, or can be an embedding model the gateway serves. Changing the embedding model later means re-embedding every document.
Its own key: the stack gets a gateway key of its own, limited to the models picked for it. SeaGit registers it from inside the gateway’s cluster, so the gateway’s master key never leaves the gateway, and revokes it when the stack is deleted. Calls show in the gateway’s call log under seagit-rag-<stack id>.
The Stack tab shows the key’s state. Retry registers it again if the registration failed; Edit Gateway models picks the chat models again from what the gateway serves now and redeploys the API.
A gateway with Allowed IPs must allow the outbound IPs of this stack’s cluster when it runs on another cluster.

Access keys

API key — for applications. Send it as Authorization: Bearer <key>.
Web UI access key — for people. The web UI exchanges it for a short-lived session; the key itself is never embedded in the page. After 10 wrong attempts, sign-in is rate-limited.

Both keys are shown once, when the stack is created. Rotate the web UI key from the Stack tab: the new key is shown once, and existing sessions stop working as soon as the change rolls out (usually under a minute).

API

Method	Path	Purpose
POST	/v1/query	Ask a question; returns the answer, the model used and numbered citations.
POST	/v1/chat/completions	OpenAI-compatible chat. `model` must be "rag" or one of the stack’s approved models. No streaming yet.
GET	/v1/models	The chat models this stack allows, and the default.
POST	/v1/ingest	Upload a file (multipart `file`). Returns `stored`; it becomes searchable per the uploads sync setting.
GET	/v1/uploads	Whether uploads are on, the sync mode and the schedule.
POST	/v1/ui/session	Exchange the web UI access key for a session token (used by the web UI).

/v1/query and /v1/chat/completions accept an optional model; without it the stack’s default model answers. Questions are limited to 8,000 characters.

curl -s https://<api-host>/v1/query \
  -H "Authorization: Bearer $RAG_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"question": "Summarise the travel policy", "model": "bedrock:us.amazon.nova-pro-v1:0"}'

Change settings safely

Change a live stack from the Stack tab with Change settings. The change is built next to the live stack and evaluated against it. Nothing switches until you adopt it, and you can discard it at any time. Settings changes are switched on per account; if the button is missing, ask SeaGit to enable it for yours.

What you can change

Setting	Valid values	Reindex?
Chat models and default model	Only models this stack can already use	No
Passages per answer (top-k)	1 to 20	No
Chunk size	200 to 4,000	Yes: builds a new index
Chunk overlap	0 up to, but below, the chunk size	Yes: builds a new index
Embedding model	Amazon Titan Text Embeddings v2, or an embedding model your LLM Gateway stack serves (when the stack takes embeddings from the gateway)	Yes: builds a new index

Chunk size, chunk overlap and the embedding model build a new index (a new collection) beside the live one. Top-k and chat models do not reindex. A model the stack doesn’t have yet needs a stack redeploy before it can be picked.

Estimate and capacity

Before the change starts, you see an estimate in US dollars at Amazon Bedrock list prices, with the document and token counts it is based on. A reindex is refused if the new index would not fit on the vector database’s disk; the form shows the space needed and the space free.

Evaluation

Pick an Evals stack that already evaluates this RAG stack, or choose Create an Evals stack for this RAG stack. The new Evals stack is created first, then the change starts. It can be created without watching any code instances.
Golden questions are optional: upload a JSONL file in the same format as .seagit/evals/golden.jsonl, up to 500 rows and 1 MB. When you don’t upload a file, the Evals stack scores both deployments on its own questions (its suite); uploading a file adds your questions for this change. The upload must have at least one row when provided.
The Evals stack’s gate sets the mode: Warn or Block.

Following a change

The change card on the Stack tab shows each step:

Building candidate
Importing documents into the new index (n of N sources), for reindex changes only
Evaluating against the live stack
Passed or Failed

An import that makes no progress for 70 minutes fails. An evaluation with no result after 2 hours fails; Run evaluation retries it. One change runs per stack at a time, and this works on single-cluster RAG stacks only.

Adopt, discard and roll back

Block: a failed or unevaluated change can be adopted only with a written reason. The reason is audited.
Warn: adopting a failed or unevaluated change asks you to confirm first. The confirmation is recorded.
Adopting a reindex change first imports any files added during the evaluation, then switches the live stack to the new index.
Discard removes the candidate and its new index. The live stack is not touched.

After you adopt a reindex change, the previous index is kept for 24 hours. Roll back (with a reason, audited) restores it, and is available for 23 hours after you adopt: it is not offered in the last hour before the previous index is removed. After 24 hours, rag-api removes the previous index by itself, inside your cluster, and Roll back is no longer offered.

While a previous index is still kept, a new reindex change can’t start or be adopted, and the message says until when. Changes that don’t reindex, such as top-k or chat models, can still go ahead.

Prompt versions

The prompt a RAG stack answers with is a versioned template, kept in the Prompt Registry of the MLflow stack the RAG stack traces to. You write and compare versions in the Prompt versions panel on the Stack tab; versions you register from a notebook under the same name show up there too.

A template has a system message and a user message. The user message must contain {{context}} (the retrieved passages, numbered [1], [2]…) and {{question}}; no other variables are filled in, and a template is at most 16 KB. Keep asking for [n] citations, or answers lose them.
Version 1 is the built-in prompt the stack has always answered with. Until a prompt change is adopted, nothing changes.
Try this version opens Change settings with that version chosen. A prompt change is scored like any other change. No reindex is needed, so it is ready in about a minute.
The text of the chosen version is copied into the candidate when the change starts: editing or adding versions later cannot change what was scored, and the live stack never needs MLflow to answer.
Adopt, Discard and Roll back work as for any change; rolling back returns to the previous prompt (or the built-in one).

Prompt versions need the RAG stack to trace to an MLflow stack, and are part of settings changes, which are enabled per account.

Troubleshooting

A new upload isn’t found — check GET /v1/uploads. With On a schedule the file is indexed on the next run; right after stack creation an event may be missed until the next sweep.
“model … is not approved for this assistant” — the model isn’t in the stack’s list; see Approved models.
“… isn’t enabled in Bedrock” — enable model access for that model in the AWS account.
“uploads are turned off for this stack” — uploads sync is set to Off.
A component stays pending — the cluster may be out of capacity; see Troubleshooting.