CruxyLearn

Retrieval-augmented generation

Embed a corpus, rank it by cosine, reorder the shortlist with rerank, then ground a chat answer in the winning passages.

Retrieval-augmented generation answers a question from your own documents: find the passages that bear on the question, then hand them to a chat model as context. This recipe chains three endpoints: Embeddings for a first-pass search, Rerank to sharpen the shortlist, and Chat to write the answer.

1. Index the corpus

Split your source material into chunks and embed them once. Store each vector beside the text it came from; you embed the corpus at index time, not per query.

const BASE = 'https://api.cruxy.ai'
const auth = {
  Authorization: `Bearer ${process.env.CRUXY_API_KEY}`,
  'Content-Type': 'application/json',
}
 
async function embed(input: string[]): Promise<number[][]> {
  const res = await fetch(`${BASE}/api/v1/embeddings`, {
    method: 'POST',
    headers: auth,
    body: JSON.stringify({ model: 'cruxy-embed', input }),
  })
  const { data } = await res.json()
  // One entry per input; data[i].index is the input's position. Sort by it
  // so the returned vectors line up with `input`, whatever order they arrive.
  return data.sort((a, b) => a.index - b.index).map((d) => d.embedding)
}
 
const chunks = [
  'The gateway resolves each public tier to an upstream model on the server.',
  'Rerank scores every document against the query and returns them best-first.',
  'Embeddings turn text into a vector you can compare with cosine similarity.',
  'A bearer key authenticates every request against api.cruxy.ai.',
  // ...the rest of your corpus, one string per chunk
]
 
const vectors = await embed(chunks) // persist these alongside `chunks`

2. Retrieve by cosine

Embed the query with the same model, then score it against every stored vector. Cosine similarity ranks the whole corpus cheaply; keep the top handful as candidates.

function cosine(a: number[], b: number[]): number {
  let dot = 0
  let na = 0
  let nb = 0
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i]
    na += a[i] * a[i]
    nb += b[i] * b[i]
  }
  return dot / (Math.sqrt(na) * Math.sqrt(nb))
}
 
const query = 'How does the gateway decide which model runs?'
const [queryVec] = await embed([query])
 
const CANDIDATES = 20
const shortlist = chunks
  .map((text, i) => ({ text, score: cosine(queryVec, vectors[i]) }))
  .sort((a, b) => b.score - a.score)
  .slice(0, CANDIDATES)

3. Rerank the shortlist

Cosine over embeddings is fast but coarse. Send the shortlist to rerank, which scores each candidate against the query directly and returns them ordered by relevance. Cap the result to the few passages you will actually pass on.

const documents = shortlist.map((c) => c.text)
 
const res = await fetch(`${BASE}/api/v1/rerank`, {
  method: 'POST',
  headers: auth,
  body: JSON.stringify({ model: 'cruxy-rerank', query, documents, top_n: 5 }),
})
const { results } = await res.json()
 
// Each result's `index` points into `documents` (the shortlist you sent),
// NOT the original `chunks`. Map through `documents` to recover the text.
const context = results.map((r) => documents[r.index])

4. Generate the answer

Give the reranked passages to chat as context and constrain the model to answer from them. stream: false returns the reply as the top-level message.

const res = await fetch(`${BASE}/api/v1/chat`, {
  method: 'POST',
  headers: auth,
  body: JSON.stringify({
    model: 'kavi',
    stream: false,
    messages: [
      {
        role: 'system',
        content:
          'Answer only from the provided context. If the answer is not in it, say you do not know.',
      },
      {
        role: 'user',
        content: `Context:\n${context.join('\n\n')}\n\nQuestion: ${query}`,
      },
    ],
  }),
})
 
const { message } = await res.json()
console.log(message.content)

Why rerank after vector search, not instead of it

The two retrieval endpoints trade off differently, so the pipeline uses both. Embeddings compress a passage into a single vector, which makes cosine search cheap enough to run over a whole corpus, but that compression blurs fine distinctions, so the top cosine hit is not always the most relevant passage. Rerank scores each document against the query directly, which is far more discriminating, but it reads every candidate on every call, so running it over the full corpus does not scale.

Chaining them keeps each in its lane: embeddings recall a broad shortlist cheaply, rerank orders that shortlist precisely. Rerank comes after vector search, never instead of it, because it ranks a candidate set you hand it; it does not index your corpus or retrieve from one.

Mapping indices back

Both retrieval endpoints return positions, not text, and they point into different arrays. An embedding's data[i].index is the position in the input you sent to /embeddings. A rerank result's index is the position in the documents you sent to /rerank: here, the shortlist, not the original corpus. Map each rerank index through documents; mapping it through chunks returns the wrong passage whenever the shortlist reordered the corpus, which is almost always. The Rerank guide flags this same trap for the single-endpoint case.

Next steps

  • Embeddings: the vectors and the batch input shape.
  • Rerank: the score ordering and top_n.
  • Chat: request options and the flattened response.

Was this page helpful?