Real-Time Streaming REST API

Our streaming API allows you to stream grounded answers token-by-token directly into your custom applications, chatbots, or terminal tools with sub-50ms latency.

1. HTTP Request Format

Send a POST request to the streaming endpoint with your live API key in headers:

Endpoint: POST http://localhost:8081/api/v1/query/stream

Headers:

BASH

Content-Type: application/json
X-API-Key: tw_rag_live_YOUR_API_KEY

Request JSON Body:

JSON

{
  "ragId": "67b738192a01f",
  "query": "What is the return window for damaged items?"
}

•ragId (string, required): The ID of your Knowledge Base from the dashboard URL.
•query (string, required): The user's question (max 500 characters).

2. Standard JSON Response Format (Non-Streaming)

If you call the standard endpoint (POST /api/v1/query), you receive a complete JSON payload upon completion:

JSON

{
  "success": true,
  "query": "What is the return window for damaged items?",
  "answer": "Damaged items can be returned within 30 days of delivery with a full refund.",
  "sources": [
    {
      "sourceName": "refund_policy.json",
      "contentSnippet": "Refund is 100% allowed within 30 days of delivery..."
    }
  ],
  "modelUsed": "Groq LPU [Key #1 · openai/gpt-oss-120b]",
  "chunksRetrieved": 2,
  "latencyMs": "142ms"
}

3. Live SSE Stream Event Format (Streaming)

When calling /api/v1/query/stream, the server streams three distinct event types over text/event-stream:

TEXT

// 1. Token events (streamed word-by-word as generated)
data: {"type":"token","token":"Damaged"}
data: {"type":"token","token":" items"}
data: {"type":"token","token":" can"}
data: {"type":"token","token":" be"}
data: {"type":"token","token":" returned"}
data: {"type":"token","token":" within"}
data: {"type":"token","token":" 30"}
data: {"type":"token","token":" days."}

// 2. Sources & Metadata event (sent at the end of answer)
data: {"type":"sources","sources":[{"sourceName":"refund_policy.json","contentSnippet":"Refund is 100%..."}],"modelUsed":"Groq LPU","latencyMs":"118ms"}

// 3. Completion signal (stream close)
data: {"type":"done"}

4. Complete Frontend Code Example (Live Typewriter)

Here is a complete, copy-paste JavaScript example showing how to connect to the stream, append tokens in real-time, and show citations on screen:

JAVASCRIPT

async function askAssistant(question) {
  const res = await fetch("http://localhost:8081/api/v1/query/stream", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-API-Key": "tw_rag_live_YOUR_KEY"
    },
    body: JSON.stringify({
      ragId: "YOUR_RAG_ID",
      query: question
    })
  });

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let fullText = "";
  let buffer = "";

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n\n");
    buffer = lines.pop() || "";

    for (const line of lines) {
      if (line.startsWith("data: ")) {
        const data = JSON.parse(line.slice(6));
        
        if (data.type === "token") {
          fullText += data.token;
          document.getElementById("chat-box").innerText = fullText; // Live update
        } else if (data.type === "sources") {
          console.log("Citations:", data.sources);
          console.log("Latency:", data.latencyMs);
        }
      }
    }
  }
}

Ready to build your Custom RAG?

Deploy in under 3 minutes with 2 free RAG knowledge bases.

Create Free RAG