Orama as an Answer Engine

With Orama 3.0, we introduced a new feature called AnswerSession that allows you to perform ChatGPT-like experiences on your website.

It uses a free feature from Orama Cloud called Secure Proxy to proxy your queries to the OpenAI API, so you don’t need to worry about sharing the API key on the client-side.

The APIs are designed to be as close as possible to the Orama Cloud APIs, so you can easily migrate your projects from Orama Cloud to Orama Open Source and vice-versa.

Getting Started

Orama implements a full-text, vector, and hybrid search engine as well as a complete RAG (Retrieval-Augmented Generation) pipeline to generate answers from your documents. All with minimum dependencies and configuration, so you can focus on building your project.

To get started, you will need to create an account on Orama Cloud and generate an API key from the “Secure Proxy” section. Then, you can use the API key to create an AnswerSession and start generating answers.

Follow this guide to get your API key for free and start using the AnswerSession APIs!

AnswerSession

Creating an answer session is as simple as:

import { create, insert } from "@orama/orama";
import { pluginSecureProxy } from "@orama/plugin-secure-proxy";

const secureProxy = await pluginSecureProxy({
  apiKey: "my-api-key",
  defaultProperty: "embeddings",
  models: {
    embeddings: "openai/text-embedding-3-small",
    chat: "openai/gpt-4o-mini"
  }
})

const db = await create({
  schema: {
    name: 'string'
  } as const,
  plugins: [secureProxy]
})

await insert(db, { name: "John Doe" })
await insert(db, { name: "Michele Riva" })

const session = new AnswerSession(db, {
  // Customize the prompt for the system
  systemPrompt: 'You will get a name as context, please provide a greeting message',
  events: {
    onStateChange: console.log
  }
})

const response = await session.ask({
  term: 'john',
})

console.log(response) // Hello, John Doe! How are you doing?

The onStateChange: console.log event will log the state of the session, allowing you to reactively update your UI based on the current state of the session.

In the example above, the onStateChange will be triggered for every new object in the following array (the state):

[
  // As soon as you call the `.ask` method, the state will be populated as follows:
  {
    interactionId: "cm2anntif000008l84lvqfrvc", // Unique interaction ID for the session
    aborted: false, // If the session was aborted
    loading: true, // If the session is loading
    query: "john", // The query that was sent to the API
    response: "", // The response from the API, which is empty until the API responds
    sources: null, // The sources used to generate the response
    error: false, // If there was an error
    errorMessage: null, // The error message, if any
  },
  // Then, Orama will perform search and push the sources to the state:
  {
    interactionId: "cm2anntif000008l84lvqfrvc",
    aborted: false,
    loading: false,
    query: "john",
    response: "",
    sources: { // The sources used to generate the response, in the same format as the search result from Orama
      count: 1,
      elapsed: { raw: 0.123, formatted: "100μs" },
      hits: [
        {
          id: "1-19238",
          score: 0.8,
          document: { name: "John Doe" }
        }
      ]
    },
    error: false,
    errorMessage: null,
  },
  // Then, Orama will update this message with incoming chunks from OpenAI (via the secure proxy):
  {
    interactionId: "cm2anntif000008l84lvqfrvc",
    aborted: false,
    loading: false,
    query: "john",
    response: "Hello, John Doe!",
    sources: {
      count: 1,
      elapsed: { raw: 0.123, formatted: "100μs" },
      hits: [
        {
          id: "1-19238",
          score: 0.8,
          document: { name: "John Doe" }
        }
      ]
    },
    error: false,
    errorMessage: null,
  }
]

RAG implementation

Orama provides a comprehensive toolkit for building high-performance RAG (Retrieval-Augmented Generation) pipelines tailored to your document processing needs.

Below is a simple example demonstrating how to set up your application to leverage Orama’s database capabilities and the Secure Proxy Plugin for document-based answer generation.

import { create, insert, search, AnswerSession } from "@orama/orama";
import { pluginSecureProxy } from "@orama/plugin-secure-proxy";

// Configuration options for better maintainability
const CONFIG = {
  API_KEY: process.env.ORAMA_SECURE_PROXY_API_KEY,
  VECTOR_DIMENSIONS: 1536,
};

// Sample documents
const SAMPLE_DOCS = [
  { description: "John Doe is a programmer, and he has 14 years." },
  { description: "Mitch Smith is a programmer, and he has 32 years." },
];

/**
 * Initialize the Orama database with secure proxy plugin
 * @returns {Promise<Object>} Configured database instance
 */
async function initializeDatabase() {
  const secureProxy = await pluginSecureProxy({
    apiKey: CONFIG.API_KEY,
    embeddings: {
      model: "openai/text-embedding-ada-002",
      defaultProperty: "embeddings",
      onInsert: {
        generate: true,
        properties: ["description"],
        verbose: true,
      },
    },
    chat: {
      model: "openai/gpt-3.5-turbo",
    },
  });

  return create({
    schema: {
      description: "string",
      embeddings: `vector[${CONFIG.VECTOR_DIMENSIONS}]`,
    },
    plugins: [secureProxy],
  });
}

/**
 * Populate the database with documents
 * @param {Object} db - Database instance
 * @param {Array} documents - Array of documents to insert
 */
async function populateDatabase(db, documents) {
  const insertPromises = documents.map((doc) =>
    insert(db, { description: doc.description })
  );
  await Promise.all(insertPromises);
}

/**
 * Perform vector search and generate response
 * @param {Object} db - Database instance
 * @param {string} userPrompt - User's question
 * @returns {Promise<string>} Generated response
 */
async function generateResponse(db, userPrompt) {
  try {
    const searchResults = await search(db, {
      mode: "vector",
      term: userPrompt,
    });

    const formattedPrompt = `### Context: \n\n${JSON.stringify(
      searchResults?.hits
    )}\n\n### Prompt:\n\n${userPrompt}`;

    const session = new AnswerSession(db, {});
    return await session.ask({ term: formattedPrompt });
  } catch (error) {
    console.error("Error generating response:", error);
    throw error;
  }
}

/**
 * Main execution function
 */
async function main() {
  try {
    // Initialize database
    const db = await initializeDatabase();

    // Populate with sample data
    await populateDatabase(db, SAMPLE_DOCS);

    // Example query
    const userPrompt = "Who is John Doe?";
    const response = await generateResponse(db, userPrompt);

    console.log("Response:", response);
  } catch (error) {
    console.error("Application error:", error);
  }
}

// Execute the application
main();