Requests & Responses

Requests

Request Body

The request schema, represented as a TypeScript type, serves as the body of the POST request to the /api/v1/chat/completions endpoint. An example of this can be found in the Quick Start above.

type Request = {
  // Either "messages" or "prompt" is required
  messages?: Message[];
  prompt?: string;
  model?: string; 
  response_format?: { type: 'json_object' };
  stop?: string | string[];
  stream?: boolean; 
  max_tokens?: number; 
  temperature?: number; 
  top_p?: number; 
  top_k?: number; 
  frequency_penalty?: number; 
  presence_penalty?: number; 
  repetition_penalty?: number; 
  seed?: number; 
  tools?: Tool[];
  tool_choice?: ToolChoice;
  // Additional optional parameters
  logit_bias?: { [key: number]: number };
  transforms?: string[];
  models?: string[];
  route?: 'fallback';
  provider?: ProviderPreferences;
};
type TextContent = {
  type: 'text';
  text: string;
};
type ImageContentPart = {
  type: 'image_url';
  image_url: {
    url: string; // URL or base64 encoded image data
    detail?: string; // Optional, defaults to 'auto'
  };
};
type ContentPart = TextContent | ImageContentPart;
type Message = {
  role: 'user' | 'assistant' | 'system' | 'tool';
  content: string | ContentPart[];
  name?: string;
};
type FunctionDescription = {
  description?: string;
  name: string;
  parameters: object; 
type Tool = {
  type: 'function';
  function: FunctionDescription;
};
type ToolChoice =
  | 'none'
  | 'auto'
  | {
      type: 'function';
      function: {
        name: string;
      };
    };

Request Headers

fetch("https://ai.api.4everland.org/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${YOUR_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional
    "X-Title": `${YOUR_SITE_NAME}`, // Optional
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Who are you?"},
    ],
  })
});

Responses

The responses align closely with the OpenAI Chat API, ensuring that choices are always presented as an array, even when the model returns only one completion. Each choice will include a delta property if a stream was requested, and a message property otherwise. This design simplifies the code usage across all models.

4EVERLAND streamlines the schema across various models and providers, thereby requiring the learning of only one schema.

Response Body

It's important to note that the finish_reason may vary depending on the model provider. The model property provides information about the model used within the underlying API.

Here's the response schema, represented as a TypeScript type:

type Response = {
  id: string;
  // Depending on whether you set "stream" to "true" and
  // whether you passed in "messages" or a "prompt", you
  // will get a different output shape
  choices: (NonStreamingChoice | StreamingChoice | NonChatChoice | Error)[];
  created: number; // Unix timestamp
  model: string;
  object: 'chat.completion' | 'chat.completion.chunk';
  // For non-streaming responses only. For streaming responses,
  // see "Querying Cost and Stats" below.
  usage?: {
    completion_tokens: number; // Equivalent to "native_tokens_completion" in the /generation API
    prompt_tokens: number; // Equivalent to "native_tokens_prompt"
    total_tokens: number; // Sum of the above two fields
    total_cost: number; // Number of credits used by this generation
  };
};
// Subtypes:
type NonChatChoice = {
  finish_reason: string | null;
  text: string;
};
type NonStreamingChoice = {
  finish_reason: string | null; // Depends on the model. Ex: 'stop' | 'length' | 'content_filter' | 'tool_calls' | 'function_call'
  message: {
    content: string | null;
    role: string;
    tool_calls?: ToolCall[];
    // Deprecated, replaced by tool_calls
    function_call?: FunctionCall;
  };
};
type StreamingChoice = {
  finish_reason: string | null;
  delta: {
    content: string | null;
    role?: string;
    tool_calls?: ToolCall[];
    // Deprecated, replaced by tool_calls
    function_call?: FunctionCall;
  };
};
type Error = {
  code: number; // See "Error Handling" section
  message: string;
};
type FunctionCall = {
  name: string;
  arguments: string; // JSON format arguments
};
type ToolCall = {
  id: string;
  type: 'function';
  function: FunctionCall;
};

Here's an example:

json
{
  "id": "gen-xxxxxxxxxxxxxx",
  "choices": [
    {
      "finish_reason": "stop", // Different models provide different reasons here
      "message": {
        // will be "delta" if streaming
        "role": "assistant",
        "content": "Hello there!"
      }
    }
  ],
  "model": "openai/gpt-3.5-turbo" // Could also be "anthropic/claude-2.1", etc, depending on the "model" that ends up being used
}