> For the complete documentation index, see [llms.txt](https://docs.4everland.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.4everland.org/ai/ai-rpc/requests-and-responses.md).

# Requests & Responses

## Requests

### Request Body

The request schema, represented as a TypeScript type, serves as the body of the POST request to the `/api/v1/chat/completions` endpoint. An example of this can be found in the [Quick Start](/ai/ai-rpc/quick-start.md) above.

```typescript
type Request = {
  // Either "messages" or "prompt" is required
  messages?: Message[];
  prompt?: string;
  model?: string; 
  response_format?: { type: 'json_object' };
  stop?: string | string[];
  stream?: boolean; 
  max_tokens?: number; 
  temperature?: number; 
  top_p?: number; 
  top_k?: number; 
  frequency_penalty?: number; 
  presence_penalty?: number; 
  repetition_penalty?: number; 
  seed?: number; 
  tools?: Tool[];
  tool_choice?: ToolChoice;
  // Additional optional parameters
  logit_bias?: { [key: number]: number };
  transforms?: string[];
  models?: string[];
  route?: 'fallback';
  provider?: ProviderPreferences;
};
type TextContent = {
  type: 'text';
  text: string;
};
type ImageContentPart = {
  type: 'image_url';
  image_url: {
    url: string; // URL or base64 encoded image data
    detail?: string; // Optional, defaults to 'auto'
  };
};
type ContentPart = TextContent | ImageContentPart;
type Message = {
  role: 'user' | 'assistant' | 'system' | 'tool';
  content: string | ContentPart[];
  name?: string;
};
type FunctionDescription = {
  description?: string;
  name: string;
  parameters: object; 
type Tool = {
  type: 'function';
  function: FunctionDescription;
};
type ToolChoice =
  | 'none'
  | 'auto'
  | {
      type: 'function';
      function: {
        name: string;
      };
    };
```

### Request Headers

```typescript
fetch("https://ai.api.4everland.org/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${YOUR_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional
    "X-Title": `${YOUR_SITE_NAME}`, // Optional
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Who are you?"},
    ],
  })
});
```

## Responses

The responses align closely with the [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat), ensuring that `choices` are always presented as an array, even when the model returns only one completion. Each choice will include a `delta` property if a stream was requested, and a `message` property otherwise. This design simplifies the code usage across all models.

4EVERLAND streamlines the schema across various models and providers, thereby requiring the learning of only one schema.

### Response Body

It's important to note that the `finish_reason` may vary depending on the model provider. The `model` property provides information about the model used within the underlying API.

Here's the response schema, represented as a TypeScript type:

```typescript
type Response = {
  id: string;
  // Depending on whether you set "stream" to "true" and
  // whether you passed in "messages" or a "prompt", you
  // will get a different output shape
  choices: (NonStreamingChoice | StreamingChoice | NonChatChoice | Error)[];
  created: number; // Unix timestamp
  model: string;
  object: 'chat.completion' | 'chat.completion.chunk';
  // For non-streaming responses only. For streaming responses,
  // see "Querying Cost and Stats" below.
  usage?: {
    completion_tokens: number; // Equivalent to "native_tokens_completion" in the /generation API
    prompt_tokens: number; // Equivalent to "native_tokens_prompt"
    total_tokens: number; // Sum of the above two fields
    total_cost: number; // Number of credits used by this generation
  };
};
// Subtypes:
type NonChatChoice = {
  finish_reason: string | null;
  text: string;
};
type NonStreamingChoice = {
  finish_reason: string | null; // Depends on the model. Ex: 'stop' | 'length' | 'content_filter' | 'tool_calls' | 'function_call'
  message: {
    content: string | null;
    role: string;
    tool_calls?: ToolCall[];
    // Deprecated, replaced by tool_calls
    function_call?: FunctionCall;
  };
};
type StreamingChoice = {
  finish_reason: string | null;
  delta: {
    content: string | null;
    role?: string;
    tool_calls?: ToolCall[];
    // Deprecated, replaced by tool_calls
    function_call?: FunctionCall;
  };
};
type Error = {
  code: number; // See "Error Handling" section
  message: string;
};
type FunctionCall = {
  name: string;
  arguments: string; // JSON format arguments
};
type ToolCall = {
  id: string;
  type: 'function';
  function: FunctionCall;
};
```

Here's an example:

```typescript
json
{
  "id": "gen-xxxxxxxxxxxxxx",
  "choices": [
    {
      "finish_reason": "stop", // Different models provide different reasons here
      "message": {
        // will be "delta" if streaming
        "role": "assistant",
        "content": "Hello there!"
      }
    }
  ],
  "model": "openai/gpt-3.5-turbo" // Could also be "anthropic/claude-2.1", etc, depending on the "model" that ends up being used
}
```