# Requests & Responses

## Requests

### Request Body

The request schema, represented as a TypeScript type, serves as the body of the POST request to the `/api/v1/chat/completions` endpoint. An example of this can be found in the [Quick Start](/ai/ai-rpc/quick-start.md) above.

```typescript
type Request = {
  // Either "messages" or "prompt" is required
  messages?: Message[];
  prompt?: string;
  model?: string; 
  response_format?: { type: 'json_object' };
  stop?: string | string[];
  stream?: boolean; 
  max_tokens?: number; 
  temperature?: number; 
  top_p?: number; 
  top_k?: number; 
  frequency_penalty?: number; 
  presence_penalty?: number; 
  repetition_penalty?: number; 
  seed?: number; 
  tools?: Tool[];
  tool_choice?: ToolChoice;
  // Additional optional parameters
  logit_bias?: { [key: number]: number };
  transforms?: string[];
  models?: string[];
  route?: 'fallback';
  provider?: ProviderPreferences;
};
type TextContent = {
  type: 'text';
  text: string;
};
type ImageContentPart = {
  type: 'image_url';
  image_url: {
    url: string; // URL or base64 encoded image data
    detail?: string; // Optional, defaults to 'auto'
  };
};
type ContentPart = TextContent | ImageContentPart;
type Message = {
  role: 'user' | 'assistant' | 'system' | 'tool';
  content: string | ContentPart[];
  name?: string;
};
type FunctionDescription = {
  description?: string;
  name: string;
  parameters: object; 
type Tool = {
  type: 'function';
  function: FunctionDescription;
};
type ToolChoice =
  | 'none'
  | 'auto'
  | {
      type: 'function';
      function: {
        name: string;
      };
    };
```

### Request Headers

```typescript
fetch("https://ai.api.4everland.org/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${YOUR_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional
    "X-Title": `${YOUR_SITE_NAME}`, // Optional
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      {"role": "user", "content": "Who are you?"},
    ],
  })
});
```

## Responses

The responses align closely with the [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat), ensuring that `choices` are always presented as an array, even when the model returns only one completion. Each choice will include a `delta` property if a stream was requested, and a `message` property otherwise. This design simplifies the code usage across all models.

4EVERLAND streamlines the schema across various models and providers, thereby requiring the learning of only one schema.

### Response Body

It's important to note that the `finish_reason` may vary depending on the model provider. The `model` property provides information about the model used within the underlying API.

Here's the response schema, represented as a TypeScript type:

```typescript
type Response = {
  id: string;
  // Depending on whether you set "stream" to "true" and
  // whether you passed in "messages" or a "prompt", you
  // will get a different output shape
  choices: (NonStreamingChoice | StreamingChoice | NonChatChoice | Error)[];
  created: number; // Unix timestamp
  model: string;
  object: 'chat.completion' | 'chat.completion.chunk';
  // For non-streaming responses only. For streaming responses,
  // see "Querying Cost and Stats" below.
  usage?: {
    completion_tokens: number; // Equivalent to "native_tokens_completion" in the /generation API
    prompt_tokens: number; // Equivalent to "native_tokens_prompt"
    total_tokens: number; // Sum of the above two fields
    total_cost: number; // Number of credits used by this generation
  };
};
// Subtypes:
type NonChatChoice = {
  finish_reason: string | null;
  text: string;
};
type NonStreamingChoice = {
  finish_reason: string | null; // Depends on the model. Ex: 'stop' | 'length' | 'content_filter' | 'tool_calls' | 'function_call'
  message: {
    content: string | null;
    role: string;
    tool_calls?: ToolCall[];
    // Deprecated, replaced by tool_calls
    function_call?: FunctionCall;
  };
};
type StreamingChoice = {
  finish_reason: string | null;
  delta: {
    content: string | null;
    role?: string;
    tool_calls?: ToolCall[];
    // Deprecated, replaced by tool_calls
    function_call?: FunctionCall;
  };
};
type Error = {
  code: number; // See "Error Handling" section
  message: string;
};
type FunctionCall = {
  name: string;
  arguments: string; // JSON format arguments
};
type ToolCall = {
  id: string;
  type: 'function';
  function: FunctionCall;
};
```

Here's an example:

```typescript
json
{
  "id": "gen-xxxxxxxxxxxxxx",
  "choices": [
    {
      "finish_reason": "stop", // Different models provide different reasons here
      "message": {
        // will be "delta" if streaming
        "role": "assistant",
        "content": "Hello there!"
      }
    }
  ],
  "model": "openai/gpt-3.5-turbo" // Could also be "anthropic/claude-2.1", etc, depending on the "model" that ends up being used
}
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.4everland.org/ai/ai-rpc/requests-and-responses.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.