Parameters

Overview

This page outlines the various parameters that can be utilized to customize and control the behavior of a model. Understanding and correctly setting these parameters is crucial in influencing the model's responses to cater to specific requirements and use cases.

List of LLM Parameters

1. Temperature

• Type: Optional

• Data Type: Float

• Range: 0.0 to 2.0

• Default: 1.0

• Description: Influences the variety in the model's responses. Lower values lead to more predictable and typical responses, while higher values encourage more diverse and less common responses. When set to 0, the model always gives the same response for a given input.

2. Top_p

• Type: Optional

• Data Type: Float

• Range: 0.0 to 1.0

• Default: 1.0

• Description: Limits the model's choices to a percentage of likely tokens. A lower value makes the model's responses more predictable, while the default setting allows for a full range of token choices. It can be compared to a dynamic Top-K mechanism.

3. Top_k

• Type: Optional

• Data Type: Integer

• Range: 0 or above

• Default: 0

• Description: Limits the model's choice of tokens at each step, making it choose from a smaller set. A value of 1 means the model will always pick the most likely next token, leading to predictable results. By default, this setting is disabled, making the model consider all choices.

4. Frequency_penalty

• Type: Optional

• Data Type: Float

• Range: -2.0 to 2.0

• Default: 0.0

• Description: Controls the repetition of tokens based on how often they appear in the input. It aims to use less frequently those tokens that appear more in the input, proportional to how frequently they occur. Negative values will encourage token reuse.

5. Presence_penalty

• Type: Optional

• Data Type: Float

• Range: -2.0 to 2.0

• Default: 0.0

• Description: Adjusts how often the model repeats specific tokens already used in the input. Higher values make such repetition less likely, while negative values do the opposite.

6. Repetition_penalty

• Type: Optional

• Data Type: Float

• Range: 0.0 to 2.0

• Default: 1.0

• Description: Helps to reduce the repetition of tokens from the input. A higher value makes the model less likely to repeat tokens, but excessively high values can make the output less coherent.

7. Min_p

• Type: Optional

• Data Type: Float

• Range: 0.0 to 1.0

• Default: 0.0

• Description: Represents the minimum probability for a token to be considered, relative to the probability of the most likely token.

8. Top_a

• Type: Optional

• Data Type: Float

• Range: 0.0 to 1.0

• Default: 0.0

• Description: Considers only the top tokens with "sufficiently high" probabilities based on the probability of the most likely token.

9. Seed

• Type: Optional

• Data Type: Integer

• Description: If specified, the inferencing will sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed for some models.

10. Max_tokens

• Type: Optional

• Data Type: Integer

• Range: 1 or above

• Description: Sets the upper limit for the number of tokens the model can generate in response.

11. Logit_bias

• Type: Optional

• Data Type: Map

• Description: Accepts a JSON object that maps tokens to an associated bias value from -100 to 100. The bias is added to the logits generated by the model prior to sampling.

12. Logprobs

• Type: Optional

• Data Type: Boolean

• Description: Determines whether to return log probabilities of the output tokens or not.

13. Top_logprobs

• Type: Optional

• Data Type: Integer

• Range: 0 to 20

• Description: Specifies the number of most likely tokens to return at each token position, each with an associated log probability.

14. Response_format

• Type: Optional

• Data Type: Map

• Description: Forces the model to produce specific output format. Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is valid JSON. Note: when using JSON mode, you should also instruct the model to produce JSON yourself via a system or user message.

15. Stop

• Type: Optional

• Data Type: Array

• Description: Stops generation immediately if the model encounters any token specified in the stop array.

API Example

fetch("https://ai.api.4everland.org/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${YOUR_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional
    "X-Title": `${YOUR_SITE_NAME}`, // Optional
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "huggingfaceh4/zephyr-orpo-141b-a35b",
    "messages": [
      {"role": "user", "content": "What is the meaning of life?"},
    ],
    "top_p:" 1,
    "temperature:" 0.9,
    "frequency_penalty:" 0.7,
    "presence_penalty:" 0.7,
    "repetition_penalty:" 1,
  })
});