> For the complete documentation index, see [llms.txt](https://documentation.astera.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.astera.com/astera-intelligence/llm-generate/ai-sdk-options.md). # AI SDK Options ### Overview Astera’s [LLM Generate](/astera-intelligence/llm-generate.md) object allows integration with LLM providers such as OpenAI, Llama, or custom LLM APIs. As part of its configuration, the object includes *Ai SDK Options* that provide fine-tuning controls over the LLM model’s behavior and output quality.
These settings allow users to: * Manage token limits * Control randomness and creativity of the output * Access confidence metrics like *LogProbs* and *Perplexity* This document covers all *AI SDK options* available within the *LLM Generate* object. ### AI SDK Options Expanding the AI SDK Options group box reveals the following options: * *Max Tokens*: Limits the output tokens. This limit can be adjusted according to the maximum tokens we want in the output. * *Temperature*: ***C***ontrols randomness in model predictions. At temperature 0 (default), the model's output is deterministic and predictable, at higher temperatures randomness and creativity increases in the output. * *Top P:* Also called nucleus sampling. Controls the diversity of the generated output by selecting tokens from a subset of the most likely options. Top P 0.1 (default) means the model will only consider the smallest set of tokens whose cumulative probability is at least 10%. This significantly narrows down the possible token choices, making the output more focused and less random. Increasing the Top P results in less constrained and more creative responses. * *Evaluation Metrics:* Enabling this option introduces three additional fields in the output: * *Output Tokens*: Total number of tokens in the generated result. * *LogProbs*: Log-probabilities associated with each generated token. These represent the likelihood (on a logarithmic scale) of a specific token being generated, based on the model's understanding of the input and context. They indicate the model’s confidence in generating each token. * *PerplexityScore*: Measure of how well the language model predicts a sequence of tokens. Lower perplexity (closer to 1) indicates better predictive performance, while higher perplexity suggests greater uncertainty in predicting the next token. ### Use Case To understand the use of AI SDK Options, let’s look at a use case. Building on the ticket classification use case we worked on in the last document, [LLM Generate](https://documentation.astera.com/astera-intelligence/llm-generate), we now want to record the confidence score of the classification as well. To achieve this, we will use the *AI SDK Options*. ### Using AI SDK Options in LLM Generate The evaluation metrics is an array of metrics for each output token. We want to limit the output tokens to 1 so that we only need to work with a single metrics record. 1. To do this, we have updated the categories slightly, so the result is no more than 1 token.
2. On the next screen, under the *AI SDK Options* group box, we will specify the output token limit as 1 and enable the evaluation metrics.
3. Notice that additional output fields are visible under the Prompt node. We can use these fields in downstream processing. 4. Let’s say we want to include the confidence score of the AI-generated category in the delimited destination. #### **Log Probs** In its raw form, log probs is a JSON structure containing the metrics of each token.
To use it for evaluating the confidence score, we can parse the *LogProbs* and calculate its exponential. A value closer to 1 means higher confidence of the AI-generated category. We can parse it using a [JSON Parser](/dataflows/text-processors/json-parser.md). 5. Drag and drop a *JSON Parser* from the toolbox and map *LogProbs* to J*SON Parser*’s *Text* input node.
6. To generate the output structure of the *JSON Parser*, copy the *LogProbs* from the *Data Preview* window, go to *Properties* of the *JSON Parser* object, click *Generate Layout by Providing Sample Text*, paste the copied text and click *Generate* to generate the layout.
7. Next, we take the exponential of *LogProb* to obtain a value between 0 and 1, with a value closer to 1 indicating a higher confidence of the result. 8. Drag and drop an [Expression](/dataflows/transformations/expression-transformation.md) object to the designer and map *LogProb* to it. 9. Go to the Properties of the *Expression* object, open the *Expression Builder* by clicking on ![](/files/qsRFvSH4akvzPf6W7bdx) under the *Expression* column for the mapped field, *LogProb* and type the expression for calculating the exponential.
10. The output of the *Expression* object is a confidence score on a scale of 0 to 1 and can be used in downstream processing. 11. Let’s say we want to map the confidence score to the destination. To do this, remove the inbound maps to the *Delimited Destination* object, add all the required fields to the *JSON Parser*, followed by mapping them to the *Expression* object and finally to the *Delimited Destination* object.
12. Preview the output of *Delimited Destination* object to verify the results.
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://documentation.astera.com/astera-intelligence/llm-generate/ai-sdk-options.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.