Tools Operations

Tools | Use Native Template

The Tools | Use Native Tempalte operation is useful if you want to create autonomous agents that can use external tools whenever a prompt cannot be answered directly by the AI model. This operation only provides a request to execute the Tools provided in the payload. It doesn't execute them.

Input Configuration

Module Configuration

This refers to the MAC Inference LLM Configuration set up in the Getting Started section.

General Operation Fields

Data: The prompt to be sent to the LLM, along the tools array. The result will suggest a chaining of the tool as a request to fulfill the users request.

Tools Array

Below is an example of the Tools Array in the Payload.

{
   "tools":[
      {
         "type":"function",
         "function":{
            "name":"get_current_temperature",
            "description":"Get the current temperature for a specific location",
            "parameters":{
               "type":"object",
               "properties":{
                  "location":{
                     "type":"string",
                     "description":"The city and state, e.g., San Francisco, CA"
                  },
                  "unit":{
                     "type":"string",
                     "enum":[
                        "Celsius",
                        "Fahrenheit"
                     ],
                     "description":"The temperature unit to use. Infer this from the user's location."
                  }
               },
               "required":[
                  "location",
                  "unit"
               ]
            }
         }
      },
      {
         "type":"function",
         "function":{
            "name":"get_rain_probability",
            "description":"Get the probability of rain for a specific location",
            "parameters":{
               "type":"object",
               "properties":{
                  "location":{
                     "type":"string",
                     "description":"The city and state, e.g., San Francisco, CA"
                  }
               },
               "required":[
                  "location"
               ]
            }
         }
      },
      {
         "type":"function",
         "function":{
            "name":"get_delivery_date",
            "description":"Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'",
            "parameters":{
               "type":"object",
               "properties":{
                  "order_id":{
                     "type":"string",
                     "description":"The customer's order ID."
                  }
               },
               "required":[
                  "order_id"
               ]
            }
         }
      }
   ]
}

XML Configuration

Below is the XML configuration for this operation:

<mac-inference:tools-native-template doc:name="Tools native template" doc:id="ff267b16-b0f7-4a8d-8dd8-004a8269862b" config-ref="Portkey">
    <mac-inference:template ><![CDATA[#[payload.template]]]></mac-inference:template>
    <mac-inference:instructions ><![CDATA[#[payload.instructions]]]></mac-inference:instructions>
    <mac-inference:data ><![CDATA[#[payload.dataset]]]></mac-inference:data>
    <mac-inference:tools ><![CDATA[#[payload.tools]]]></mac-inference:tools>
</mac-inference:tools-native-template>

Output Configuration

Response Payload

This operation responds with a json payload containing the main LLM response. Additionally, attributes such as token usage and tool usage (boolean value) are included as part of the metadata (attributes), but not within the main payload.

Example Response Payload (Tools not used)

If the prompt is a general question that can be answered with public knowledge, such as:

"What is the capital of Switzerland?"

{
    "response": "The capital of Switzerland is Bern."
}

Example Response Payload (Tools used)

If the prompt requires accessing external data through a tool, such as:

"Show me the latest account created in Salesforce."

The AI model will use the configured tool to fetch the information, resulting in:

{
    "response": {
        "tools": [
            {
                "function": {
                    "name": "get_delivery_date",
                    "arguments": "{\"order_id\":\"220\"}"
                },
                "id": "call_ylw95cO0kxCt7NELy91cF7bX",
                "type": "function"
            }
        ]
    }
}

Attributes

Along with the JSON payload, the operation also returns attributes, which include information about token and tool usage:

{
    "attributes": {
        "tokenUsage": {
            "outputCount": 16,
            "totalCount": 220,
            "inputCount": 204
        },
        "additionalAttributes": {
            "finish_reason": "tool_calls",
            "model": "gpt-4o-2024-08-06",
            "id": "chatcmpl-AQwXGmrzcO7XG5Ic1Hp0PKIrRoL4O"
        }
    }
}

tokenUsage: The token usage metadata, returned as attributes.
- outputCount: The number of tokens used to generate the output.
- totalCount: The total number of tokens used for input and output.
- inputCount: The number of tokens used to process the input.
additionalAttributes: Other relevant metadata for the request.
- finish_reason: Indicates whether external tools were used in processing the request.
- model: The id of the model used.
- id: The id of the request.

Example Use Cases

This operation can be particularly useful in various scenarios, such as:

Automating Routine Tasks: Create autonomous agents that handle routine tasks by calling appropriate APIs.
Customer Support: Automate responses to common queries by integrating tools that provide necessary information.
Inventory Management: Use tools to check inventory levels or order status based on user prompts.
Employee Management: Retrieve employee information or manage employee-related tasks through API calls.
Sales and Marketing: Access CRM data or manage leads and accounts efficiently using predefined tools.

Chat MAC Vectors