Embedding Operations

Supported Model Providers

Azure OpenAI (opens in a new tab) (Microsoft): Managed access to OpenAI models through Azure's cloud platform.
Einstein (opens in a new tab) (Salesforce): AI platform integrated into Salesforce for CRM automation and insights.
OpenAI (opens in a new tab) (OpenAI): Developer of advanced AI models like GPT and DALL·E for diverse applications.
Mistral AI (opens in a new tab) (Mistral): Open-weight LLMs optimized for efficiency and customization in AI projects.
Nomic (opens in a new tab) (Nomic): Tools for understanding and visualizing large datasets with embeddings and AI.
Hugging Face (opens in a new tab) (Hugging Face): Community-driven hub for machine learning models, datasets, and tools.

Embedding | Generate from Text

The [Embedding] Generate from text operation optionally split the text into chunks of the provided size and creates numeric vectors for each text chunk.

How to Use

The [Embedding] Generate from text operation can be followed by either the [Store] Add or the [Store] Query operations. The output payload is ready to be used by both [Store] operations without any transformation.

Add Text to Store

When used in combination with [Store] Add operation the text along with the generated embeddings can be ingested into a vector store.

Generate Embeddings from Text - Add to Store

Query from Store

⚠️

When generating an embedding from text for query purposes do not provide any segmentation parameter. Leave blank Max Segment Size (Characters) and Max Overlap Size (Characters).

When used in combination with [Store] Query operation the provided text is first used to generated an embedding that is then used to perform a query against the vector store.

Generate Embeddings from Text - Query from Store

Input Fields

Module Configuration

This refers to the MuleSoft Vectors Embedding Configuration set up in the Getting Started section.

General

Text: The text to generate embeddings for.

Segmentation Fields

Max Segment Size (Characters): The segment size of the document to be split in.
Max Overlap Size (Characters): The overlap size of the segments to fine tune the similarity search.

Embedding Model

Embedding Model Name: Indicates the embedding model to be used (default is text-embedding-ada-002).

XML Configuration

Below is the XML configuration for this operation:

<ms-vectors:embedding-generate-from-text
  doc:name="[Embedding] Generate from text"
  doc:id="92c7a561-7b99-4840-8ffb-f680c9e392dc"
  config-ref="MuleSoft_Vectors_Connector_Embedding_config"
  maxSegmentSizeInChar="3000"
  maxOverlapSizeInChars="300"
  embeddingModelName="sfdc_ai__DefaultOpenAITextEmbeddingAda_002">
  <ms-vectors:text ><![CDATA[#[payload.text]]]></ms-vectors:text>
</ms-vectors:embedding-generate-from-text>

Output Fields

Payload

This operation responds with a json payload.

Example

Here an example of the JSON output.

{
    "embeddings": [
      [-0.00683132, -0.0033572172, 0.02698761, -0.01291587, ...],
      [-0.0047172513, -0.03481483, 0.02046227, -0.037395656, ...],
      ...
    ]
    "text-segments": [
        {
            "metadata": {
                "index": "0"
            },
            "text": "In the modern world, technological advancements have become .",
        },
        {
            "metadata": {
                "index": "1"
            },
            "text": "E-commerce giants like Amazon and Alibaba have redefined ..",
        },
        ...
    ],
    "dimension": 1536
}

embeddings: The list of generated embeddings
- list-item (embedding)
text-segments: The list of segments.
- list-item (text-segment):
  - text: The text segment
  - metadata: The metadata key-value pairs.
    - index: The segment/chunk number for the uploaded data source.
dimension: The dimension of the selected embedding model.

Attributes

EmbeddingResponseAttributes:
- embeddingModelDimension: The dimension for the embedding model used.
- embeddingModelName: The embedding model name used.

Embedding | Generate from Document

The [Embedding] Generate from Document operation creates numeric vectors for provided document's text segments.