Introduction

Project Overview

The aim of the MuleSoft AI Chain (MAC) project is to help organisations to design, build, and manage AI agents in MuleSoft's Anypoint Platform. This project is built on the idea that MuleSoft users shouldn’t have to leave MuleSoft to get the most out of AI, they should be able to leverage their existing skills and the power of Anypoint Platform to innovate. What does MuleSoft AI Chain do?

• Simplify Agent Development: MuleSoft AI Chain simplifies the process of building AI agents. It aims to make the orchestration of applications, LLMs, and vector databases easy.
• Unify AI Ecosystems: MuleSoft AI Chain unifies AI ecosystems, allowing developers to optimize their AI usage by incorporating business logic across multiple LLMs. Businesses don't want to be tied to a single LLM - having the flexibility to leverage multiple LLMs based on a desired use case or pricing model is critical to the long term success of an AI strategy.
• Govern the Full Agent Lifecycle: MuleSoft AI Chain allows businesses to govern the full agent lifecycle. Security around AI initiatives is top of mind for every organization. Businesses can seamlessly manage, secure, and monitor their AI deployments on the Anypoint platform.
• The end result? Faster time to value for AI-driven innovation, an omni-channel, best-of-breed experience with all the latest innovative tools in the AI space, and a unification layer for AI that makes managing & monitoring your AI experiences easy.
• The MuleSoft AI Chain (MAC) Project consists of multiple solutions. While the project started with MuleSoft AI Chain as MuleSoft Custom Connector, the vision evolved and it transitioned to a project with a broader scope. We will be introducing more AI related capabilities to the MuleSoft Developers to have a No Code / Low Code experience in Anypoint Studio / Platform.

MAC Project Connectors Support

The MAC Project offers a suite of AI-powered MuleSoft connectors, including MuleSoft AI Chain, MuleSoft Inference MAC Vectors, Einstein AI, Amazon Bedrock, MAC Whisperer and MAC WebCrawler. These connectors ease the way autonomous agents are built, by connecting LLMs, Vector Databases, AI Services, Chains, Tooling and Conversations. Through the unification layer of MuleSoft AI Chain Connectors, customers can easily switch between LLMs and Vector Databases with zero effort.

MAC Project Model Support

With the integration of key AI frameworks into MuleSoft, we make sure to support a broad variety of Large Language Models (LLM) providers to cover the need of the MuleSoft Community. We will continuously invest in adding new model providers to our connectors.

MAC Project Capabilities

Combining the MAC connectors with existing features of the MuleSoft Anypoint Platform, results in a powerful and secure enterprise platform for building AI Agents.

MAC Project Architecture

Building agents using MuleSoft, the connectors can be combined based on the design requirements for the agent. The following architecture demonstrates how different capabilities can be leveraged to implement advanced AI agents by combining the connectors.

Released Connectors

MuleSoft AI Chain

Our flagship project integrates LangChain4j capabilities into MuleSoft, allowing for:

• Easier interaction with LLMs and Vector Store (In-memory only, external Vector Databases are supported with MAC Vectors)
• Optimized usage in MuleSoft applications
• Access to a wide range of AI Services, Tools, and Chains

MuleSoft AI Chain leverages the MuleSoft ecosystem to provide additional capabilities on top of the LangChain4j project. These include:

Agentforce

Agentforce Connector enables developers to easily integrate with AI agents in Salesforce from within Anypoint Platform. It allows you to:

• Seamlessly connect to your Salesforce org where the agents of interest are deployed
• Quickly identify a list of agents that are active and available for integration
• Begin a new session, or agent conversation, to enable application-to-application communication
• Allow for back and forth interaction between external apps and agent to drive improved accuracy and reliability of agent outputs
• Construct prompts that programmatically receive data from external systems, enabling the agent to complete tasks on behalf of human users
• Manage agent sessions by closing conversations with agents to avoid unnecessary open connections serving as a control on both cost and security

Einstein AI

MAC Connector for Salesforce Einstein AI to interact with the models API of the Salesforce platform and benefit from its trust layer and automation capabilities. It allows you to:

• Leverage the Salesforce Trustlayer
• Generate Embeddings with Embedding Models
• Build Prompt Templates in your workflow
• Perform adhoc RAG using Embedding Models
• Perform RAG using MAC Vectors Connector
• Build Chat Memory capabilities
• Optimized usage in MuleSoft applications

MAC Vectors

MAC Vectors Connector for external vector databases (i.e. Milvus, Chroma, Elastic, PGVector, Pinecone, etc.) and for embedding models (i.e. OpenAI, MistralAI, etc.).

Leverage various external Vector Databases / Stores:

• Azure AI Search (opens in a new tab) (Microsoft): Cloud-based AI-powered search with semantic search capabilities.
• AlloyDB (opens in a new tab) (Google): AlloyDB is a fully managed relational database service that offers high performance, seamless integration, and impressive scalability. AlloyDB is 100% compatible with PostgreSQL.
• Chroma (opens in a new tab) (Open Source): Open-source vector database for AI and embeddings management.
• Elasticsearch (opens in a new tab) (Elastic): Scalable search engine for structured/unstructured data and analytics.
• Milvus (opens in a new tab) (Zilliz): Vector database optimized for similarity search and AI workloads.
• Amazon OpenSearch (opens in a new tab) (Amazon Web Services): Managed search service for full-text, structured data queries.
• PGVector (opens in a new tab) (Open Source): PostgreSQL extension for storing and querying vector embeddings.
• Pinecone (opens in a new tab) (Pinecone): Scalable vector database with high-speed similarity search capabilities.
• Qdrant (opens in a new tab) (Qdrant): Vector database with advanced filtering for semantic search applications.
• Weaviate (opens in a new tab) (Weaviate): Weaviate is an open-source, cloud-native vector database that enables storing, indexing, and searching high-dimensional data objects and vector embeddings for AI applications.
• MongoDB Atlas (opens in a new tab) (MongoDB) MongoDB Atlas is a fully managed, multi-cloud database service built on MongoDB’s flexible document model, offering automated scaling, security, and integrated features like full-text and vector search to simplify and accelerate modern application development across AWS, Azure, and Google Cloud.
• Ephemeral File (opens in a new tab) (LangChain4J): Ephemeral file based vector store. Implemented leveraging LangChain4j In-memory store implementation.

Enable to:

• Parse Documents
• Split Documents
• Generate embeddings
• Ingest files into the Vector Database
• Ingest folders into the Vector Database
• Ingest text into the Vector Database

Amazon Bedrock

MAC Connector for Amazon Bedrock to design, build and manage Amazon Bedrock agent workflows, llm interaction, knowledgebase and RAG directly in the MuleSoft Anypoint Platform. The Amazon Bedrock connector allows you to:

• Build and Manage AI Agents
• Dynamic and Flexible Tooling
• Leverage the Amazon Bedrock Security Capabilities
• Ingest files into the supported Embedding Models
• Build Prompt Templates in your workflow
• Perform adhoc RAG using the Vector Database
• Build Chat Memory capabilities
• Optimized usage in MuleSoft applications

MuleSoft Inference

MuleSoft Inference provides access to Inference Offering i.e.

• Github Models,
• Groq,
• Hugging Face,
• Ollama,
• Open Router,
• Portkey and more.

MAC Whisperer

A connector offering Speech-to-Text and Text-to-Speech capabilities. In the beginning it will only support OpenAI.

MAC WebCrawler

A connector offering web crawling capabilities to extract and process data from web pages based on their structure.

Work in Progress

Future Plans

Our goal is to continuously enhance the MAC project by releasing new connectors and improving existing ones. We aim to provide comprehensive AI capabilities within the MuleSoft ecosystem, empowering developers to build more intelligent and versatile applications.

Getting Involved