MAC WebCrawler Connector Operations

The WebCrawler connector supports 6 operations. Here's a structured overview of these operations:

Crawl website - MAC Whisperer is a highly customizable web-crawling connector for MuleSoft. It allows users to easily crawl websites, retrieve specific content, and automate web scraping processes. This connector is ideal for users looking to gather large-scale web data efficiently. Additionally, it includes several helper operations that allow fine-tuned crawling, such as targeting specific elements and even downloading images from the crawled websites.
Download image - Extract and download images from web pages directly to your system. Download all images from a webpage, or only a specified image (or document!) by providing a direct link to the image / file.
Generate sitemap - Automatically create a sitemap from a website to aid in SEO and site structure analysis. This information can be used to customize the way you want to crawl the website (eg specific pages)
Get page contents - Retrieve all relevant textual content from a given webpage.
Get page insights - Analyze and extract key information from a page, such as word count, link structures, and more.
Get page meta-tags - Collect metadata from a webpage, including title, description, keywords, and other SEO-related information.