Reference

Batch scrape multiple URLs

Efficiently extract content from multiple web pages simultaneously with customizable filtering and output formats. Ideal for gathering data from several sources without manually visiting each page.

Extract structured data

Automatically extracts and organizes specific data from web pages using natural language instructions or a predefined schema, transforming unstructured web content into structured, usable information.

Perform deep research

Automatically investigate any topic by analyzing multiple web sources and synthesizing findings into comprehensive insights with citations. Ideal for getting well-researched answers without manually browsing dozens of pages.

Scrape URL

Extracts content from web pages in your desired format, with optional automation of browser interactions and intelligent data structuring via AI.

Start a web crawl (v2) [NEW]

Automatically crawl and extract data from websites using natural language instructions, with intelligent discovery of related pages and subdomains. The system handles the entire crawling process and delivers structured results automatically.

Start an agent job

Launches an autonomous agent that can navigate websites, perform searches, and extract data across multiple pages based on your instructions. This is ideal when you need to gather complex information that requires clicking through multiple pages or interacting with web interfaces.

Cancel a batch scrape job

Stops a batch web scraping job that's currently running, allowing you to halt unnecessary data collection and free up resources. Use this when a scrape job is taking too long, targeting the wrong content, or no longer needed.

Cancel a crawl job

Stop a running web crawl operation when you no longer need the data or want to free up resources. This helps manage crawling jobs that are taking too long or consuming unnecessary processing power.

Cancel a crawl job

Stop an in-progress or queued web crawl job to prevent it from consuming resources or completing unnecessary data collection. Use this when you need to halt a crawl that's taking longer than expected or was started with incorrect parameters.

Cancel an agent job

Stops a currently running web scraping or data collection task when you need to halt an operation that's taking too long or is no longer needed.

Generate LLMs.txt for a website

Generates an LLMs.txt file that converts your website content into a machine-readable format optimized for AI language models to understand and reference your site.

Get agent job status

Check the progress and completion status of a web data extraction job, retrieving the results once the agent has finished searching and collecting information from web pages.

Get all active crawl jobs

Retrieve a list of all currently running web crawl jobs for your account, allowing you to monitor ongoing data collection operations.

Get batch scrape status

Check the progress and retrieve results of a batch web scraping job you've submitted, allowing you to monitor multi-page scraping tasks and access the collected data when ready.

Get crawl job status

Check the progress and retrieve results from a web crawl job, including all extracted data and metrics about how the crawl performed. Use this to monitor ongoing crawls or collect completed data for further processing.

Get deep research status

Check the progress and retrieve the results of a deep research analysis job you've initiated. This helps you track when your research is complete and access the findings without waiting indefinitely.

Get errors from a crawl job

Retrieves a list of errors and failed pages from a web crawl job, helping you identify which URLs encountered issues during scraping.

Get errors from batch scrape job

Retrieve detailed error information from a batch scraping job to identify which URLs failed and why, helping you diagnose issues and retry problematic pages.

Get extract job status

Check the progress and retrieve results of a data extraction job you previously submitted. This helps you monitor long-running extraction tasks and access the extracted data once processing is complete.

Get historical team credit usage

Retrieve your monthly credit usage history to understand how your web scraping consumption patterns change over time. This helps you track spending trends and plan your budget for API usage.

Get historical team token usage

View your team's monthly token usage history to understand consumption patterns and identify optimization opportunities. This helps you track how many tokens your crawling operations are using over time.

Get LLMs.txt generation job status

Check the completion status of an LLMs.txt generation job and retrieve the generated content once ready. This helps you monitor automated website documentation tasks without manual polling.

Get team credit usage

Monitor your team's remaining credits and understand your current consumption to manage your Firecrawl API usage effectively.

Get team queue status

Check the current status of your team's web scraping queue, including job counts, processing speed, and capacity limits to monitor your scraping workload.

Get team token usage

Check your remaining token credits and balance for Firecrawl's Extract feature to monitor your usage and avoid running out of credits.

Get the status of a crawl job

Check the progress and completion status of a web crawl job you've started, allowing you to know when your website data extraction is ready to use.

Map multiple URLs

Automatically discover and map all pages across a website by crawling from a starting URL, with customizable options to focus on specific content, limit scope, or include subdomains. This helps you understand your site structure and identify all pages for indexing, content management, or analysis.

Preview crawl parameters

Test and preview your website crawl settings before executing a full crawl, so you can verify the correct pages will be captured based on your requirements.

Search

Search the web for specific information and automatically extract clean, structured content from the top results, saving you time on manual research and data collection.

Start a web crawl

Automatically crawl and extract content from websites, gathering all pages and data in bulk from a starting URL. Useful for collecting information across multiple pages without manual copying.