Ingestion Presets

Manage ingestion/scraping configuration presets for knowledge bases.

Import JSON

New Preset

Configure ingestion settings.

Name *

Description

Crawling Settings

Follow links

Max Pages

Max Links Per Page

URL Filtering

Include Patterns (regex, one per line)

Exclude Patterns (regex, one per line)

Same domain only

Respect robots.txt

Content Extraction

Use smart extraction (AI) for web pages

Main Content Selector

CSS selector for the main content container

Exclude Selector

CSS selector for elements to exclude

Detail Page Main Selector (optional)

Override main selector for detail pages (pages with query parameters)

Detail Page Exclude Selector (optional)

Additional exclude selector for detail pages. Combined with base exclude selector.

Document Naming Strategy

How to name documents when scraping. Falls back to other options if selected option is not available.

Extract images

Extract links

Chunking Settings

Chunk Size (tokens)

Chunk Overlap (tokens)

Chunk Strategy

Indexing Settings

Index immediately

Scraping Behavior

OCR Language

Override KB language

Scraper Timeout (seconds)

Rate Limit Delay (seconds)

Retry Attempts

Retry Delay (seconds)

Authentication (Optional)

Auth Headers (JSON)

Cookies (JSON)

Metadata

Set as default