Connect a Source
Content reaches Pragence in two ways: a connector that is kept in sync, or
documents added by hand (uploaded files and pasted text). In the console, both
are under Sources (/manage/sources); see the Sources guide.
Connector types
The server has one catalogue of connector types. GET /api/v1/connectors/types
returns it with the credential and settings fields each type takes, and is
authoritative for your deployment.
| Type | Needs | Indexes |
|---|---|---|
sitemap | settings.sitemap_url | Every page listed in a sitemap.xml |
firecrawl | settings.url; a Firecrawl API key (per connector or deployment-wide) | A crawled website, including pages that need JavaScript |
university_website | settings.url; a Firecrawl API key | An institution's admissions pages, with deadlines and costs picked out. For Admissions |
google_drive | A Google sign-in, or credentials.service_account | Google Docs, Sheets and Slides, PDF, Word, Excel, PowerPoint, text and Markdown. See Google Drive |
microsoft_365 | A Microsoft sign-in | Files in a SharePoint site's library, or the signed-in person's OneDrive. See SharePoint and OneDrive |
confluence | An Atlassian sign-in; settings.space_keys | Pages of the chosen Confluence Cloud spaces. See Confluence |
notion | credentials.token | Pages shared with the integration, or one database. See Notion |
filesystem | settings.root_path | Files in a folder mounted on the server |
Workplace tools (dlt:slack, dlt:jira, ...) | Varies | Slack messages, Jira issues, HubSpot and Pipedrive records, Zendesk and Freshdesk tickets, GitHub issues and pull requests, Google Sheets rows. See Workplace tools |
| Help centres, sites and trackers | Varies | Zendesk, Intercom, ServiceNow, Airtable, GitHub, GitLab, Canvas, WordPress and Ghost. See Help centres, sites and trackers |
When ENABLE_DLT is on (it is off by default), the generic
dlt:sql_database, dlt:rest_api and dlt:filesystem types are also listed.
Before the first sync
- The caller needs the Knowledge product grant and write access to the destination collection. Otherwise creating or syncing a connector returns
403. - A fresh install has no billing rates. Until an operator applies the billing catalogue, starting a sync or an ingest returns
503withdetail.code = "billing_catalogue_unconfigured". This is deliberate: nothing is priced by default.
Sources that sign in
Google Drive, SharePoint and OneDrive, and Confluence sign in through the
provider. In the console, choose the source under Add source and sign in.
Through the API, call POST /api/v1/connections/{provider}/authorize, send the
operator's browser to the returned address, and pass the connection_id that
comes back when you create the connector. The deployment must have the provider's client id and secret
configured; see the Connectors API.
Create and sync a connector
Test the configuration first with POST /api/v1/connectors/test, then create
the connector with POST /api/v1/connectors/:
{
"connector_type": "sitemap",
"name": "Product docs",
"collection_id": "<collection-id>",
"schedule": "0 3 * * *",
"settings": {"sitemap_url": "https://example.com/sitemap.xml"},
"credentials": {}
}Start a sync with POST /api/v1/connectors/{connector_id}/sync. It returns
202 with a sync_id and a status_url. Poll
GET /api/v1/ingestion/runs/{sync_id} until the status is no longer queued
or running.
Schedules
schedule is a cron expression evaluated in UTC. Stored schedules are
executed: by the API process when jobs run in-process, or by the job worker's
source-sync-schedule deployment when JOB_DISPATCH_BACKEND=prefect. Set
ENABLE_SOURCE_SCHEDULES=false to turn scheduled syncs off. A scheduled sync
that cannot be paid for or cannot be priced is recorded as a failed run
(insufficient_credits or billing_unconfigured); it is not left queued.
A connector that fails three syncs in a row is paused until someone resumes it.
What a sync does
- Documents whose content has not changed are skipped.
- A source that refuses the sign-in or cannot be read fails the run. It is never reported as a successful sync that found nothing.
- For
sitemap,google_drive,microsoft_365,confluence,notion,filesystemand the help centre, site and tracker types, documents that are no longer at the source are deleted from the index.firecrawlanduniversity_websitedo not detect deletions. The workplace tool types (dlt:slackand so on) do. - If the changes and deletions together exceed the large-change threshold (45% of what is indexed, by default), the run ends
held_for_reviewand nothing is applied until it is approved. - Plan document limits apply. Once the limit is reached, new documents fail with
document_limit_reached; updates to documents already indexed still apply.
Upload a file or paste text
Use POST /api/v1/ingest/upload (multipart) or POST /api/v1/ingest/document
(JSON). Both return 202 with a run_id and status_url; see the
Documents API. Uploads above
INGESTION_MAX_UPLOAD_MB (default 100) are refused with 413.
Limitations
- Sign-in sources need the deployment's own registered app for that provider. Without it, Google Drive can still use a service account; SharePoint, OneDrive and Confluence cannot be connected.
- Only Google Drive and SharePoint/OneDrive can mirror per-person sharing, and only when it is turned on. Every other connector makes what it indexes available to everyone who can use the project.
- SharePoint, OneDrive, Confluence, the workplace tool types and the help centre, site and tracker types have not yet been verified against live vendor accounts.
- Notion access is limited to what the integration has been invited to.
- A connector must use a type from the catalogue. There is no endpoint for registering connector code at runtime.