Pragence

Google Drive

Index files from Google Drive. Connector type: google_drive. There are two ways to sign in.

Setup

Sign in with Google. The connector reads as the account that signed in, with read-only Drive access, and its token is kept fresh. The deployment must have SOURCE_GOOGLE_CLIENT_ID and SOURCE_GOOGLE_CLIENT_SECRET set. In the console, choose Google Drive under Add source and then Sign in with Google. Through the API, start the sign-in with POST /api/v1/connections/google/authorize, send the operator's browser to the returned address, and pass the connection_id that comes back when creating the connector. See Sign-in connections.

Or use a service account.

  1. Create a Google service account with read-only Drive access and download its JSON key.
  2. Share the folder (or shared-drive content) with the service account's address.
  3. Create the connector with the key in credentials.service_account.

Either way, set settings.folder_id to limit the connector to one folder. Without it the connector reads every file the account can see. Then start a sync with POST /api/v1/connectors/{connector_id}/sync, or choose Sync now in the console.

{
  "connector_type": "google_drive",
  "name": "Policies",
  "collection_id": "<collection-id>",
  "schedule": "0 */6 * * *",
  "credentials": {"service_account": { "...": "the service account JSON key" }},
  "settings": {"folder_id": "<folder-id>", "mirror_permissions": false}
}

What gets indexed

File typeHow it is read
Google DocsExported as plain text
Google SheetsExported as CSV
Google SlidesExported as plain text
PDFText extracted
Word (DOCX), Excel (XLSX), PowerPoint (PPTX)Text extracted
Plain text and MarkdownRead as is

Other file types are not listed and are not indexed. A file that cannot be turned into text (a scanned PDF, a password-protected file) is reported on the run with a reason and left out.

Sync behaviour

  • Incremental. A sync reads files modified since the last applied sync started. Pass since to choose another point.
  • Scope. The folder in folder_id and every folder inside it, including content in shared drives. Files in the trash are left out. A folder tree of more than 2,000 folders fails with too_large rather than syncing part of it.
  • Deletions. A file that is no longer in scope, including one moved to the trash, is deleted from the index. A sync started with limit does not remove anything.
  • Slowing down. Requests Google throttles are retried with backoff, and an expired sign-in is renewed during a long sync.
  • Schedule. A stored cron schedule is executed. The catalogue suggests every six hours (0 */6 * * *).
  • Refusals. If Drive rejects the sign-in or the file listing, the run fails with source_refused. It is not reported as an empty folder.

Respecting Drive sharing

By default every document the connector indexes is available to everyone who can use the project. Set settings.mirror_permissions to true to limit each document to the people Drive shares it with. Only an organization admin can turn this on or off.

With mirroring on, each sync reads the sharing of every file in scope, not only changed files:

Drive sharingWho can get answers from the file
Anyone with the linkEveryone in the organization
A domainUsers whose email is in that domain
A personThe user with that email
A groupMembers of that group, as supplied below

Things to know:

  • Group members are supplied by you. Drive reports that a file is shared with a group but not who is in it. An admin supplies memberships with PUT /api/v1/connectors/{connector_id}/access-groups or on the source's Access tab. A group with no members listed grants nobody access. GET .../access-groups lists groups that still have none.
  • Sharing must be re-confirmed. A restricted document whose sharing has not been confirmed by a sync within ACL_MAX_STALENESS_HOURS (default 24) is hidden until the next successful sync. Schedule the connector more often than that.
  • A failed permission read fails the sync with permissions_failed.
  • Turning mirroring off makes the connector's documents available to the whole organization again on the next sync.
  • People are matched by the email address of their Pragence user.
  • Organization admins can read every document. Public channels (widget, portal, phone) have no signed-in person, so they answer only from files shared with anyone with the link.

The Google API client packages must be installed in the deployment.

On this page