Managing indexers 

General indexer configuration 

An indexer configuration typically defines:

• source type or indexer identifier;
• AI connection for embeddings;
• vector store connection;
• target collection;
• chunk size, overlap and limits;
• source-specific settings such as root pages, import path or connector fields;
• additional metadata.

Chunking 

Chunking splits long source documents into smaller pieces. Smaller chunks can improve precision. Larger chunks can preserve context. Use overlap when answers need information that may cross chunk boundaries.

Source state 

The source state tracks indexed sources and helps the indexer decide whether a source has changed. The extension differentiates source identity and content changes so a re-index can update existing vectors rather than create duplicates.

JSON and JSONL sources 

JSON and JSONL indexing can separate searchable text from structured metadata. This is useful when imported records contain both answer content and additional fields such as authors, departments, categories, URLs or contact details.

json_text_fields

Defines which JSON values become indexed text.

json_metadata_fields

Defines which JSON values are written to additional metadata in the vector payload.

Both options are configured as comma-separated lists using dot notation. Each list entry is prefixed with the configuration option it belongs to. Entries can also be written on separate lines in the additional metadata field.

Examples:

json_text_fields.title,json_text_fields.description,json_text_fields.content.body
json_metadata_fields.author.name,json_metadata_fields.department,json_metadata_fields.keywords.*

Dot notation supports nested structures, numeric array indices and wildcards:

json_metadata_fields.author.name
json_metadata_fields.items.0.title
json_metadata_fields.keywords.*

Metadata fields can also be mapped to custom payload keys:

json_metadata_fields.author.name=contact_name,json_metadata_fields.author.email=contact_email,json_metadata_fields.meta.department=department

Example for a JSON record with type, content.text, keywords, meta.category and id fields:

json_text_fields.type
json_text_fields.content.text
json_text_fields.keywords.*
json_metadata_fields.type
json_metadata_fields.meta.category=category
json_metadata_fields.id=external_id

If no json_text_fields are configured, the JSON document is flattened and indexed as text. For production setups, explicit text fields are recommended because they usually produce more precise retrieval results.

Operational advice 

• Use a separate test collection before changing production chunk settings.
• Rebuild a collection when source identity rules or chunking strategy changes.
• Keep pipeline logs enabled during setup and reduce logging later.
• Use deterministic source identifiers for custom sources.
• For JSON and JSONL sources, configure text fields and metadata fields explicitly whenever possible.