# AI publishing and Markdown

Configure llms.txt, literal content, feeds, discovery manifests, and retrieval publications.

Configure llms.txt, literal content, feeds, discovery manifests, and retrieval publications.

The [complete endpoint reference](https://cybermaps.dev/docs/endpoints/) also includes robots.txt, sitemap families, REST routes, and authorization.

## How it works

The **Enable AI Publication Hub** setting is the master control for fixed discovery publications, localized LLMS routes, RAG chunks, and public discovery REST routes. Publications are assembled from WordPress content and configured publisher data; no external AI account or API key is required.

These representations omit theme chrome, navigation, scripts, and other repeated page layout. Compatible agents can retrieve URLs, summaries, identity, actions, freshness, and publisher guidance with substantially less payload and token overhead than rendering every full HTML page.

### Fixed publications

| Path / route family | Format | Availability | Static mode |
| --- | --- | --- | --- |
| /.well-known/agent-skills/cybermaps-site-guide/SKILL.md | text/markdown | Publication Hub and publication configuration | well\_known |
| /.well-known/agent-skills/index.json | application/json | Publication Hub and publication configuration | well\_known |
| /.well-known/ai-catalog.json | application/json | enable\_discovery\_hub | well\_known |
| /.well-known/api-catalog | application/linkset+json | Publication Hub and publication configuration | well\_known |
| /.well-known/mcp/server-card.json | application/mcp-server-card+json | Publication Hub and publication configuration | well\_known |
| /.well-known/oauth-authorization-server | application/json | Publication Hub and publication configuration | well\_known |
| /.well-known/oauth-protected-resource | application/json | Publication Hub and publication configuration | well\_known |
| /ai-actions.json | application/ld+json | Publication Hub and publication configuration | well\_known |
| /ai-catalog.json | application/json | enable\_discovery\_hub | dynamic |
| /ai-discovery | application/json | Publication Hub and publication configuration | well\_known |
| /ai-discovery.json | application/json | Publication Hub and publication configuration | dynamic |
| /ai-sitemap.xml | application/xml | Publication Hub and publication configuration | all |
| /ai-usage.json | application/json | Publication Hub and publication configuration | well\_known |
| /ai.json | application/json | Publication Hub and publication configuration | well\_known |
| /api-catalog | application/linkset+json | Publication Hub and publication configuration | dynamic |
| /auth.md | text/markdown | Publication Hub and publication configuration | dynamic |
| /cybermaps-openapi.json | application/vnd.oai.openapi+json | Publication Hub and publication configuration | dynamic |
| /feed.json | application/feed+json | Publication Hub and publication configuration | dynamic |
| /knowledge-graph.json | application/ld+json | Publication Hub and publication configuration | all |
| /llms-full.txt | text/markdown | enable\_llms\_full | all |
| /llms-tldr.txt | text/plain | enable\_llms\_tldr | all |
| /llms.txt | text/markdown | Publication Hub and publication configuration | all |
| /news/archive.jsonl | application/x-ndjson | Publication Hub and publication configuration | all |
| /news/changelog.json | application/json | Publication Hub and publication configuration | all |
| /news/llms.txt | text/markdown | Publication Hub and publication configuration | all |
| /news/speakable.json | application/ld+json | Publication Hub and publication configuration | all |
| /skill.md | text/markdown | Publication Hub and publication configuration | dynamic |
| /updates.json | application/json | Publication Hub and publication configuration | all |

`/ai.json` is the **Cybermaps AI Discovery Manifest 1.0**. It is a documented Cybermaps vendor extension, not a claim of an external protocol or independent standard.

`/ai-discovery.json` is the separate community AI Discovery Protocol 3.0 Level 3 publication. It links the site’s knowledge graph, LLMS context, robots directives, JSON Feed, bounded updates, AI sitemap, and the four required `/news/*` context, speakable, changelog, and JSONL archive publications. The protocol frequency hint is `daily`; Cybermaps also invalidates its cached publication when relevant site content or settings change. MCP is a separate, default-off capability; its transport and discovery metadata are available when the administrator enables an MCP mode. `/updates.json` reports only current public content changed in the last seven days and does not infer deletions without an event ledger.

The canonical nested `SKILL.md` uses valid Agent Skills frontmatter. The draft well-known index binds its entry to the exact guide bytes with SHA-256. `/skill.md` remains available for existing links. Consumer discovery and use still depend on the client.

Eligible singular resources also have literal `text/markdown` alternates. Pretty permalinks use `/{permalink}/index.md`, filename permalinks append `.md`, and plain permalinks use `cybermaps_markdown=1`. The representation includes the canonical source URL, content type, language, modified time, and structural Markdown derived from stored content. Headings remain headings, safe links stay clickable, and relative links resolve against the canonical resource. Unsupported URL schemes remain plain text. It does not execute shortcodes or dynamic blocks and is not materialized by the Static File Engine.

The endpoint registry records each fixed path, aliases, handler, body format, media type, enablement setting, static targets, maturity, adoption description, advertising state, and request-cost tier. Routing, status checks, static publication, analytics classification, and manifest generation consume this same registry.

When the AI Publication Hub is enabled, eligible HTML pages advertise their Markdown alternate and describing `llms.txt` publication through both HTML `<link>` elements and RFC 8288 `Link` response headers. The optional Header-Based Discovery control adds the broader feed and API Catalog headers. The API Catalog supplies its registered `rel="api-catalog"` relation, media type, and RFC 9727 profile on GET and HEAD responses at the RFC 9727 well-known location. Its dynamic compatibility alias is retained for existing integrations. The Linkset advertises Cybermaps’ public OpenAPI contract; administrative and secret-authenticated routes are omitted.

### LLMS output and custom AI instructions

`/llms.txt` lists eligible resources in stable publication order with links to their literal Markdown alternates and compact extracts. Its configurable limit is clamped to 20–500 links, with a default of 100. The output reports selected resources and observed coverage. It examines at most 1000 candidates, counted before SEO checks. When a link or candidate limit leaves more content to explore, the output discloses that boundary and links the complete XML sitemap. Configured title, mission, sitemap reference, license, and publisher guidance are included when available.

`/llms-full.txt` is disabled by default because it can become large. It includes the same structural stored-content Markdown for eligible resources without executing shortcodes or dynamic blocks. LLMS inventory is traversed in bounded, non-caching post batches rather than loaded as one corpus.

#### LLMS output ceilings

| Publication | Output ceiling |
| --- | --- |
| /llms.txt | 4 MiB minus 1 byte (4,194,303 bytes) |
| /llms-full.txt | 32 MiB minus 1 byte (33,554,431 bytes) |

These are output ceilings, not guaranteed available capacity. Generation also checks PHP memory headroom for extraction and string handling, so available memory can impose a lower limit. Exceeding the applicable output ceiling or memory-headroom limit makes a dynamic request return **HTTP 507** with `Content-Type: application/problem+json`. In this case, 507 indicates a publication-generation limit, not necessarily a full disk.

Generation produces a complete publication or fails; it does not silently cut entries or truncate the full corpus to fit. The summary’s configured link and candidate-scan limits remain separate selection controls, with coverage reported in the summary.

The same summary/full distinction applies to localized publications such as `/{language}/llms.txt` and `/{language}/llms-full.txt`, and to static publication. Static reconciliation records `publication_too_large` without writing a partial file. See the [Static File Engine guide](https://cybermaps.dev/docs/static-file-engine/) for handling of previously generated files.

**Documentation correction:** these limits already existed in v7.4.1; this is not a new limit introduced in v7.4.2. Verified against the tagged [v7.4.2 LLMS implementation](https://github.com/Alex9001/cybermaps/blob/v7.4.2/src/Discovery/LLMS.php), [v7.4.1 implementation](https://github.com/Alex9001/cybermaps/blob/v7.4.1/src/Discovery/LLMS.php), and [v7.4.2 technical guide](https://github.com/Alex9001/cybermaps/blob/v7.4.2/docs/documentation.md).

#### Cache storage and the budgeted briefing

Cache-storage thresholds do not determine publication output ceilings. The summary’s separate **512 KiB** database-cache threshold controls whether a body can be stored without an external object cache. The full publication bypasses that body cache entirely: it is never stored in a transient or retained in the static generator’s request cache. The summary cache’s size check does not give `/llms-full.txt` a 4 MiB output limit.

The optional budgeted briefing:

*   publishes up to 100 configured pinned IDs first;
*   follows the configured content-type order, modified date descending, and ID ascending;
*   uses fixed-size literal excerpts;
*   includes only complete entries that fit the budget;
*   estimates tokens as `ceil(UTF-8 bytes / 4)`; and
*   reports eligible, selected, omitted, and partial counts.

The briefing examines at most 250 candidates across pinned and regular passes, including candidates excluded by SEO checks. Its Candidate-Scan field reports actual examined rows and whether another candidate remained beyond the limit. Generated briefings larger than 512 KiB are cached only when WordPress uses an external object cache, avoiding oversized database transients.

**Custom AI Instructions** are stored once and published as publisher guidance in `/llms.txt`, enabled `/llms-full.txt`, `/skill.md`, the discovery manifest, the discovery index, and `/wp-json/cybermaps/v1/discovery`. The separate Site Guide addition appears only in `/skill.md`. Schema-bound feeds, XML, usage, action, Linkset, knowledge-graph, and chunk formats stay within their defined field structures.

### Granular publication controls

AI Publishing also provides controls for:

*   LLMS title, mission, content license, included post types, concise link limit, taxonomy filters, sitemap link, and pinned briefing resources;
*   global AI-publication exclusions by post ID and by term ID or slug; the UI consolidates the two historical post-ID fields without dropping either saved list;
*   budgeted-briefing token allowance;
*   manifest endpoint selection, business description, topic labels, and capability labels;
*   Knowledge Graph controls for exposing the administrator as a `Person` and linking the configured primary entity as the website publisher;
*   JSON Feed item limit from 1–100, author fields, and full-content versus summary output;
*   AI sitemap content types, a 1–2,000 eligible-item limit per selected type, up to 100 custom external links, transparent metadata excerpts, and vendor media hints; custom links do not fabricate a `lastmod` value;
*   training, retrieval, and commercial-use preferences plus an optional licensing contact;
*   up to 100 action-to-URL mappings using the supported Schema.org action types.

### Localized LLMS and translation relationships

With the Multilingual AI Hub enabled, active WPML or Polylang languages receive:

*   `/{language}/llms.txt`;
*   `/{language}/llms-full.txt` when the full file is enabled; and
*   `/{language}/llms-tldr.txt` when the briefing is enabled.

The translation integration can also add sitemap `hreflang` alternates. Cybermaps maintains its own translation relationship registry and can synchronize supported WPML duplicates; the editor panel supports manual grouping where needed. Sitemaps settings include the primary language code used in supported `hreflang` output. Language tags are normalized to bounded BCP 47-style values, and an alternate is emitted only while its target remains an eligible, public sitemap resource. Relationship and eligibility changes invalidate the sitemap caches of every connected site.

### Text chunks and search

When enabled, `/discovery/chunks/{post_id}.json` divides eligible structural stored-content Markdown into configurable overlapping character windows. It preserves headings and safe links without executing shortcodes or dynamic blocks. A resource is authorized only when it belongs to the exact bounded inventory selected for the AI sitemap, so advertised chunk links, dynamic authorization, and static publication stay aligned.

Chunk size is normalized to 100–12,000 characters; overlap is normalized from zero to at most half the selected chunk size.

`/wp-json/cybermaps/v1/search` uses bounded WordPress text search over configured public content. `q` is required and limited to 200 characters and UTF-8 bytes; `limit` accepts 1–100 results, defaulting to 20.

On an unresolved front-end URL, a registered crawler signature or conservative `bot`, `spider`, or `crawler` User-Agent receives an RFC 9457 `application/problem+json` response. Cybermaps can include up to three eligible alternatives found by literal WordPress search of the requested slug.

## Concurrent builds and retries

Cold LLMS builds share a site- and publication-specific ownership lock across dynamic, REST, localized, and forced/static generation. A competing request serves an existing valid cached body when available, or returns HTTP 503 with `Retry-After: 5` and `Cache-Control: no-store`.

Generation checks a cooperative 20-second deadline between queries, candidate checks, and rendered entries. Lost ownership, stalled pagination, or an exceeded deadline fails without caching a partial body or replacing an existing static publication. A blocked database or cache driver cannot be interrupted at these checkpoints; its connection and read timeouts still matter. The full corpus has no candidate-count cap.