GitHub Pages SEO and IndexNow

This repository publishes the Docusaurus site to GitHub Pages and includes a small SEO/deployment layer on top of the standard Docusaurus build.

What the deploy publishes

The production docs build publishes the following search-related assets automatically:

• robots.txt from docs/docusaurus/static/robots.txt
• sitemap.xml generated by the Docusaurus sitemap plugin
• site-level metadata and social-card tags from docs/docusaurus/docusaurus.config.ts
• homepage-specific metadata from docs/docusaurus/src/pages/index.tsx

For the live site, those assets are available under the GitHub Pages project path:

• https://nick2bad4u.github.io/eslint-plugin-typefest/robots.txt
• https://nick2bad4u.github.io/eslint-plugin-typefest/sitemap.xml

INDEXNOW_KEY secret

IndexNow is optional and is enabled only when the repository-level GitHub Actions secret INDEXNOW_KEY exists.

When that secret is present, the deploy workflow:

1. builds the docs site,
2. writes indexnow-key.txt into the built site output, and
3. deploys GitHub Pages.

The separate manual Run IndexNow Submission workflow is responsible for:

1. generating the route manifest,
2. diffing changed public routes when requested,
3. verifying the published key file is reachable, and
4. submitting either changed URLs or the full sitemap to IndexNow.

The key file is not committed to the repository. It is generated at build time from the secret so the published site can prove ownership without storing the key in source control.

The route-manifest step is now intentionally more portable across similar ESLint plugin repositories:

• it resolves Docusaurus @site/... sources relative to the configured docs app,
• it can infer the public site URL from package metadata when no explicit override is provided,
• and it can optionally narrow git-diff scanning through INDEXNOW_CONTENT_PATHS_JSON if a copied repository wants stricter path scoping.

First-time verification timing

The first IndexNow submission for a newly published key can take a little while to become usable because Bing still needs to fetch the key file and complete site verification.

• The protocol documents a pending-validation state (202 Accepted).
• In practice, Bing can also return a temporary 403 with SiteVerificationNotCompleted while verification is still propagating.

That response usually means the setup is structurally correct, but Bing has not finished verifying the published key yet.

The automation retries that specific first-run verification state for a bounded window before failing the workflow.

Why the key file uses keyLocation

This site is hosted as a GitHub Pages project site under:

https://nick2bad4u.github.io/eslint-plugin-typefest/

That means the repository controls a path on nick2bad4u.github.io, not the host root itself. The workflow therefore publishes the IndexNow verification file at:

https://nick2bad4u.github.io/eslint-plugin-typefest/indexnow-key.txt

and submits it through the IndexNow keyLocation field instead of assuming a root-host verification file.

Manual IndexNow workflow

IndexNow submission is now split into its own manual workflow:

• Run IndexNow Submission

That workflow exposes manual inputs for:

• submission_mode (delta or sitemap)
• base_ref (optional, for delta mode)
• head_ref (optional, for delta mode)
• content_paths_json (optional, to narrow delta scanning)

Use it when you explicitly want to notify IndexNow after a deploy. Normal docs deploys no longer submit automatically.

Why the workflow no longer re-submits the full sitemap on every push

Continuously re-submitting an entire site sitemap is noisy and unnecessary for a normal docs deployment pipeline.

The manual IndexNow workflow can derive a delta of changed public URLs from:

• the git diff range for the push, and
• a route manifest generated from the current Docusaurus build metadata.

By default the helper diffs the repository broadly and then intersects those changed files with the generated route manifest. That keeps the logic portable across repos whose public content might not live in exactly the same folders.

If a copied repository wants to constrain the diff to specific content roots, it can set INDEXNOW_CONTENT_PATHS_JSON to a JSON string array of repo-relative paths.

That means you can submit targeted changes without replaying the full sitemap every time.

This keeps notifications targeted:

• new URLs are submitted,
• modified URLs are re-submitted,
• renamed URLs are submitted at their new location,
• unchanged existing pages are not replayed.

If you ever need a full replay, the submit-sitemap helper still exists as a manual script path, but it is no longer the default workflow behavior.

Behavior summary

Workflow	INDEXNOW_KEY present	Result
Deploy Docusaurus To GitHub Pages on push	no	Deploy site only
Deploy Docusaurus To GitHub Pages on push	yes	Deploy site and publish the verification key file
Deploy Docusaurus To GitHub Pages on manual dispatch	no	Deploy site only
Deploy Docusaurus To GitHub Pages on manual dispatch	yes	Deploy site and publish the verification key file
Run IndexNow Submission in delta mode	yes	Submit changed public URLs derived from the selected diff
Run IndexNow Submission in sitemap mode	yes	Submit all URLs from the deployed sitemap

Operational notes

• If the secret is missing, the deploy workflow skips writing the public verification key file and the manual IndexNow workflow fails fast with a clear error.
• The submission workflow validates the deployed key file and sitemap before notifying IndexNow.
• The submission workflow retries Bing's temporary SiteVerificationNotCompleted response instead of failing immediately on the first verification attempt.
• If delta mode has no usable diff baseline or no changed public URLs, the submission workflow exits cleanly without posting anything.

Related files

• Developer docs index
• Docusaurus config
• Deploy workflow
• Manual IndexNow workflow