Publishing to the portal¶

Generated documentation lives in the repository it describes. This portal is an aggregator: it clones each registered repository, copies the documentation into docs/repos/<type>/<name>/, rebuilds navigation, and publishes a static site plus an AI search index.

This guide covers getting a newly documented repository onto the site.

Prerequisites¶

The documented repository must already have:

Its doc gen pull request merged into its default branch (master or main). The portal clones a branch, not a pull request.
A docgen-manifest.md at the repository root containing a ## Documentation type section with a **Type:** value. The portal reads this file to decide where the repository goes. Without it, the repository is skipped with a warning.
The generated README.md, optional packages.md, and docs/ tree at the repository root.

Recognized types map onto portal sections as follows:

Documentation type	Portal section
`Framework`, `Component`	Package Libraries
`Front End`	Web
`Rest API`	REST APIs
`Console App`	Console

An unrecognized type still publishes, but it creates its own section — if you see a new tab appear unexpectedly, the manifest type is probably wrong. Fix it in the source repository rather than working around it here.

API client packages documented under docs/clients/ are the one exception to the one-repository-one-section rule: they appear on the REST APIs page under the API they wrap, and again in the API Clients section of Package Libraries, because a client is both owned by its API and a package you can consume.

Let an agent do it¶

Steps 1 through 4 are mechanical, so a Copilot agent can run them end to end. Open a session on this repository (saif-corp/legends), fill in the three values at the top, and send:

Publish a newly documented repository to the Legends documentation portal.

Repository name: <IT.BOF.Framework>
Repository URL:  <https://SAIFCorporation@dev.azure.com/SAIFCorporation/SAIF/_git/IT.BOF.Framework>
Branch:          <master>

Do the following, stopping and reporting back if any step fails:

1. Confirm the source repository is ready to import. Check that the branch
   above exists and that a docgen-manifest.md sits at the repository root with
   a "## Documentation type" section and a "**Type:**" value. Tell me the type
   you found and which portal section it maps to. If the manifest is missing or
   the type is unrecognized, stop — that has to be fixed in the source
   repository, not here.
2. Create a branch named repo/<repo-slug> from an up-to-date main.
3. Run ./scripts/Add-RepoDocumentation.ps1 with -RepoName, -RepoUrl, and
   -Branch set to the values above, and -SkipCommit so I can review before
   anything is committed.
4. Run python -m mkdocs build --strict. This repository has a known baseline of
   pre-existing link warnings in imported docs/repos content; compare against
   that baseline and only treat new warnings as failures.
5. Show me git status and git diff --stat. The change should be limited to the
   docs-repos.yml entry, the new docs/repos/<type>/<name>/ tree, and the
   regenerated docs/index.md, section index pages, and mkdocs.yml nav. Flag
   anything outside that set instead of committing it.
6. Once the diff is clean, commit, push the branch, and open a pull request
   against main summarizing what was imported.

Do not hand-edit docs/index.md, the section index pages, or the nav block in
mkdocs.yml — those are generated by scripts/clone-repos.ps1. If they are wrong,
fix the script.

The agent stops at the pull request

Merging and the production indexing approvals are deliberately left to a person. Steps 5 and 6 below are yours.

The rest of this guide is the same work done by hand, and is worth reading once so you can tell whether the agent got it right.

1. Branch the portal repository¶

git clone https://github.com/saif-corp/legends.git
cd legends
git checkout -b repo/<repo-slug>

Install the site dependencies once per machine:

pip install -r docs/requirements.txt

2. Register and import the repository¶

scripts/Add-RepoDocumentation.ps1 does the whole import: it adds the entry to docs-repos.yml, clones the repository, detects the documentation type, copies the documentation in, rebuilds navigation and the home page, runs a strict MkDocs build, and commits and pushes on success.

./scripts/Add-RepoDocumentation.ps1 `
    -RepoName "IT.BOF.Framework" `
    -RepoUrl  "https://SAIFCorporation@dev.azure.com/SAIFCorporation/SAIF/_git/IT.BOF.Framework" `
    -Branch   "master"

Option	Effect
`-SkipBuild`	Skip the strict MkDocs validation build.
`-SkipCommit`	Import and build only; leave the changes uncommitted.
`-All`	Re-import every repository already listed in `docs-repos.yml`.

Use -All to refresh the whole portal after a batch of source repositories publish new documentation:

./scripts/Add-RepoDocumentation.ps1 -All

If you are on main

The script creates repo/<repo-slug> (or repo/refresh-all-docs for -All) automatically. On any other branch it commits where you are.

Doing it by hand¶

If you would rather stage the change yourself, add the entry to docs-repos.yml:

- Name: IT.BOF.Framework
  Url: https://SAIFCorporation@dev.azure.com/SAIFCorporation/SAIF/_git/IT.BOF.Framework
  Branch: master

then run the clone-and-import script:

./scripts/clone-repos.ps1

To rebuild navigation and the home page without re-cloning anything:

./scripts/clone-repos.ps1 -OnlyBuildMenu

Both paths regenerate docs/index.md, the per-section index pages, and the nav: block in mkdocs.yml. Those files are generated — edit the scripts, not the output.

3. Preview locally¶

mkdocs serve

Then check:

The new repository appears under the expected section tab.
Its card on the home page shows the right repository count.
Readme, Packages, and per-library or per-screen links all resolve.
mkdocs build --strict passes — a broken internal link fails the build, and the pipeline will fail the same way.

mkdocs build --strict

Mermaid fences

Doc gen emits standard ```mermaid fences, which render on both the portal and Azure DevOps, so current documentation needs no conversion. Azure DevOps only gained support for those fences in May 2026 — anything documented before then uses Azure DevOps ::: fences, which MkDocs cannot render. The hooks/fix_mermaid_fences.py build hook rewrites any that remain, so you do not need to convert them by hand in the source repository.

That hook rewrites the files under docs/, not just the build output, so build before you commit when importing an older repository. Committing straight after the copy stores ::: fences that differ from the rest of the site.

4. Open a pull request to `main`¶

Push the branch and open a pull request against main.

Keep the diff reviewable: an import PR should contain the docs-repos.yml entry, the new docs/repos/<type>/<name>/ tree, and the regenerated docs/index.md, section index pages, and mkdocs.yml nav — nothing else. If unrelated files changed, something rebuilt more than it should have.

The pull request validation build runs the same strict MkDocs build, so a PR that is green locally should stay green.

5. Publish¶

Merging to main is what ships the documentation. The .azdo/azure-docs.yml pipeline then:

Builds the MkDocs static site and publishes it to the documentation storage account.
Runs the AI search stages for each environment: uploads docs/repos/*.md with its metadata to that environment's legends-docs container, deploys the search/ definitions, and starts the search indexer.

Environments run Test → QA → UAT → Production, and each one is gated by its azure_<env> Azure DevOps environment approval. The production site publish waits behind the QA and UAT stages as well.

Pull request validation builds the site but does not index. Indexing only happens on main and on manual queues.

Indexing is incremental¶

Each stage syncs the documentation to blob storage and then runs the indexer over what changed. Blob storage change detection picks up new and edited documents, and the data source's deletion detection policy removes documents whose blobs are gone, so a normal publish does not reprocess the whole corpus.

The pipeline does not wait for indexing to finish¶

Indexing the whole corpus takes well over an hour, so the pipeline starts the run, watches it for a few minutes to catch an immediate failure, and then finishes. A green pipeline means the content was uploaded and indexing began, not that it completed. Give it time before concluding that a document is missing.

An indexer is also single-instance and a run cannot be cancelled once it starts — not even by resetting it. If a run is already going when the pipeline triggers the indexer, the step can fail; queue the pipeline again once the earlier run is idle.

Nothing is lost when that happens. The indexer runs on its own schedule every six hours, so content the pipeline could not index is picked up on the next scheduled run. The schedule is the safety net; the pipeline is just what makes a normal publish show up promptly.

If you changed search/skillset.json or search/index.json, already-indexed documents need to be re-enriched, which requires a full rebuild. Let the merge to main run first, so that every environment has the new definitions, then queue the pipeline manually and tick Rebuild the search index from scratch. Because a reset needs an idle indexer, that option will fail the stage if a run is already in progress — wait for it to finish and queue again. Expect the rebuild to take considerably longer, because every document is re-embedded.

A rebuild does not empty the index. Search keeps answering throughout, and each document's chunks are replaced as it is reprocessed. A newly added field is therefore filled in on some chunks and empty on others until the run finishes. Adding a field is backward-compatible; renaming or retyping one is not, because the index is then deleted and recreated. See the AI Search section of the repository README.md for the chunk fields and how to verify them.

6. Verify in production¶

Once the Prod stage completes, confirm both halves are live:

legends list repos
legends search "<something specific to the new repository>" --repo-name IT.BOF.Framework

If the pages render on the site but search returns nothing, the most likely explanation is simply that indexing has not finished yet — the pipeline does not wait for it. Check the indexer's status before assuming something broke:

az rest --method GET --resource https://search.azure.com `
  --uri 'https://<search-service>.search.windows.net/indexers/legends-docs-indexer/status?api-version=2025-05-01-preview'

Read lastResult.status: inProgress means it is still working. The top-level status field is not what you want — running there only means the indexer is enabled.

If the last run failed, check the Prod indexing stage in the pipeline rather than re-importing. The Run search indexer step reports anything that went wrong in the first few minutes of the run, and warns rather than fails when it had to defer to the schedule.

Keeping documentation current¶

When a source repository regenerates its documentation, re-import it:

./scripts/Add-RepoDocumentation.ps1 `
    -RepoName "IT.BOF.Framework" `
    -RepoUrl  "https://SAIFCorporation@dev.azure.com/SAIFCorporation/SAIF/_git/IT.BOF.Framework" `
    -Branch   "master"

The script removes the existing docs/repos/<type>/<name> folder before copying, so renamed or deleted documents do not linger. Repositories already in docs-repos.yml are not duplicated.