SRE 知识工作流
建立 SRE 故障知识库：整理 runbook 与故障复盘
把 runbook、故障复盘与批准的事件笔记整理成有来源、可审核且不会悄悄过期的值班知识。
关于这篇指南
工作流程
维护 runbook、故障复盘与值班知识的 SRE、平台工程与 on-call 团队
9 分钟阅读
01
用一个服务、环境与故障类型固定来源边界。
02
让诊断步骤、缓解限制与验证信号回到当前 runbook 或故障复盘。
03
每次故障或系统变更后审核过期步骤，不让旧建议悄悄留在值班流程。
完整示例
建立 SRE 事件知识库
这是三个文件的虚构桌面演练：用只读观察整理积压，遵守中止上限，并把原因保留为待验证假设。
试做这个任务
只根据三个来源文件，整理观察到的积压影响，提出有时间和取样上限的只读观察计划，并区分假设与必须由事件负责人决定的事项。每个结论都用 [[filename]] 引用。
预期推理
把 8.6% 的 5xx、1.8 秒 p95 和 420 的队列深度记为观察；遵守 15 分钟、三个视图、20 行数据的限制；遇到变更操作或暴露敏感数据就中止；把 09:05 部署视为待比对的关联假设，而不是已证明的原因。
为什么要保持这些内容关联
单个桌面演练资料可以手动复核。当观察、假设、中止规则和后续时间线修正需要保持关联，又不能把笔记变成操作授权时，来源链接的事件 wiki 才有帮助。
审核边界
核对观察数值、15 分钟／三个视图／20 行上限、中止条件和时间先后；升级、回滚和负责人仍由事件负责人判断。
在 Wenlan 中查看这个示例
Wenlan v0.18.3 界面，呈现隔离测试后读回的数据。来源是虚构教学数据，参考答案为这次练习人工编写；画面没有展示 AI 自动生成或审批完成。
引用的来源变更后，Wenlan 将页面标记为过期，正文仍保留原文。这里的「updating…」表示等待重建，不代表已完成修正。请先检查变更的来源，再重建并审核新答案。
来源文件
打开文件即可阅读完整的人工编写 Markdown。
三种语言视图使用相同的人工编写英文来源数据集。
01incident-observations.mdNorthstar API — tabletop observations
# Northstar API — tabletop observations Status: fictional teaching observation sheet; no live service or customer data is involved. At 09:10 UTC, the alert window showed 8.6% HTTP 5xx responses and 1.8 seconds p95 latency for the Northstar API. The normal reference window in this exercise is 0.4% 5xx and 350 milliseconds p95. Queue depth was 420 items; the reference value is 40. These are observations, not a root-cause finding. This packet contains no customer identifiers, raw logs, or evidence of a particular deployment causing the change.
02read-only-triage.mdRead-only tabletop triage guardrails
# Read-only tabletop triage guardrails The responder may inspect the incident dashboard, a redacted log sample, and the deployment timeline. The observation window is at most 15 minutes, with at most three views and 20 redacted log rows. Record timestamps and observed values; do not turn correlation into causation. Do not restart a service, change configuration, invalidate a cache, roll back a release, acknowledge a customer action, or write to an incident system. Abort immediately if a view requests a mutation, exposes an unredacted identifier or secret, or cannot show whether its data is fictional and redacted. A human incident owner decides whether to escalate, page another team, or authorize a change.
03deploy-timeline.mdNorthstar API — fictional deployment timeline
# Northstar API — fictional deployment timeline A cache-configuration deployment completed at 09:05 UTC. The incident alert began at 09:10 UTC. The timeline records those two times but does not contain a causal test, rollback approval, or owner decision. The five-minute ordering makes the deployment a hypothesis to compare against read-only observations. It does not prove that the deployment caused the 5xx responses or latency. Rollback eligibility and the responsible owner are not specified in this packet.
参考答案
Reference answer: observe within limits, preserve uncertainty
The exercise supports a bounded read-only backlog observation: compare the alert values with the reference window, inspect only the named redacted views, and abort on mutation, secrets, or unredacted identifiers. The 09:05 deployment is a temporal hypothesis, not a proven cause; escalation, rollback, and ownership remain human decisions.
Northstar API — triage reference
Observed state
The fictional alert window records 8.6% 5xx, 1.8 seconds p95 latency, and queue depth 420, compared with reference values of 0.4%, 350 milliseconds, and 40. These are observations and do not establish a root cause. incident-observations.md
Safe observation plan
A responder may inspect the incident dashboard, one redacted log sample, and the deployment timeline for at most 15 minutes, three views, and 20 redacted rows. The responder records timestamps and values only. Any requested mutation, unredacted identifier or secret, or unclear fictional/redaction status is an abort condition. read-only-triage.md
Hypothesis and human decisions
The cache-configuration deployment completed at 09:05 UTC, five minutes before the alert. That makes it a candidate correlation to compare, not proof of causation. The packet does not authorize a rollback, identify an owner, or decide whether to escalate; a human incident owner must make those decisions. deploy-timeline.md read-only-triage.md
变更后的来源: deploy-timeline.md
这项变更后的预期更新
The reference's statement that the 09:05 deployment was a candidate cause becomes stale. The 09:05 event was only a review and the actual deployment followed the alert, so it cannot explain the initial onset. The read-only limits, observed values, and human decisions about escalation or rollback remain unchanged; the incident cause is still unknown.
# Northstar API — fictional deployment timeline, revision 2 A cache-configuration review was recorded at 09:05 UTC, but it made no production change. The cache-configuration deployment completed at 09:15 UTC, after the incident alert began at 09:10 UTC. The timeline still contains no causal test, rollback approval, or owner decision. The corrected ordering removes this deployment as an explanation for the alert onset. Other causes remain unknown. Rollback eligibility and the responsible owner are not specified in this packet.
01
先回答：一个故障类型维护一份可复现的知识
每个服务或运维边界建立一份故障知识库。只加入批准的 runbook、故障复盘、架构笔记与脱敏事件摘要；维护症状、前置条件、只读诊断、缓解限制、停止条件、验证信号、负责人和审核日期。
Wenlan 能让支持的文档连接到有来源 Pages、引用、修订、stale 状态、lint 与人工审核。它不会监控生产环境、接收告警、导入实时 telemetry、执行 runbook、批准变更或替代故障管理系统。
02
故障稳定后，再整理来源
从一个服务、环境与故障类型开始。把实时监控、原始日志、凭证、客户数据与变更批准留在专用系统，只把经过批准且可以引用的文档加入来源边界。
保存每份来源的日期、修订、负责人和适用环境。把观察事实、假设、缓解措施与确认原因分开，不要为了写出顺畅摘要而把多个 contributing factors 压成一个原因。
03
建立故障到 runbook 的审核闭环
每次故障或系统变更后，只刷新受影响的知识，并让负责人重新验证步骤。
- 记录服务、环境、故障类型、来源修订与知识负责人。
- 保存症状、适用条件、只读检查、风险、停止条件与验证信号。
- 来源或系统改变时，先把受影响内容标成 stale。
- 在 staging 或桌面演练中测试步骤、链接、预期输出与回退方式。
- 值班使用前打开准确来源；检索到 runbook 不等于获准执行生产变更。
完成 Wenlan 与 AI 客户端设置后
wenlan status
wenlan sources add ~/Ops/approved-incident-knowledge
/distill <服务与故障类型>
/pages <服务故障 runbook>
/lint
/curate04
保留生产操作的安全边界
监控、告警分派、telemetry、故障指挥、权限、变更批准、rollback 执行与紧急访问仍应留在专用系统。Wenlan 不会对生产环境验证命令，也不能保证 runbook 安全。
操作前由具名负责人核对服务版本、环境、权限、风险、预期输出、停止条件与 rollback。这份知识提供可检查上下文，不是自动执行授权。
让一个故障教训能够复现
选定一个服务与故障类型，连接当前 runbook 与故障复盘，再在下一次值班前审核每个运维主张。
常见问题