SRE 知識工作流
建立 SRE 事故知識庫：整理 runbook 與事故復盤
把 runbook、事故復盤與核准的事件筆記整理成有來源、可審查且不會默默過期的值班知識。
關於這篇指南
工作流程
維護 runbook、事故復盤與值班知識的 SRE、平台工程與 on-call 團隊
9 分鐘閱讀
01
用一個服務、環境與事故類型固定來源邊界。
02
讓診斷步驟、緩解限制與驗證訊號回到目前 runbook 或事故復盤。
03
每次事故或系統變更後審查過期步驟，不讓舊建議默默留在值班流程。
完整示範
建立 SRE 事件知識庫
這是三個檔案的虛構桌上演練：用唯讀觀察整理積壓，遵守中止上限，並把原因保持為待驗證假設。
試做這個任務
只根據三個來源檔，整理觀察到的積壓影響，提出有時間與取樣上限的唯讀觀察計畫，並分開假設與必須由事件負責人決定的事項。每個結論都用 [[filename]] 引用。
預期推理
把 8.6% 的 5xx、1.8 秒 p95 與 420 的佇列深度記為觀察；遵守 15 分鐘、三個畫面、20 行資料的限制；遇到變更操作或暴露敏感資料就中止；把 09:05 部署視為待比對的關聯假設，而非已證明的原因。
為什麼要保持這些內容連結
單一桌上演練資料可以手動複核。當觀察、假設、中止規則與後續時間線修正需要保持連結，又不能把筆記變成操作授權時，來源連結的事件 wiki 才有幫助。
審查邊界
核對觀察數值、15 分鐘／三個畫面／20 行上限、中止條件與時間先後；升級、回滾和負責人仍由事件負責人判斷。
在 Wenlan 裡查看這個範例
Wenlan v0.18.3 介面，呈現隔離測試後讀回的資料。來源是虛構教學資料，參考答案為這次練習人工編寫；畫面沒有展示 AI 自動生成或審批完成。
引用的來源變更後，Wenlan 將頁面標記為過期，正文仍保留原文。這裡的「updating…」表示等待重建，不代表已完成修正。請先檢查變更的來源，再重建並審查新答案。
來源檔案
打開檔案即可閱讀完整的人工編寫 Markdown。
三種語言檢視使用相同的人工編寫英文來源資料集。
01incident-observations.mdNorthstar API — tabletop observations
# Northstar API — tabletop observations Status: fictional teaching observation sheet; no live service or customer data is involved. At 09:10 UTC, the alert window showed 8.6% HTTP 5xx responses and 1.8 seconds p95 latency for the Northstar API. The normal reference window in this exercise is 0.4% 5xx and 350 milliseconds p95. Queue depth was 420 items; the reference value is 40. These are observations, not a root-cause finding. This packet contains no customer identifiers, raw logs, or evidence of a particular deployment causing the change.
02read-only-triage.mdRead-only tabletop triage guardrails
# Read-only tabletop triage guardrails The responder may inspect the incident dashboard, a redacted log sample, and the deployment timeline. The observation window is at most 15 minutes, with at most three views and 20 redacted log rows. Record timestamps and observed values; do not turn correlation into causation. Do not restart a service, change configuration, invalidate a cache, roll back a release, acknowledge a customer action, or write to an incident system. Abort immediately if a view requests a mutation, exposes an unredacted identifier or secret, or cannot show whether its data is fictional and redacted. A human incident owner decides whether to escalate, page another team, or authorize a change.
03deploy-timeline.mdNorthstar API — fictional deployment timeline
# Northstar API — fictional deployment timeline A cache-configuration deployment completed at 09:05 UTC. The incident alert began at 09:10 UTC. The timeline records those two times but does not contain a causal test, rollback approval, or owner decision. The five-minute ordering makes the deployment a hypothesis to compare against read-only observations. It does not prove that the deployment caused the 5xx responses or latency. Rollback eligibility and the responsible owner are not specified in this packet.
參考答案
Reference answer: observe within limits, preserve uncertainty
The exercise supports a bounded read-only backlog observation: compare the alert values with the reference window, inspect only the named redacted views, and abort on mutation, secrets, or unredacted identifiers. The 09:05 deployment is a temporal hypothesis, not a proven cause; escalation, rollback, and ownership remain human decisions.
Northstar API — triage reference
Observed state
The fictional alert window records 8.6% 5xx, 1.8 seconds p95 latency, and queue depth 420, compared with reference values of 0.4%, 350 milliseconds, and 40. These are observations and do not establish a root cause. incident-observations.md
Safe observation plan
A responder may inspect the incident dashboard, one redacted log sample, and the deployment timeline for at most 15 minutes, three views, and 20 redacted rows. The responder records timestamps and values only. Any requested mutation, unredacted identifier or secret, or unclear fictional/redaction status is an abort condition. read-only-triage.md
Hypothesis and human decisions
The cache-configuration deployment completed at 09:05 UTC, five minutes before the alert. That makes it a candidate correlation to compare, not proof of causation. The packet does not authorize a rollback, identify an owner, or decide whether to escalate; a human incident owner must make those decisions. deploy-timeline.md read-only-triage.md
變更後的來源: deploy-timeline.md
這項變更後的預期更新
The reference's statement that the 09:05 deployment was a candidate cause becomes stale. The 09:05 event was only a review and the actual deployment followed the alert, so it cannot explain the initial onset. The read-only limits, observed values, and human decisions about escalation or rollback remain unchanged; the incident cause is still unknown.
# Northstar API — fictional deployment timeline, revision 2 A cache-configuration review was recorded at 09:05 UTC, but it made no production change. The cache-configuration deployment completed at 09:15 UTC, after the incident alert began at 09:10 UTC. The timeline still contains no causal test, rollback approval, or owner decision. The corrected ordering removes this deployment as an explanation for the alert onset. Other causes remain unknown. Rollback eligibility and the responsible owner are not specified in this packet.
01
先回答：一個事故類型維護一份可重做的知識
每個服務或操作邊界建立一份事故知識庫。只加入核准的 runbook、事故復盤、架構筆記與去識別事件摘要；維護症狀、前置條件、唯讀診斷、緩解限制、停止條件、驗證訊號、負責人與審查日期。
Wenlan 能讓支援的文件連到有來源 Pages、引用、修訂、stale 狀態、lint 與人工審查。它不會監控正式環境、接收告警、匯入即時 telemetry、執行 runbook、核准變更或取代事故管理系統。
02
事故穩定後，再整理來源
從一個服務、環境與事故類型開始。把即時監控、原始 log、憑證、客戶資料與變更核准留在專用系統，只把經核准且可引用的文件加入來源邊界。
保存每份來源的日期、修訂、負責人與適用環境。把觀察事實、假設、緩解措施與確認原因分開，不要為了寫出順暢摘要而把多個 contributing factors 壓成一個原因。
03
建立事故到 runbook 的審查閉環
每次事故或系統變更後，只刷新受影響的知識，並讓負責人重新驗證步驟。
- 記錄服務、環境、事故類型、來源修訂與知識負責人。
- 保存症狀、適用條件、唯讀檢查、風險、停止條件與驗證訊號。
- 來源或系統改變時，先把受影響內容標成 stale。
- 在 staging 或 tabletop 演練中測試步驟、連結、預期輸出與回復方式。
- 值班使用前打開確切來源；檢索到 runbook 不等於獲准執行正式變更。
完成 Wenlan 與 AI 客戶端設定後
wenlan status
wenlan sources add ~/Ops/approved-incident-knowledge
/distill <服務與事故類型>
/pages <服務事故 runbook>
/lint
/curate04
保留正式操作的安全邊界
監控、告警分派、telemetry、事故指揮、權限、變更核准、rollback 執行與緊急存取仍應留在專用系統。Wenlan 不會對正式環境驗證命令，也不能保證 runbook 安全。
操作前由具名負責人核對服務版本、環境、權限、風險、預期輸出、停止條件與 rollback。這份知識提供可檢查情境，不是自動執行授權。
讓一個事故教訓能被重做
選定一個服務與事故類型，連接目前 runbook 與事故復盤，再於下一次值班前審查每個操作主張。
常見問題