LeoBeta Join the beta

Library/Case studies/Sourcegraph

What Leo saw at Sourcegraph, 2. Every incident update, written five times

During an outage, Sourcegraph's incident messenger wrote each update for the status page and then again for four Slack channels, every 30 minutes. Leo suggested writing it once. Fifteen days later, the repeat posting was gone.

We gave Leo Sourcegraph's public handbook exactly as it stood on 31 December 2021, and nothing later.

Sourcegraph had no part in this

We used Sourcegraph's handbook because it is public and openly licensed (Apache 2.0). Sourcegraph did not take part, and nothing here is Sourcegraph's view of Leo. Every quote below links back to Sourcegraph's own pages and commits.

The claims, and where to check them

Claim Where to check it
During an incident, the person keeping customers informed updated the status page at least every 30 minutes, and posted again in #customer-support, #ce, #sales and #marketing Sourcegraph's incidents page, 31 Dec 2021 (the page at that commit)
Incident records lived in three places: the incident tool, postmortems in a Drive folder, and some write ups as handbook pages The same page
On 15 January 2022 the messenger's role was rewritten, and the repeat posting went Sourcegraph's commit de4b12b1
In mid 2024 postmortems were still kept in a Drive folder, apart from the incident tool Sourcegraph's incidents page, June 2024

The work as it was

When something broke, one person kept everyone informed. The handbook told them how: update the public status page, "These are best offered at least every 30 minutes," and then post again in #customer-support, #ce and #sales, and again in #marketing for anything customers would notice.

So every update was written once for customers and four more times for colleagues, every half hour, in the middle of an outage.

What Leo saw

Leo's page for incident response, joined from the incident process, the playbooks and the on call log.
Leo's page for incident response, joined from the incident process, the playbooks and the on call log.

Leo joined the incident page with the playbooks and the on call pages, and saw two things: the same update written five times, and the record of each incident spread across three places, so nobody could easily count them.

The ways forward

  • Redesign: write the update once. Each status page update reaches the internal channels as it is posted, so the messenger writes it once.
  • Improve: one incident record. The log and the postmortem are drawn from one record, instead of three.

What happened next

The card after Sourcegraph's commit was played back: the redesign way, seen in action on 15 January 2022.
The card after Sourcegraph's commit was played back: the redesign way, seen in action on 15 January 2022.
  • 15 January 2022. Fifteen days after the handbook Leo read, the messenger's role was rewritten. The four repeat posts went; internal channels were now for reviewing drafts and keeping colleagues informed.
  • 2023 and 2024. The channels were renamed, the process stayed.
  • Mid 2024. The records were still in three places: the incident tool, a Drive folder of postmortems, and a few handbook pages.

One of the two ways was taken within weeks. The other stayed on the table.

What this does not show

The commit does not say why the role changed, or how much time it saved. It shows the repeat posting going quickly, and the split record staying for years.

Leo learns how a business really runs by reading what its people write and talking with them, lays out the ways the work could go, side by side, and keeps the choice with the people who make it. When they take a path, its card records it, including one Leo did not offer.