What Leo saw at Kafka, 3. The release manager's playbook, from the release managers

Kafka had a written release process. Its wiki page in October 2019 explained how to set up signing keys, cut a branch, run the release script and publish to the mirrors. What it did not say was how a release is actually steered: what gets in, what gets moved out, what counts as a blocker, and what to do about the tests that fail for no reason. That part lived in the release managers' mail.

The claims, and where to check them

Claim	Where to check it
Kafka announced seven releases in 2019; its three feature releases (2.2.0, 2.3.0, 2.4.0) each had a different volunteer release manager	The [ANNOUNCE] and release plan mails on the dev list, 2019
The 2.2.0 plan set four dates: KIP freeze 24 Jan, feature freeze 31 Jan, code freeze 14 Feb, release 28 Feb 2019 (tentative); 2.2.0 was announced on 26 Mar	The 2.2.0 release plan mail (8 Jan 2019) and the announcement (26 Mar 2019) on the dev list
On 17 Feb 2019, 108 unmerged tickets were moved out of 2.2.0, each with the same comment	Kafka's JIRA: comments of 17 Feb 2019 reading "Moving all major/minor/trivial tickets that are not merged yet out of 2.2 release"
To get green builds for 2.2.0, one developer opened 82 flaky test tickets between 12 Feb and 21 Apr 2019	The same JIRA, filtered by reporter and date
On 13 Nov 2019 the 2.4.0 release manager was "hesitant to include as blocker" an issue "since this is existing issue"; it was fixed and included the next day	The 2.4.0 release thread on the dev list, 13 Nov 2019; KAFKA-9046, resolved 14 Nov 2019
Kafka's Release Process page in 2019 covered the mechanics and said nothing about flaky tests; the step was added on 11 Jan 2024 as "Monitor Tests"	Release Process, version 159 (7 Oct 2019) and version 258 (11 Jan 2024, "add section mentioning the need to monitor tests early on")

The work as it was

Each feature release had a volunteer release manager, a different person each time. They posted a plan with four dates (KIP freeze, feature freeze, code freeze, release), tracked it by hand on a wiki page, chased the owners of blockers, cut the branch, and called the votes. For 2.2.0 the tentative date was 28 February; the release was announced on 26 March, after three candidates.

Much of it was the same keystrokes, many times. On 17 February 2019, after code freeze, 108 tickets that had not merged were moved out of the release, each with the identical comment "Moving all major/minor/trivial tickets that are not merged yet out of 2.2 release". Many had been carrying a fix version forward for years. And because approved pull requests could not merge without a green build, the same release manager spent ten weeks filing 82 flaky test tickets.

What Leo saw

Four of the sixteen developers ran a release in 2019 (2.2.0, 2.3.0, 2.2.2 and 2.4.0), and Leo joined their accounts into one playbook: twelve steps from volunteering to the announcement, with where each one hurts. It also has the rules nobody wrote down, as the managers applied them. In Leo's words, from their mail:

• After code freeze, only blockers go in, and a blocker is "a regression on the release branch or anything that can lose data. A bug that already existed in earlier releases is not a blocker."
• Except when it is. On 13 November 2019 the 2.4.0 release manager wrote, "Since this is existing issue, I am hesitant to include as blocker"; the Connect developers explained it stopped users upgrading, and it was fixed and included the next day. Another developer's view, as Leo recorded it: an old bug "should not be ruled out as a release blocker for that reason alone". Leo keeps both lines, from both people.
• Before a candidate, the release manager works to get stable test runs on the release branch, opening a ticket for each failure and asking the people who know each area whether it blocks.

The ways forward

• Improve: one sweep, not 108 comments. Moving unmerged tickets out at each freeze becomes one bulk change or script run by the release manager.
• Eliminate: nothing to sweep. A fix version goes on a ticket only once it has a pull request close to merging, so years old tickets never ride from release to release.

What happened next

• 2020. The wiki's Time Based Release Plan was edited once; its rule for code freeze still said only "fixes for blocking bugs", without saying what makes a bug blocking.
• 11 Jan 2024. The Release Process page gained a section, Monitor Tests: "Due to flakes, this isn't always possible - so it's necessary to triage the tests & recognize what are flakes. The earlier you do this, the easier it'll be later." The step Leo had drawn from 2019 mail as step 7 was written down four years later. On the card it is recorded as a path Leo did not offer: writing the step down, rather than changing it.

What this does not show

• The ticket sweeps may have been done with a bulk edit that posts one comment per ticket. Leo asked; the record does not say. Either way, it was work moving tickets that should not have been in the release.
• We checked two of Kafka's wiki pages (Release Process and the Time Based Release Plan), not every page or doc it has. "Not written down" means not on those two pages, at those dates.
• Kafka is famous and the model knows a great deal about it. Everything here is from Kafka's own dated mail, JIRA and wiki, and the claims box says where.

Leo learns how a business really runs by reading what its people write and talking with them, lays out the ways the work could go, side by side, and keeps the choice with the people who make it. When they take a path, its card records it, including one Leo did not offer.