Back in spring 2018 I wrote a note on my experience using Henrik Knibergs Scrum Checklist as a tool for agile coaching. Having empirical data of how a team progresses in its journey towards agility and adoption of Scrum enables long term optimisation in stead of only relying on Sprint-to-Sprint optimisations. Revisiting the Scrum Checklist periodically gives a track record showing which actions or changes in environment impacted the perceived agility and Scrum-ability.

In the initial edition I used the scale of 0-2 ranging from “statement doesn’t fit with the team” to “team does more than the statement”. This has proven be ambiguous and has been replaced by an expanded Likert scale from 1-10:

Each question from the Scrum Checklist is assessed periodically, e.g. “Delivering what business needs most” is rated 10 if we strongly agree that this is the case and 1 if we strongly disagree. The assessment is accompanied by a reflection on what caused the number to decrease, stagnate or increase: was it a deliberate action, an external event, etc.

Using the assessment – and any other assessment – one should always be aware of caveats:

• The actual number does not matter; the change is important. Are we improving or should we improve?
• Never compare the numbers between two teams; observe global trends. Which questions have the highest variance, lowest or highest average?
• Always supplement assessments with comments; numbers cannot stand alone. Was the change caused by an external event or did the team deliberately change?
• You get what you measure. When using the Scrum Checklist to identify actions and looking for improvements you’ll swiftly find that any team deviating from Scrum will get low scores; e.g. if a team uses Kanban without Scrum.

Aggregating the information from several teams one will be able to spot possible improvements. Below is an aggregated view of a set of teams. Each bar in the graph represents a question in the assessment.

Items marked with a red circle are possible improvement area. One area of improvement could be “Team has all skills needed to bring backlog items to Done. Team members not locked into specific roles.”. To improve this one would have to dig into the premises of the data and involve the development team to a) realize the issue and b) find a way to solve or ease the problem.