Measuring performance in kanban

Question

Let's assume I decide to manage a certain project using Kanban. This happens to be a customer service kind of project where the dev team delivers continuous improvement so Kanban is definitely well suited and scrum would probably be over the top and ultimately unnecessary.

Let's also assume I want to measure the performance of the team. I know someone prescribes to take a, say, two weeks activity and count the number of tasks delivered in that timespan. You can repeat that process every two weeks and so monitor performance. It is advisable to break down stories so that they are kind of comparable and basically become a unit of measure.

I have never used Kanban before but being able to break down stories into similar sized ones sounds like a not very feasible goal to me. In fact the more I think about it the more it seems difficult.

What about adding story points to each task on the board before starting development as we do in Scrum? Would that be a theoretically sound practice?

What would you consider "good performance"? You need to know what you think is important before you start to measure things, or you'll be judging people on things that don't matter. — Erik, May 19 '19 at 20:21
Neither Scum nor Kanban measure team performance. They both estimate delivery based on predictable, localized cadences. — Todd A. Jacobs, May 19 '19 at 23:48
@ToddA.Jacobs oh really? So why does Scrum measure velocity and works hard to increase it? Obviously velocity allows predictability but that's not the whole story, is it? Performance (and therefore velocity) monitoring is a matter of interest in Scrum. — nourdine, May 19 '19 at 23:59
@ToddA.Jacobs interesting answer ... ultimately and although it is not a performance metric to be used to continuously push the limit, it seems to me velocity is a performance metric all the same. It surely can be used to spot things that start to get wrong in the team as well. — nourdine, May 20 '19 at 07:11

score 5 · Accepted Answer · answered May 19 '19 at 22:56

5

The idea of managing a project with kanban is subtly off the mark and may cause frustration for you. Kanban is a method of optimizing process. This means you absolutely can apply kanban to the development process and improve that process. A more efficient process will often support a more successful project, there is a ton about project management that kanban doesn't cover that you need to account for.

I'm not certain how the size of items is connected here. "Small batch sizes is an idea that comes from lean and most kanban teams find that the size of their items when they start are problematic and need to be smaller, but size is relative and you will have to decide in your context. Similarly, story points often don't add value in teams using kanban, but if they add value for you, great.

On to performance measurements. Some very common ones are:

Throughput: amount of items done in some period of time (5 backlog items per week or 3 features per quarter for example)
cycle time: time it takes from start of work until delivery
lead time: time from when a need is identified until it is delivered
escaped defects: number of bugs that slip past the team
value delivered: like throughput, but measured in some value units realized instead of work items done.

It is important to keep in mind that kanban is much more than the board. If you aren't familiar with the kanban method, you probably want to learn more about the practices in it. In my opinion, it can be harder than scrum because, while it is very lightweight, it takes a lot of discipline and transparency. Also, if you are starting with Kanban, I would highly recommend looking into STATIK as an approach to starting with kanban.

answered May 19 '19 at 22:56

Daniel

16,867
2
21
49

The performance measurement you mentioned are ok I suppose. You seem to know the subject way better than I do, so I take those measurements are popular and accepted among Kanban people. On the other hand, simply from the perspective of a rational person, I find it hard to consider reliable, or even acceptable, the throughput measurement to name one. If during week 1 the team completes 10 very simple tasks and in week 2 the same team completes 3 horribly complex bugs what conclusion can I draw? Nothing I suppose. It's like asking if it's faster a train or heavier an elephant. See what I mean? – nourdine May 19 '19 at 23:06
1

I know exactly what you mean. This is a common challenge with kanban. It is great for optimizing a process toward some end. Want to deliver more value per quarter? Awesome. Want to reduce turn-around time? Great. Want to know if your programmers are any good? This gets fuzzy. – Daniel May 19 '19 at 23:13
In truth, this isn't much different anywhere else. Knowing if a developer is good has always been a difficult task. I can identify the ones in the extreme (never get anything done, create a lot of bugs or always pulls through amazing work and creates very few bugs) but real objective measurement has always been elusive. – Daniel May 19 '19 at 23:16
Coming from Scrum it would seem sensible to me, to add an estimate of the task effort using poker. Devs then get fed estimated tasks and carry on using kanban as normally prescribed. At regular intervals I work out velocity and boom! If it works for Scrum why shouldn't it work for kanban. It's just an estimate on the story after all. Any programming techniques or other technical choices the team makes will get reflected on the velocity and I will get a way of monitoring the guys and the workflow will still be handled in a lightweight manner with kanban. I cannot see what could go wrong. – nourdine May 19 '19 at 23:25
@nourdine That simply won’t work. Kanban optimizes for flow and throughput, not time-boxing or variadic packing. At best, you’ll have a frustrating framework mismatch if you treat the framework components as hot-swappable practices. There’s some overlap, but the frameworks are fundamentally different, and Daniel did a good job capturing the core metrics for Kanban. Velocity isn’t one of them, because Kanban cycles are continuous. – Todd A. Jacobs May 19 '19 at 23:37
@Todd, do you have an answer for the doubts I expressed in my first comment to this answer? – nourdine May 19 '19 at 23:40
@nourdine Daniel already answered it. If you want to understand why effective queues require standardized batch sizes, or how to decompose your work better, please open a new question. Meanwhile, if you can’t standardize the size of your backlog items then Kanban is probably the wrong framework for your project. – Todd A. Jacobs May 19 '19 at 23:45
1

You can estimate in both Scrum and Kanban if you finds it adds value. Most teams don't. I usually warn people off of using velocity or throughput for measuring team performance because I've seen to many teams with high velocity that can't get a feature to the customer for anything. I'd much rather judge them on value delivered in a period Nd get them to rally around that. Also, measuring teams or larger groups over individuals allows you to promote healthy behavior like knowledge sharing and assisting each other, which usually leads to better results. – Daniel May 20 '19 at 01:47
All that said, I feel I should reiterate that Kanban is designed to help you optimize an existing process. If that isn't your goal, that is likely why these metrics don't seem to meet your needs. That doesn't mean the team wouldn't benefit from Kanban, but you might need a different tool in addition. – Daniel May 20 '19 at 01:52
@Daniel I do need to optimize an already relatively well oiled process. As I said it is a technical customer service "project" where devs sort out customers' issues (bug fix and various others things like helping them configuring things etc etc). It is a never ending project and it's not a product per se (which is way Scrum looks over the top to me in this scenario). All I was looking for is a way to be able to estimate if a certain bunch of tickets can be fulfilled in a certain time frame (2 weeks?). Being tickets very different in size that seems impossible, hence the idea of using estimates – nourdine May 20 '19 at 07:18
@nourdine That's not a project, that's service delivery. There are entire frameworks based around service delivery, such as ITIL. With Kanban, you must have a standardized batch size for your queue metrics to mean anything. You can, however, have different SLAs for various swimlanes and multiple kanbans, but that's well outside the scope of the question you asked. As comments are not for extended discussion, please either start opening related questions, or this thread will shortly get moved to chat. – Todd A. Jacobs May 20 '19 at 21:28
@ToddA.Jacobs I do not understand how the batch size is related to my last comment. – nourdine May 20 '19 at 21:59
@nourdine “Tickets very different in size” means you aren’t optimizing your batch size for flow. – Todd A. Jacobs May 20 '19 at 22:48

score 0 · Answer 2 · answered May 19 '19 at 20:07

My suggestions for measuring performance in a Kanban team would be:

Customer feedback - including internal customers like people from the business side of your organisation
Collaboration - time spent supporting other team members, etc.
Adaption and innovation - track ideas and solutions arising from the team

Measuring performance in kanban

2 Answers2