10

Some background: At work, there are several epic (user visible) stories whose effort will be greatly reduced by improving the roles and permissions infrastructure in our code base. However, the effort of shoe-horning any one of these epics into existing infrastructure is less than the effort of improving the infrastructure.

Our team has been tasked with improving our roles and permissions. Additionally, since we would like to follow an Agile methodology, we have been given one of the mentioned epics to work on and are attempting to follow Scrum while building it. However, we are struggling with coming up with user stories that produce work that:

  • is user facing (as all user stories should be),
  • improves our infrastructure in any significant way, and
  • fits into one sprint.

It seems to be a 2 out of 3 deal: Either we make things user facing and fit into one sprint at the expense of making things harder to fix in the future, or we improve things in 1 sprint that we can't show anybody, or we improve things and make changes user facing but it doesn't fit into 1 sprint.

This makes me wonder if there is an Agile methodology that would work for us better than Scrum. Do you have any suggestions?

des4maisons
  • 201
  • 2
  • 3
  • 1
    Not all stories need to be user-facing. Non-functional requirements are fine; just spend a bit of time thinking about how to demo the results during Sprint Review. – Todd A. Jacobs Jul 25 '13 at 22:43
  • 3
    Adding to CodeGnome's remark, the key to remember is that what ultimately matters is business value. User stories are a good way of keeping close to value because a user story represents a useful, complete piece of functionality. But if nonfunctional requirements are costing you, then that's untapped business value. Tap that sucker. – Jacques Chester Jul 26 '13 at 00:57
  • 1
    I would downvote this if I had the rep. It is possible to do development using Scrum. You should ask advice about the specific user stories that youre struggling with instead of just trying to confirm your anti-scrum-bias. – Dave Hillier Jul 26 '13 at 08:58
  • 2
    Is your large refactoring going to leave the codebase broken for many weeks? That sounds like a bad idea – Dave Hillier Jul 26 '13 at 09:06
  • Refactoring may change the implementation, but doesn't change an application's behavior. Maybe you're having a hard time decomposing your epics because you're entangling behavior and implementation. Something to think about, anyway. http://en.wikipedia.org/wiki/Code_refactoring – Todd A. Jacobs Jul 28 '13 at 04:10

4 Answers4

13

Stories are about concrete aims you want to achieve with your product. Of course, refactoring may well be required by concrete aims!

There is really no specific way of handling large refactorings in agile on purpose. Let me give you two examples of large refactorings; one is perfectly fine, the other isn't:

  • Change an old and broken data access library and put in a new one, in order to reduce bugs and improve performance

  • Change an old and broken data access library in an application that is not currently developed or maintained

Both examples represent the same technical requirement but they are two completely different stories. In fact, it's impossible to express the second in story form!

This is why agile focuses on what you are trying to achieve and it's constantly hostile towards expressing needs in terms of solutions.

Refactoring is an activity that development teams should do constantly, without a specific story. Refactoring is a normal part of development and every story should include it - some stories have more refactoring than others, but then some other stories have more testing, or more design, etc. The problem only ensues because one takes a task, refactoring, and tries to make it into a story.

In your particular case I would approach the problem in this order:

  1. Does the refactoring and one of the stories that benefit from it fit in an iteration? Just do the story and the refactoring.
  2. Is the refactoring too big to fit in an iteration? Break it down and only do what fits in an iteration - refactoring is always a series of small steps that keep the application in a working state, so there's no big bang in refactoring.
  3. If your refactoring is "special" and you can't break it down, it's not a refactoring. It's a rewrite. Make a business case for it - establish a business value - and use that to create stories. Typically your value is on "verticals" and this allows you to break the rewrite down properly in stories.
Sklivvz
  • 2,454
  • 11
  • 19
  • 3
    In other words, there is nothing specific about scrum in my answer. Units of work that exceed a reasonable time are simply bad ideas, even outside agile. Break it down. – Sklivvz Jul 26 '13 at 09:30
4

If you are making large scale changes to your underlying infrastructure code then The Mikado Method might be useful. Methods and Tools did a review on it.

There is a book on it Mikado Method for Legacy Code by Ola Ellnestam & Daniel Brolund

Another book Working Effectively with Legacy Code by Michael Feathers is also a useful resource.

Geoff Burns
  • 601
  • 2
  • 8
  • 18
4

There is obviously technical debt in the product which has negativity impacted the business resulting in the need to a large refactor. The issue here is not to get carried away and rewrite the entire application, but to focus on the debt areas that are hurting the most.

  1. Build a backlog of known issues

  2. Order them dealing with the most valuable to business first.

  3. Refactor starting at the top and working down until the product is stable.

  4. During refactoring, you will come across many other debt areas. DO NOT fix them part of the current refactor, but rather add them to the backlog. Obviously if it is a issue related to the current item, then fix it. The problem here is that developers easily go down a path and very quickly get into a massive job. Mitigate your risk.

  5. Focus on improving the quality and the unit tests around it. If there is an issue, write a test to prove it (failing test). Then refactor until it passes.

There are many unknowns and I would suggest using t Kanban method to limit WIP and focus on one refactor at a time.

Communicate a lot and manage expectations.

Brett Maytom PST
  • 665
  • 3
  • 6
4

TL;DR

You're gravely misunderstanding a variety of aspects of Scrum, especially the respective roles of user stories, the Product Backlog, and the Sprint Review meeting. Your process needs refactoring a lot more than your infrastructure does.

The problems you're facing aren't unique to Scrum. Unless you address the underlying causes, switching methodologies probably won't buy you anything.

Identified Problems

You have a number of procedural and process problems with your Scrum implementation that stem from improper application of principles. Some specific examples include:

  1. User stories must have a point-of-view, but the perspective does not have to be that of the end-user.
  2. You keep using the word "refactoring," but I do not think it means what you think it means.
  3. You have difficulties with decomposition.
  4. You have difficulties with identifying and/or measuring incremental steps.
  5. Your system design and project methodologies haven't fully embraced the iterative development model.

This may sound critical, but you've actually described most of these problems yourself. I'm suggesting that you need to look deeper into your process to understand why you're having these issues, and to come up with solutions that will work for your team.

Potential Solutions

There's no silver bullet. However, here are some potential solutions for the problems identified above.

  1. Identify the beneficiary of your improved roles and permissions infrastructure.

    Write your user stories from that point of view in order to provide the story with valid exit criteria and help your Product Owner measure the story's value.

  2. Stop using the word refactoring.

    You're not really talking about "refactoring." What you're describing is either a change to non-functional requirements, or possibly a change in the behavior or architecture of your system.

    It's up to the Product Owner to decide if the value of this change set is high enough to off-set the cost of pushing other features down the stack, or if the long-term drag of the technical debt is worth it to avoid having a non-shippable product at the end of a given Sprint. The answer may be "no," in which case you don't have to spend any more time worrying about it since the responsibility for this technical debt will have been accepted by the Product Owner.

  3. Embrace iterative development.

    Iterative development involves trade-offs. One of the often-overlooked trade-offs is that you are often trading time for stability. For example, an iterative rewrite/redesign of your roles and permissions may take longer than starting from scratch in order to maintain the product in a potentially-shippable state at the end of each Sprint.

    This may involve more work, as incremental improvements can involve progressive re-work. It may also involve more time when compared to a full rewrite. It's up to the Product Owner to allocate project resources through story scope and prioritization, so this is ultimately the P.O.'s trade-off to make.

    It's up to the Product Owner to define value to the project, and to prioritize user stories accordingly. By writing stories with clearly-identified value to the project or the team, you can make it more likely that the stories will be prioritized rather than the technical debt accepted as a cost of doing business.

  4. Decompose harder.

    Small, incremental steps require that you have a way to measure success or improvement. In addition, you need to think more about how you will demo non-functional requirements during the Sprint Review.

    As an example, rather than a full rewrite of your permissions system, perhaps a single story will involve improving the speed of your permissions checks. The Development Team and the Product Owner can work together to define "fast enough," and the demo could involve some live or documented benchmarks showing the improvement.

There are certainly other ways to address your issues. Your team should be using Sprint Retrospectives to identify other process problems, and to brainstorm about ways to improve your development process and your Scrum implementation.

Todd A. Jacobs
  • 50,264
  • 7
  • 59
  • 177