Every company is a software company[1], just look at the top 4 companies by market capitalization[2] being (as of 31st March 2019), Microsoft, Apple Inc., Amazon.com, Alphabet Inc. (Google), but that doesn’t mean “just doing software” is going to work - just ask General Electric, Intel, P&G and HP who made significant investments in IT but without results. To be successful, companies need to be nimble and be able to adjust to the changing marketplace. If they’re not, their competitors sure are! And it’s not just the big boys who are doing Agile (through whatever means they call it, “Start-up culture”, “The Google Way”), but it’s the new kids on the block as well such as Airbnb, Lyft and Tesla to name but a few.
The ability to adapt to the market is critical in the modern economy. With advances in technology this creates lower barriers to entry where newcomers can create disruption and shake things up. As a result, according to Yale professor Richard Foster, the lifespan of a company in the S&P500 has shrunk by more than 50 years in the last century, from 67 years in the 1920s to a more modest 15 years[3]. S&P500 companies, you have been warned – 1 of you is being replaced every two weeks, and for the newcomers … the opportunity is there for the taking!
Agile is a software development concept that evolves software through collaboration between the business and development. It’s methodology encourages an evolutionary approach to solution development (as opposed to a more waterfall method that would typically have a larger design period). The idea is to get a deliverable to the customer asap from which iterative improvements can be made. Having early deliverables will mean everyone is much more clearer about what needs to be developed further and where improvements can be made.
Agile was popularized by the Manifesto for Agile Software Development which stated several values and principles.
Agile is an adaptive methodology that’s highest priority is the satisfy the customer through early and continuous delivery of value. By working in short sprints, the team can deliver value quickly in “chunks” whilst also giving the customer the ability to provide feedback and adjust future requirements accordingly. The “chunks” are sprints that typically are periods of 2 to 3-weeks.
The methodology aims to get business and development working together closely throughout the project. The team are given the suitable environment in order to make decisions in the delivery and succeed in satisfying the customer.
There are daily meetings (ideally face-to-face and standing up) to ensure everyone reports back on what they did yesterday, what they are going to do today, and if they have any impediments that they need help unblocking. The team are allowed to self-organise to pick-up tasks to deliver the work planned for that sprint.
Work is organised into a product backlog (basically a list of requirements that we call user stories) and a set of user stories are chosen for a sprint. At the end of the sprint, the team reflect on how the sprint went and if any improvements could be made.
The Agile Manifesto[4] declares following values:
-
Individuals and Interactions over processes and tools
-
Working Software over comprehensive documentation
-
Customer Collaboration over contract negotiation
-
Responding to Change over following a plan
The Agile Manifesto Principles[5] these values are built on are:
-
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
-
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
-
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
-
Business people and developers must work together daily throughout the project.
-
Build projects around motivated individuals.
-
Give them the environment and support they need, and trust them to get the job done.
-
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
-
Working software is the primary measure of progress.
-
Agile processes promote sustainable development.
-
The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
-
Continuous attention to technical excellence and good design enhances agility.
-
Simplicity—the art of maximizing the amount of work not done—is essential.
-
The best architectures, requirements, and designs emerge from self-organizing teams.
-
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.
However, within a larger Enterprise SAFeR believes that “best architectures, requirements, and designs” do not necessarily emerge from self-organizing teams in contrast to the Agile principles. For an Enterprise, you need to have a common design principle across all teams and that is why this function is therefore proposed to be driven from outside of the team by Enterprise, Solution, and System Architects/Engineers.
But it’s not all rainbows and unicorns. There are pro’s and con’s to any methodology and some scenarios may suit other methodologies. For example, if you are developing nuclear reactors, missiles or alike you may want to choose a different strategy. Some of the pro’s and con’s of agile can be summarised as follows:
| Pro’s | Con’s |
|---|---|
No long design duration before build and can be changed iteratively |
Specification not tightly defined |
Tangible built components visible to business quickly and can be modified |
Components may not be 100% as expected by business |
Bugs raised can fixed iteratively during the build process. |
Solution will not be bug free. |
As outlined in “Thinking of Building a Cloud Operating Model”[6] from Dan Scarfe et al (I couldn’t have said it better myself!):
Having teams operating using an Agile methodology is great, but how does an enterprise facilitate these teams working in this new framework? How can the enterprise manage these teams and ensure that value is being delivered across the organisation and have a high-level overview of what is happening?
This is where the SAFeR is a framework for managing multiple business-focussed streams of work within an enterprise. A kind of backlog of backlogs. It defines these within the context portfolios of epics or major initiatives. An epic might be a new product that you want to launch. It might be a major technology refresh programme you want to undertake. It might be a new market you want to enter. It can be anything.
SAFeR provides a mechanism to manage and then prioritize these major initiatives and provide a bridge into both the IT and business teams that might enable this epic to be delivered.
Underneath the portfolio (epic) level is the program level. Epics are just that, epic! Each epic is broken down into a series of features and enablers which are grouped into a program increment, which is a time-bound period where you deliver some form of business value. This value is managed by an agile release train. Within each program increment, you have a number of scrum teams who actually deliver the work. Over, you’ll see a diagram showing the high-level view of this process.
The thing that is most interesting about SAFeR is its applicability across such a broad range of initiatives you might be running an as organisation. The fact that by using SAFeR, Agile can be applied to far more than just software development. Agile budgeting and planning can now be used to represent and manage the entire innovation pipeline within your organisation.
As an IT organisation, your responsibility becomes standing up, feeding and watering a given number of downstream agile sprint teams. These multi-disciplinary teams will span different technologies. They will have different skillsets. They will do different things. But they will be there, willing and able, to work through the constant stream of requirements. No more functional requirement. No more detailed business cases. No more friction between the business and IT. Peace, and hopefully, harmony.
This brief description doesn’t begin to do justice to the intricacies, or power, of SAFeR. When you fully understand it and see it in action, you will appreciate its power and how useful it can be. For more information on SAFeR, please see www.scaledagile.com.
Getting back to the team again. Agile software development methodology covers a range of specific practices such as Kanban, Extreme Programming, Lean software development and Scrum for example. Scrum gives some additional practices to our Agile method ideally suited to software development teams of around 5 to 10 team members utilizing practices such as sprints and daily stand-ups (or daily scrums). It uses iterative sprints of 2 to 4-weeks where a team takes a set of requirements (the sprint backlog taken from a larger product backlog).
Whilst Scrum is organised into Sprints of a few weeks, Kanban is more about seeing progression in work done. We use Kanban to see progression of Stories, Features and Epics. With Kanban, the focus is more on the flow of workload. Some projects are better suited to Kanban if they are very iterative. For example, quite often Power BI projects I have worked on have better suited Kanban where there is a lot of back and forth with a stakeholder on getting a report “just right” with change being the word, and it’s therefore not always possible to organise a “sprints” worth of requirements because of the nature of the work. In addition, this work is often carried out by a single person and doesn’t warrant the overhead of the various scrum roles.
Alternatively, if you want that member to be part of a team (perhaps delivering a different Feature or set of Stories to the rest of the team, but working in the same sprint) you could define a rather loose Story of “I want a certain report” with tasks underneath that move across more like a Kanban board. Our daily stand-up boards are essentially Kanban boards after all, just we have Stories.
Although the majority of the methodology is geared towards Scrum, consider that there may be instances where just running Kanban could be better suited for some workloads.
Sprints are generally used for tracking progress of Stories, Tasks, Improvements and Bugs.
Our sprints should align to the lifecycle we identified earlier, therefore; Sprint 0 will generally always be a Prep & design sprint. So we can generalise sprints into the following types (although we don’t need to be strict about the specific content of the sprints, it just serves as a good generalisation):
Prep & design sprint(s) are where we are adding to the backlog (via requirements capture), creating information architecture, wireframes, some technical designs and creating graphic designs (or brand design guidelines).
Engineering sprint(s) for technically building and developing the solution - these should form most of the sprints to deliver an Epic.
Testing sprint(s) are stabilization / dedicated testing sprints mid-way through for larger projects. As a rule, you should have one of these sprints for every 3 or 4 engineering sprints. Do not be confused – we are doing testing right from the start! This is a sprint were we are concentrating on improving quality.
Operational sprint(s) are where we’re are going live! Typically done at the end of the project unless you want multiple releases to production in a single project. This could include tasks such as documentation, training, and alike.
In the T-Minus-15 methodology we advocate the general use of a 3-week sprint (15-man days, hence T-Minus-15). Feel free to use a T-Minus-10 or T-Minus-20 but from experience the 3-week sprint allows for a long enough duration to “get stuck in” without an overhead of sprint planning meeting time to organise the next sprint.
To give a high-level overview of the project (Epic) you can put together a Gantt chart down to the level of the sprints along with specific milestones or events such as “Go live”, “Release to production”, etc. However, as we talk about later when we go into DevOps principles, where possible reduce the “big bang” launch date. Rather, an iterative deployment to product is preferred with a comms date for announcing to the wider business (which is not the day after the first release to production!).
As an aside, in line with SAFeTM you may also have an Innovation & Planning (IP) sprint. This sprint acts as an estimating buffer for meeting PI Objectives and provides dedicated time for innovation, continuing education, PI Planning, and Inspect and Adapt (I&A) events[7]. For more information on IP sprints, see https://www.scaledagileframework.com/innovation-and-planning-iteration/.
The Sprint Burndown Chart gives the team a quick graphical representation of the remaining work left in that sprint. It shows us in a view if the team are on track to deliver the promised functionality by the end of the sprint. The ideal burn down diagonal is shown in white and the actual remaining effort (typically shown in hours) in the green bars.
What does the burndown chart above tell us? Well, it looks like the team had a little bit of a slow start, then picked up speed and got ahead of schedule. They then hit some blockers by the looks of it and started to slip behind schedule, but managed to pick it up with a good effort at the end and completed the sprint.
If a team is evidently not on track to deliver the effort estimated by the end of the sprint, the scrum master must approach the product owner to discuss the options available, namely:
-
To descope functionality (Stories) from this sprint, or,
-
To extend the sprint to allow for time to complete the work.
This can be a difficult conversion and one of the sought after skills of a scrum master to tackle this event head-on. The scrum master (and team) needs to address this head on weekly in the checkpoint meeting (rather than leaving it to the end of the project!). This is the primary reason why the weekly checkpoint meeting is indeed weekly (and not just at the end/start of sprints).
The Product Backlog is our single list of requirements for the business. It is a hierarchical list of Epics, Features, Tasks, Bugs and Improvements.
These items can be prioritized into an order of delivery where the User Stories and above should only be updated with the consent of the Product Owner. The Scrum team will use the currency of Story Points to estimate the workload for items. We’ll cover Story Points later, but in essence they are a unit of prescribed varying sizes (a modified Fibonacci sequence) to help the team estimate work sizes. These estimates also help the Product Owner prioritize work items.
We also have the concept of “Enablers” for Epics, Features and Stories. Enablers are way of putting architectural, infrastructure, code refactoring, research, prototyping, compliance, and other work that users may regard as “invisible” but are important in delivering HVHQ solutions. To read more about enablers with regards to SAFeR please see https://www.scaledagileframework.com/enablers/.
In this methodology we recommend treating bugs and improvements and children of a Stories. That is, every bug should be related to a Story as should an Improvement.
“Pruning” the product backlog and keeping it an object of beauty is the reasonability of the team. The backlog gives clarity to the business and the team of the work that needs to be undertaken. You should ensure the product backlog is nicely populated (in terms of metadata for the various items), well structured (not a mess of items at any 1 particular level) and no orphaned items (e.g. bugs and improvements that don’t belong to Stories).
Why do we need this epic? What value are we delivering to the business? These are the two important questions you need to ask about a project initiative before you start to figure out the “How”! The Epic is the overarching workload that we are taking to the customer (whether internal or external) for approval. Therefore, the budget holder needs it handed on a plate to them why they are approving this project! So, let’s make it easy for them. If possible, put in financial savings or other facts and figures that with help you win the pitch.
An Epic can be thought of as an initiative to be undertaken that will initially be a Minimum Viable Product (MVP). An Epic will be delivered through the use of lean start-up principles.
To see the Epic metadata please see Appendix.
To find out more about Epics, please see https://www.scaledagileframework.com/epic/.
A Feature is a unit of value to be delivered to the business. It is described as a feature and benefit. They are not written in a user story voice (“As a someone, I would like something”) as they typically support multiple roles. A Feature is undertaken by a single team and delivered during a release.
To see the Feature metadata please see Appendix.
To find out more about Features according to the SAFeR framework, please see https://www.scaledagileframework.com/features-and-capabilities/.
User Stories are requirements that are written in user readable language. Stories are the centrepiece of our requirements backlog. This is the requirement that the business are asking for, that the engineers build against and the test pilots validate against.
To see the User Story metadata please see Appendix.
Tasks are activities that can be assigned to team members that are deemed necessary to fulfil a user story. All tasks should be carried out to meet a requirement. So think carefully if you have orphaned tasks as to why you are doing them. If there a missing requirement that you are aware of but haven’t documented?
To see the Task metadata please see Appendix.
First thing, we need to set expectations with the client. Much as our lovely developers may try, there is no such thing as bug free software.
To see the Bug metadata please see Appendix.
Enhancements are like Bugs, except a Bug is a failure to meet a User Story, whereas an Enhancement is something that does not specifically cause a failure to a user story. We could argue to the cows come home if something is a bug or an enhancement, but if we’ve structured our project right (in terms of contractual obligations), it doesn’t really matter too much how we classify it! Why is it useful to keep bugs and enhancements separate? Because it is a way for the business (and team themselves) to request updates without affecting the quality report and to most likely help prioritize these kinds of changes below bugs.
To see the Enhancements metadata please see Appendix.
We track risks and issues that are covered in the weekly T-Minus-15 Checkpoint meeting.
A risk is something that has a potential to occur with a subsequent consequence. For a risk, we therefore need to consider the potential impact, likelihood of occurrence, and what steps we can take to mitigate or reduce the likelihood.
An issue is something that has already occurred. For an issue, we therefore need to consider what steps we shall take to resolve the issue.
These are groups of users that we identify. When we come to capture the user stories, these are the users that they relate to. Personas help us think about specific roles in the organisation, and how our solution is going to meet their needs. Therefore, user stories should have metadata that connects that requirements to one or more specific personas.
These personas should match the swim lanes in any Information Architecture workflows.
A Story Point is a method for the team to size the effort for the related Story. Typically, the smallest story is assigned a size of 1 and then other Stories are sized of this using a modified Fibonacci sequence (0.25, 0.5, 1, 2, 4, 8, 16). The Story point usually represents a single number that represents a combination of qualities of volume (how much effort), complexity (how hard), knowledge (what’s known) and uncertainty (what’s unknown).
However, contrary to traditional story points, the T-Minus-15 methodology draws a comparison of the story point and engineering effort, where 1 story point equates to 1 day of effort. This allows the team to more easily visualise and size story sizes. It also allows a way to provisionally pre-plan sprints in advance and calculate the capacity of the team for the number of story points that can be undertaken.
To calculate the total story points that can be carried out in a sprint, we have to appreciate that the engineering time is not the only time required to size a user story! Use the following formula to calculate the total effort for a Story:
From the above, in summary the total effort is approximately twice the engineering effort we put in the Story Points. This is useful for calculating the total effort for Features and capacity of Sprints. As a general rule for example, you can see that for every 4 Engineers, on average you’re going to need 1 Test Pilot.
We use Kanban boards to track progress of Stories, Features and Epics. A Kanban board is simply a visual representation to view the progress of items with columns relating to the progression of states and items related as cards on that board.
Stories in particular also have a “Definition of done” for each column that we will look at in section
Who and how do we update items in the backlog? Here are our work items rules:
-
Only the Epic Owner can update an Epic.
-
Only the Business Owner can update a Feature.
-
Only the Prepper and Scrum Master can update User Stories.
-
The scrum master is responsible for closing a Story.
-
There should be no time remaining for Closed items children.
-
All members of the A-Team should have specific tasks in the sprint.
-
Stories can only be set to “Done” when they meet their definition of done for the relevant statuses.
-
The Test Pilot and Tech Lead is responsible for confirming a release is ready for Production.
All items in the product backlog have various states associated with them to track progress on a Kanban board. In the T-Minus-15 methodology we use the following states:
As your team update the states of work items on a Kanban board, it’s important to agree with what we mean by “done”. By specifying what we call a “Definition of Done” for each column in the Kanban board it is clear for all the team as to what the acceptance of the item being complete is.
In the T-Minus-15 methodology we declare the Definition of Done for Stories as follows:
Work-In-Progress (WIP) limits are used to impose limitations on the number of items that can be moved into active progress. Therefore, items must be moved out of the active state before the next item can be brought in.
WIP limits are important to ensure that the team do not undertake hundreds of items to work on (because you have to say “yes” right?!), but actually complete only a rather small fraction of these. Much better to release a smaller amount of high-quality high-value (HQHV) than a large amount of low-quality. Working on too many items simultaneously will mean the team are not focused on ensuring each work item is high value.
In actual fact, having a WIP limit enables you to deliver things quicker. Think of juggling. I can juggle 1 to 3 balls pretty well and quickly. If I were to try to juggle 4 or 5 balls, I wouldn’t be very effective at juggling because I would actually spend most of my time picking the balls up rather than juggling!
So, next time as a team member you get asked “can you start this” you can say “I would love to, but I’ve hit my WIP limit, so will prioritize that next” (something along those lines!).
What WIP limits should be set? You should set WIP limits for your business for Epics, Features and Stories. Typically each will have a Kanban board associated this those items to show progress. In addition, because we have columns in our board that are primarily associated with different roles, we should probably adjust the WIP limit per column.
In most tools, such as Azure DevOps, it’s possible to set WIP limits on your Kanban boards (and per column).
As a rule of thumb, I would recommend a WIP limit for Stories of 2 per person, a Feature of 2 per Team. So, for example, if you have 2 Test Pilots in your team, you may decide to have a WIP limit of 4 for Stories that are ready for Testing.
So what happens if the engineers have already shipped 4 Stories to the Test Pilot? Then it’s time for some pair programming on other Stories with a colleague, peer reviews, or roll your sleeves up and help with getting those Stories through testing. With a T-shaped skills cross-section that we talk about later in section 8.4 Skills it means you have a degree of flexibility to help out with adjacent tasks accordingly.
So, here are some of the advantages of implementing WIP limits within the team specifically:
-
Reduce the amount of context switching for team members and ultimately mean we are delivering more of which is a higher quality.
-
Code reviews don’t build up and the original developers loose context of that code.
-
Merges to the master branch are more frequent ultimately resulting in small potential releases to production.
-
Encourage the team to work collaboratively to move work items along.
You can complete the WIP limits for your organisation in the Appendix.