Softwire Blog
Why enterprises can be disrupted
25 May 2017, by Andy Patterson
Software engineering is the foundation that allows new consumer services, efficient back-office systems, and market-disrupting products to be created. As with all foundations, it’s important to get the basic principles correct, and we believe that these principals are not only technical but creative. There are so many possible implementations of software applications that an environment which stimulates creative and innovative solutions will deliver the best results.
If we start with the humble software engineer, we can look at what motivates them to do their job effectively and create software to make a difference. A large body of research exists (summarised neatly in Drive[2]) to demonstrate that extrinsic motivators (reward and punishment) work well for simple rule-based tasks, but work very poorly for creative or complex tasks. For these types of task, we can measure the presence (or lack of) three types of intrinsic motivation:
- Autonomy: the urge and capability to direct our own work and make our own decisions
- Mastery: the desire to master our tools and improve our skills
- Purpose: the need to do something with meaning, and with clear direction
These motivators provide us with a language to discuss how to create an environment that is productive and efficient, producing brilliant software; conversely, it provides terms to explain why companies can have environments that are stagnant and inefficient, leading to software that no one wants to use.
Autonomy
We start with the hardest motivator: autonomy. It might seem confusing that autonomy is important – isn’t the job of a software engineer to implement something that the business has decided is necessary? Allowing engineers to dictate their own workloads is only a single example of autonomy (and, incidentally, one that Google and others have used to great effect[3]). Indeed, most engineers are happy to admit that UX or business analysis are not their areas of expertise. However, autonomy is much deeper than this – it’s about giving engineer the freedom to decide how to implement what’s required (what processes or frameworks to use), and the responsibility for delivering working software to end users. Teams with high autonomy are more efficient, as they’re empowered to remove processes that slow development effort down, and free to adopt better tools or frameworks as they become available.
One of the reasons that agile practices (and by extension, practices like Continuous Delivery and DevOps) have been so successful is that they push autonomy down to the team level, focus on cross-functional teams, and encourage continuous change. This isn’t to say that agile methods such as scrum or kanban are the only methods for achieving autonomy: they’re just a particularly well-known and effective one.
Mastery
For software engineers, mastery of a toolset, skill, or framework is immensely satisfying. However, Mastery is more than perfecting one skill: it’s heavily reliant on continuous improvement. It should be obvious that technology is a rapidly-changing industry, and that adopting new tools and skills as they become available is the key to success.
Mastery of technical improvements
By now, the analogy of technical debt[4] is well known and addresses one aspect of mastery (the desire to improve the code being worked on). Long-running codebases with low technical debt are strongly correlated with products that are reliable, high-quality, and respond predictably to change requests – Google is an excellent example of a company that invests heavily in reducing debt[5]. However, technical debt is an imperfect analogy as it is unquantifiable, and must be measured by its side effects: slow, unreliable delivery, and disenfranchised, unmotivated developers. Business that focus solely on putting features in front of customers win in the short term, but over the long term incur sufficient debt that delivering even simple features takes a huge effort.
Mastery of personal development
While developers will generally self-improve over time, this can be greatly accelerated with the right support and processes, including formal and informal training. These two approaches differ in what they aim to achieve, and personal development is normally most effective when they’re combined. Training courses and conferences aim to teach a specific skill, or give a breadth of knowledge about the industry. They also provide opportunities to network and engage with companies working on similar problems. By contrast, coaching and mentoring are much more individual, helping developers improve their strengths and mitigate weaknesses. These approaches can deliver huge benefits to teams and organisations:
- Ensuring that knowledge is spread across the team, removing single points of failure and improving consistency
- Improving skills and abilities
- Allowing individuals to take on more responsibility, resulting in more capable teams
The rate at which software development changes is phenomenal; the tools and technologies for most new projects have changed in the last 5 years, and are radically different to those 10 years ago. The NodeJs ecosystem is the canonical example; just over 5 years old, NodeJs and npm now have over 4 million active developers, and are used by thousands of companies.
Mastery deals less with each individual change, and is more concerned with the process of adapting to changes over time. It’s important to determine where best to invest in changing technology, but as we saw from autonomy, the correct place to make this decision is within an engineering team.
Purpose
Given that most software engineering is product or service-focussed, providing a clear backlog and direction of work gives a purpose to both individual and team efforts. In addition, a frequent and well-defined release schedule provides a visible goal to focus on. A purpose can be highly effective when combined with the goals of Mastery; to not only deliver software, but to get better at the process of delivering software. Teams with a strong sense of purpose are more likely to succeed, and less likely to get distracted and solve unrelated problems.
Scaling and disruption
Autonomy, mastery, and purpose are useful tools as they allow us to examine teams (or individuals) separately, and suggest targeted improvements. Some teams may have a strong sense of purpose, but are held back by poor tooling, whereas other teams may be able to make wide-reaching decisions but lack a clear goal. The ability to address the needs of each team in isolation allows this approach to scale. There are obvious technical challenges with scaling development work across multiple teams (particularly communication overhead made famous by the Mythical Man Month), but teams that are motivated to succeed will overcome these problems for themselves.
Looking at the world through the lens of intrinsic motivation it’s possible to see why a startup can deliver a disruptive product, but an industry behemoth can struggle to update its flagship products and services. The startup’s developers have autonomy (freedom from constraining structure and processes), mastery (time and money to improve skills and tooling), and purpose (the vision of the startup). Large organisations can be disruptive – Apple, Amazon, and Google regularly deliver ground-breaking products – but to do so requires continual effort and focus.
[1] The perils of ignoring software development: http://www.mckinsey.com/industries/high-tech/our-insights/the-perils-of-ignoring-software-development
[2] Drive: the surprising truth about what motivates us: http://www.worldcat.org/title/drive-the-surprising-truth-about-what-motivates-us/oclc/311778265
[3] Google’s “20 percent time” in action: https://googleblog.blogspot.co.uk/2006/05/googles-20-percent-time-in-action.html
[4] TechnicalDebt: http://martinfowler.com/bliki/TechnicalDebt.html
[5] Searching for Build Debt http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/37755.pdf
Top CIO Priorities 2017 – Infographic
1 March 2017, by Karl Graham
Tips for managing technical people – Blog Post wrap up
14 October 2016, by Zoe Cunningham
Over the past few months, we’ve been posting excerpts from my new book, “Galvanizing the Geeks – Tips for Managing Technical People”. You can buy the full book on my website, here.
This post serves as a reference to all the snippets that are freely available here.
Tips for managing technical people – Train technical leadership
21 September 2016, by Zoe Cunningham
The following is an excerpt from my new book, “Galvanizing the Geeks – Tips for Managing Technical People”. You can buy the full book on my website, here.
You need to ensure that there’s a technical career path as well as a management one open to your managees. Many companies nowadays ensure that there’s a path by which their technical people can reach the very top of the company, whether this is via a Technical Advisory Board (favoured by companies such as Thoughtworks and UBS) or by other means.
Creating the technical path and putting people on it isn’t enough. You also need to train people to fill the relevant roles. A common misunderstanding of the technical lead role is to see it as a sort of glorified developer, who gets to tell the other developers what to do. While a progression to project manager is seen as a move to a different role, the role of the technical lead can be seen as ‘more of the same’.
Visualising your project’s progress with Burnup Charts
17 August 2016, by Helen Hosein
One of the most important aspects of Project Management is being able to answer the question, “How is the project going?” Is it on schedule? Is it on budget? A very popular tool for keeping track is a Burnup Chart.
Here’s a simple example:
Every burnup chart has the same basic anatomy:
- The x-axis denotes time. Usually it’s measured in sprints, but it can be weeks or even days depending on how much granularity you need for reporting (and how frequently you can measure your progress).
- The y-axis is a metric of progress. In this example, the metric is story points completed which is a common choice but can be somewhat subjective. Other possibilities include number of stories completed, or amount of budget used. You can choose any metric you like as long as it’s a good representation of progress.
- The scope line (shown in black, and labelled “Scope (points)” in this example). This represents everything that needs to get done. In this case, it’s the total story point estimate for the whole project. If you’re measuring story count, it’s the total number of stories. If you’re measuring budget, it’s the maximum budget for the project. In this example the scope line is nice and flat, but in some cases, scope can change over the course of a project, so the scope line will change accordingly. This is one reason why burnup charts might be preferable to burndown charts – it’s easier to represent changes in scope.
- The burn line (shown in blue, and labelled “Story points” in this example). This is a cumulative plot of work done, according to our chosen metric. In a project that’s running to schedule, we expect the burn line to meet the scope line around (or ideally just before) the end of the project. The gradient of the burn line indicates velocity, or how fast we’re burning through work. Velocity can help project a rough end date for your project and is another useful indicator of how things are going. If your velocity is much lower than expected, there’s a good chance your project will be late. Also, if your velocity changes unexpectedly, it may indicate that something has happened that is worth investigating. Your burn line should be relatively smooth if things are going well.
Here are two projects with similar scopes and schedules but different velocities. Adding a trendline to the burn line makes it clear which is likely to be delivered early, and which is in danger of being late.
Another way to visualise this is with a target line. Here’s a project that ran pretty much exactly on schedule.
It’s normal for the burn line to wobble above and below target (staying perfectly on target would be a bit spooky – and may be a sign that your team is gaming the system), so a small wobble usually isn’t cause for concern.
One way to be sure is to add multiple target lines – one for the worst case (shown in dashed red, labelled “Pessimistic”), one for the best case (shown in dashed green, labelled “Optimistic”) and one for the average case (shown in dashed grey, just labelled “Target”).
Now we can see that once the project gained momentum, it stayed within our expectations, so we know that the wobbles in progress were definitely ok.
This other project, however, has been below the red line for all 8 sprints so far. Work is being completed, but the burnup shows that it’s not happening at the rate we would expect in order for the project to be delivered on time.
It looks like things might be improving: the trendline shows that at our current velocity, we should just about finish before the worst case end date, but it’s a pretty narrow margin and it assumes nothing else will go wrong.
Because we can visualise the project’s progress so clearly, we have the opportunity to take action before it’s too late. This project’s PM was able to intervene and turn things around. In the end, it finished ahead of schedule!
It’s easy to see the point at which effective changes were made to the project because the gradient of the burn line (the velocity) changed sharply. Burnup charts are also a great way to check whether the changes you make as a PM are having the effect you want.
Another handy use for burnup charts is keeping track of Work in Progress (WiP). If you’re familiar with Lean or Kanban, you’ll know that too much Work in Progress slows your project down, so it’s useful to be able to visualise how much WiP you have. The way to do this on a burnup chart is to plot a second burn line for work that’s in development. Just as the usual burn line shows cumulative work completed (using your metric of choice), the “In dev” line shows cumulative work in development or further, using the same metric. The distance between the two lines is your Work in Progress at that moment in time.
In this example, we can see that we had a lot of WiP in Sprint 6 (20 points!) which might have been slowing us down a bit. In Sprint 7, we picked up fewer stories and pushed more of the in-progress stories through the pipeline, which helped get velocity back on track. Here’s that project’s burnup again with target lines added.
One final tale of caution – Burnup charts are an excellent tool for visualising progress, but they’re only as good as the metrics you choose and your interpretation of them. Here are two burnup charts of the same project, one plotting story count and one plotting story points.
What’s going on here? Is the project running early or running late? Should you be worried?
What these charts are telling us is that, in Sprints 6 and 7 in particular, we got through a lot of stories but not a lot of story points. A look at the backlog shows that we completed a lot of smaller stories then, which gave us some quick wins. Does that mean we’re now ahead of schedule? Or are the remaining, larger stories going to bite us in the end, meaning we’re actually behind schedule?
Looking at the work remaining after Sprint 8, there are 3 small stories, 7 medium stories and 9 large ones. Large stories carry a lot of risk, so the project is starting to look precarious. At this point, it’s critical to remove any blockers and progress those large stories. In this case, the story points chart is probably a truer picture of what’s going on because it captures the risk associated with the size of the stories, and that’s where the trouble with this particular project lies. If you report the story count chart to your stakeholders, they may get their hopes up that the project will be early, and will actually be disappointed if it’s late. They also may not understand the urgency of unblocking the large stories if it looks like everything’s fine.
How did it end? The large stories were unblocked after Sprint 8, and things got better. Here are what those charts looked like at the end of the project.
The story points chart shows an increase in velocity when the large stories were unblocked. Things stayed pretty much on target after that.
The story count chart shows how the team got as much done as they could while they were blocked, but then ate through that buffer as they finished off the large stories. It shows that the rate of story completion was not sustainable, because of all the large stories still to be done.
Make sure that you choose the right metrics for your project so that the shape of your burnup chart is a true representation of its status. Then, use your burnup chart as a tool to help you identify when investigation and action are needed. Finally, use your burnup chart to verify the results.
Tips for managing technical people – Talk to the Duck
10 August 2016, by Zoe Cunningham
The following is an excerpt from my new book, “Galvanizing the Geeks – Tips for Managing Technical People”. You can buy the full book on my website, here.
‘Rubber duck’ technique is one of the innovations that I have seen introduced by a project manager who was willing to try something new. He didn’t invent the practice, but no-one works in a vacuum; the best project managers build up their skillsets through others’ experiences as well as their own.