---

Your Build Smells: Avoiding Painful Merges

---

23 December 2015, by Tim Perry

This post is the first post in a series looking at Build Smells – signs of major issues in team build processes – and the things we can do to try and solve them. Watch this space, or follow us on twitter to see more of these as we publish them.

Painful Merges

If you’re not careful, it’s easy to end up in a situation on a project where you end up consistently wasting time fighting merge conflicts, rather than getting useful things done. This isn’t unavoidable, and it’s a build smell: a sign that all is not quite right here.

We see cases where this is not just occasional, but actually becomes completely standard, and there’s simply an accepted expectation that teams working together will burn time every couple of weeks, as somebody spends a day merging everybody’s changes together (or a week a month, or worse). It’s not uncommon to see cases where this is even encouraged: where everybody in the team automatically starts a branch every sprint, and everybody merges them all together all at once at the end (nearly two weeks later). This is a recipe for pain.

This wastes time and saps morale. It’s even worse than that in reality though, because if you’re continually expecting to hit merge conflicts then you can’t really refactor; any wide-ranging changes to the codebase either require everybody to down tools, or to risk huge time-wasting conflicts. You end up in a situation where your build processes and tools are pressuring you not to refactor, where your own build is encouraging you to write worse code. Renaming a method becomes a painful and potentially costly process, so you don’t.

This is a build smell. Merges should be frequent, small, and typically easy. If you see people burning large amounts of time merging, actively avoiding merging their code, and racing each other to push changes to avoid having to deal with the repercussions, you’re seeing this in action.

How do these happen?

We can examine this more closely at this by looking at some of the most common forms of merge conflict:

• Textual conflicts

  Conflicts where two changes have been made that are totally independent, but which don’t have a clear resolution when merging textually.

  For example: two people adding two new fields to a class in Java, on the same line. The fact they chose the same line typically makes no difference, and the fields typically independent, but all major merge tools will ask you to resolve this manually, as they have no way to decide which line should appear above the other, or whether that can be done safely.

  Interestingly, this conflict transparently disappears if we make these changes in sequence. If one person adds a field, and pushes the code, and another pulls the code, and adds a field, then there are no merge conflicts, and the situation is invisibility resolved with zero extra effort from either party.

• Simple semantic conflicts

  Conflicts where two changes have some interaction that produces incorrect behaviour, but aren’t necessarily in direct opposition. Changing a getDistance() method to return a distance in meters instead of centimeters for example, while somebody else adds some code that calls that method.

  These changes may well not textually conflict, but do have a relationship: code that’s expecting centimeters will (probably) not act correctly when given a different unit.

  (As an aside: issues like this are a good reason to include units of measure in your types, to use things like F#’s built-in unit of measure support where possible in languages that support that, or to at least use clearer method names).

  Again though these conflicts are trivial to resolve if the changes are made in sequence. If you are already aware that the method returns meters, because that change has already been shared, then you’ll ‘resolve’ the conflict without even noticing, by writing your code with consideration of that built-in from the start.

• Fundamental semantic conflicts

  These are the conflicts where you and somebody else are trying to make changes that are fundamentally in opposition.

  Perhaps I need a button to be green, and you need it to be red, or I need to make a major refactoring of a class that makes some functionality impossible, and you’re writing some new code that depends fundamentally on that functionality existing.

  These are conflicts without easy resolutions. To solve this, we’re going to have to have a discussion, and come to some compromise in our goals and overall approach.

  These issues are also far rarer than people imagine though. Try to think of a conflict recently that would’ve been an issue if you’d made your changes entirely in sequence, rather than in parallel. It’s rare that two developers on the team are chasing mutually exclusive goals at the same time.

  Notably when this does happen it’s still much easier if we can resolve these immediately. If I find out that we can’t make buttons red when I make the first button red, I’ve got a problem to solve. If I find out that we can’t make it red after I’ve spent three days redesigning the whole application in shades of burgundy then I’ve wasted a huge amount of time, I have to undo all that, and I still have the same problem to solve too.

When we hit painful merges, it’s rarely because we’ve actually hit fundamental conflicts. Most of the time, we’ve just built up a large set of textual and relatively simple semantic conflicts all together, and they’ve snowballed into a huge conglomerated mess that we now have to unravel.

The underlying conflicts here come from isolation. Code that’s kept on a branch or kept locally unpushed is code that’s isolated from everybody else on the team. They’re secret changes you’ve made, that you’re not sharing with everybody else. Because you haven’t shared them, nobody can take them into account in their own changes, and conflicts quickly proliferate.

This gets dramatically worse as isolation increases. If you keep code on a branch for a day, you have one day of your changes to potentially combine with one day of everybody else’s on the team. If you all stay on your branches for a week, you’ve got a whole week’s worth of your own changes to combine, with every other person’s week-sized chunk of work too. This quickly gets out of hand.

What can we do about it?

The solution here comes neatly out of the above: large conflicts are typically a collection of many relatively simple conflicts, and each of those would almost disappear if the changes were made in sequence. Therefore, if we want to get rid of merge conflicts, we need to move as much possible away from isolating our codebases, and get as close to being a team of people making small changes in sequence as we can.

To fix this smell, share your code early. As early as possible; I’d suggest you integrate your code with the rest of your team every time you have code that doesn’t break anything. As soon as you have code that compiles, passes your tests, and functionally works (to a level where you’d be happy showing it to a customer): share it with your team. Sharing this won’t break anything (by definition), and completely eliminates any risk of conflicts within that code.

That’s all well and good, but much easier said than done. Let’s look at some examples:

• Sharing tests

  If you’ve written a couple of new passing tests in a clean codebase (for example, just before you’re about to refactor something), you can commit them and share them immediately. They’re passing tests, so you can be pretty confident you’re not going to break anything, and once pushed you remove any risk that they’ll conflict with others in the team.

  If you don’t do this for a while, you build up isolated changes. When somebody attempts to refactor test code in the meantime, touch any of the dependencies you’re using, or actually touch the code you’re testing, they have no way to easily take your changes into consideration.

  Meanwhile if you have pushed your tests then it’s easy for others to check that their changes don’t break it, or to apply their wider refactorings to your test too (often automatically, with automatically applied refactorings).

  Doing TDD, and you’ve written a failing test instead? There’s reasonably mileage to be gained in many cases in ignoring the test (e.g. with @ignore in JUnit), and sharing that (although this does require discipline to ensure you don’t leave tests ignored long term).

  You don’t gain any guarantees in this case that the behaviour under test won’t change, but you do ensure your code will continue to compile, and others making wider changes that affect your code will fix it in the process of those changes like any other code in the codebase. By sharing your code early here, you allow people making other changes to do so with respect to yours immediately, rather than leaving you to tidy up the mess later.

• Sharing model and component changes

  If you’ve just added a field, or a new method to the models or internal components of your system, you’re almost certainly very quickly back in a functionally working state where your changes could be shared. A remarkable amount of the code you add as you progress towards building new functionality doesn’t break your application for your users or other developers, and can be easily shared immediately.

  Added a new calculateCost() method to your Invoice class as the first step in the feature you’re building? Share that now, and the other developer who needs to refactor Invoice tomorrow for their changes won’t step on your toes, and you’ll both live happier lives.

  This is especially relevant in wider-ranging cross-cutting changes. If you’ve refactored big chunks of code, changing how logging works, or renamed some fields, it’s should be very unlikely you’ve made any user-facing changes to your application at all, and your tests should be catching any such issues if you have. In addition, the stakes are higher; refactorings particularly quickly collect conflicts, while the majority of them won’t have any fundamental conflicts at all (rename field/variable, extract method/class/interface, etc), and fundamental refactoring conflicts that do appear will explode far more painfully if left to ferment for a few weeks.

  Sharing changes like early drastically reduces the potential chance and damage of conflicts, and is well worth pushing for where possible.

• Sharing trickier changes

  These are two examples where it’s particularly easy, but they don’t cover every case. Often you want to make a change that is fairly large in itself, and individual changes within that will result in broken or incomplete functionality. Rewriting the interface for a page of your application is a good example, performing a huge refactoring or rewriting a major hefty algorithm within your application; in many cases this might be a large and challenging change, and nobody wants to share a broken or incomplete page or algorithm with the rest of their team (let alone their users).

  There are things we can do in this case however. Using feature flags/toggles and branching by abstraction are two particular approaches. Both revolve around sharing code changes immediately, but isolating the user-facing functionality within the codebase (rather than isolating the functionality and the code changes, using version control).

  Branch by abstraction works by introducing an abstraction layer above the code you’re going to change, which delegates to either the old or new implementations. Initially this calls the old implementation (changing nothing), but once standalone parts of the new implementation are completed and ready the abstraction can be changed to call through to new code instead, and eventually the old implementation (and potentially the abstraction) can be removed.

  Feature flags work somewhat similarly, but dynamically with configuration, either application-wide, or in per-user settings. A conditional statement is added at a relevant branching point for your change, switching between the two implementations based on the current configuration (return flags.newUserPageEnabled ? renderNewUserPage() : renderOldUserPage()).

  This has a few advantages, notably making local development of visible features like new pages easier (just enable the config property only on your machine), and allowing you to take this even further to become a feature in itself (as done by Github, Flickr and Travis CI) providing you fine-grained control over your feature release process, for progressive rollouts or A/B testing.

All of these techniques help you share code earlier, and sharing code early enough can almost totally remove conflicts from your team’s process, and make them easier to deal with when they do happen.

I should be clear that I’m not saying branching is always a bad idea. Branching has many sensible practical and effective uses, and it’s an incredibly powerful tool. Unfortunately many teams fall back to it as their goto tool for every change they make, and they start to build the way they work around actively keeping things on branches.

Branching comes with a cost, that rapidly increases over the lifetime of the branch. That’s fine if you’ve got a specific benefit you want to get from branches, but not a good idea if you don’t. Some good reasons to use branches for example: very experimental code that’s unlikely to make it into production, situations where you can’t allow everybody push access to master directly (as in open-source and some regulatory environments), and situations where your review processes don’t work effectively with it (on of the main reasons I see for not following this, as Crucible is one of few good tools that will easily let you review commits on master).

Nonetheless, in each of these cases it’s still worth aiming to merge more frequently. Encourage many small pull requests to add a feature, don’t allow experiments to go on too long with deciding whether they’re worth committing to, and push for more smaller reviews with minimal turnaround time before merging.

In general, smaller and more frequently shared changes disproportionately reduce the risk of each change, make refactoring vastly easier, and can often make conflicts disappear with no extra effort. A large proportion of merge conflicts simply are simply invisible if the changes are made in sequence instead, and sharing code earlier moves you closer and closer to this situation.

Finally, it’s notable that this way of working isn’t just a benefit for conflict resolution. It is a good idea all the time to aim to break your work into many small chunks, with many steps at which the code is fully working. Doing so helps avoid yak shaving, makes each of your changes easier to understand, and has lots of benefits for time management and estimation too. There’s a lot of upsides.

Caveats

Hopefully that’s an interesting insight into the kind of issue we see in projects, and some of our approaches to improving them!

We do encourage all our teams to find their own solutions independently for their problems however, and I’d be lying if I claimed this is our standard go-to approach in every case. It is an approach that many of us have found effective in places though, and it’s well worth considering as an option if you’re facing these same challenges in your team.

I should be clear also that I’m not suggesting that branches are never useful, that this is the One True Way of managing version control, or that you should dive into this 100% immediately. If your team is facing concrete issues with conflicts and merges, experiment with reducing isolation and sharing code earlier and more frequently, and measure and discuss the changes you see in practice. We’ve found benefits in many cases, but that is by no means definitive, and there will be cases where this is not the right choice.

If you’re interested in more thoughts on this topic, I do recommend the writings of Martin Fowler, particularly around feature branches, and Jez Humble and Dave Farley’s book ‘Continuous Delivery‘.

Let us know your thoughts all this below, or on twitter! In the next post, coming soon, we’ll take a look at the next smell in the series: Nervous Pushes.

Softwire’s Charity Quiz Night for Schistosomiasis

---

21 December 2015, by Laura Bethke

For the third time in its history, Softwire hosted a Charity Quiz in late November this year, with all proceeds donated to the Schistosomiasis Control Initiative.

As in previous years, the quiz was written and hosted by director Tom Steer, whose general knowledge is so great that he has probably been banned from all pubs in the area. He can now only use his quizpertise to selflessly write quizzes for other people, and he does that very well.

There were four ordinary rounds, one table round (where the teams had to identify films from cryptic descriptions, books from their first line and words from their chemical element names) and as is traditional for Softwire quizzes, a taste test. Tom lovingly prepared pick and mix bags full of jellybeans with 9 different flavours for each team, and this was probably the only time in history anyone ever ordered a 2kg bag of liquorice flavour – I can assure you they are disgusting.

In London, 13 teams competed for the crown, while our sister office in Bristol held a parallel event with 4 competing teams. It was a tight race with the three leading London teams, Flux Quizpacitor, The tall and the wise and Quiz Akabusi being within 5 points of each other after two rounds and the table round. However, an absolutely stellar performance with 25 out of 25 possible points in round 3 eventually secured Quiz Akabusi the victory. In Bristol, Quiztosomiosis Control InQuiztive was the winner.

The best team name of the night should definitely be awarded to “Hopefully not last”; even though they did not try to come up with a clever quiz pun, they did manage to hit their goal exactly and end up in 12^th, out of 13 possible places.

The quiz was also accompanied by a raffle, with half the money donated to SCI and the other half up for grabs by a lucky raffle winner, should they manage to answer a final question. Rich Bradley, part of the winning team Quiz Akabusi, had a very lucky day indeed with his ticket being the chosen one and he managed to answer correctly that the Dachshund was originally bred to hunt badgers (an easy question for any German speakers, as it literally translates to “badger dog”).

Overall it was a fantastic night in the office, with lovely food prepared by our fantastic kitchen staff, and with a really entertaining quiz that managed to attract enough interest to raise £1347.50 for SCI! Hopefully we can convince Tom to carry on writing quizzes so we can host similar events for many years to come.

What does it mean for an app to feel like an app?

---

21 December 2015, by James Griffiths

Here at Softwire, we’ve been busy building a mobile app for a chain of health clubs, and this is the second post in a four-part series about our experience and what we’ve learned. Watch this space for the next instalment, or follow us on twitter for updates!

We’ve chosen to build the app using Apache Cordova, so our code can easily run on both iOS and Android platforms.

As a quick recap: Cordova is cross-platform way of building mobile apps using web technologies. As a developer, you write a regular website using HTML, CSS and javascript, and Cordova takes this website and creates an app that is a wrapper around the website.

For the iOS app to be available on the App Store it needs to be approved by Apple. One of Apple’s requirements that is particularly relevant for Cordova apps is that the app “feels like an app, not like a website”.

What does this mean for us? There are a lot of components to this, but in this post I want to look at how it’s affected our approach to client-server interaction.

No 404s

We’re lucky to have some really nice designs for the app, and we wanted the entire UI to live up to these.

To keep this user experience really smooth one thing we really have to ensure is that we don’t ever show the user an ugly 404 page.

On a regular website, a user moves from one page to another. Each page is loaded individually from the server and contains the data they have asked for, as well as the code needed to take them to other pages.

This wasn’t an option for us. Our app users might be on the move, underground or have a slow data connection. If we built the application extremely minimally as a wrapper around a traditional server-hosted application then we’d be making these page requests in an environment where they might fail. We can’t risk showing them an half-loaded page or, even worse, a 404.

Load data, not code

To solve this, we’ve built the application as a single-page website. All the code is bundled into the initial download – that way, all our pages load instantly, without having to wait for any code to be downloaded. This approach is an important part of good Cordova development, and helps solve the error handling issue above and reduce latency for users to keep the app feeling snappy.

The only communication we have with our servers is to send and receive data (rather than code). We have a RESTful API that manages all our operations, like loading class timetables, making bookings and getting club opening hours.

If any of these API requests fail we can show the user a pretty error message, and allow them to retry, without the app looking broken.

Smooth transitions

The app looks beautiful so we want it all to perform well on the phone. That includes loading pages, but we also want scrolling, swiping and transitions to all be smooth, and to avoid the screen juddering.

In the next post we’ll look at how we achieved this. Keep an eye on our blog for the next edition, or follow us on twitter for updates.

24 Pull Requests: The Hackathon

---

18 December 2015, by Tim Perry

We at Softwire care about open-source, and for the last 3 years every December we’ve been very involved in 24 Pull Requests, an initiative to encourage people to give back to the open-source community in the 24 days of advent up to Christmas.

That’s still ongoing (although we’re doing great so far, and watch this space for details once we’re done), but this year we decided to kick it up a notch further, and run an internal full-day hackathon, which we recently ran, on the 7th. This came with free holiday for employees (with 1/2 a day of holiday from our morale budget and 1/2 a day from our holiday charity matching scheme), to make it as easy as possible for people to get involved, and prizes for the first 3 contributions and the best 3 contributions (judged at the end).

We kicked the day off with a cooked breakfast buffet at 9, following a brief bit of warm up and setup, and then a team stand-up with the 20 or so participants. Everybody quickly went round and outlined what they were looking at (or what they wanted, if they didn’t have an idea), so people doing similar things could find each other, and people with more enthusiasm than specific plans could find inspiration or other people to pair with and help out. And from there, we’re off!

We then managed to get the first prizes for the 3 pull requests all claimed by 11.30, with Rob Pethick adding in missing resources in Humanizer, Hereward Mills removing some unused code from Cachet, and Dan Corder moving AssertJ from JUnit parameterized tests to JUnitParams (quickly followed by a stream more of PRs coming in just slightly behind).

This pace then accelerated further through the afternoon, as people found projects and got more set up, resulting in a very impressive 46 pull requests total by the end of the day (taking us to a total of 89 for the month so far).

These covered all sorts of projects and languages, and included changes like:

• Adding auto-reversing migration tests to FluentMigrator
• Decoding and printing non-binary entity text in Akka
• Improving Scalacheck to handle very large double ranges
• Creating TypeScript type definitions for Chai-Things

Plus a whole range of other interesting and slightly larger changes, along with the many small typo fixes, documentation updates, setup instruction fixes and so on that are so important in really making open-source projects polished, reliable and usable.

Great work all around. We enjoyed an excellent day of open-sourcing, made a lot of progress, and we’re now in a great place to do more and more open-source over the coming weeks, and beyond once 24PRs is complete.

We’re now up to 96 pull requests in total, and counting, and we’re hoping to keep making progress up the 24PRs teams leaderboard as it continues!

Watch this space for more updates on the tools we’ve been working on and interesting changes we’ve been making, or follow us on twitter for extra details.

Write an app without writing an app

---

16 December 2015, by James Griffiths

Here at Softwire, we’ve been busy building a mobile app for a chain of health clubs, and this is the first post in a four-part series about our experience and what we’ve learned. Watch this space for the next instalment, or follow us on twitter for updates!

We were recently asked by a large chain of health clubs to build an app for both iOS and Android platforms. We chose to build the app using Apache Cordova, rather than writing separate native applications. Why did we choose Cordova, and how do you get started with it on a new project?

The problem with native apps

At Softwire, we’re used to building native mobile phone apps, but building native apps comes with a number of difficulties:

• Each platform requires you to learn and use a specific programming language, development environment and set of libraries.
• You have to maintain a separate codebase for each platform and keep the code in sync between platforms.
• Each platform has its own quirks and bugs will typically be different in each platform.

All of this makes native development potentially more expensive and risky, and we’ve been interested in investigating other approaches for a while.

What’s the alternative?

(more…)

Getting more Women into Technology

---

9 December 2015, by Zoe Cunningham

As a successful woman working in technology, I often get asked how to attract more women into tech roles. I get asked by diversity officers of large corporates like BT and I get asked to share my experiences as a women with young people through organisations like STEMNet and TeenTech. “Where are the women?” has become a topic across many areas of modern life, such as business, academia and politics, as well as technology. My favourite question on the topic was from a young founder of a tech company: “There are only four of us in the startup and we’re all white males – what do we do?”.

Step one: Don’t panic It’s not your job to fix the whole tech industry. There are fewer women than men in tech right now, and you’re not going to change that by redesigning your marketing materials or having more female-friendly office perks. If there are only four of you and you care enough to accost a speaker to ask what you can do differently, your company make up likely reflects the industry and you will be OK.

Step two: Realise that there is no magic “women thing” that you can do One mistake that crops up time and time again and will alienate rather than attract women is the tendency to think of them as a homogenous group. I’m a mid-thirties woman without children so painstakingly outlining your maternity policy will not make me feel more welcome, whereas for some women this will be a key factor in considering where to work. Similarly booking in “girly” events or perks without finding out what your female employees like can seriously backfire. As one of our developers said recently “women in tech don’t tend to want manicures”. (Of course I’m not saying that generalisation is true either – just remember that different women like different things.)

Step three: Find information specific to your business What do the women who work for you already think you do well, or not so well? Perhaps even more tellingly, why do women leave your business? Make sure you hold exit interviews and when you do try to ascertain whether you are finding out what people really think. Consider using a member of the HR team or someone who hasn’t been involved in the day to day work of the exiting team member, to make sure that they are not holding back from criticising the person who is holding the interview.

Step four: And if all that fails… If you are really have problems recruiting and retaining women, here are some suggestions.
• Train all of your staff in relationship building and personal interaction. Soft skills are classically considered female territory, but even more importantly these skills will help you to uncover what people really think and help them to feel welcomed.

• Value diversity. The best way to think of diversity is not whether an employee is a woman, or of a different race or sexual orientation, but whether they actually think differently from you. Hunt out different opinions. Think through the “obviously stupid” ideas for a bit longer – maybe you are missing something. Be suspicious when everyone you hire agrees with you.

• Finally, quotas for hiring are risky for all kinds of reasons, but quotas for shortlists are not. Force yourself to find diverse candidates, especially for senior roles in the company (note that you may find this very hard – don’t give up!) and you may well find yourself surprised to end up hiring a woman.

“Masters of Code” Hackathon

---

2 December 2015, by David Simons

A long time ago, I got a message from one of our clients asking if I wanted to go to the Mastercard “Masters of Code” hackathon: I definitely did, and it finally happened this month.

Hackathons are times where people from all across the development life cycle get together to make something solving a problem. The catch? It’s in 24 hours. I really enjoy coding and getting something in your hands at the end of it. Hackathons are also a great place to learn loads of different skills (coding, and otherwise) from the wide range of people that attend.

Despite grand plans from a lot of people, we ended up with a target crack team of 2 people. We were challenged to make the day-to-day lives of London Mastercard easier by integrating with a combination of the TfL and Mastercard developer APIs. These are sources of some really interesting data and abilities – letting you plan routes, see the city and make payments with vendors.

24 hours is not a lot of time, and so we set to work straight away on working out what to build. We settled fairly early on using Mastercard’s loyalty offers API, allowing us access to personalised offers of the user that will use the app. All too often, people get these offers but don’t take advantage of them. A higher take-up rate on these offers will benefit both Mastercard and the end-user. The user gets savings, and Mastercard get people using these offers and – in turn – spending more on their card. Win-win.

We used TfL’s journey planner app to help users plan their journey and get advice as to the best route from A to B in London. However, when users change lines or transport routes at various stations, we looked out for opportunities for users to stop off if they had things to do at these interchange points. We went further and advised people how much would be added to their journey if they went out of their way to tick items off their to-do list. If it’s only 10 minutes, then why not make the extra hop and avoid a 2-hour round trip at the weekend?

In terms of the technical nature, we decided to write the application in NodeJS, serving a web interface written using KnockoutJS and Bootstrap. It was the technical framework with which we had the most experience – and with only two of us we were working solidly throughout the 24 hours as it was! When we added persistence, I got the chance to play with a new tech I’d been meaning to checkout – Firebase, Google’s distributed Database as a Service platform. It’s lightweight but first to set up and was perfect for our needs!

We were happy with our end product – an MVP and even some of the nice-to-have features that were a pipe dream just 24 hours earlier!

Other applications including a gamified social-media charity app (“Give to charity or I’ll shame you on Facebook”), an app that encourages you to walk by secretly saving the money you would’ve spent on your train fare and eventual winners “All of Us,” – a Night Out as a Service application, that integrates events, journey and tabs across a range friends.

Our product actually came second in the end – which we were happy about (not least because of the $1,000 prize!), but the whole weekend was amazing to learn a lot about APIs and other developers’ mindsets. The clarity of the idea was praised. We’re proud of what we made. We’re going to give the app a bit of distance for a week or so, but we’re excited about the potential of it – we even used it to plan our journey from the venue back home!