Softwire Blog


Softwire’s trip to Ghana with Ashanti


27 January 2016, by

Bonkron2I have just returned from a week in Ghana along with Zoe, helping out a charity called Ashanti Development that provides development support to villages in the Ashanti region. Through the charity, Softwire sponsor a village called Bonkron and amongst other things we have provided latrines, hygiene training, a kindergarten, micro credit, bee keeping training and a hardship fund.

Our journey started with a flight from London to Accra and a quick overnight stay before boarding a flight the following morning to Kumasi. From there it was just over an hour’s drive to the town of Mampong, a quick stop for some lunch and then a 10 minute bumpy ride down a dirt track to Gyetiase. We were staying in a clinic that was built by the charity in 2007 to provide accommodation to visiting volunteers and basic medical services to the local villages and which now specialises in ophthalmology.

Although fairly basic by western standards, the accommodation at the clinic was clean and comfortable and staffed by a team of wonderful locals for whom nothing was too much trouble. After meeting the other volunteers who were staying with us at the clinic (Penny, David, Bill and Marie) it was off to meet with the village chief and the elders in order to be granted permission to stay in the village. Tradition dictated that we had to answer the question of “What is your mission” (even though they knew the answer) and shake hands with them all many times both when arriving and leaving!

A wild fire the night before had knocked out a couple of electricity poles so without power as darkness fell, we dined by torch light and planned a trip to the local electricity company the next morning in order to expedite getting the power back on for the surrounding villages. A few of us headed out at 8am and after some pleading, the electricity company agreed that if the local community could provide the labour to dig the holes needed for the replacement poles then they would get to work that afternoon to restore the power. A community team was quickly gathered and by evening we had power!

During the rest of the week we travelled to many villages to help out and to see the work of the charity.

20160112-00007(1)We made many trips to Bonkron and on Wednesday they held a huge Durbar (Party / Reception) in our honour during which we were shown (and joined in with!) many of the village customs, including games, dances etc. As the event drew to an end I was surprised (and honoured) to find out they were planning to make me a chief of the village. They presented me with a number of gifts and after a brief ceremony and lots of photos I was made “Nana Kwadjo Phil”

The highlight of the trip, however, was on Friday when we had arranged to present to the school a laptop, projector and internet access and to use it to show a short cartoon to the local children and have a Skype call back to the Softwire office in London. The event was planned for 5pm and we decided to get to the village at 4pm in order to set everything up as we weren’t sure what to expect. When we arrived, we were pleasantly surprised to see many of the children were already there sitting on loads of desks that had been taken from the classroom and set up as cinema style seating. A generator was whirring away around the corner (the power was out again!) and a sound system was available for us to connect to.

Bonkron3We showed a short episode of “Shaun the sheep” and then connected over Skype to London. The internet was a little poor, probably due to the lack of power in the area, and did drop out a few times but that did not dampen the spirits of the children who enjoyed chanting “Soft” “Wire”, singing songs, seeing everyone in the office and chatting to Dan. We showed a couple more episodes of Shaun and then brought up Google Earth to show the village and then zoomed out to show Ghana, Africa and then world then back in to show the UK, London and our offices. Overall it was a great success and we look forward to future Skype calls with Bonkron.

On Saturday, we visited the chief and elders of the village to ask their permission to leave and then said an emotional goodbye to the other volunteers before leaving on our journey back to the UK.

Overall it was an amazing experience and one I hope others at Softwire will have the chance to share on future visits.

Software Development. Where do you even start?


21 January 2016, by

Kicking off a new project can be challenging. There are lots of options to consider and often many obstacles to overcome. With years of experience delivering successful projects under our belts, we asked our developers what steps they believe people should be taking before they start a project. They had all sorts of ideas to share but the three main points which came up were; think hard about what you want, plan ahead and work with your developers. So what did they mean?

Think hard about what you want!
Getting your requirements together is a hugely important step towards starting a successful project, so it’s important to get this early step right;

  • Try and define requirements primarily in terms of business needs.
    o  The fine details of implementation are best fleshed out later in the process, in collaboration with your designers and developers.
  • Try not to make your requirements list too generic, unless it’s strictly necessary. Often very simple sounding requirements like ‘users can define their own reports’ can be very complex to implement and may not be needed.
  • Don’t be constrained by any existing solution! Now is the time to think differently and perhaps the best solution involves a whole new approach. By focussing primarily on your business goals you give the developers room to translate them into the concrete implementation ideas which will suit you best.
  • Prioritise ruthlessly – for each feature ask “if the whole system was ready to launch tomorrow but this wasn’t there, would I delay launching to add this in?”
  • Make sure you understand what your users really want – particularly if your users are customers outside your organisation.
    o Note that what they actually want might not be the same as what they say they want – if possible get some hard data from tracking usage or watching how they work.
    o And try to avoid “design by committee” – understanding a variety of viewpoints is valuable, but ultimately just one or two people should be responsible for pulling things together in a coherent way.

Plan ahead!
It would be nice to live in a world where all projects could be approached footloose and fancy free, but sometimes it pays to be cautious to give your project the best chance of being a success;

  • Allow some slack in your timelines and budgets. Things often change over the course of the project and giving yourself some extra room for these changes gives your project a greater chance of success.
  • Try and favour incremental approaches wherever you can. Releasing partial solutions early so that users have a chance to test them and give feedback, gives you a chance to creat the best solution possible.
  • If you’re replacing a system it can also be useful to have a period during which the old and the new system are running at the same time. This can help flag up any tiny details you may have missed and it also give people a chance to transition.
  • If you’re doing a big marketing launch, plan to run the system for a short while before making any announcements so that it’s been tested by real users.

Work with your developers.
When it comes down to it, it’s your developers who are going to be responsible for the delivery of a successful project. Involving your developers from the early stages is important;

  • Whether it’s in-house or third party you should consider getting some expert help define your requirements. Your developers will be able to help you identify the things which really matter and eliminate risks early on in the process.
  • Get some help with design and UX and don’t leave it until last. The quality of the visual design and ease of use can have a massive impact on user engagement when you launch. You might want to consider commissioning user workshops with a UX professional to map out the most engaging journey.
  • If you’re inviting proposals you’ll get much more satisfying results if you help the developers to do a good job. Give them as much relevant information to work with as possible and tell then what matters to you most, so that they can focus on it when responding.
  • Stay engaged once your project starts. The best results come from projects where there’s a contant flow of ideas and discussion over the course of the project between developers and customers/product owners.

24 Pull Requests 2015


20 January 2016, by

At Softwire we make an effort to be involved in the open-source community that powers so much of what we do. For the last three years, we’ve spent December getting involved in 24 Pull Requests – an initiative encouraging developers to contribute pull requests to projects they use and love through the run up to Christmas – and this year is no exception!

Even more than last year though, this year has been an enormous success. We’ve even managed to bring ourselves up into an amazing 4th place position overall, out of the 9,934 organisations taking part. (This may have been strongly encouraged by the suggestion of a team croquembouche appearing if we reached the top 5. We at Softwire are certainly not immune to the charms of an exciting cake!)

Particular commendation should go to the top contributors, each of whom put in:

  1. Hereward Mills – 25 pull requests
  2. Tim Perry – 24 pull requests
  3. David Simons – 17 pull requests
  4. Rob Pethick – 14 pull requests
  5. David Broder-Rodgers – 9 pull requests

Pull Requests

A selection of particularly interesting pull requests within the 161 pull requests we contributed in December:

You can check out the full list of everything we got up to at 24pullrequests.com/organisations/Softwire

Bonus Stats

24 days of pull requests
161 PRs from the Softwire team (so 7 per day, on average)
36 contributors contributing at least one PR (so 4 each, on average)
78 projects contributed to (including 17 PRs to DefinitelyTyped, 14 to Workflowy2Web and 8 to AssertJ & 24PullRequests)

Fantastic work all round!

Smooth transitions in Cordova / Knockout


15 January 2016, by

Here at Softwire, we’ve been busy building a mobile app for a chain of health clubs, and this is the third post in a four-part series about our experience and what we’ve learned. Watch this space for the next instalment, or follow us on twitter for updates!

We’ve chosen to build the app using Apache Cordova, so our code can easily run on both iOS and Android platforms. Take a look at the first post to read about what Cordova is or the second to see some of the other practices we use to make Cordova feel as app-like as possible.

One of the important requirements was for scrolling and transitions to be smooth and to avoid the screen juddering. Lack of polish and smoothness is one of the most common objections raised against Cordova; it’s an easy trap to fall into, but simpler than you might expect to solve once you’re aware of it.

Work with the phone

Modern phones have powerful graphics processors, so we want to offload as much of rendering onto the GPU as we can, to keep the CPU free and ensure our rendering goes at top speed.

Let’s say we’re using CSS to move an element ‘my-page’ off the right of the screen by applying the ‘off-right’ class.

We could write:

#my-page {
    left: 0;
    transition: left 1s;
}
#my-page.off-right {
    left: 100%;
}

This will work, but it’s not typically run on the GPU by most browsers, which means performance may take a hit if you have many or large elements moving like this across the page.

Fortunately we can replace this equivalent to get GPU acceleration, with:

#my-page {
    transition: transform 1s;
}
#my-page.off-right {
    transform: translateX(100%);
}

The ‘transform’ property here is designed to be easy to handle on the graphics processor, and so runs much more smoothly.

Don’t stop me now

We’re building our app as a single-page website, within Cordova. As part of this, we’re using Knockout.js to do data bindings. Knockout is a simple way of keeping your javascript object models in sync with the HTML (or ‘binding’ them together, in Knockout lexicon).

It’s important to understand how this works to improve performance though, with Knockout or other similar frameworks. When you change the value of a Knockout-observed variable, Knockout will immediately make the corresponding change to the HTML, and each of these changes in the HTML will cause the browser to re-render the relevant part of the page. If a large part of the page is moving when this re-render is trigger then it will appear to stop, re-render and then continue moving, causing a noticeable judder.

This is particularly noticeable if you update properties across an array of objects. Knockout interprets each of these as a separate change and causes the page to re-render multiple times, resulting in a very juddery transition! Doing this is generally bad for rendering performance, but it’s a very easy pattern to fall into with many modern single-page app frameworks.

To solve this, in our app we’ve been careful to make sure we never change knockout-bound values whilst we’re transitioning. When we move between pages, we’ve adopted a workflow like this:

  • 0 ms
    Create new page, start loading data and binding each part to the page.
  • 100 ms
    Pause data bindings. Put any data change requests in a queue.
    Start transition (transition lasts 500ms).
  • 600 ms
    Resume data binding. Process any queued requests.

In our testing we’ve found that the first 100ms is typically long enough for data to be fetched from the cache and rendered, when it’s already cached, without being long enough for the user experience to suffer.

Nothing to see here

Generally, we’ve been very happy with the performance of Knockout, but it’s performance can degrade with increased load. Every data-bound value causes extra work for it, and slightly reduces performance. For best performance, we should minimise the amount of data that Knockout is binding.

For us, this means segmenting the app into different ‘journeys’ and ‘pages’ and ensuring Knockout is only data-binding the journey and page currently in use, and disposing of the data and DOM elements not relevant to the current page. Within some of our larger pages, we take this further, and change what data is bound as the user moves around the page. This requires some discipline for us to remember to destroy pages and journeys as soon as we’ve finished viewing them, but has been well worth it in performance benefits. If there’s “nothing to see here” then it’s best if Knockout doesn’t have to keep track of it.

All this has come together for us fantastically; despite being built with Cordova the app feels snappy and quick across a wide range of devices, and our customer is very happy with the result.

In the fourth and final post we’ll look at how we architected the app to ensure maintainability for the future. Keep an eye on our blog for the next edition, or follow us on twitter for updates.

Ice Skating at Somerset House


12 January 2016, by

As is becoming increasingly traditional, just before Christmas Softwire took a winter trip ice skating to Somerset House.

The turnout was record breaking – 33 of us (plus a smattering of guests) opted in, and all headed off one Monday afternoon to the rink. Not only that, but the group consisted of a great spread of people, from new starters, the members of the sales and admin team, developers – the MD even came along! Though most surprisingly, most of us managed to stay upright the whole time! (I may withhold my own records, there…)

While it might not have been a great winter all round (too warm and wet for my liking, though I may regret saying that in the weeks to come), we were blessed by perfect weather, and ended up skating around the rink as the sun set. Afterwards, most of us headed back out for hot drinks and a warm room, with a few finally peeling off to share a dinner of the most Christmassy food – burritos.

Happy new year from Softwire!

iceskating

Your Build Smells: Nervous Pushes


6 January 2016, by

This post is the second post in a series looking at Build Smells – signs of major issues in team build processes – and the things we can do to try and solve them. You might also be interested in the first post on painful merges, and do watch this space or follow us on twitter to see more of these as we publish them.

Nervous Pushes

Actually pushing code frequently is important for lots of reasons. As we covered in our previous post, sharing code earlier with your team can significantly reduce the time you spend fighting your version control systems, but even outside that, pushing code brings you closer to actually releasing useful code, and generally getting anything useful done.

Anything that stops you pushing code is something that’s slowing and hurting your team, but unfortunately it’s easy to get into a situation where your build does exactly that. If pushing code normally means something breaks then not only do you lose the time required to investigate and fix the issue (if there even is one), but you also lose the focus and concentration you had up until that point, as you have to context switch out of whatever new task you’d picked up in the meantime back to the fallout from your last one.

All this time and effort is something we humans instinctively avoid, and very quickly you’ll find if pushing code creates hassle, then people will push code less often. At this point, your build is making your team worse at development. This is a build smell.

Confidence

All of this fundamentally undermines your confidence in your build. If you have confidence in your build then you can write code, quickly find out whether it works (and quickly deal with it if it doesn’t), and iterate at high speed to code that actually solves your problem. Confidence lets you go fast, focusing on each task in hand sequentially and quickly moving toward success there, rather than having to keep one distracted eye nervously watching everything else to see what’ll break next.

This is one of the underlying components of flow, the much-discussed feeling of complete focus and being ‘in the zone’ that lets you zoom through development tasks with speed and control. This only works if you can be quickly confident in whether your code works as you write it; you need to be able to keep moving forwards without distractions and uncertainty, have your mind full with only what you’re building, and concentrate on your goals. Without confidence in your build, that’s not possible.

When confidence really falls apart you end up assuming the build is wrong more than it’s right and this gets much worse; you lose more than just the required time to fix issues, but you normalize build failures, start to ignore them, and lose the ability to deal effectively with actual failures when they come up. Once you start automatically just rerunning the build a couple of times when it fails then fixing the build starts to take even longer still, this problem gets even worse, and the whole situation quickly spirals.

If we can set up our builds to give us confidence instead, then we can build better software, faster (and enjoy ourselves more). If your build doesn’t give you this, then you have a problem to fix.

How can we fix this?

This all revolves around the core quality-checking steps of your build, which I mean to include anything that could really fail because of potentially incorrect code changes: automated tests, linting and style checks, and even compilation phases.

To get confidence your build, these steps need to give you accuracy, thoroughness, speed and simplicity.

Accuracy

If the code actually works, the build should pass.

The easiest way to totally undermine confidence is to have a build that fails half the time. As soon as this becomes normal, it becomes very difficult to quickly or easily get confidence in the code you write, and your development speed (and quality) drops significantly. There’s a few things that we’ve found particularly effective for helping here:

  • Reduce test dependencies – The most common cause of inaccuracy is intermittent issues with something outside your codebase, be that external systems, time and timezones, or the current operating system. Dependencies on any of these will bite you eventually, and working to remove or limit those really helps. Setup and teardown dependencies within tests, and take care about the assumptions you make about your environment.
  • Keep environments identical – Builds that fail only on the build server are hard to trust, frustrating, and difficult to fix. Try and keep environments consistent to avoid that, at the very least make it possible to create an environment locally that’s identical to CI, for debugging. There’s lots of great tools nowadays to support this; Docker and Vagrant are both good options, for example.
  • Have zero tolerance for inaccuracy – Lack of accuracy will snowball; once you’ve got one test that randomly fails more will join it without you noticing, and the frustrations of this make it difficult for people to invest in test quality generally, making it worse. You have to disproportionately fight to beat these issues when they happen, or they’ll get far worse.

Thoroughness

If the build passes, the code should actually work.

Passing tests need to be something that gives you good confidence in the quality of the codebase, so you’ve got passing tests you can move forwards and focus on the next problem, with nagging doubts that you’ve broken something and not noticed.

  • Monitor Coverage (a bit) – Code coverage is an oft-rejected metric, and strict adherence to arbitrary goals can be harmful, but overall coverage trends and patterns reliably contain useful data. It’s very useful to know that coverage is going down recently, or that one area is much better covered than another. Measure (and examine) these patterns to get the real value, and keep your team focused on thoroughness.
  • Ensure every potential risk is covered – The key things here is to not skip testing things because it’s hard. Sometimes you can’t usefully unit test something, but that doesn’t mean you should skip testing it. Add thorough integration tests instead, and ensure every potential risk gets caught somewhere.
  • Test what might fail – Overtesting can bite you too, increasing maintenance cost, and sapping morale and team investment in good testing. If there are no reasons you can think of why a test could fail, it’s not useful (even Kent Beck agrees), and you should potentially move the test up to a higher level. Instead of writing tests for every simple getter, test them as part of the code using them.

Speed

Builds should run fast enough to let you quickly know their result.

Speed powers confidence, allowing you to quickly check how you’re doing and iterate. Time spent waiting for builds is typically unproductive time, but also creates painful context switching: If you get pulled back to your previous change three hours later when the build fails then you’re no longer in the best place to fix it, and you’ve lost any focus on what you moved onto next too.

  • Quick feedback – Effective team flow and the ability to move quickly depends on a quick feedback cycle. Waiting 4 hours to know whether your change worked makes it very hard to quickly iterate on changes. Anything you can do to reduce the time to feedback makes this much easier, and helps you and your team develop faster and better. Tools like Wallaby (for JS) and NCrunch (for .Net) are fantastic for this.
  • Watchable unit tests – Unit tests that you are prepared to look at while they run (so 10 seconds max, unless they’re somehow fantastic to watch) are tests that you’ll be much happier to run all the time. Longer, and your mind will wander – taking your focus with it – and resulting in you running the tests far less. Fight to keep unit tests running fast.
  • Smoke test suites – You won’t manage to get integration or system test suites down to this time. Once you’ve got a 4 hour system test suite though, you can be sure that nobody on the team is running it before every commit. You can make that easier by picking a very small representative set of these to use as smoke tests, which are quick enough that people can run them as they work. You can also set these up as a early build step in CI, to ensure major problems are caught quickly in CI (and to avoid waiting 4 hours to discover that 100% of your tests failed).

Simplicity

Builds should be easy enough to check locally and fix that people actually do.

People are lazy, and that includes you, me, and everybody on your team. If running your build takes work, they’ll avoid it. Keeping the build simple and easy to use encourages people to actually go and use it, rather than avoiding it as an obstacle on their road to getting things done.

  • One click – If your build takes a series of actions, people will forget one, do them wrong, or skip to ‘only the important ones’. It’s typically easier than you’d expect to get it down to a single click (or simple command line step), and providing that kind of simple interfaces makes it drastically more likely than you’d expect that people will use it.
  • Zero setup – Manual setup wastes time, and you’ll do it wrong anyway. It also You’ll also have big problems when the setup changes, you have to migrate users from setup A to setup B, and you start finding tests that only pass in one setup. Again, this is often easier than you’d expect (especially with tools like Docker/Vagrant and Puppet/Ansible/Chef), and has big value.
  • Easy debugging – Eventually builds will fail, and making it easy to work out why will make people fix them more quickly and more effectively. Have Selenium take screenshots at the end of failing tests, make sure your build and CI setup provides detailed info on what exactly failed, have any logs from the system under test easily accessible, and ensure it’s easy to run individual tests standalone locally.

Put all this together, and you can put your project in a state where your tests are easy and quick to run, and reliably give you a good clear idea of whether your system works. All of this powers confidence, and makes it much easier to quickly and successfully build high-quality software every day.

Hopefully that’s an interesting insight into the kind of issues we see in projects, and some of our approaches to solving them!

We do encourage all our teams to find and play with their own solutions for problems they face, and we studiously avoid dogmatically sticking to any one approach, so I should note that these are just an example of a range of practices we’ve found effective in different cases. Don’t blindly follow advice from the internet just because it sounds convincing! Take a look at the concrete problems your team in facing, and consider whether some of the approaches and practices here might help you to effectively solve your real issues.

Let us know your thoughts all this below, or on twitter! In the next post, coming soon, we’ll take a look at the next smell in the series: Rare Releases.