Softwire Blog


Using sqlite for dev nodeJS environments


29 November 2016, by

This post will explain how to get one click DBs working in nodeJS.

Problems with your zero to hero

I was recently working on a NodeJS project, and it was set up to use a mySQL database. This was fine for production, but meant that people joining the project had several manual steps to get the project working.

  • Install SQL server
  • Set up a SQL database
  • Update the NodeJS config to point to the database.

This isn’t too difficult to do. But I wanted to reduce the ramp up to getting the project up and running on a dev machine.

This meant moving to a database that could configure itself from whatever was checked into the codebase.

One option would have been to translate the steps above into code so that running a command would create a SQL database that was configured to be used by the code.

This would have been overkill, as there are much better options.

I chose to instead configure dev and test environments to use a local sqlite database.

Local sqlite database

A sqlite database can be added as an npm dependency, so that it is installed as part of dependency management when a new developer checks out the code.

"dependencies": {
  "sqlite3": "3.1.4"
}

Now that sqlite has been added as a dependency, it can be called from the ORM that we’re using. In this example, I’ve used Sequelize as the ORM, as once set up, it allows easy mapping of Javascript objects to database objects, as well as having built in support for database migrations.

The next thing to do is to have a configuration file which details how to connect to the database in different environments:

module.exports = {
  development: {
    dialect: 'sqlite',
    storage: 'data/dev-db.sqlite3'
  },
  test: {
    dialect: 'sqlite',
    storage: 'data/test-db.sqlite3'
  },
  production: {
    username: process.env.RDS_USERNAME,
    password: process.env.RDS_PASSWORD,
    database: 'ebdb',
    port: process.env.RDS_PORT,
    host: process.env.RDS_HOSTNAME,
    dialect: 'mysql'
  }
};

This config.js file shows that in dev and test mode, we’ll connect to the sqlite database which is stored in the file “data/<env>-db.sqlite3”.

In production, it instead connects to an RDS instance (the production machine in this project was running on AWS), with the connection details stored in the environment on the cloud machine (rather than being checked into the code).

Now we need to setup sequelize to use the correct database when it is initialised. This is done in the index.js file under the models folder:

var path = require('path');
var Sequelize = require('sequelize');
const basename = path.basename(module.filename);
const env = process.env.NODE_ENV || 'development';
const config = require(path.join(__dirname, '..', 'config', 'config.js'))[env];
const db = {};
var sequelize = new Sequelize(config.database, config.username, config.password, config);

This code fetches the relevant section of the config.js file, and uses it to initialise the ORM. What’s not shown above is the code that loads all the models into Sequelize. We’ll leave that as an excercise for the reader.

As you can see. This file is using an environment variable to determine what mode we are running the code in. All that’s required to run in production is set NODE_ENV in the environment variables to be equal to “production”.

The app can now be started using “npm start”, which will start, using the dev database.

Migrations

Just a quick note here on using Sequelize for database migrations. You can hook your migrations into the “npm start” command above by adding the following to your package.json file:

"scripts": {
    "prestart": "sequelize db:migrate -c config/config.js"
}

Summary

So what have we learnt?

  • You should try and ensure setting up a project is as easy as possible for new developers
  • You can use different environments variables to separate out dev/test/production environments (and default to test).
  • This method can work, even if you are deploying to the cloud.
  • You can easily add database migrations into the start command of your application.

This post originally appeared on Chris Arnott’s personal blog.

What I did on my second volunteering day – Zoe


25 November 2016, by

Inspired by my first day volunteering with Barnardo’s in Marylebone, I decided to use my second volunteering day to pursue something else that I have been keen to do for a while – spending more time with older people. Now that I’m in my thirties I’m starting to realise what a different perspective can be gained with age and so I’m very keen to hear from people who have maybe four times as much experience as I do!

Once again I used the fabulous employee volunteering platform Benefacto. It was easy to find something that matched my desired area, and one click and I was booked to help run The Lunch Club with Finsbury & Clerkenwell Volunteers. FCV have been organising volunteers to help others in need since 1971 and one of their current responsibilities is to run a lunch club for local elderly residents every Monday and Thursday.

Like at Barnardo’s, a lot of the jobs that need doing are quite simple and just need manpower. So my first job was to lay the tables ready for people to arrive.

FCV-table

Second, myself and a fellow volunteer from Accenture were given the task of peeling a mountain of potatoes for the Shepherd’s Pie, followed by a large basket of cooking apples brought in by a volunteer. Luckily two regular volunteers were able to chef these into the main course and an apple strudel for dessert.

 

FCV-potatoes

At about 12pm guests started to arrive for lunch. Some are able to make their own way in, but most are brought in a minibus driven by another volunteer. We helped them in and to their seats, gave them a cup of tea or coffee and then served lunch.

 

FCV-guests1

FCV-guests2

The number of volunteers helping out through FCV is absolutely incredible. There is Andy, who seems to be the main person behind the day to day running of the lunch club. There are two chefs (there used to be three) and a driver. There is Jan, who helps with everything and Irenie, who runs the raffle. Then there is the board. This includes Felicity who started lunch club tens of years ago, Andrew the chair and Leslie the treasurer; all three dropped in to see how lunch was going on the day I was there. Judy, an ex-employee of the charity, dropped in (she brought the apples we peeled). While we were peeling, she explained how well the charity worked, not just for the clients, but also for the volunteers.

 

After lunch, Andy ran several games of bingo and I learnt some new bingo calls. Then the guests were driven home and we cleared up and washed up. There was a lot of washing up! Overall it was a great day and a fantastic experience to be part of this community of people doing good, even just for a day.

Softwire uprising!


15 November 2016, by

First day at a new job is not all that hard to imagine. Most of it is what you would expect: a bit of admin work, introduction to the new team and the wider company, setting up your workstation, kicking off with a few training exercises. But on my first day, between emails with links to handy materials and those telling me about welcome drinks, I found something I would have never guessed would be there: an invitation to a sword-fighting event. …what? Feeling as if I was back at the University Freshers’ Fair again, I obviously signed up straight away.

And I was right to! Who wouldn’t want to look like this for a few hours?!

Swordfighting 1

But I’m getting ahead of myself – can you tell I’m excited? Back to the beginning though. What is actually going on?

It’s really quite simple: Softwire has a morale scheme designed to bring ideas like this to life. Anyone can come up with one, get it organised and Softwire will cover or subsidize it. Moreover, each employee gets a yearly allowance of holidays solely to be spent on such activities and outings. In the past those have included theatre plays, climbing and laser tag, etc.

Now, sword-fighting. …what? Yeah, “…what?” pretty much describes all of my initial attempts at figuring out what this will be. It hasn’t really cleared up all that much until the day of the event (one of the them, as the interest was so high that this happened twice!), when a bunch of us got up from our desks half-way through the day, grabbed weapons provided by our about-to-be-teacher and set off to parade down through the streets to the nearby Hampstead Heath. There, we were given a quick introduction and soon enough were whacking each other with the newly obtained items of friendly destruction.

The weapons themselves were a very surprising combination of softness, making them completely safe and sturdiness, allowing proper fencing work. Plus, they were realistically looking to boot! We had swords, daggers, shields of different sizes, batons, maces, axes, spears and, best of all, complete freedom to mix-and-match and try out whatever we liked. This meant we learnt the basics of actual fencing (much as would be done with real weapons); we could go all out without worrying we’ll hurt each other; and we looked AWESOME doing it! But don’t take my word for it:

Swordfighting 2

Swordfighting 3

The event itself consisted of a series of different games, some promoting teamwork and cooperation, others more focused on the individual skills. Amongst them we’ve played weapon-equipped variations of British Bulldog and tag, as well as a re-enactment of the Battle of Thermopylae (with rather devastating results for “Team Greeks”).

Swordfighting 4

One of the best things about this event was that it was joined by people from across company; “newbies” like myself, directors, seasoned employees and managers all alike, all equally tired and happy at the end of the games, yearning for more! It was a great chance to get to know everyone better and well, from a perspective one doesn’t get to see in the office. Surely, some people were hooked on more than others; some have done it before, others, myself included, will try to do it again, but at the same time I’m super excited about the next thing that Softwire will surprise me with!

How to become a supplier for the public sector


9 November 2016, by

Have you ever wanted to work for the public sector, but didn’t think that as an SME you would have the same opportunities as a large supplier?  Softwire’s experience as a software supplier to the public sector has shown us that this is not the case. In this blog post we explore why bespoke software developers, design agencies, cloud product vendors or freelancers should consider working for the public sector.

  1. Firstly, the Government has a genuine commitment to increase public sector spend with SME’s. The target for 2020 is 33% * this follows an upward trend of 25% in 2015.
  1. Even if you have no experience in the public sector this should not detract you from applying. Softwire had little experience in the public sector but we have recently been able to win multiple varied and exciting projects. Experience in the private sector is extremely valid along as long as you can demonstrate your expertise through case studies and testimonials.
  1. There are procurement portals which list all the current opportunities available. Some good starting points are:

Explore all the frameworks on offer  http://ccs-agreements.cabinetoffice.gov.uk/

Freelancers, bespoke providers, cloud product providers and cloud consultants should head to the digital marketplace https://www.digitalmarketplace.service.gov.uk/ and to search for larger digital projects from across the EU: http://ted.europa.eu/TED/misc/chooseLanguage.do=

  1. In most cases the projects have been scoped and funded in advance of selecting a supplier, mitigating your risk. Requirements and budgets are usually stated upfront allowing you to assess which projects would be a good fit for your organisation and there’s a clear and open process for asking questions.
  1. The scope and range of public sector work is very diverse. It’s not just local authorities and large public bodies. Public sector includes research institutions, infrastructure providers, regulators and many more.
  1. There is a strong government directive for digital transformation and you could be working on projects using leading-edge technologies, methodologies and design techniques. The Government Digital Service manual explains and encourages best practice to ensure project success. https://www.gov.uk/service-manual
  1. Public sector projects have a strong social impact. Through digital transformation the Government wants to push the boundaries of technology to improve and make public services more effective and efficient.
  1. The government mystery shopper portal ensures that the procurement process is fair to SME’s. https://www.gov.uk/government/publications/mystery-shopper-results-2016
  1. techUK works with the Government on behalf of the industry and is an advocate for the needs of small and medium sized tech companies.
  1. With the Government digital transformation budget expected to be around £1.8bn next year – what are you waiting for?

*https://www.nao.org.uk/blog/wp-content/uploads/2016/03/Governments-spending-with-small-and-medium-sizes-enterprises.pdf

Launching the Minimum Viable Diversity Pledge


3 November 2016, by

Diversity in tech is a major problem, and tech isn’t alone in this. One of the places where this is most visible is in speaker lineups and panels at events. Across industry after industry, speakers at events are extremely rarely from marginalized groups, including women, people of colour, people with disabilities and members of the LGBT community. Not only are they underrepresented, in many cases events don’t feature any diversity at all.

This is bad. Encouraging diversity not only makes the world a happier, fairer place, it also has concrete benefits for productivity and problem solving. When you never see a lineup of role models who are anything like you, you don’t feel included, it’s hard to be inspired, and you end up being slowly pushed out of the industry.

Diversity matters at Softwire, and we want to do everything we can to improve this. Today with the help of the Women in Engineering Society and the 30% Club we’re launching a new initiative to take a concrete step forwards on event diversity: the Minimum Viable Diversity Pledge.

Minimum Viable Diversity Pledge logo

The goal of the Minimum Viable Diversity Pledge is to totally stop the worst offenders for speaker diversity. By pledging, you’re committing to never actively supporting a paid event or panel that includes zero diversity whatsoever. This is a minimum bar, and we’d encourage people to go further, but the low bar is key.

We can all agree that such events have clear problems, and that finding at least one underrepresented speaker for any given topic is achievable. A low bar focused on this makes it easy for as many speakers, attendees, events and companies to sign up. Once that reaches a critical mass, running a paid event and totally ignoring diversity becomes impossible.

The world we’re aiming for here is one where every event organiser gets at least 2 or 3 of their speakers accept their invite on the condition that there’s at least some diversity in their lineup, along with attendees checking there’ll be at least some diversity included before they buy tickets. Once that happens, you can’t run an event without thinking about diversity, and you can’t host a lineup filled with a range of identical voices without a few of them publically dropping out. This won’t solve diversity overnight, but does make life far more difficult for those who totally ignore it, and provides steady pressure on every event to actively put in at least a little effort towards this issue.

There are four pledges, for speakers, attendees, events themselves, and companies, so everybody can get involved:

  • Speaker: I will never speak at any paid conferences or panels as part of a homogeneous group of speakers.
  • Attendee: I will never attend any paid conferences or panels with a homogeneous group of speakers.
  • Event: We will never organise an event lineup or panel with a homogeneous group of speakers.
  • Company: We will never sponsor or organise paid conferences or panels with a homogeneous group of speakers, we will strongly encourage our employees not to attend or speak at such events, and we’ll support them in raising diversity concerns with events directly.

This is where we need your help. This only works if enough people sign up and get involved. Take a concrete step on diversity, help shut down the worst offenders, and make it impossible to run a paid event that ignores its responsibilities. Sign the pledge.

Better CSS-only tabs, with :target


2 November 2016, by

CSS-only tabs are a fun topic, and :target is a delightfully elegant declarative approach, except for the bit where it doesn’t work. The #hash links involved jump you around the page by default, and disabling that jumping breaks :target in your CSS, due to poorly defined behaviour in the spec, and corresponding bugs in every browser.

Or so we thought. Turns out there is a way to make this work almost perfectly, despite these bugs, and get perfect CSS-only accessible linkable history-tracking tabs for free.

What? Let’s back up.

What’s :target?

:target is a CSS psuedo-class that matches an element if its id is the hash part of the URL. If you’re on http://example.com#my-element for example,div:target would match <div id="my-element"> and not <div id="abc">.

This ties perfectly into <a href="#my-element">. All of a sudden you can add links that apply a style to one part of the page on demand, with just simple CSS.

There’s lots of fun uses for this, but an obvious one is tabs. Click a link, the page updates, and that part of the page is shown. Click another link, that part of the page disappears, and a different part is shown. All with working history and linkability, because the URL is just updating like normal.

Implementation is super easy, and works in 98% of browsers, right back to IE9. It looks like this:

<!DOCTYPE html>
<html>
<head>
  <style>
    div:not(:target) {
      display: none;
    }
    
    div:target {
      display: block;
    }
  </style>
</head>
<body>  
  <a href="#tab-1">Tab One</a>
  <a href="#tab-2">Tab Two</a>

  <div id="tab-1">
    Tab one contents
  </div>
  
  <div id="tab-2">
    Tab two contents
  </div>  


</body>
</html>

Hop, Skip, and Jump

If you try to do this in any page longer than the height of your browser though, you’ll quickly discover that this can jump around, which can be annoying. Try http://output.jsbin.com/melama — you’ll have to scroll back up after hitting each tab. It’s functionally fine, and in some cases this behaviour might be ok too, but for many it isn’t exactly what you want.

This is because links to #hash links not only update the URL, but also scroll down to the element with the corresponding id on the page. We can fight that behaviour though, with a little extra JavaScript.

JavaScript, in our beautiful CSS-only UI? Sadly yes, but it’s important to understand why this is still valuable. With a tiny bit of JS on top we still retain the benefits that a CSS-based approach gives us (page appearance independent of any logic, defined declaratively, completely accessible, and with minimal page weight) and then progressively enhance it to improve the UI polish only when we can.

In environments where JavaScript isn’t available the tabs still work fine; they swap, and the browser jumps down to the tab you’ve opened. When you do have JavaScript, we can enhance them further, to tune the exact behaviour and polish that as we like.

Our JavaScript enhancement looks something like this:


var hashLinks = document.querySelectorAll("a[href^='#']");
[].forEach.call(hashLinks, function (link) {
  link.addEventListener("click", function (event) {
    // Disable jumping around and URL updates
    event.preventDefault();
    // Update the URL only ourselves
    history.pushState({}, "", link.href);
  });
});

This disables the normal behaviour on every #hash link, and instead manually updates the hash and the browser history. Unfortunately though it doesn’t work. The URL updates, but the :target selector matching doesn’t!

You end up with #tab-1 in the URL, but <div id="tab-1"> doesn’t match :target. The spec doesn’t actually cover this case and every singlebrowser currently has this crazy behaviour (although both the spec authors and browser vendors are looking at fixing this). Game over.

Two steps forward, one step back

We can beat this. Right now we can disable jumping around, but it breaks :target. We need a way to disable jumping around, but still update the hash in a way that :target will listen to.

There’s a few ways we might be able to work around this. Ian Hansson has a clever trick where you position: fixed and hide the targeted element, to control the scroll, but depend on its :target status with sibling selectors. Meanwhile Chris Coyier has suggested capturing the scroll position and resetting it, and I think there might be a route through if you change the element’s id to something else and back at just the right time too. These are all very hacky though; it’d be nice to come up with a way of just fixing the JS we want to use above, so it actually works properly.

Fortunately a helpful comment from Han mentions some interesting current browser behaviour that might help us out, while examining compatibility issues with fixing this for real:

I’ve found a good reason to believe that virtually no webpages would be broken by the “breaking” change of updating :target selectors upon pushState: though they are currently not updated, if you however hit Back and then Fwd again, then :target rules do get applied

Moving in the browser history give us the :target behaviour we (and all sane people) are expecting. If we can find a way to transparently go back and forward without breaking the user’s experience, we can fix :target.

From there’s it’s easy. We can build a simple workaround in JavaScript to do exactly this, and save the day:

var hashLinks = document.querySelectorAll("a[href^='#']");
[].forEach.call(hashLinks, function (link) {
  link.addEventListener("click", function (event) {
    event.preventDefault();
    history.pushState({}, "", link.href);
    
    // Update the URL again with the same hash, then go back
    history.pushState({}, "", link.href);
    history.back();
  });
});

Here we add the hash to the history twice, and immediately remove it once.

This isn’t perfect, and it would be nice if :target worked properly without these workarounds. As-is though, this gives perfect behaviour, with the only downside being that the Forward button in the user’s browser isn’t displayed as disabled, as they might expect. Actually going forward won’t do anything though (it’s the same URL they’re already on), and your users are not typically going to notice this.

This will keep working even if/when :target is fixed, and you’ll then be able to remove the extra workaround here, to lose that slightly messy extra behaviour. If this does break, or any users don’t have JavaScript, they’ll get working tabs with jumping-to-tab behaviour. Slightly annoying, but perfectly usable.

So what?

This lets you build amazingly simple & effective CSS-only tabs.

Minimal clean markup & CSS, perfect accessibility, working in IE9, with shareable tab URLs and a fully working history for free. Enjoy!

Full code:

<!DOCTYPE html>
<html>
<head>
  <style>
    div:not(:target) {
      display: none;
    }
    
    div:target {
      display: block;
    }
    
    /* Make the div big, so we would jump, if the JS was still broken */
    div {
      height: 100vh;
    }
  </style>
</head>
<body>  
  <a href="#tab-1">Tab One</a>
  <a href="#tab-2">Tab Two</a>

  <div id="tab-1">
    Tab one contents
  </div>
  
  <div id="tab-2">
    Tab two contents
  </div>
  
  <script>
  // Stop href="#hashtarget" links jumping around the page
  var hashLinks = document.querySelectorAll("a[href^='#']");
  [].forEach.call(hashLinks, function (link) {
    link.addEventListener("click", function (event) {
      event.preventDefault();
      history.pushState({}, "", link.href);
      history.pushState({}, "", link.href);
      history.back();
    });
  });
  </script>
  </body>
</html>