What “engineering maintenance day” is, and how it keeps us sane

Author: “You should keep your dependencies up to date”
Reader: “No shit, tell me something I don’t know :)“

However, most teams I’ve worked in fall behind on maintenance over time.

Why?

Because keeping dependencies up-to-date takes time and effort. In an environment when shipping is the primary objective, maintenance takes a back seat. Sure, when the effects of outdated dependencies are apparent (something is breaking, or development is slowed down significantly), it’s easy for anyone in the organization to say “we should stay up-to-date with these things!”.

A couple of examples when:

  • The new major version of Ruby on Rails has been released and six months later, your other dependencies stop releasing updates for your old major version.
  • The new shiny version of Webpack 2 (or 3) hit the streets, and all your loaders all of a sudden stop supporting Webpack 1.

You don’t necessarily need to upgrade, but the day you do (security issues perhaps), it’s going to take you way longer to do the migrations. Compared to if you had done it regularly. The challenge is to be proactive enough that the team isn’t put in those situations.

With that in mind, what are the actions we as developers can take that ensures that the outcome never becomes a production halt? For us, the main action is Engineering Maintenance Day (EMD).

EMD occurs one day a month, per project, where we take the time to update dependencies, do risk assessment and look at our current technical debt.

We do it because we don’t want to fall behind on our dependencies and software as the pain of getting back on track has a tendency to accumulate quite heavily.

Software development these days is moving at a rapid speed and the more code you touch, the more likely you are to introduce bugs, even if your intent is to improve it. That means increased development time, testing, and therefore cost ($$$). And if it slips through all the way to a release with a medium to high-defect, everyone need to foot a huge bill.

On top of that, one of the larger macro-trends over the past decade, is that the use of third-party dependencies has exploded. For example, the open-source-JavaScript community right now is just the largest, most active open source community that has ever existed.

Check out the stats that GitHub announced last year: https://octoverse.github.com/ 

According to the statistics published by GitHub, open source JavaScript activity as measured by pull requests has doubled in the past two years. It’s more than the next two languages (Java and Python) combined.

“Open source JavaScript activity as measured by pull requests has doubled in the past two years”

Given we are heavily dependent on JavaScript at Sparta — we decided to implement something called Engineering Maintenance Day in order to ensure we minimise defects that are shipped to production.

We have three different products within our team, that each require a variety of tasks during an EMD.

API (Ruby, Ruby on Rails)

  • Heroku (Ruby version) — Are there any new major or minor releases that we could benefit from?
  • Database (Postgres) — Same as above (we recently updated to 9.5+ because we needed JSONB as a column type)
  • Gems — Go through the changelog’s on a regular basis. Use a lockfile. One of the tools you can take a look at is Gemnasium which helps automate this

Web (Node/Express)

  • Heroku (Node) — Are there any new major or minor releases that we could benefit from?
  • NPM packages — Go through the changelog’s of the updated dependencies. Lock your versions with a yarn lockfile or npm-shrinkwrap. Take a look at Greenkeeper.io which helps you automate this.

Mobile (Swift)

  • Cocoapods — Are there any new major or minor releases that we could benefit from?

Verification during an EMD

This is one of the times where proper test suites really shine, since we can update dependencies and quickly find any issues.

  • Unit tests
  • End to end tests

Forks

There will always be reasons for a fork to exist. In a larger organisation it will be hard to keep track of the different reasons. As always, it’s good to maintain a documented list of these in case the maintainer at your company decides to move on.

Example:

(Dependency name). Reason for fork: Forked because we decided to expose a private attribute. Pull request upstream has been submitted at 12/2 2017 by <Name>

We make sure to always put the forks in our own organisation’s namespace on GitHub. That way we as an organisation maintain control of them.

Locked versions

There might be reasons that some dependencies have been locked to a specific version. Perhaps it involves a larger refactoring because it exposes a new API. Either way, these needs to be documented somewhere.

If your dependency file (Gemfile, package.json, Podfile) supports comments, that would be a good place. If not, you should put it in a shared wiki of some sort.

Risk assessment

Even if we do EMD on a regular basis, we can not avoid that some work are just too big to fit into one or two days. This is why we keep a separate document where we list our risks and/or technical debt that we need to deal with at some point.

Its organised in a way that lists “Risk for <X>, because <Y>, which means that <Z>. Therefore we need to <A>”

Example:

“Risk for timeouts to occur, because the data layer (SQL queries and relations) is not optimized for the size of customers we are taking on, which means that scaling to bigger accounts will be harder. Therefore we need to spend some time on optimizing these parts”

These items can then be reviewed perhaps on a bimonthly schedule where you go through them and allocate proper time and resources to the bigger maintenance projects.


This blog post was written together with my colleague Jonas Martinsson at Sparta. Shoutout to Erik Hedberg for technical proofreading.

My experience at Uppsala Hackathon, and how to build a mobile app in a day

This post originally appeared at Swedish Startup Space.

Swedish Startup Space wrote about the Uppsala Hackathon back in June and I decided that I wanted to participate in this “gathering of programmers to collaboratively code in an extreme manner over a short period of time”.

The Uppsala Hackathon was attended by about 60-70 participants. Some people had teams and ideas before they arrived, but I had not found a team, so I joined one that consisted of the Valtech ConsultantsSara IngmarDaniel LervikPetter Norman and Sandra Nyström.

Design and Motive

In the beginning of the hackathon we knew that we wanted to build an app that would help solve difficulties when using public transportation. To solve the problem, we needed to use a few different API’s. Instead of collecting information from different API’s and combing results on the mobile application itself – we decided to build our own API.

In doing so, we keep the amount of data down to a minimum which is good because traveller’s/commuters doesn’t always have high bandwidth networks available. We can also be confident that we are sending the correct information to the device and avoid wasting precious data. This also gives us the opportunity to extend our service with more public transport agencies while maintaining control of performance and bandwidth of the app. Another benefit of using your own API, is that it makes it extremely easy to build another app for another platform since we have full control of the API.

Backend solution

We chose to build the API on .NET because the team had most experience on the platform. Daniel had previously participated in a project at Valtech where ServiceStack was used to build the API. ServiceStack is an open source REST web services framework which had lots of good stuff we could use. To further speed up the development process we used AppHarbor to host the API.

Mobile app

The mobile app was going to use the API generated by the ServiceStack. Both Daniel and myself have previous experience of developing mobile applications, so we focused on this part. The API was consumed using RestKit, an amazing framework for dealing with RESTful APIs.

The mobile app was developed using CocoaPods, because there is no sane person that enjoys dealing with dependencies and build flags in Xcode. Enter SLussen, see the demo of the app in the video below.

Disclaimer: the lag you see in the film is because of the delay that appeared when doing screen recordings.

We won! The jury wrote the motivation:

The SLussen project helps commuting travelers to calculate total ticket prices when traveling through different zones in Stockholm county. This project has was picked for its potential value in Uppsala county.

Read more

Thanks Petter for a great summary of the backend!

Video

Remove Previous, Next and Done buttons from keyboard in Phonegap in iOS

I’ve been working on my first Phonegap project together with a friend the last week or so. He’s been developing most of the app for Android and Eclipse, and my job is to make sure it runs smoothly in iOS. It contains in-apps and some other goodies that I will post about later.

Hide buttons

Since everything in Phonegap is a UIWebView, I needed a way to get rid of the silly previous, next and done buttons when focusing my keyboard. This uses private API:s so it will make your app rejected if you submit with this code.
6X1vv

The solution is to modify your AppDelegate.m to something like this:

Remove and add focus on keyboard

When bringing a keyboard in and out, use the following to remove focus from it

$('#searchform-1').blur();

And use this to bring focus to an input field:

$('#searchform-1').focus();

Integrate Testflight, Flurry and Corona SDK

Testflight

I use Testflight to coordinate pre-releases of software I develop for the iOS platform. Testflight is an amazing tool and I’m surprised Apple hasn’t bought the company yet. You simply create a group of people, send out invites and have them register their devices with Testflight. Unfortunately, the provisioning profile in the iOS ecosystem is a huge clusterf*ck, so you have to manually and not to mention painfully add these identifiers to your provisioning profile in the Apple Developer Portal.

When the profile is updated with the correct devices, you upload the .IPA to Testflight and add some release notes and hit send. The people participating in your beta release will then get an email that they can download a new version.

Testflight and Corona SDK

At work, we use the Corona SDK to develop multi platform games. Corona Labs describes their product as follows:

Corona SDK is the leading mobile development framework. Build high-quality apps and games for iOS, Android, Kindle Fire and Nook in record time.

It works remarkably well, not too much problems and it’s easy to deploy. Anyway, to integrate Corona SDK and Testflight you need to do a couple of things:

  • Build your software and generate the application and it’s zip file
  • Create a new folder called ‘Payload’ and move the application file into it
  • Compress that file and change its extension to .ipa instead of .zip
  • Upload the IPA to Testflight

Flurry

Screen Shot 2013-02-10 at 13.48.39
Flurry’s error log

Flurry is a great tool for gathering statistics, I’ve used it in all applications I’ve created for iOS and made sure its been used on Android as well. One of the best features for me as a developer is the fact that I get access to the crash reports (see the picture below).

To use Flurry with Corona, you use their analytics library.

-- Import library and initialize with personal application key 
require "analytics"
analytics.init( "YOUR_APP_KEY" )
-- Log event ID
analytics.logEvent( "Event ID" )

One thing I sometimes wish I would’ve implemented earlier in the project, is a log inside the app. The log should save the current state and the roadmap with the relevant server/client calls that took them there. It should also include the option to send this log to the developer.

2013-02-10 13.45.02
Evernote’s log

Evernote has implemented this very neat and it could be of tremendous help when debugging the communication between the server and the clients.

Använd Mogenerator med Core Data och Xcode

Sedan i början av sommaren har jag jobbat med att utveckla iOS-applikationer och har spenderat en hel del tid de senaste månaderna med Objetive-C, Xcode och Core Data. Core Data är utvecklat av Apple och har funnits med sedan ett par releaser av iOS tillbaka. Core Data är enkelt uttryckt Apples svar på modell-delen i MVC-paradigmen (Model View Controller). Det hanterar migreringar, serialisering av data och manipulation samt läsning av data i databasen.

Det funkar riktigt bra för det mesta, däremot har jag dragit mig för att göra vissa ändringar i modellen för att det blir så rörigt med mina “custom entity files”. Då kan vi passa på att säga välkommen till Mogenerator!

mogenerator generates Objective-C code for your Core Data custom classes. Unlike Xcode, mogenerator manages two classes per entity: one for machines, one for humans. The machine class can always be overwritten to match the data model, with humans’ work effortlessly preserved

Xmo’d seamlessly integrates mogenerator into Xcode

Mogenerator hjälper oss med att generera nya entity filer utifrån vår modell, nu slipper jag alltså göra det manuellt (och ofta göra fel på vägen). Dessa filer håller sig synkroniserade och uppdaterade mot min modell.

Börja med att installera Mogenerator från webbplatsen.

Installera i Xcode

  • Öppna projektinställningarna, skapa ett nytt target (“Add Target”)
  • Skapa ett “Aggregate target”, finns under Other.
  • Välj ett namn och tryck “Done”.
  • Välj det target som du nyss skapade och tryck “Add Build Phase” och sedan “Add Run Script”.
  • Fyll i följande information i scriptet, justera såklart med vad som gäller för din modell:
cd dinapp/coredata/
mogenerator -m app.xcdatamodeld/app.xcdatamodel/

Nu kan du sedan välja detta target uppe i vänstra hörnet och trycka Command+B (Build) och luta dig tillbaka medans Mogenerator skapar dina filer.

Glöm inte bort att välja samma klassnamn i fältet för “Class” när du modifierar din entity i modellen, annars genererar inte Mogenerator några filer. Första gången du skapar dina modeller kommer du behöva lägga till filerna i Xcode manuellt, sedan sköts detta genom att filerna bara skrivs över.

För mer information om fördelarna med att använda Mogenerator så rekommenderar jag följande inlägg: Getting Started with Mogenerator.