Category Archives: Code

Build Status and Deploy Button with a Raspberry Pi, Octopus and CruiseControl.net

I made a build status and deploy button using a Raspberry Pi that connects to our Octopus deploy server and our CruiseControl.net build server.

Why? Well it all started a while ago I got a Raspberry Pi mini computer to muck around with at home. I was pretty blown away with it for a number of reasons

  1. Cheap – enough said.
  2. Power consumption – they don’t use much, you can even run one on a backup phone battery.
  3. You can swap SD cards to swap the OS – It’s like the olden days when your machine used to boot from floppy (but you only need one thank goodness).
  4. They are small – well smallish.
  5. They have GPIO pins (General Purpose Input Output pins).

They had me at cheap! It wasn’t until it arrived that I realised how cool being able to mess with programmable pins was. This started my journey, at first I was reading articles on the web and eventually I bought this book Teach Yourself Electricity and Electronics, 5th Edition to get a more comprehensive understanding about electronics.

A few months later and we are using Raspberry Pi’s at work for fun and profit. Having a device on my desk at work for testing gave me the idea for a project that was suitably simple. I wanted to make something to learn about electronics and practise some soldering. The idea was to make some status LEDs indicate the health of the build at work and have a button for deploying our software. I managed to get it working, this is what it looks like, some people have said that it bears some resemblance to a Nyan Cat.

Raspberry Pi Deploy Button

Raspberry Pi Deploy Button

A few people have been asking about how it went. I made a video so you can see for your self.

Software

I wrote the software that runs the button and the LEDs in Javascript using node.js. I used node because it has heaps of packages that communicate via http and with the GPIO pins. Getting the software up and running was rather quick.

I found it amusing to operate electronics using a high level language. To read and write from the GPIO pins I tried a number of packages and ended up using onoff  because it supports reading via an interrupt rather than polling, which thrashes the CPU.

We are using CruiseControl.net for our build server and Octopus Deploy for our deploy server. I used request.js for getting the build and deploy status via http.

To start a deploy, Octopus has a command line tool written in .net that lets you remote control the server. I thought this would be the most stable interface to code against, so I tweaked the code to get it to run in mono, but I discovered there is an issue in mono that I am still waiting on a fix for. It should be possible to use request.js for creating and deploying a release, but I can wait a little longer.

Hardware

I thought software was hard, then I started learning about electronics. It’s amazing computers work at all.

For the LEDs, I am using an Integrated Circuit with a Darlington Array of transistors on it, this allows a higher voltage to power the LEDs and protects the CPU which is connected to the pins. Each LED requires a different resistor according to its specifications. I found this handy site that talks about GPIO pin interfacing.

You would think the button would be pretty easy, it is either on or off right? it’s a bool! Well not quite, the button needs to be grounded so that it can be read correctly. Also you want to know what the value is when it changes without using your CPU to poll all the time. So you need an interrupt which signals your program when the pin changes state. To top it all off, when you go to read the value, it might not be correct as electricity bounces. To counter bounce you can write some code to trade off responsiveness with accuracy or use a low pass filter (a circuit).

Result

I really enjoyed making this, it started off as a simple idea to practice soldering, but turned into something that is rather useful. I think for our team, it sets the bar for how easy the deployment of our software should be. It is a physical symbol that the deploy is one step, and any features that get deployed, need to a part of that step. If anyone makes it more complicated than that, then they are going to have to get the soldering iron out.

Create a Chrome Extension in 5 Steps

If you follow along, by the end of this post you’ll have a working Chrome extension.

One of the great things about web programming is that the parts are in a standard structure. Because of this standard structure, it’s possible to modify other people’s programs. This client side modification is usually done with user scripts.

Why

There are various reasons why you might want to create an extension. For example, you could make an extension to:

  1. Improve your browsers general functionality to be more powerful and useful
  2. Make a specific site more useful
  3. Circumvent poor security
  4. Automate something

Other browsers

This post will show you how to get started with Chrome extensions, however you can do similar things in other browsers, for example in Firefox has grease monkey plugins.

Creating your first Chrome extension

Step 1
Create a directory. Everything your extension needs will go into this directory.

Step 2
Create a file in the directory called manifest.json with these contents:

{
 "name": "Hello World",
 "version": "1.0",
 "manifest_version": 2,
 "description": "Say hello.",
 "background": { "scripts": ["background.js"] },
 "browser_action": {
 "default_icon": "icon.png"
 },
 "permissions": ["tabs"]
}

Step 3
Create and copy in an icon named icon.png

Step 4
Create a file called background.js with these contents:

chrome.browserAction.onClicked.addListener(function(tab) {
 alert("hello world");
});

Step 5
Open Chrome, click the spanner, extensions, click developer mode.
Click ‘load unpacked extension’, find your folder and click ok.

That’s it, your extension is done. Your code will load when Chrome starts, a button will show up next to the navigation bar, and the onClicked event will fire when your button is clicked.

Three weeks of Python programming

I recently discovered the 50apps challenge, a year long challenge that publishes weekly programming exercises.  I hope to be able to participate in as many of the exercises as my schedule permits. The first three weeks of the challenge were focused on exploring the Python programming language. These are my notes.

Week 1

I wrote a web crawler in Python. It works by getting the contents of a web page, finding all the links and following them to a specified depth, while scanning for some search text.

I found Python pretty straightforward to work with despite not having used or even read it before. I was surprised that I managed to complete the exercise in two hours. The documentation was good. Finding the regex method to use took the most time, and having to explicitly cast was annoying.

Week 2

I created a Django website for the web crawler created the week before.

I used Django, which is a Python web framework. It was a larger framework than I was expecting, coming in over four megabytes, but it did seem to have a lot of features. It took a little longer than I wanted to get something going: I spent two hours getting a basic form and another two hours adding some more advanced features. I personally prefer Sinatra-style frameworks like Express as they seem to make more sense to me.

I made my application reuse the code I wrote the week before. I am wary that it could create a security hole, depending on how Django cleans form input data. The code to reuse the previous functionality was a little more complicated than it should have been because of the way I wrote the code from the week before, but I wanted to see if I could reuse it without changing it.

I spent over an hour trying to get Django working with Google app engine. I quit while I was ahead because the proposed solutions I found looked hacky or required another package to install. It seemed pretty painful compared to other hosting sites like no.de or heroku. To Google app engine’s credit, the logging was good.

Week 3

This week we explored Python’s functional programming side. I wrote a website that graphs word statistics for a given page. The logic for gathering the statistics was to be written without using looping constructs. I used a primitive form of TDD just by using the command line and the assert function in Python. Later I integrated the statistic logic into the web interface, this time I avoided trying to reuse previous code as it was rather different and I was running out of time.

It took about an hour to get a filtered list of words with a count and an hour to get rid of duplicates, limit the results to only ten words and find shortest and longest words. To hook it all up to the web interface was another hour. A lot of the time was spent re-reading the documentation and head scratching.

By the way I ran the tool over my blog and I use the word ‘I’ a lot. :)

Service Wait – servicewait.com

Over the break I made Service Wait , a simple smart phone timer application with a start and stop button that sets values you can tweak later to get the elapsed time. It’s useful to see how much time you have wasted waiting for something, or for working out times for time sheets. I made it for my mum for Christmas.

JavaScript Models

In other projects I have used knockout.js to create view models. This time I just worked with jQuery and jQuery mobile directly and for a simple problem I would recommend starting out by working this way. I prefer to add a framework or tool when it solves an issue.

jsHint

I used this project to experiment with jsHint, which does some static analysis of your JavaScript to help find problems. jsHint is a fork of jsLint that provides more flexibility around which rules should be enforced. I started off using it strictly but have since come up with my own set of options that I use on other projects to improve readability.

Hack

I built the site using jQuery mobile which has great support for most smart phones. However I was primarily concerned with optimising for use on iPhones/iOS. I found that on iOS you could show a time keyboard by specifying the type as a time like so.

<input type="time" name="timestart" id="starttime" title="" />

This will show the time input keyboard in iOS. However there seems to be an issue in iOS 5.1 where the value of a html input time field will not display when it is set via JavaScript. To get around this issue I wrote a hack where I float a span over the top of the field when I want to show a value and then hide it if the user enters something via the time keyboard.


TODO

One idea I had that may come in the future is a tweet button, which basically allows you to whinge on twitter about how long you had to wait with a great deal of precision.

Contributing to WordPress Support Tickets

Recently I was working on a quick one-off project. I found a WordPress plugin called Support Tickets that pretty much did everything that the project required. The last time I touched php was nearly a decade ago so it’s not my usual bag, but it works, so why bother being snobby about the technology used?

Late in the game a few issues cropped up, I managed to figure them out and fix them. Looking on the support forum a few people are using it and suggesting all sorts of fixes.

I though I would share the open source love and contribute my fixes. So I got the original author’s permission, and made my own fork on github.

I hope it helps.

Debug JavaScript in MVC

Most JavaScript libraries come with a minified version that is quick to load, and a debug version that lets you debug issues. In ASP.net MVC you really want to be using the debug JavaScript while developing, and the minified Javascript when your site goes into production. I thought up this handy tip that gives a good return, for little investment.

Step 1: Change your layout pages to reference your JavaScript files via a UrlHelperExtensions class.

Step 2: In the UrlHelperExtensions class write some code like this:

public static string Scripts(this UrlHelper helper, string fileName)
{
    return helper.Content("~/Content/Scripts/" + fileName);
}
public static string knockout(this UrlHelper helper)
{
    if (System.Diagnostics.Debugger.IsAttached)
        return Scripts(helper, "knockout-1.2.1.debug.js");
    return Scripts(helper, "knockout-1.2.1.js");
}
public static string CDNjquery(this UrlHelper helper)
{
    if (System.Diagnostics.Debugger.IsAttached)
       return "//ajax.googleapis.com/ajax/libs/jquery/1.6/jquery.js";
    else
        return "//ajax.googleapis.com/ajax/libs/jquery/1.6/jquery.min.js";
}

The jQuery example here is using a CDN in the hopes that the visitor already has it in their browser cache.

Now your JavaScript will load quickly and you can still debug during development.

Skipping MVC web.config files with msdeploy

This post describes how to deploy a ASP.net MVC web application with msdeploy, while skipping the overwriting of the root web.config file. Sounds easy, what could possiblie go wrong? Well, msdeploy can be a real pain to try and figure out, but at least it’s the sort of thing you write once and keep in the back pocket.

This example deploys from an existing ‘last build’ staging area to a ‘beta’ staging area but doesn’t copy across the configuration or error files.

"c:\Program Files\IIS\Microsoft Web Deploy V2\msdeploy.exe" -verb:sync -source:iisApp="LastBuild" -dest:iisApp="Beta" -skip:objectName=dirPath,absolutePath="elmah" -skip:objectName=filePath,absolutePath="^((?!Views).)*web\.config$" -skip:objectName=filePath,absolutePath="settings-override.xml"
pause

The tricky bit that this command is handling is that MVC doesn’t just have the one web.config file. Each view folder also contains a web.config file used by the view engine, so simply ignoring files name web.config will deploy a broken web application. The example above works because msdeploy allows regular expressions, this regular expression skips over the web.config in the root directory but still copies over the others that are in ‘Views’  folders.

You can see how the regular expression works at this handy site.

Client Side Validation of Dynamically Loaded Input Fields in MVC3

In MVC3 it’s possible to enable client side validation of dynamically loaded input fields by marking up the associated models with attributes via unobtrusive javascript. The result is validation code that is executed straight away on the client side to give the user feedback as well as on the server side to stop bad data from being entered. This approach is a double edged sword, since on one hand you get client and server side code for free, but you’re also limited to a few different kinds of input validation. The other catch associated with this approach is if you are dynamically loading form fields into your page that you want validated, it will not work automatically. 

To understand why, let me tell you a story that explores the depths of your model’s views and controllers as well as the life of your page.

First, your controller is accessed by the incoming request.

Then your controller will load up a model that has properties marked up with validation attributes.

The model is passed to the view.

When the view engine starts pumping out the resulting html it sees the attributes and the unobtrusive javascript settings and adds extra attributes to your html.

The request is fulfilled and the page content is loaded into the browser.

When the page is loaded the unobtrusive and validation javascript scripts are loaded and the validation script adds event handlers to the fields on the page.

It’s at this point that validation should work as expected.

However, if you make an AJAX request that gets a different form or more fields to show on the page, the new content will not run the client side validation.

This is because the content we loaded via the AJAX request is not hooked up like the fields that were there when the page first loaded.

So to get around the issue, all we really need to do is hook up the fields that were dynamically loaded into the page. This can be achieved by adding a bit of javascript to where you are dynamically loading your new fields:

$("#contentid").load("/ContentUrl", postData, function (responseText, textStatus, xmlhttpRequest) {
  if (textStatus == "success") {
    jQuery.validator.unobtrusive.parse("#contentid");
  } else if (textStatus == "error") {
    $("#contentid").html(responseText);
  }
});

The important bit in this code snippet is where we call jQuery.validator.unobtrusive.parse, this hooks up the validation functions to the control events that are required for unobtrusive validation. Now that you can validate input, don’t be a validation nazi.

Game of Life in NodeJS

Last weekend I went to Brisbane coding event called code smash. It coincided with node knockout. The event was provided us with a space to work, people to work with and pizza. What we did at the event was up to us. In the spirit of node knockout I was keen to do some serious team coding and get results. Luckily I managed to team up with Andrew who had similar goals. Andrew works with ruby on rails full time and I work on asp.net mvc full time. So clearly the sensible thing to do was to write a Game of Life in NodeJS. Just to keep us on our toes.

We had slightly less than 5 hours. So we decided to take a client side javascript version of the game of life that was already written and make it shared from the server side. We got it running on the server first. That was pretty straightforward as the client side javascript was well-written. Then we got the game displaying as quickly as possible by having clients poll the server. Then we added the ability for formations of cells to be added to the game, so it would be more interesting. Andrew seemed to know a bit about the Game of Life, and was aware of interesting formations that would travel. So we added some of these as possible formations that could be added to the game. We got it working and had something to show; we were happy with the day.

This weekend I was still energetic about the project. So I

  • Changed the polling to push based sockets using socket.io;
  • Made the packets a smaller;
  • Made the field size as big as I could;
  • Switched out the heavy duty game library and did the painting with jCanvas; and
  • Gave our Game of Life a home.

I’m not usually a big fan of games like I used to be back in the day. This is probably because I lack the time. This game is a set-and-forget sort of game, there’s no big time investment required. I’m still thinking about how to make it more interesting by adding to the multi-player aspect of the game.

The game suffers from entropy over time and relies on people visiting to add cells to make the result dynamic and interesting.

I have though of a few other interesting possibilities, that could still be explored:

  • Calculate a big field and be able to zoom and scroll around parts of it.
  • Be able generate a still image of a big field frame of the game. Maybe keep some cell history and heat map the image if it’s sparse.
  • Introduce colours to the cells that mix as cells reproduce.
  • Introduce a multi-player aspect where each player can add spawn points that emit predetermined cell formations.
  • Make the server send only the unpredictable state updates, and get the clients to do the other predictable calculations related to display.
  • Be able to have an infinite sized field that scales as servers are added to calculate more of the field.

The code is on github.

Computing is Cheap

Over a decade ago I used to run a Linux server (Redhat at first and later Gentoo) that did some routing, stored files and hosted my website and a squid proxy. The Internet link was a terrible upload capped cable connection with a dynamic domain name service to update the host name when the IP address changed.

The server was a bit of a beast with 4 hard drives in it so I imagine it would have used some electricity. Using the current electricity rates (0.19c per kwh * 24hrs * 31 days * 0.250kw) I estimate running the server would cost around $35. And that does not factor in the cost of the connection.

This long weekend I got myself a Virtual Private Server for around $7 a month (with the current good AUS exchange rate).

Here is the comparison:

  • The upload connection is 2mbps rather than 128kbps.
  • The disk space is a modest 5GB rather than 160GB.
  • Software these days seems to be pretty memory intensive and needed tweaking to run on 128MB efficiently; on the older server I think I had 512MB.
  • The CPU is 2.4GHz compared to a 350MHz but I am sharing it with other virtual machines.

Computing power is cheap and virtualisation is making it cheaper.