Week 1

I wrote a web crawler in Python. It works by getting the contents of a web page, finding all the links and following them to a specified depth, while scanning for some search text.

I found Python pretty straightforward to work with despite not having used or even read it before. I was surprised that I managed to complete the exercise in two hours. The documentation was good. Finding the regex method to use took the most time, and having to explicitly cast was annoying.

Week 2

I created a Django website for the web crawler created the week before.

I used Django, which is a Python web framework. It was a larger framework than I was expecting, coming in over four megabytes, but it did seem to have a lot of features. It took a little longer than I wanted to get something going: I spent two hours getting a basic form and another two hours adding some more advanced features. I personally prefer Sinatra-style frameworks like Express as they seem to make more sense to me.

I made my application reuse the code I wrote the week before. I am wary that it could create a security hole, depending on how Django cleans form input data. The code to reuse the previous functionality was a little more complicated than it should have been because of the way I wrote the code from the week before, but I wanted to see if I could reuse it without changing it.

I spent over an hour trying to get Django working with Google app engine. I quit while I was ahead because the proposed solutions I found looked hacky or required another package to install. It seemed pretty painful compared to other hosting sites like no.de or heroku. To Google app engine’s credit, the logging was good.

Week 3

This week we explored Python’s functional programming side. I wrote a website that graphs word statistics for a given page. The logic for gathering the statistics was to be written without using looping constructs. I used a primitive form of TDD just by using the command line and the assert function in Python. Later I integrated the statistic logic into the web interface, this time I avoided trying to reuse previous code as it was rather different and I was running out of time.

It took about an hour to get a filtered list of words with a count and an hour to get rid of duplicates, limit the results to only ten words and find shortest and longest words. To hook it all up to the web interface was another hour. A lot of the time was spent re-reading the documentation and head scratching.

By the way I ran the tool over my blog and I use the word ‘I’ a lot.

keza.net

JavaScript Models

In other projects I have used knockout.js to create view models. This time I just worked with jQuery and jQuery mobile directly and for a simple problem I would recommend starting out by working this way. I prefer to add a framework or tool when it solves an issue.

jsHint

I used this project to experiment with jsHint, which does some static analysis of your JavaScript to help find problems. jsHint is a fork of jsLint that provides more flexibility around which rules should be enforced. I started off using it strictly but have since come up with my own set of options that I use on other projects to improve readability.

Hack

I built the site using jQuery mobile which has great support for most smart phones. However I was primarily concerned with optimising for use on iPhones/iOS. I found that on iOS you could show a time keyboard by specifying the type as a time like so.

<input type="time" name="timestart" id="starttime" title="" />

This will show the time input keyboard in iOS. However there seems to be an issue in iOS 5.1 where the value of a html input time field will not display when it is set via JavaScript. To get around this issue I wrote a hack where I float a span over the top of the field when I want to show a value and then hide it if the user enters something via the time keyboard.

TODO

One idea I had that may come in the future is a tweet button, which basically allows you to whinge on twitter about how long you had to wait with a great deal of precision.

keza.net

Three hours later you are a bit miffed when you have run aground on a desert island.

When you are cruising around on a boat, the wind, tides and currents affect where your boat ends up. You can’t just set a bearing and expect to end up precisely where you intended. Instead you need to recalculate where you are and refine your bearings. The more recalculations you do, the closer you stay on course.

Releasing software is similar but even the destination changes. It’s obvious when you’re in control of a boat, but when releasing software it’s easy to forget that getting regular user feedback is essential to staying on track.

Communication is surely an important piece among the many moving parts in the releasing software machine.

keza.net

Late in the game a few issues cropped up, I managed to figure them out and fix them. Looking on the support forum a few people are using it and suggesting all sorts of fixes.

I though I would share the open source love and contribute my fixes. So I got the original author’s permission, and made my own fork on github.

I hope it helps.

keza.net

Step 1: Change your layout pages to reference your JavaScript files via a UrlHelperExtensions class.

Step 2: In the UrlHelperExtensions class write some code like this:

public static string Scripts(this UrlHelper helper, string fileName)
{
    return helper.Content("~/Content/Scripts/" + fileName);
}
public static string knockout(this UrlHelper helper)
{
    if (System.Diagnostics.Debugger.IsAttached)
        return Scripts(helper, "knockout-1.2.1.debug.js");
    return Scripts(helper, "knockout-1.2.1.js");
}
public static string CDNjquery(this UrlHelper helper)
{
    if (System.Diagnostics.Debugger.IsAttached)
       return "//ajax.googleapis.com/ajax/libs/jquery/1.6/jquery.js";
    else
        return "//ajax.googleapis.com/ajax/libs/jquery/1.6/jquery.min.js";
}

The jQuery example here is using a CDN in the hopes that the visitor already has it in their browser cache.

Now your JavaScript will load quickly and you can still debug during development.

keza.net

The idea came to me when I worked for a large software company that had huge end of year parties. Last year, on the day of the end of year party when I was experimenting with Ruby, I thought it would be neat to use the set of templates that let you generate a basic application (called scaffolding) to write a web application to log the beverages that I would drink at the office Christmas party. It was pretty quick to get up and running.

I hosted my application at heroku so I could use it that night. At the party I found it difficult to use on my phone, so I ported the interface to jQTouch. This made it mobile friendly and much easier to use.

A few months later I was showing some people the application and they wanted to be able to use it. So I decided to make it multi-user and integrate with services like Twitter and Facebook. I started this project using Ruby on Rails just after a new release, which made it frustrating trying to find stable packages. So I decided to rewrite what I had in node.js.  As a bonus node.js made the serialising of ajax responses nice and easy.

A little while later I purchased the domain name and started hosting the application on my own VPS. At the time there were very few node.js hosts. It was an interesting learning exercise to host it myself. I made my own deploy script using ssh, git and upstart.

I wanted the UI to be responsive and not hit the server on every action, but the client side code quickly became out of control. I found that knockout.js was a handy way to organise my code into models that bind to parts of the html document. These bound elements get automatically updated as changes are made. As the logic is encapsulated in a model,  I could potentially test complicated logic without the browsers DOM being required or being susceptible to changes to the structure of the document.

After battling with some bugs in the jQTouch framework I switched to jQuery Mobile, it was easy to switch over and it works on a wider range of devices. As the project spanned the better part of a year it was interesting to see the landscape of dependencies that I was building on change beneath me.

Later I added the ability to use the browsers geolocation capabilities with Geo.js so that locations of drinks could be stored and mapped.

This project has been a great opportunity to learn about how to use stuff like google analytics, landing pages, logos and even outsourcing. One interesting anecdote I have relates to the logo, there is a cool site called fiverr.com where people post work they will do for $5, I used the site to find a kid in the states who I hired to design my logo for a fiver.

Overall beveragelog.com has been a great learning experience, there is still a lot of scope for more enhancements and features but for now I am relieved to finally get it released. I have found starting something is hard and takes enthusiasm, but finishing something is harder and takes stamina.

I have already used some of the skill I gained in other work and hope that they continue to serve me well.

keza.net

This example deploys from an existing ‘last build’ staging area to a ‘beta’ staging area but doesn’t copy across the configuration or error files.

"c:\Program Files\IIS\Microsoft Web Deploy V2\msdeploy.exe" -verb:sync -source:iisApp="LastBuild" -dest:iisApp="Beta" -skip:objectName=dirPath,absolutePath="elmah" -skip:objectName=filePath,absolutePath="^((?!Views).)*web\.config$" -skip:objectName=filePath,absolutePath="settings-override.xml"
pause

The tricky bit that this command is handling is that MVC doesn’t just have the one web.config file. Each view folder also contains a web.config file used by the view engine, so simply ignoring files name web.config will deploy a broken web application. The example above works because msdeploy allows regular expressions, this regular expression skips over the web.config in the root directory but still copies over the others that are in ‘Views’  folders.

You can see how the regular expression works at this handy site.

keza.net

I created Benchy because I wanted to do some performance tuning, but I was reluctant to start without having a reliable way to measure the results of my changes over time. I also wanted to be able to see when changes were introduced that had negative impacts on performance. Benchy is my first complete and released open source project. Starting things is hard, finishing things is even harder.

It was fun to be able to start a fresh green fields project that has potential to be useful. As Benchy is a simple system and a new project, I found it was pretty straightforward to write tests first and do Test Driven Development. (TDD with .net, it exists!) I often feel uneasy making things more complex than they need to be, so I was glad that I used Moq for the tests. I really liked not having to add a whole bunch of extra cruft to mock out dependencies during testing.

I shamelessly stole most of the framework concepts from nUnit.

I used flot for graphing the benchmark results.

Read all about it on its dedicated page at http://www.keza.net/benchy/

Or have a look at the source at https://github.com/kezakez/Benchy

Thanks to the company I work at for supporting my ideas and open source.

keza.net

To understand why, let me tell you a story that explores the depths of your model’s views and controllers as well as the life of your page.

First, your controller is accessed by the incoming request.

Then your controller will load up a model that has properties marked up with validation attributes.

The model is passed to the view.

When the view engine starts pumping out the resulting html it sees the attributes and the unobtrusive javascript settings and adds extra attributes to your html.

The request is fulfilled and the page content is loaded into the browser.

When the page is loaded the unobtrusive and validation javascript scripts are loaded and the validation script adds event handlers to the fields on the page.

It’s at this point that validation should work as expected.

However, if you make an AJAX request that gets a different form or more fields to show on the page, the new content will not run the client side validation.

This is because the content we loaded via the AJAX request is not hooked up like the fields that were there when the page first loaded.

So to get around the issue, all we really need to do is hook up the fields that were dynamically loaded into the page. This can be achieved by adding a bit of javascript to where you are dynamically loading your new fields:

$("#contentid").load("/ContentUrl", postData, function (responseText, textStatus, xmlhttpRequest) {
  if (textStatus == "success") {
    jQuery.validator.unobtrusive.parse("#contentid");
  } else if (textStatus == "error") {
    $("#contentid").html(responseText);
  }
});

The important bit in this code snippet is where we call jQuery.validator.unobtrusive.parse, this hooks up the validation functions to the control events that are required for unobtrusive validation. Now that you can validate input, don’t be a validation nazi.

keza.net

We had slightly less than 5 hours. So we decided to take a client side javascript version of the game of life that was already written and make it shared from the server side. We got it running on the server first. That was pretty straightforward as the client side javascript was well-written. Then we got the game displaying as quickly as possible by having clients poll the server. Then we added the ability for formations of cells to be added to the game, so it would be more interesting. Andrew seemed to know a bit about the Game of Life, and was aware of interesting formations that would travel. So we added some of these as possible formations that could be added to the game. We got it working and had something to show; we were happy with the day.

This weekend I was still energetic about the project. So I

• Changed the polling to push based sockets using socket.io;
• Made the packets a smaller;
• Made the field size as big as I could;
• Switched out the heavy duty game library and did the painting with jCanvas; and
• Gave our Game of Life a home.

I’m not usually a big fan of games like I used to be back in the day. This is probably because I lack the time. This game is a set-and-forget sort of game, there’s no big time investment required. I’m still thinking about how to make it more interesting by adding to the multi-player aspect of the game.

The game suffers from entropy over time and relies on people visiting to add cells to make the result dynamic and interesting.

I have though of a few other interesting possibilities, that could still be explored:

• Calculate a big field and be able to zoom and scroll around parts of it.
• Be able generate a still image of a big field frame of the game. Maybe keep some cell history and heat map the image if it’s sparse.
• Introduce colours to the cells that mix as cells reproduce.
• Introduce a multi-player aspect where each player can add spawn points that emit predetermined cell formations.
• Make the server send only the unpredictable state updates, and get the clients to do the other predictable calculations related to display.
• Be able to have an infinite sized field that scales as servers are added to calculate more of the field.

The code is on github.

keza.net