Tag Archives: github

Three weeks of Python programming

I recently discovered the 50apps challenge, a year long challenge that publishes weekly programming exercises.  I hope to be able to participate in as many of the exercises as my schedule permits. The first three weeks of the challenge were focused on exploring the Python programming language. These are my notes.

Week 1

I wrote a web crawler in Python. It works by getting the contents of a web page, finding all the links and following them to a specified depth, while scanning for some search text.

I found Python pretty straightforward to work with despite not having used or even read it before. I was surprised that I managed to complete the exercise in two hours. The documentation was good. Finding the regex method to use took the most time, and having to explicitly cast was annoying.

Week 2

I created a Django website for the web crawler created the week before.

I used Django, which is a Python web framework. It was a larger framework than I was expecting, coming in over four megabytes, but it did seem to have a lot of features. It took a little longer than I wanted to get something going: I spent two hours getting a basic form and another two hours adding some more advanced features. I personally prefer Sinatra-style frameworks like Express as they seem to make more sense to me.

I made my application reuse the code I wrote the week before. I am wary that it could create a security hole, depending on how Django cleans form input data. The code to reuse the previous functionality was a little more complicated than it should have been because of the way I wrote the code from the week before, but I wanted to see if I could reuse it without changing it.

I spent over an hour trying to get Django working with Google app engine. I quit while I was ahead because the proposed solutions I found looked hacky or required another package to install. It seemed pretty painful compared to other hosting sites like no.de or heroku. To Google app engine’s credit, the logging was good.

Week 3

This week we explored Python’s functional programming side. I wrote a website that graphs word statistics for a given page. The logic for gathering the statistics was to be written without using looping constructs. I used a primitive form of TDD just by using the command line and the assert function in Python. Later I integrated the statistic logic into the web interface, this time I avoided trying to reuse previous code as it was rather different and I was running out of time.

It took about an hour to get a filtered list of words with a count and an hour to get rid of duplicates, limit the results to only ten words and find shortest and longest words. To hook it all up to the web interface was another hour. A lot of the time was spent re-reading the documentation and head scratching.

By the way I ran the tool over my blog and I use the word ‘I’ a lot. :)

Contributing to WordPress Support Tickets

Recently I was working on a quick one-off project. I found a WordPress plugin called Support Tickets that pretty much did everything that the project required. The last time I touched php was nearly a decade ago so it’s not my usual bag, but it works, so why bother being snobby about the technology used?

Late in the game a few issues cropped up, I managed to figure them out and fix them. Looking on the support forum a few people are using it and suggesting all sorts of fixes.

I though I would share the open source love and contribute my fixes. So I got the original author’s permission, and made my own fork on github.

I hope it helps.

Game of Life in NodeJS

Last weekend I went to Brisbane coding event called code smash. It coincided with node knockout. The event was provided us with a space to work, people to work with and pizza. What we did at the event was up to us. In the spirit of node knockout I was keen to do some serious team coding and get results. Luckily I managed to team up with Andrew who had similar goals. Andrew works with ruby on rails full time and I work on asp.net mvc full time. So clearly the sensible thing to do was to write a Game of Life in NodeJS. Just to keep us on our toes.

We had slightly less than 5 hours. So we decided to take a client side javascript version of the game of life that was already written and make it shared from the server side. We got it running on the server first. That was pretty straightforward as the client side javascript was well-written. Then we got the game displaying as quickly as possible by having clients poll the server. Then we added the ability for formations of cells to be added to the game, so it would be more interesting. Andrew seemed to know a bit about the Game of Life, and was aware of interesting formations that would travel. So we added some of these as possible formations that could be added to the game. We got it working and had something to show; we were happy with the day.

This weekend I was still energetic about the project. So I

  • Changed the polling to push based sockets using socket.io;
  • Made the packets a smaller;
  • Made the field size as big as I could;
  • Switched out the heavy duty game library and did the painting with jCanvas; and
  • Gave our Game of Life a home.

I’m not usually a big fan of games like I used to be back in the day. This is probably because I lack the time. This game is a set-and-forget sort of game, there’s no big time investment required. I’m still thinking about how to make it more interesting by adding to the multi-player aspect of the game.

The game suffers from entropy over time and relies on people visiting to add cells to make the result dynamic and interesting.

I have though of a few other interesting possibilities, that could still be explored:

  • Calculate a big field and be able to zoom and scroll around parts of it.
  • Be able generate a still image of a big field frame of the game. Maybe keep some cell history and heat map the image if it’s sparse.
  • Introduce colours to the cells that mix as cells reproduce.
  • Introduce a multi-player aspect where each player can add spawn points that emit predetermined cell formations.
  • Make the server send only the unpredictable state updates, and get the clients to do the other predictable calculations related to display.
  • Be able to have an infinite sized field that scales as servers are added to calculate more of the field.

The code is on github.

Server-Sent Events in ASP.net MVC

Google Chrome (Chromium) recently released a version of their browser with support for server-sent events. I used this technology and ASP.net MVC to create a WebSlides application.

I created a nifty throw-away demo called WebSlides that switches between slides (PNG images). The slide show presenter navigates to a URL on their smart phone and is able to tap on back or forward links to change the current slide. The audience and the computer running the projector run a browser that points to another URL. As the presenter changes the slide all the clients connected see the updated slide.

What is a server-sent event?

Basically a server-sent event is a draft web standard that (on the ball) web browsers support to allow updates from the server to be “pushed” out to the browser. It is similar to WebSockets which has been all the rage (with cool experiments) but it is not a duplex communication channel. When you see the term “pushed” in regard to server-sent events you should note that what is actually happening is that the browser is actually polling a server URL. You can easily see this when you intercept traffic with fiddler. What it means though is you will not have to write the JavaScript to do the polling and it also gives you a way to only send data to the client when it is required. In my case I use it to update the slide the browser is showing when the presenter changes the slide.

How did I do it?

I created the WebSlides demo using ASP.net MVC, I believe I am the first to create something like this as support for server-sent events is pretty fresh. I found that this guy created a chat client with cold fusion on the server-side which is pretty cool.

In my demo I created a new ActionResult called ServerSentEventResult that does all the heavy lifting you need to do make the browser happy.

So all you have to do is use it like this in your controller.

ServerSentEventResult eventStream = new ServerSentEventResult();
Follower followerModel = new Follower();
eventStream.Version = followerModel.Version;
eventStream.Content = () =>
{
    JavaScriptSerializer serializer = new JavaScriptSerializer();
    return serializer.Serialize(followerModel);
};

Another interesting point to note about my implementation is the lambda used to get the content, you may remember that the browser will poll the server to get events, so why make the server serialize data if it is not necessarily going to send it every request.

The rest of the magic happens on the client side with jQuery, If you have come this far I will leave it to you to dig around on github to see how that works.

PDFRender on github

A webpage that can render itself in PDF?

During a recent blog reading session, I stumbled upon some information about jsPDF. It’s basically a nifty JavaScript API that can generate PDF documents. It works by manually creating a base64 encoded datauri. What this means is that modern web browsers like Chrome, Firefox and Safari can create PDF documents on the client side.

I had a cool idea for a useful project that I don’t believe yet exists. I thought it would be awesome to progressively enhance (in a browser that supports datauri) an html page to show a link that would render the text in the current page in PDF, ready for efficient paper saving printing. I can see such a project being useful for resumés, shopping lists, bills and receipts.

So I created a prototype and started up a new github project http://github.com/kezakez/PDFRender

If you haven’t heard about github, it is free source control in the cloud with the killer feature of being able to reference code to the line number via URL. Exciting stuff!