A very geeky web project

Update: the Glasgow version of the map is now live!

My interest in railways started off about 3 years ago, as simply a desire to squeeze into disused and supposedly-sealed-up tunnels and take photos of them. Normal enough, you might think. But since then it’s grown into a more general interest. I’ve collected a lot of books on railways, especially the ones around Edinburgh and Glasgow (in fact, so many that I’m starting to fear for the structural integrity of my bookshelf). I haven’t yet graduated to skulking on station platforms in all weathers wearing a cagoule and meticulously writing down the numbers of all the passing trains, but it may just be a matter of time now.

Maybe I inherited it from my mother. She writes a whole blog about trains and railways, here.

My rapidly growing collection of railway books (minus a few that are scattered around the house, wherever I last read them)

My rapidly growing collection of railway books (minus a few that are scattered around the house, wherever I last read them)

One thing I found while researching the history of the rail network was that I always wanted more maps to help me visualise what was going on. There were a few good ones in the books, but I often found myself struggling to imagine how things were actually laid out in the past, and how the old lines fitted in with the present day railways. I wished there was some sort of interactive map out there that would let you change the date and watch how the railway network changed over time, but I couldn’t find anything like that (the closest thing I found was a “Railway Atlas” book that has a map of the present day network in each area with a map from 1922 on the opposite page). So I decided to make one.

(Actually, I decided to make two: one for Edinburgh and one for Glasgow. The Glasgow one is taking a bit longer due to the more complex network on that side of the country, but I’m hoping to release it soon).

The project fitted in well with some other things I’d been wanting to do as well. I’ve always had an interest in maps and have been collecting the Ordnance Survey 1:50000 series (among others) for most of my life now, so when I discovered that Ordnance Survey now release a lot of their data for free, I was excited at the possibilities. I knew that the OS OpenData would make a good basis for my railway maps. I’d also been wanting to experiment with some of the newer web technologies for a while, and coding the viewer for the maps seemed like a good opportunity to do that.

My (mostly) Ordnance Survey map collection. I don't have a problem. Honest, I don't. I can stop any time I want to.

My (mostly) Ordnance Survey map collection. I don’t have a problem. Honest, I don’t. I can stop any time I want to.

As with a lot of projects, it seemed simple at first but once I actually started work on it, I quickly realised it was going to take longer than I thought. There were two main elements to it:

  1. The data sets. To be able to draw the map, I would need detailed data on all of the railway lines and stations in the Edinburgh and Glasgow areas, past and present, including their names, opening and closing dates, which companies built them, and so on. As far as I knew, this information didn’t even exist in any one single source, and if it did it was sure to be under copyright so I wouldn’t be able to just take it and use it. I was going to have to create the data sets pretty much from scratch.
  2. The viewer. Once I had the data, I needed to make a web page that could display it in the form I wanted. I already had quite a clear idea in my head of what this would look like: it would show the map (of course), which could be scrolled and zoomed just like Google or Bing Maps, and there would also be a slider for changing the date. The lines on the map would be colour coded to show which company they were owned by, or their current status, and special lines like tunnels and freight routes would also be shown differently.

It turned out I also needed to build a third major element as well: an editor for creating the data sets. Previously when I’d drawn maps, I’d either used the Google map maker (which has copyright problems if you want to actually use your creations for anything), or drawn them using Inkscape (which, great though it is, isn’t really designed for making maps in). I didn’t think either of those was going to cut it for this project… I needed something better, something that had all the features I needed, but was free from copyright issues. So I decided to make a map editor first.

Step 1: The Editor

At this point, anyone who’s a software engineer and has had it drummed into them “Don’t re-invent the wheel!” is probably shaking their head in exasperation. “You built your own map editor? Why would you do that? Surely there must be one out there already that you could have used!”. To be honest, I’m sure there was, but I don’t regret my decision to make my own. I had three good reasons for doing it that way:

  1. I would learn a lot more.
  2. I could make an editor that was very well suited to the maps I wanted to make. It would have all the features I needed, but wouldn’t be cluttered with extra ones I didn’t need. And I would know exactly how to use it, and would be able to change it if anything started to annoy me.
  3. It would be fun!

I’d had my eye on the Qt GUI toolkit for a while, wanting to give it a try and see if it was better than the others I’d used in the past. So I downloaded Qt Creator and got building.

Of course, I needed some map data first, so I downloaded one of the Ordnance Survey OpenData products: “OS OpenMap Local”, for grid squares NS and NT. (Ordnance Survey products don’t use the latitude and longitude co-ordinates familiar to users of Google Maps or OpenStreetMap; they have their own “National Grid” system that divides the UK into hundred kilometre squares, and uses numerical co-ordinates within those squares). These came in the form of two enormous (nearly a gigabyte altogether) GML files.

GML stands for “Geography Markup Language”, and is a standard XML grammar used for expressing geographical information. The contents of the OpenMap Local files are actually pretty simple conceptually; there’s just a hell of a lot of them! They mostly consist of great long lists of map elements (which can be areas such as forests or lakes or buildings, linear items like roads or railways, or point locations like railway stations) with names, national grid references, and any other relevant information. I wanted to use this information to display a background map in my map editor, on top of which I could draw out the railway routes for my interactive map.

I knew that parsing several hundred megabytes of XML data was likely to be pretty slow, and I didn’t really want the editor to have to do this every time I started it up, so I wrote a Python script that would trawl through the GML files and extract just the bits I was interested in, saving them in a much more compact file format for the actual editor to read.

Now I was onto the fun part: actually displaying the map data on the screen. Thankfully, Qt’s excellent graphics functionality was a great help here. After writing a quick function to translate OS national grid references to screen co-ordinates, and using it to project the map data onto the screen, I was looking at a crude map of Edinburgh. I spent a while tweaking the details to get it to look the way I wanted it: changing the colours of each type of element, changing the line widths for different types of road, hiding the more minor details when the view was zoomed out (OpenMap Local is very detailed and includes the outline for every single building, so trying to display all of that when you’re zoomed out far enough to see an entire city results in a very cluttered map, not to mention one that displays very slowly!).

Edinburgh, courtesy of Ordnance Survey's OpenData, and my map editor.

Edinburgh, courtesy of Ordnance Survey’s OpenData, and my map editor.

Once I had the background map displaying to my satisfaction, I turned my attention to the actual editing functions and finding a suitable way to store the data for the railway map…

Step 2: The Data

The data model for the interactive map is pretty simple. The three main concepts are: segments (simple sections of track without any junctions), stations (pretty self explanatory I hope) and events. An event is a change in one of the segments’ or stations’ properties at a certain date. For example, the segment that represents Scotland Street Tunnel has an event in 1847 when it came into use (a “change of status” event), another in 1862 when it was taken over by the North British Railway company (a “change of company” event), and another in 1868 when it was abandoned (another “change of status”). When the events are complete and accurate, this gives the viewer all the information it needs to work out how the map should look at any particular date. For a file format, I decided on JSON – it was straightforward, easy to access from both Qt and JavaScript, and easy to inspect and edit by hand for debugging.

Editing the data for Scotland Street Tunnel

Editing the data for Scotland Street Tunnel

I considered storing the data in a database rather than a file and having the viewer page query it in the background to retrieve whatever data it needed. But for this particular application, the data is relatively small (about 150KB for the Edinburgh map), and the viewer needs almost all of it pretty much straight away, so throwing a database into the mix would just have added complexity for no good reason.

Creating the data set was by far the most time-consuming part of the whole process. Every railway line and station, past and present, had to be painstakingly added to the map, and then all of the event dates had to be input. I collated the information from many different sources: present-day railway lines are part of the Ordnance Survey OpenData that I was using for the background map, so it was easy enough to trace over those. However, disused lines are not included, so I had to refer to old maps to see their routes and then draw them onto my map as best I could. For the dates, I referred to several books and websites – “An Illustrated History of Edinburgh’s Railways”, and the corresponding volume for Glasgow, were particularly valuable. Where possible, the event dates are accurate to the nearest day, although the current viewer only cares about the year.

The whole data set for Edinburgh, loaded into the editor

The whole data set for Edinburgh, loaded into the editor

I think I made the right choice in creating my own map editor – if I’d used existing software, it’s doubtful that I would have got the maps done any more quickly. There would have been a learning curve of course, but even after I’d got past that, it’s doubtful that I would have been as productive in a general map editor as I was in my specialised one.

Step 3: The Viewer

The viewer was the final piece of the jigsaw, and although I’d given it some thought, I didn’t properly start work on it until the Edinburgh map data was nearly completed. Unlike for the editor, there was only one real choice of technology for the viewer – if I wanted it to run on a web page and work across virtually all modern devices, it was going to have to be HTML5.

HTML5 extends previous versions of HTML with new elements like the canvas tag, which allows graphics to be rendered in real-time from JavaScript – in days gone by, this required a plug-in such as Flash or Java, but now it can be done in a vanilla browser without anything added. I hadn’t used the canvas before, but a bit of quick experimentation confirmed that it was more than capable of doing everything I needed for my interactive map. I also made use of the JQuery library to simplify operations such as fetching the map data from the web server in the background.

First, I wrote a small library of line drawing routines for all the different sorts of railways: dashed lines for tunnels, crossed lines for freight, and dashed-line-within-solid-line for single track railways (as used on some OS maps). These aren’t supported directly by the canvas, but it only took just over a hundred lines of JavaScript code to add them. Then I was ready to build a map renderer on top.

Different line styles and their uses

Different line styles and their uses

I had a basic version up and running pretty quickly, but it took a lot longer to implement all the features I wanted: background images, scrolling and zooming, the slider for changing the date, clicking on items for more information. Getting the background images lined up perfectly with the lines and stations turned out to be the trickiest part, though it really shouldn’t have been hard. It took me an embarrassingly long time of debugging before I realised I was truncating a scaling factor to two decimal places in one place but not in another, and once that was fixed everything looked fine.

It lives! The finished product

It lives! The finished product

There are still a few things that annoy me about the end product (the mobile browser and touch screen support, especially, could be better), but overall I’m pretty happy with it. It was fun to create and I learned a lot in the process… about the history of the local railways of course; about how geographical data is stored and processed; about programming GUIs with Qt; and about creating interactive graphics using HTML5.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.