My take on “Codes of Conduct” for software projects

Posted on September 20, 2018 by gcat

The news that the Linux kernel development project has adopted a new code of conduct has prompted a lot of comment. As someone who’s been a software developer for all my working life and who’s written about vaguely related stuff before, I thought I would stick my oar in as well, at least to address what I think are some widespread misconceptions.

First off, I’ll say a bit about myself and my own experience. I’ve been a software professional for 16 years. During that time I seem to have impressed a lot of the people I’ve worked with. I have more than once “rescued” projects that were previously thought to be doomed and turned them into success stories. Collaborators who have worked with me in the past have frequently requested to work with me specifically when they approach my organisation for further consultancy. Last year I was promoted to a fairly senior technical position, and also last year I did my first paid freelance project, receiving glowing praise from the client for the way I handled it.

I’m not saying this to brag. I’m normally a pretty modest person and believe me, talking about myself in those terms doesn’t come easily. I’m saying it because it’s going to be relevant to what I say next.

I’m also, by pretty much any definition, a snowflake. (That’s the term these days, isn’t it?). I don’t like confrontation and I tend to avoid it as much as I can. I find it hurtful being on the receiving end of harsh words or blunt criticism and I also tend to avoid situations where this is likely to happen. When it does happen I find I need to retreat and lick my wounds for a while before I feel ready to face the world again.

I didn’t choose to be this way, and if I’d been given the choice I wouldn’t have chosen it, because to be honest it’s pretty damned inconvenient. But it’s the way I am, the way I’ve always been for as long as I can remember. (Again, this may not seem relevant yet, but trust me, I’m bringing it up for a reason).

It’s maybe not surprising, then, that I’m broadly supportive of any initiative that tries to make software development a friendlier place. I don’t follow Linux kernel development closely enough to have a strong opinion on it, but some open source communities certainly have acquired reputations for being quite harsh and unpleasant working environments. This probably is a factor in my choosing not to contribute to them – although I have contributed a bit to open source in the past, these days if I want to code for fun I prefer to just tinker with my own solo projects and avoid all that potential drama and discomfort.

Not everyone agrees, of course, and sites like Slashdot are awash with comments about how this is a disaster, how it’s going to destroy software quality, and how it’s the beginning of the end of Linux now that the Social Justice Warriors have started to take over. I’m not going to attempt to address every point made, but I would like to pick up on a few common themes that jumped out at me from reading people’s reactions.

Fear of harsh criticism makes people deliver

The main justification put forward for keeping the status quo seems to be that people will up their game and produce better code if they’re afraid of being flamed or ridiculed. I don’t believe this works in practice, at least not for everyone.

I remember years ago when I was learning to drive, my first instructor started acting increasingly like a bully. When I made mistakes (as everyone does when they’re learning something new), he would shout at me, swear at me and taunt me by bringing up mistakes I’d made weeks before. But far from spurring me on to improve my driving, this just wound me up and made me stressed and flustered, causing me to make even more mistakes, in turn causing him to throw more abuse my way, and so on. It got so bad that I started to dread my driving lessons and when I was out in the car with him I lost all confidence and became terrified of making even the tiniest mistake.

After a few weeks I got fed up with this so I phoned the driving school and told them I wanted a different instructor, someone who would build up my confidence rather than destroy it. They assigned me to a great instructor, an experienced and patient older man who I got on very well with, and the contrast was dramatic. My driving improved straight away and I started to actually look forward to my lessons. Within a few weeks I was ready to take my test, which I passed on the first attempt. I always remember this experience when I hear someone express the opinion that abuse will make people perform better.

Of course, everyone responds differently to these situations. I knew someone who said he was glad his driving instructor shouted at him because, after all, it was potentially a life-or-death situation and this would help him to take it seriously. So I’m not saying everyone’s experience will be the same as mine, just pointing out that not everyone responds positively under that sort of pressure.

Furthermore, someone who goes to pieces in the face of abuse might still be perfectly capable in other circumstances. I was able to drive just fine once I got away from that first instructor, and since then I’ve driven all over the country, driven minibuses and towed caravans without incident.

People will use the code of conduct to blow grievances out of all proportion and seek attention

Personally, as someone who hates conflict and hates being the centre of attention, I can’t imagine anything I’d be less likely to do than go out of my way to draw attention and publicity to myself. If anything I think I’d more likely be far too reticent about seeking help if someone was violating a code of conduct, and I imagine it would be the same for most of the people who would actually benefit the most from the code.

That’s not to say everyone would be the same, of course. There might well be a vocal minority who would act in this way, but that shouldn’t stop us from trying to improve things for people who genuinely do need it. In any case, whether a given behaviour really constitutes gratuitous “attention seeking” or whether it’s out of proportion is very much a subjective judgement.

Emotionally fragile people have nothing to offer anyway

I hope my description above of my own working life has shown that we do have something to offer. I think this belief is due to confusion between “people who are good at software development” and “people who are good at being loud and obnoxious”. If you create a working environment so toxic that 70% of people can’t cope with it and leave, that doesn’t mean you’ve retained the 30% best developers, it means you’ve retained the 30% of people best equipped to thrive in an abusive environment. I see no reason to think there’s going to be much correlation there.

I think a similar argument can be made about the contentious “safe spaces” I’ve written about before. Many of their opponents argue that it’s healthier to be exposed to a diverse range of different points of view rather than living in a bubble. I completely agree, but I disagree about how best to achieve that. A complete free-for-all isn’t necessarily a reliable way to foster open debate – you can easily end up with a situation where the loudest, most abrasive people come to dominate and everyone else is reluctant to express a contrary opinion for fear of being abused and ridiculed. If you genuinely want (and I’m not convinced many of the detractors actually do want this) to hear as wide range a of opinions as possible, you need an environment where everyone feels comfortable expressing themselves.

Maybe if there were unlimited good software developers in the world you could make a case for only working with the emotionally hardy ones and avoiding the possible difficulties of dealing with us “snowflakes”. But there aren’t. In most places developers are highly in demand, so it makes no sense to dismiss people who might actually be able to make a valuable contribution.

It’s not up to us to accommodate your emotional frailties, it’s up to you to get over them

Of all the views expressed in these discussions, I think this is the one that irks me the most. It implies that anyone who reacts badly to harsh words and insults could easily “get over it” if they chose to do so, and that just doesn’t tally with my experience at all.

I’ve spent many decades trying to “get over” the problems I’ve had. I’ve spent a five figure sum of money on therapy. I’ve read more self help books than I care to remember and filled notebooks cover-to-cover with the exercises from them. I’ve forced myself into numerous situations that terrified me in the hope that they would be good for me. I’ve practised mindfulness, attended support groups, taken medication, taken up exercise, talked things over with friends and family, spent long hours in painful introspection. You name it, I’ve probably tried it.

And you know what? I’m a lot better than I was. At the start of the process I could barely even hold a conversation with someone unless I knew them well, and I certainly wouldn’t have been able to hold down a job. Now I function reasonably well most days, I do pretty well at work and I have a decent social life as well. But despite all this progress, I’m still pretty emotionally sensitive, and I still don’t cope well with insults and intimidation. Maybe I’ll get even better in the future (I certainly hope to and intend to), but I suspect I will always find that kind of situation unpleasant enough to want to avoid it when possible, even if I no longer find it as debilitating as I once did.

So it makes me pretty angry when people who don’t even know me assume that, because I still get upset more easily than most, I obviously just haven’t tried hard enough. It’s noticable that these people almost never mention how you should “get over it”. Some of them seem to just assume that if you keep putting yourself in the situation that upsets you then you’ll eventually adjust and be OK with it, but this has never worked particularly well for me – as with the driving lessons example I gave above, it typically just leads to me feeling more stressed and harassed.

Basically, I think this one is an example of the just-world fallacy. It’s uncomfortable to realise that some people might struggle with certain situations through no fault of their own and that there might not be any easy solution open to them. It raises all kinds of awkward questions about whether we should be making adjustments to help them and so on, not to mention the fear of “maybe this could happen to me too some day”. It’s much neater to pretend that those people must have done something to deserve their problems, or at the very least that they must be “choosing” to forego a perfectly good solution.

Whilst I do have a tiny bit of sympathy for some of the objections to the way things are going (I wouldn’t personally relish software development becoming yet another field where social skills and confidence are valued over actual technical ability, for example), overall I find it really hard to take most of the objectors seriously. They moan and whinge about what a disaster it would be to have to treat others with basic civility, then go on to accuse the other side of being over-sensitive and blowing things out of proportion. They heap disdain on people for having problems they never asked for and almost certainly don’t want, but fail to put forward any useful suggestions on how to deal with those problems.

Uppsala and Stockholm (again)

Posted on April 27, 2016 by gcat

I was in Sweden last week. I seem to have been there a lot lately; that was my third work trip there and I’ll have my third midsummer trip in a couple of months as well.

Although I usually like to explore new places while I’m away, I was more in the mood for just chilling out and doing nothing this time, what with life getting very busy back home with work, Beltane and wedding stuff all at once. So I decided after the meeting I would book myself into my favourite hostel in Stockholm for a couple of days and spend them doing nothing at all. (Sometimes I find it easier to relax and unwind away from all the distractions and half-ticked-off To Do lists at home). The work part of the trip felt familiar as well, as I was staying in the same hotel and having a meeting in the same venue as I did two years ago.

At first it looked as if I was going to have to be at least a bit adventurous after all: when I first looked at accommodation options, my normal hostel was fully booked for the nights I wanted. But there must have been some cancellations as when I went back to actually book, they had two beds available. I booked one quickly before they changed their minds again. Although some of the other hostels looked alright, I knew that this one had a good lounge for relaxing in, whereas some of the others apparently didn’t have much common space, or didn’t allow alcohol in it. Plus City Backpackers is supposedly the best hostel in Sweden so I felt that going elsewhere after staying there twice probably would have been a bit of a come-down.

The Journey

I booked onto a nice quick direct flight to Stockholm Arlanda. As luck would have it, not just one but two things came along later and annoyingly clashed with my chosen flight: firstly, an all staff meeting at work. I wouldn’t normally be too upset about missing a meeting, but this was an interesting one as there’s a lot going on right now (reorganisations, pending move to a new building). Ah well. I’ll read the minutes later.

And secondly, the London Transport Museum announced that they were releasing tickets for their next wave of tours of abandoned tube stations and hidden tunnels, which I really wanted to go on (I like that sort of thing, you see) but which I was sure would sell out insanely quickly. The time that this would go live? 10am Tuesday, exactly the time I was boarding my plane. Grrr. Why did they have to pick the ONE day in several months that I wasn’t able to be in front of a computer at 10am?

(Laura kindly volunteered to try and get me tickets instead. Despite a surge of demand reminiscent of the Raspberry Pi launch 4 years ago, she succeeded in getting us tickets for the Down Street tour, the one I most wanted, and apparently the most sought-after one by far. So that’s something to look forward to, although it’s not til December. It’s expensive, but what the hell, you have to treat yourself sometimes. I have to admit I find it slightly amusing that the last people to be caught sneaking into a disused tunnel in London were apparently fined less money than I’ve just paid for a legitimate underground tour).

Anyway. On the plus side, it was a lovely clear morning for a flight (a rare treat when flying from Edinburgh), and the plane was amazingly empty (a rare treat when using a budget airline), which is always nice. I had a whole three seats to myself and was able to enjoy the view of West Lothian spread out below me like a map (though not get any decent photos of it, since the window was very dirty and a jet engine was blocking a lot of it). I was also able to see miles and miles of stationery traffic on all the local motorways, even though it was way past rush hour time… apparently it’s been chaos due to accidents this morning, though thankfully I avoided most of the chaos on my way to the airport. I wonder if the plane was empty because all the other would-be occupants were stuck in the jams down there.

We made it to Uppsala, where our meeting’s being held, pretty easily (though it seemed slightly bizarre that we had to enter our names into the ticket machine when buying tickets for a less than 20 minute train journey! It also seems slightly bizarre that, in contrast to the blandly corporate or edgily cool jingles they use to precede the announcements in most airports and stations, the one at Arlanda sounds like Grandpa Flump playing two quavering notes on his flumpet). Uppsala is actually the fourth largest city in Sweden, but it really doesn’t feel much like a big city at all to me… though it only has about a third of the population of Edinburgh, so I suppose it is small compared to what I’m used to. The hotel was nicer than I remembered, and it turns out it has fast Eduroam access in the rooms which is great for me – when Eduroam’s available it usually seems way faster and more reliable than whatever random public networks you can find. So I didn’t have to attempt any accidental dodgy hacker tricks in order to get online this time. Ahem.

I noticed the disgusting old white sock and the miniature Jaegermeister bottle on the lower storey roof outside my window straight away, but it was a bit longer before I noticed the (bare foot!) foot prints in a chaotic pattern on the other half of the roof. There surely has to be a story behind those…

The Meeting

We were treated to some lovely Swedish weather (clear and sunny, though still cold) as we walked down to the meeting venue, on the Uppsala University campus. The walk was a picturesque and relaxing one, along the river with its pretty bridges and boats (although it’s not quite so relaxing if you do what one of my colleagues and I did the first time we came here and walk right down the wrong side of the river, assuming there’ll be another bridge further down, then find there isn’t). I always think the campus itself looks more like a woodland summer camp than one of the top universities of northern Europe (in case it’s not clear, I do mean that as a compliment!). Apparently it was originally built as some kind of army base, so it makes sense that the layout is a bit unusual for a university.

The meeting itself was an interesting one, and since I’d actually got a decent night’s sleep for a change, I didn’t even come close to falling asleep at any point during the proceedings. It seems to do my brain good being away for a bit, because I always seem to come up with lots of new ideas for all my projects when I’m travelling. I made sure to note them down for later.

Our dinner was in an old station building. It was in slightly better condition than the old stations I normally find myself in.

The Holiday

After the day and a half of meeting, it was off to Stockholm for my little holiday. The train was on time and very nice, as they usually seem to be in European countries other than Britain, and after finally excusing myself from a crazy old woman on the platform who seemed determined to talk to me in Swedish and completely unconcerned by the fact that I couldn’t understand a word, I was on my way.

Coming back to somewhere I’ve visited before sometimes does strange things to my perception of time; I remember on my second visit to Madrid it felt like ages since I’d been there before, when in reality it was only just over two months, but coming back to Stockholm after 18 months, I didn’t feel as if much time had passed at all. The hostel “upgraded” me from the eight bed dormitory I’d booked to a 6 bed “apartment”. They were using the apartments as extra dorms, probably because they were so busy, so I still had to share with other people. But it did mean we had our own private loo, shower, small kitchen, and even a sauna (which I didn’t dare to use as I didn’t have a clue how to work the thing, though one of my room mates did manage to get it working).

I slept better than I normally do in a hostel room. I hadn’t had any plans for Friday at all, but when I discovered my pyjama top was missing, and confirmed via email that it was still at my hotel in Uppsala, I decided I was going to go and get it back. (I’ve no idea how I managed to do this; I’m normally ultra-careful not to leave anything behind when I stay in a hotel, to the point of even checking inside cupboards that I know perfectly well I’ve never opened before I leave). I probably wouldn’t have bothered as it would have been cheaper just to buy a new one than to pay for the extra return train ticket, but I felt bad as it was a present from Laura. Anyway, I didn’t really mind relaxing on the train for a couple of hours. There are worse ways to spend a morning.

I spent most of the time just relaxing, either in the hostel lounge or in a nearby bar, and was glad that I’d ripped my mother’s Reginald Perrin DVDs to my laptop to keep me entertained. It was what I felt I needed. Of course, I’d done most of the important stuff around the hostel on my previous visits here anyway – for example, photographing the local tunnel:

And the local unfortunately-named cafe:

My second day in Stockholm was slightly more energetic, though I still found the time for plenty of Reggie Perrin as well. I went for my first run since the horrible flu/chest infection/laryngitis that I was suffering from last month. Although I’d worked up to being able to run for 40 minutes non-stop before the illness, I didn’t want to do anything that strenuous after over a month’s break, so I did a gentle 20 minutes (with short pause to remove gravel from my trainer). It went surprisingly well and didn’t even cause me to have a coughing fit, so I was happy. I also went for a wander along the sea front, far enough to see Langholmen (a nice, mostly rural-feeling wooded island that’s surprisingly close to the city centre), but I was too tired to cross over to it this time.

The Return

My return flight left at 7:55, so I had to be up before 5am to get the bus. (It was the only direct flight of the day, so it was that or waste about 5 hours getting home). As always seems to happen when I need to be up early, my hostel roommates, who’d been perfectly well behaved throughout my whole stay, decided to pick the final night to make a lot of noise and keep me awake. All I can say is I hope they enjoyed the sound of my 4:45 alarm… I certainly didn’t.

The flight was the first time I’ve ever used wifi on a plane. I remember when the internet was only in the uni computer labs, or at home via excruciatingly slow dial up modem. Now, the number of places that you can escape from it is ever-diminishing: planes have wifi; my last two phones have been waterproof so being in the bath or shower is no excuse; hell, even one of the abandoned railway tunnels I explored had a perfect 4G signal (though admittedly that’s probably just an accident of microwave propagation rather than any deliberate desire on the part of Glasgow City Council to let urban explorers broadcast their crew shots more easily).

I came home feeling happier and more relaxed than I had done in weeks, thanks to the couple of days of doing nothing at all other than what I felt like. I decided I should book my next unwinding trip straight away (well, after next payday) so I have it to look forward to.

In Bruges (… and Ghent, and Antwerp)

Posted on October 6, 2015 by gcat

I’m just back from a week long trip to Belgium. I had a work meeting in Ghent on Monday and Tuesday, but Laura was able to come out and join me afterwards, so we decided to make it into a little holiday and explore Bruges and Antwerp before coming home.

This wasn’t my first time in Belgium; I had a similar meeting in Ghent nearly two years ago, but it ended up being my only foreign work trip in several years that I didn’t really enjoy. Nothing to do with Belgium, though, which seemed a nice enough place… mostly just because of the overwrought state I’d rather foolishly managed to get myself into by going on three other trips away in the few weeks preceding it. So I spent most of my time there (not very long as I couldn’t stay on after my meeting that time) wishing I was at home on my own and didn’t have to deal with any of it. Thankfully this time was a lot better.

I was away for seven nights in total, but I ended up staying in five different hotels in total, and my first four nights were all in different places. It wasn’t ideal, but there was a logic to it (basically, I wanted to stay near the airport the first night as I was flying in so late, and after the meeting I didn’t want to stay in the same hotel as it was expensive and work wouldn’t pay for another night). It did mean I got a bit fed up with wheeling all my luggage around with me for most of the first few days and was glad when I finally arrived in Bruges and was able to stay in the same room for more than about five minutes.

The first hotel was OK, except that it wasn’t as close to the airport as the name would have you believe, and the air in the room was so dry that I didn’t sleep very well. On the plus side, my room had a nice balcony overlooking the nearby motorway. Also on the plus side, the dryness woke me up while the lunar eclipse was going on so I managed to see some of it after all (though I didn’t, in my semi-conscious state, manage to get any decent photos of it unfortunately).

The Belgian trains seemed cheap, efficient and (the ones I ended up on, at least) mostly empty, so getting to the meeting was quite easy and painless, though I did have a long sweaty walk with all my luggage once I got to Ghent as the station isn’t very near the city centre. Our meeting venue was suitably grand:

After a good day of discussion, we had the conference meal. I had been gearing up for a big rant about this, because the venue insisted on everyone in the group having the same thing, which is one of my pet hates when it comes to catering. But the food was so delicious, and the chef’s moustache so worthy of Hercule Poirot’s native land, that I’ll forgive them this time. The starter wouldn’t have been to all tastes as it consisted mostly of raw meat, something I’d never eaten before, but it was much nicer than I expected.

After the meeting finished, I had one day to kill on my own before Laura arrived on Wednesday night. I’d decided to stay in Ghent for this since I didn’t get to see much of it last time, and I was very glad I did. My hostel (the building on the right with the crow-step gables in the photo below) was a lovely old building with friendly staff and good facilities.

(Only problem was, I think spending the night there re-awakened the part of me that loves to go hostelling, but one night didn’t feel like enough and I probably won’t get the chance to do it again for ages now, which is kind of frustrating. Bah. Oh well, I can surf around online and plan adventures for next summer, I guess).

I spent my evening doing the final week 9 run of Couch to 5K (which I’ll probably write a whole post about soon) at a nice park, then treating myself to a steak at one of the many pavement cafes to celebrate, followed by some good local beer in the hostel lounge. The next morning, I wandered round and took photos. Ghent has a very pretty old town so it was a good place to do this, and I was very lucky with the weather as well.

My favourite thing in Ghent (apart from the lovely hostel) was the belfry. Every quarter hour the bells play a tune – not just a short chime sequence like a lot of clocks back home, but a full-on polyphonic tune with melody and harmonies and everything. You can climb up and admire the view from the top (which is very pretty in most directions, but surprisingly industrial to the north), but even better, you can go in the “clock room” and watch the mechanism that works the clock faces and rings the bells. The bell sequence is controlled by a huge metal drum, a bit like a giant music box, that rotates while the tunes are playing.

(These sort of things kinda fascinate me. For some reason, when I was very small, I always wanted to be a clock maker. In reality as an adult, I don’t think I would have the patience or the dexterity for that and would probably have got bored with it by now if I was doing it full time, but it’s interesting to see examples of the craft occasionally).

Next, it was off to Bruges! I had time to check into hotel no. 4 and have a much-needed nap before Laura arrived. The hotel was unusual: all of the rooms had doors opening to the outside world rather than a hotel corridor, so it felt very like the sort of cheap American motel that you see in films (I’ve never seen one in real life but I assume they exist). It was fine though, apart from the connecting door to the next room which, although locked at least, didn’t block sound very well.

Bruges, like Ghent, was a very picturesque place to wander round and soak up the sights. Or even better, sail round. We took a boat tour round the canals, which is something I like to do whenever possible in a new city (done London, Paris, Berlin, Amsterdam and possibly others now). The only slight disappointment was that the boat didn’t go through the long-ish canal tunnel I’d noticed on the map and then couldn’t resist going to have a look at when we were in the area.

There was also a belfry that seemed quite similar to the one in Ghent, though we didn’t go inside this time. It must be a traditional Belgian thing.

We left our two days in Antwerp ’til last, and to be honest it was a bit of an anticlimax compared to the other cities. There was nothing wrong with it particularly, it just didn’t seem to have much going on compared with the other places we’d been. I had a similar experience in Frankfurt earlier in the year… nothing much wrong with it, just didn’t live up to the other German cities I’d visited and loved (Berlin, Munich, Leipzig, Heidelberg). Apologies if anyone from Antwerp (or Frankfurt) is reading!

The highlight for me was the very impressive main station building:

Our Antwerp hotel was quite a long way out, among the seemingly endless stretch of ports to the north west of the city centre. On weekends the buses didn’t even go to it and we had to get off quite a distance away and walk to it. But on the plus side, it was right next to an amazing (and very popular seeming) world buffet restaurant where I ate far too much Chinese food, doughnuts and profiteroles on the first night.

As is often the case, I felt ready to go home by the end of the trip. I’d enjoyed it, but I was starting to get travel fatigue after moving from hotel room to hotel room so many times in a week, and I had lots to do at home… during my meeting, emails seemed to be pouring in giving me new tasks on my other project.

I’d like to thank several organisations for making the trip much more interesting than it otherwise would have been. Firstly, my bank for sending my new debit card to the wrong address. The old one then expired at the end of September while I was still away, so for half the trip I couldn’t get money out from my main account. Secondly, Sony for pushing an update to my phone just before I set off that pretty much halved the battery life – just what I needed when I was going to be away from home and away from easily accessible plug sockets for several days. And last but not least, my web hosting provider for rejecting my monthly payment for absolutely no reason whatsoever (at first I thought this must be due to the debit card expiring, but it was nothing to do with that – it worked fine when I redid it manually). It would have been so dull just travelling around beautiful European cities without all that stuff to keep me occupied.

Let me be the first (err, second actually) to say I’ll miss netbooks

Posted on February 6, 2013 by gcat

I was interested to see this article in the Register. The majority of the comment online about the death of netbooks seems to be along the lines of “Tablets are so much cooler and slicker, netbooks were clunky and annoying to use and who really needs a full PC these days anyway, especially when they’re travelling? Hardly anyone, that’s who. Good riddance netbooks”. But I for one am disappointed that they’ve stopped making them… I can’t see that anything else is going to meet my needs quite so well when I’m travelling… and finally someone agrees with me!

I took my HP netbook running Xubuntu away with me several times last year. I always found it useful, but on the three trips where I combined work with pleasure, it was indispensable. It was light enough to carry around in my backpack without taking up half my cabin baggage allowance or knackering my shoulders. It was cheap enough that if it did get damaged or stolen it wouldn’t be the end of the world (yes, I do have insurance, but you never know when they’re going to worm their way out of actually paying up). Its battery lasts a genuine six hours on a single charge, even when I’m actually doing work on it. It has a proper (if fairly small) keyboard so typing emails or documents on it doesn’t make me lose the will to live. It has enough storage space to keep most of my important files locally in case I can’t get online.

Most of all, it actually runs a proper full operating system! This isn’t something I’m just arbitrarily demanding because I’m a technology snob. I really do need it and do make use of it. At my technical meeting in Madrid in September, I was running a Tomcat web server, a MySQL database server, a RabbitMQ server running on top of an Erlang interpreter, and a full Java development environment. Try doing that on an iPad or an Android tablet! You might think all of that would be pretty painful on a single core Atom with 2GB of memory, but it actually ran surprisingly well. I wouldn’t want to work like that all the time but for a three day meeting it was perfectly adequate and usable. The full OS also means I can encrypt the whole disk which gives me a lot of peace of mind that my files are secure even if the thing does get stolen.

But now I’m starting to get worried about what I’m going to replace it with when the netbook finally departs for the great electronics recycling centre in the sky. Despite the market being flooded with all sorts of portable computing devices, I can’t see any that are going to do what I want quite so well as the netbook did.

Get a tablet? Urgh, no thanks… I’m sure they have their place, but even if I added a proper keyboard there is no way I’d get all that development software to run on Android or iOS. OK, I wouldn’t be surprised if there is some way to hack some of it into working on Android, but Android is hardly a standard or well supported environment for it. It’s not going to Just Work the way it does on a standard PC running Windows or Ubuntu.

Get a Microsoft Surface Pro? This tablet actually does run a full version of Windows 8 (or will when it comes out), but at $900 it costs nearly three times as much as my netbook did. I couldn’t justify spending that on something I’m going to throw into my backpack and take all over the place with me. I’d be constantly worrying it was going to get broken or stolen.

Get an “ultrabook”? Again would do the things I need, but again would cost WAY more than the netbook, would almost certainly weigh a lot more than the netbook, and I’d be very surprised if it had comparable battery life either (at least not without spending even more money on SSDs, spare batteries, etc.). For the “pleasure” part of my Madrid trip I was staying in a hostel room with seven other people. There was ONE power socket between the eight of us. When travelling, battery life really does matter.

Get a Chromebook and install a full Linux distribution on it? This is actually the option I’d lean towards at present. Chromebooks have price, portability and battery life on their side and apparently are easy to install Linux on. The downsides would be the ARM processor (which could limit software compatibility as well as making even the lowly Atom look blazingly fast in comparison), and the lack of local storage (Chromebooks generally seem to have a few gigabytes of storage. My netbook has a few hundred gigabytes!). So, still not an ideal option, but unless some enterprising company resurrects the netbook concept, could be the best of a bad lot :(.

(I freely admit I’m in a small minority here… not many people need to run multiple servers on their computer while travelling, and not many of those that do tend to extend their business trips with nights in hostels. But that doesn’t stop it being annoying that something that met my needs perfectly is no longer being made 😉 ).

Sound Synthesis IV: Next Generation Sound Synthesis

Posted on January 22, 2013 by gcat

Last time we looked at (and listened to!) various methods of digital sound synthesis, beginning with the very primitive systems used by early computers, and ending with the sample-based methods in widespread use today. This time I’m going to talk about a new and very promising method currently in development.

What’s wrong with sample-based synthesis?

Our glockenspiel test sound already sounded pretty good using the sample-based method… do we really need a more advanced method? The answer is, although sample-based synthesis does work very well for certain instruments under certain conditions, it doesn’t work well the whole time.

Although it’s based on samples of real instruments, it’s still not fully realistic. Often the same sample will be used for different notes and different volumes, with the synth altering the frequency and amplitude of the sample as needed. But on a real piano (for example), the notes will all sound subtly different. A high C won’t sound exactly the same as a low C with its frequency increased, and pressing a key hard will result in a very different sound from pressing the same key softly – it won’t just be louder. Some of the better synths will use a larger number of samples in an attempt to capture these nuances, but the amount of data can become unmanageable. And that’s just considering one note at a time. In a real piano, when multiple notes are being played at the same time, the vibrations in all the different strings will influence each other in quite complex ways to create the overall sound.

It gets even worse for string and brass instruments. For example, changing from one note to another on a trumpet can sound totally different depending on how fast the player opens and closes the valves and it is unlikely a sample-based system will be able to reproduce all the possibilities properly without recording an unrealistically large number of samples. In some genres of music, the player may do things with the instrument that were never intended, such as playing it with a valve only part way open. A sample-based system would have no way of dealing with such unforeseen cases – if no-one recorded a sample for that behaviour, it can’t synthesise it.

The other problem with many of the synthesis methods is one of control. Even if it were possible to get them to generate the desired sound, it’s not always very obvious how to do it. FM synthesisers, for example, take a bewildering array of parameters, many of which can seem like “magic numbers” that don’t bear any obvious relation to the sound being generated. To play a note, sound envelopes and frequencies need to be set for every operator, the waveforms can be adjusted, and the overall configuration of the operators also needs to be set. Hardly intuitive stuff for people accustomed to thinking in terms of instruments and notes.

Physical Modelling Synthesis

A newer synthesis method has the potential to solve both the realism problem and the control problem, giving musicians virtual instruments that not only sound more realistic but are much easier to “play” and will correctly handle all situations, even ones that weren’t envisaged when the synth was designed. This is called Physical Modelling Synthesis, and it’s the basis for the project I’m working on just now.

The basic idea is that instead of doing something abstract that just happens to give the result you want (like e.g. FM synthesis), or “cheating” with recordings to give a better sounding result (like sample-based synthesis), you simulate exactly how a real instrument would behave. This means building a mathematical model of the entire instrument as well as anything else that’s relevant (the surrounding air, for example). Real instruments create sound because they vibrate in a certain audible way when they are played – whether that’s by hitting them, bowing them, plucking their strings, blowing into them, or whatever. Physical modelling synthesis works by calculating exactly how the materials that make up the instrument would vibrate given certain inputs.

How do we model an instrument mathematically? It can get very complex, especially for instruments that are made up of lots of different parts (for example, a piano has hundreds of strings, a sound board, and a box filled with air surrounding them all). But let’s start by looking at something simpler: a metal bar that could be, for example, one note of a glockenspiel.

To simulate the behaviour of the bar, we can divide it into pieces called elements. Then for each element we store a number, which will represent the movement of that part of the bar as it vibrates. To begin with, the bar will be still and not vibrating, so all these numbers will be zero:

We also need something else in this setup – we need a way to hear what’s going on, otherwise the whole exercise would be a bit pointless. So, we’ll take an output from towards the right hand end of the bar:

Think of this like a sort of “virtual microphone” that can be placed anywhere on our instrument model. All it does is take the number from the element it’s placed on – it doesn’t care about any of the other elements at all. At the moment the number (like all the others) is stuck at zero, which means the microphone will be picking up silence. As it should be, because a static, non-moving bar doesn’t make any sound.

Now we need to make the bar vibrate so that it does generate some sound. To do this, we will simulate hitting the bar with a beater near its left hand end:

What happens when the beater hits the bar? Essentially, it just makes the bar move slightly. So now, instead of all zeroes in our element numbers, we have a non-zero value in the element that’s just been hit by the beater, to represent this movement:

But the movement of the bar won’t stay confined to this little section nearest where the beater hit. Over time, it will propagate along the whole length of the bar, causing it to vibrate at its resonant frequency. After some short length of time, the bar might look like this:

and then like this:

then this:

As you can see, the value from the beater strike has “spread out” along the bar so now the majority of the bar is displaced in one direction or another. The details of how this is done depend on the material and exactly how the bar is modelled, but basically each time the computer updates the bar, the number in each box is calculated based on the previous numbers in all the surrounding boxes. (The values that were in those boxes immediately before the update are the most influential, but for some models numbers from longer ago come into play as well). Sometimes the boxes at the ends of the bar are treated differently from the other boxes – in fact, they are different, because unlike the boxes in the middle they only have a neighbouring box on one side of them, not both. There are various different ways of treating the edge boxes, and these are referred to as the model’s boundary conditions. They can get quite complex so I won’t say more about them here.

Above I said “some short length of time”, but that’s quite vague. We actually want to wait a very specific length of time, called the timestep, between updates to the bar. The timestep is generally chosen to match the sampling rate of the audio being output, so that the microphone can just pick up one value each time the bar is updated and output it. So, for a CD quality sample rate of 44100Hz, a timestep lasts 1/44100th of a second, or 0.0000226757 seconds.

If the model is working properly, the result of all this will be that the bar vibrates at its resonant frequency – just like the bar of a real glockenspiel. Every timestep, the “microphone” will pick up a value, and when this sequence of values is played back through speakers, it should sound like a metal bar being hit by a beater.

Here are the first 20 values picked up by the microphone: 0, 0, 0.022, -0.174, -0.260, 0.111, 0.255, 0.123, 0.426, 0.705, 0.495, 0.342, 0.293, 0.116, 0.016, 0.009, 0.033, -0.033, -0.312, -0.321, -0.030

and here’s a graph showing the wave produced by them:

To simulate a whole glockenspiel, we can model several of these bars, each one a slightly different length so as to produce a different note, and take audio outputs from all of them. Then if we hit them with our virtual beater at the right times, we can hear our test sample, this time generated by physical modelling synthesis:

pmsynth

I used a very primitive version of physical modelling synthesis to generate this sample, so it doesn’t sound amazing. I also used a bit of trial and error tweaking to get the bar lengths I wanted, so the tuning isn’t perfect. Both the project, and my knowledge of this type of synthesis, are still in fairly early stages just now! In the next section I’ll talk about what we can do do improve the accuracy of the models, and therefore also the quality of the sound produced.

Accuracy and model complexity

In our project we are mainly going for quality rather than speed. We want to try and generate the best quality of sound that we can from these models; if it takes a minute (or even an hour) of computer time to generate a second of audio, we don’t see that as a huge problem. But obviously we’d like things to run as fast as possible, and if it’s taking days or weeks to generate short audio samples, that is a problem. So I’ll say a bit about how we’re trying to improve the quality of the models, as well as how we hope to keep the compute time from becoming unmanageable.

A long thin metal bar is one of the simplest things to model and we can get away with using a one-dimensional row of elements (as demonstrated above) for this. But for other instruments (or parts of instruments), more complex models may be required. To model a cymbal, for example, we will need a two-dimensional grid of elements spaced across the surface of the cymbal. And for something big and complicated like a whole piano, we would most likely need individual 1D models for each string, a 2D model for the sound board, and a 3D model for the air surrounding everything, all connected and interacting with each other in order to get an accurate synthesis. In fact, any instrument model can generally be improved by embedding it in a 3D space model, so that it is affected by the acoustics of the room it is in.

There are also different ways of updating the model’s elements each timestep. Simple linear models are very easy and fast to compute and are sufficient for many purposes (for example, modelling the vibration of air in a room). Non-linear models are much more complicated to update and need more compute time, but may be necessary in order to get accurate sound from gongs, brass instruments, and others.

Inputs (for example, striking, bowing, blowing the model instruments) and how they are modelled can have an effect as well. The simplest way to model a strike is to add a number to one of the elements of the model for just a single timestep as shown in the example above, but it’s more realistic to add a force that gradually increases and then gradually decreases again across several timesteps. Bowing and blowing are more complicated. With most of these there is some kind of trade-off between the accuracy of the input and the amount of computational resources needed to model it.

2D models and especially 3D models can consume a lot of memory and take a huge number of calculations to update. For CD quality audio, quite a finely spaced grid is required and even a moderately sized 3D room model can easily max out the memory available on most current computers. Accurately modelling the acoustics of a larger room, such as a concert hall, using this method is currently not realistic due to lack of memory, but should become feasible within a few years.

The number of calculations required to update large models is also a challenge, but not an insurmountable one. Especially for the 3D acoustic models, the largest ones, we usually want to do the same (or very similar) calculations again and again and again on a massive number of points. Fortunately, there is a type of computer hardware that is very good at doing exactly this: the GPU.

GPU stands for graphics processing unit, and these processors were indeed originally designed for generating graphics, where the same relatively simple calculations need to be applied to every polygon or every pixel on the screen many, many times. In the last few years there has been a lot of interest in using GPUs for other sorts of calculations, for example scientific simulations, and now many of the world’s most powerful supercomputers contain GPUs. They are ideal for much of the processing in our synthesis project where the simple calculations being applied to every point in a 3D room model closely parallel the calculations being applied to every pixel on the screen when rendering an image.

Advantages of Physical Modelling Synthesis

You might wonder, when sample-based synthesis is getting so good and is so much easier to perform, why bother with physical modelling synthesis? There are three main reasons:

Sound quality. With a good enough model, physical modelling synthesis can theoretically sound just as good as a real instrument. Even with simpler models, certain instrument types (e.g. brass) can sound a lot better than sample-based synthesis.
Flexibility. If you want to do something more unusual, for example hitting the strings of a violin with the wooden side of the bow instead of bowing them with the hair, or playing a wind instrument with the valves half-open, you are probably going to be out of luck with a sample-based synthesiser. Unless whoever designed the synthesiser foresaw exactly what you want and included samples of it, there will be no way to do it. But physical modelling synthesis can – you can use the same instrument model and just modify the inputs however you want.
Ease of control. I mentioned at the beginning that older types of synthesiser can be hard to control – although they may theoretically be able to generate the sound you want, it might not be at all obvious how to get them to do it, because the input parameters don’t bear much obvious relation to things in the “real world”. FM is particularly bad for this – to play a note you might have to do something like: “Set the frequency of operator 1 to 1000Hz, set its waveform type to full sine wave, set its attack rate to 32, its decay rate to 18, its sustain level to 5 and its release rate to 4. Now set operator 2’s frequency to 200Hz, its attack rate to 50, decay rate 2, sustain level 14, release rate 3. Now chain the operators together so that 2 is modulating 1”. (In reality the quoted text would be some kind of programming language rather than English, but you get the idea). Your only options for getting the sound you want are likely to be trial and error, or using a library of existing sounds that someone else came up with by trial and error.

Contrast this with how you might play a note on a physical modelling synthesiser: “Hit the left hand bar of my glockenspiel model with the virtual beater 10mm from its front end, with a force of 10N”. Much better, isn’t it? You might still use a bit of trial and error to find the optimum location and force for the hit, but the model’s input parameters are a lot closer to things we understand from the real world, so it will be a lot less like groping around in the dark. This is because we are trying to model the real world as accurately as possible, unlike FM and sample-based synthesisers which are abstract systems attempting to generate sound as simply as possible.

Here’s a link to the Next Generation Sound Synthesis project website. The project’s been running for a year and has four years still to go. We’re investigating several different areas, including how to make good quality mathematical models for various types of instruments, how to get them to run as fast as possible, and also how to make them effective and easy to use for musicians.

Of course, whatever happens I doubt we will be able to synthesise the bassoon ;).

Sound Synthesis III: Early Synthesis Methods

Posted on January 9, 2013 by gcat

Digital Sound Synthesis

Before I delve into describing different types of synthesis, I should start with a disclaimer: I’m coming at this mainly from the angle of how old computers (and video game systems) used to synthesise sound rather than talking about music synthesisers, because that’s where most of my knowledge is. Although I have owned various keyboards, I don’t have a deep knowledge of exactly how they work as I’m more of a pianist than a keyboard player really. There is quite a bit of overlap between methods used in computers and methods used in musical instruments though, especially more recently.

To illustrate the different synthesis methods, I’m going to be using the same example sound over and over again, synthesised in different ways. It’s the glockenspiel part from the opening of Sonic Triangle‘s sort-of Christmas song “It Could Be Different”. For comparison to the synthesised versions, here it is played (not particularly well, but you should get the idea!) on a real glockenspiel:

glockenspiel

(In fact, in the original recording of the song, it isn’t a real glockenspiel. It’s the sample-based synthesis of my Casio keyboard… there’ll be more about that sort of synthesis later).

If you have trouble hearing the sounds in this post, try right clicking the links, saving them to your hard drive and opening them from there. Seriously, I can’t believe that in 2013 there still isn’t an easy way of putting sounds on web pages that works on all major browsers. Grrrr!

Primitive Methods

As we saw last time, digital sound recordings (which include CDs, DVDs, and any music files on a computer) are just very long lists of numbers that were created by feeding a sound wave into an analogue-to-digital converter. To play them back, we feed the numbers into a digital-to-analogue converter and then play back the resulting sound using a loudspeaker. But what if, instead of using a list of numbers that was previously recorded, we used a computer program to generate a list of numbers and then played them back in the same way? This is the basis of digital sound synthesis – creating entirely new sounds that never existed in reality.

Very old (1980s) home computers and games consoles tended to only be able to generate very primitive, “beepy” sounding music. This was because they were generating basic sound wave shapes that aren’t like anything you’d get from a real musical instrument. The simplest of all, used by a lot of early computers, is a square wave:

square wave sound

Another option is the triangle wave, with a slightly softer sound:

triangle wave sound

The sound could be improved by giving each note a “shape” (known as its envelope), so that a glockenspiel sound, for example, would start loud and then die away, like a real glockenspiel does:

triangle wave with envelope sound

None of these methods sound particularly nice, and it’s hard to imagine any musician using them now unless they were deliberately going for a retro electronic sort of effect. But they have the advantage of being very easy to synthesise, requiring only a simple electronic circuit or a few lines of program code. (I wrote a program to generate the sound samples in this section from scratch in about half an hour). The square wave, for example, only has two possible levels, so all the computer has to do is keep track of how long to go before switching to the other level. The length of time spent on each level determines the pitch of the sound produced, and the difference in height between the levels determines the volume.

FM Synthesis

I remember being very excited when we upgraded from our old ZX Spectrum +3, which could only do square wave synthesis, to a PC and a Sega Megadrive that were capable of FM (Frequency Modulation) Synthesis. They could actually produce the sounds of different instruments! Looking back now, they didn’t sound very much like the instruments they were supposed to, but it was still a big improvement on square waves.

FM synthesis involves combining two (or sometimes more) waves together to produce a single, more complex wave. The waves are generally sine waves and the combination process is called frequency modulation – it means the frequency of one wave (the “carrier”) is altered over time in a way that depends on the other wave (the “modulator”) to produce the final sound wave. So, at low points on the modulator wave, the carrier wave’s peaks will be spread out with a longer distance between them, while at the high points of the modulator they will be bunched up closer together, like this:

Some FM synthesisers can combine more than two waves together in various ways to give a richer range of possible sounds.

Here’s our glockenspiel snippet synthesised in FM:

fm sound

(In case you’re curious, this was done using DOSBox, which emulates the Yamaha OPL-2 FM synthesiser chip used in the old Adlib and SoundBlaster sound cards common in DOS PCs, and the Allegro MIDI player example program. Describing how to get an ancient version of Allegro up and running on a modern computer would make a whole blog post in itself, but probably not a very interesting one).

It’s certainly a step up from the square wave and triangle wave versions. But it still sounds unnatural; you would be unlikely to mistake it for a real glockenspiel.

FM synthesis is a lot more complicated to perform than the older primitive methods, but by the 90s FM synthesiser chips were cheap enough to put in games consoles and add-in sound cards for PCs. Contrary to popular belief, they are not analogue (or hybrid analogue-digital) synths; they are fully digital devices apart from the final conversion to analogue at the end of the process.

In case you were wondering, this is pretty much the same “frequency modulation” process that is used in FM radio. The main difference between the two is that in FM radio, you have a modulator wave that is an audio signal, but the carrier wave is a very high frequency radio wave (up in the megahertz, millions-of-hertz range). In FM synthesis, both the carrier and modulator are audio frequency waves.

Sample-based Synthesis

Today, when you hear decent synthesised sound coming from a computer or a music keyboard, it’s very likely to be using sample-based methods. (This is often referred to as “wavetable synthesis”, but strictly speaking this term refers to only a quite specific subset of the sample-based methods). Sample-based synthesis is not really true synthesis in the same way that the other methods I’ve talked about are – it’s more a clever mixture of recording and synthesis.

Sample-based synthesis works by using short recordings of real instruments and manipulating and combining them to generate the final sound. For example, it might contain a recording of someone playing middle C on a grand piano. When it needs to play back a middle C, it can play back the recording unchanged. If it needs the note below, it will “stretch out” the sample slightly to increase its wavelength and lower its frequency. Similarly, for the note above it can “compress” the sample so that its frequency increases. It can also adjust the volume if the desired note is louder or quieter than the original recording. If a chord needs to be played, several instances of the sample can be played back simultaneously, adjusted to different pitches.

This synthesis method is not too computationally intensive; sound cards capable of sample-based synthesis (such as the Gravis Ultrasound and the SoundBlaster AWE 32/64) became affordable in the mid 90s and today’s computers can easily do it in software. Windows, for example, has a built-in sample-based synthesiser that is used to play back MIDI sound if there isn’t a hardware synth connected. Sound quality can be very good for some instruments – it is typically very good for percussion instruments, reasonable for ensemble sounds (like a whole string section or a choir), and not so good for solo string and wind instruments. The quality also depends on how good the samples themselves are and how intelligent the synth is at combining them.

Here’s the glockenspiel phrase played on a sample-based synth (namely my Casio keyboard):

sample based

This is a big step up from the other synths – this time we have something that might even be mistaken for a real glockenspiel! But it’s not perfect… if you listen carefully, you’ll notice that all of the notes sound suspiciously similar to each other, unlike the real glockenspiel recording where they are noticeably different.

Next time I’ll talk about the limitations of the methods I’ve described in this post, and what can be done about them.

Sound Synthesis II: Digital Recording

Posted on January 3, 2013 by gcat

Digital Recording

Things changed with the advent of compact discs, and later DVDs and MP3s as well. Instead of storing the continuously changing shape of the sound wave, these store the sound digitally.

What do we mean by digitally? It means the sound is stored as a collection of numbers. In fact, the numbers are binary, which means only two digits are allowed – 0 and 1. The music on a CD, or in an MP3 file, is nothing more than a very long string of 0s and 1s.

How do you get from the shape of the sound to a string of numbers? After all, the sound wave graphics we saw last time looks very different from 1000110111011011010111011000100. First of all, you sample the sound signal. That means you look at where it is at certain individual points in time, and ignore it the rest of the time. Imagine drawing the shape of a sound wave on a piece of graph paper like this:

To sample this signal, we can look at where the signal is each time it crosses one of the vertical lines. We don’t care what it’s doing the rest of the time – only its intersections with the lines matter now. Here’s the same sound, but instead of showing the full wave, we just show the samples (as Xs):

To simplify things further so we can stick to dealing with whole numbers, we’re also going to move each sample to the nearest horizontal grid line. This means that all the samples will be exactly on an intersection where two of the grid lines cross:

So far, so good. We have a scattering of Xs across the graph paper. Hopefully you can see that they still form the shape of the original sound wave quite well. From here, it’s easy to turn our sound wave into a stream of numbers, one for each sample. We just look at each vertical grid line and note the number of the horizontal grid line where our sample is:

The wave we started with is now in digital form: 5, 9, 5, 6, 7, 1, 2, 6, 4, 6. It’s still in ordinary decimal numbers, but we could convert it to binary if we wanted to. (I won’t go into details of how to convert to binary here, but if you’re curious, there are plenty of explanations of binary online – here’s one). We can record this stream of numbers in a file on a computer disk, on a CD, etc. When we want to play it back, we can reverse the process we went through above to get back the original sound wave. First we plot the positions of the samples onto graph paper:

And now we draw the sound wave – all we have to do is join up our samples:

Voila! All ready to be played back again.

This might look very spiky and different from the original smooth sound wave. That’s because I’ve used a widely spaced grid with only a few points here so you can see what’s going on. In real digital audio applications, very fine grids and lots of samples are used so that the reconstructed wave is very, very close to the original – to show just one second of CD quality sound, you would need a grid with 65,536 horizontal and 44,100 vertical lines!

(In electronics, the device that turns an analogue sound wave into samples is called an analogue to digital converter, and its cousin that performs the inverse task is a digital to analogue converter. As you probably guessed, it’s not really done using graph paper).

But why?

At this point you may be wondering, why bother with digital recording? It seems like we just went through a complicated process and gained nothing – in fact, we actually lost some detail in the sound wave, which doesn’t look quite the same after what it’s been through! There are several advantages to digital recording:

Digital recordings can be easily manipulated and edited using a computer. Computers (at least all the ones in common use today) can only deal with digital information – anything analogue, such as sounds and pictures, has to be digitised before they will work with it. This opens up a huge range of possibilities, allowing much more sophisticated effects and editing techniques than could be accomplished in the analogue domain. It also allows us to do clever things like compressing the information so it takes up less space while still sounding much the same (this is what the famous MP3 files do).
I noted above that we lost a bit of detail in our sound wave when we converted it to digital and then converted it back. However, in real life situations digital recordings generally give much better sound quality than analogue recordings. This is because the small inaccuracies introduced in the digitisation process are usually much smaller and less noticeable than the background noise that inevitably gets into analogue recording and playback equipment no matter how careful you are. Digital is more or less immune to background noise for reasons I’ll explain shortly.
Digital recordings can be copied an unlimited number of times without losing any quality. This is closely related to the point above about sound quality. If you’re old enough to have copied records or cassettes onto blank tapes, or taped songs off the radio, you may have noticed this in action. The copy always sounds worse than the original, with more background noise. If you make another copy from that copy instead of from the original, it will be worse still. But it isn’t like that with digital recording – if you copy a CD to another CD, or copy an MP3 file from one computer to another, there is no loss of quality – the copy sounds exactly like the original, and if you make another copy from the copy, it will also sound exactly like the original. (This isn’t just a case of the loss in quality being so small you can’t hear it – there genuinely is no loss whatsoever. The copies are absolutely identical!).

Notes on background noise

I mentioned above that digital recordings are more or less immune to background noise and that’s one of their big advantages. But first of all, what is background noise, where does it come from, and what does it do to our sound signals?

Background noise is any unwanted interference that gets into the sound signal at some point during the recording or playback process. It can come from several different sources – if the electrical signal is weak (like the signal from a microphone or from a record player’s pick-up), it can be affected by electromagnetic interference from power lines or other devices in the area. If there is dust or dirt on the surface of a record or tape, this will also distort the signal that’s read back from it.

There is no getting away from background noise, it will always appear from somewhere. If we have a vinyl record with a sound signal recorded onto it that looks like this:

by the time it gets played back through the speakers, noise from various sources will have been added to the original signal and it might look more like this:

Once the noise is there, it’s very difficult or impossible to get rid of it again, mainly because there’s no reliable way to tell it apart from the original signal. So ideally we want to minimise its chances of getting there in the first place. This is where digital recording comes in. Let’s say we have the same sound signal recorded onto a CD instead of a vinyl record. Because it’s in digital form, it will be all 0s and 1s instead of a continuously varying wave like on the vinyl. So the information on the CD will look something like this:

This time there are only two levels, one representing binary 0 and the other binary 1.

There will still be some noise added to the signal when it gets read back from the CD – maybe there is dust on the disc’s surface or electrical interference getting to the laser pick-up. So the signal we get back will look more like this:

But this time the noise doesn’t matter. As long as we can still tell what is meant to be a 0 and what is a 1, small variations don’t make any difference. In this case it’s very obvious that the original signal shape was meant to be this:

So, despite the noise, we recovered exactly the original set of samples. We can pass them through the digital to analogue converter (DAC) and get back this:

a much more accurate version of the original sound wave than we got from the analogue playback. Although the noise still got into the signal we read from the CD, it’s disappeared as if by magic and doesn’t affect what we hear like it did with the record.

(Of course, digital recording isn’t completely immune to noise. If the noise level was so high that we could no longer tell what was meant to be a 0 and what was a 1, the system would break down, but it’s normally easy enough to stop this from happening. Also, we can’t prevent noise from getting into the signal after it’s converted back to analogue form, but again this is a relatively small problem as the majority of the recording and playback system works in digital form).

Does digital recording really sound better?

Not everyone thinks so. A lot of people say they prefer the sound of analogue recordings, often saying they have a “warmer” sound compared with the “colder” sound of digital. In my opinion, yes there is a difference, but digital is more faithful to the original sound – the “warmth” people talk about is actually distortion introduced by the less accurate recording method! It’s absolutely fine to prefer that sound, in the same way that it’s absolutely fine to prefer black and white photography or impressionist paintings even though they’re less realistic than a colour photograph or a painting with lots of fine detail.

“Ah”, you might say. “But surely a perfect analogue recording would have to be better than a digital recording? Because you’re recording everything rather than just samples of it”. Technically this is true… but in reality (a) there’s no such thing as a perfect analogue recording because there are so many ways for noise to get in, and (b) at CD quality or better, the loss of information from digitising the sound is miniscule, too small for anyone to be able to hear. Double-blind tests have been done where audio experts listened to sounds and had to determine whether the sound had been converted to digital and back or not. No-one was able to reliably tell.

Phew! That was longer than I meant it to be. That’s the background… next time I really will start on actual sound synthesis, I promise!

Sound Synthesis I: How Sound Recording Works

Posted on December 23, 2012 by gcat

Hello, and welcome to the first of several blog entries inspired by one of my projects, Project Noah. Actually that’s just my code name – its real name is Next Generation Sound Synthesis. I actually get paid for working on this one!

As the name suggests, the project is about new ways of synthesising sound, creating more realistic sounding digital instruments and better acoustic models. I think it’s a pretty interesting area, and the approach being taken (physical modelling synthesis) shows a lot of promise. But before I get onto that, I’d like to go back to basics a bit and talk about computer synthesis in general, giving some examples of the different ways it’s been done over the years, what they sound like, their strengths and weaknesses, etc. I’ll be talking mainly from a computer programmer’s perspective rather than a musicians, so my examples will draw mainly from the sound chips and software used to make sounds in computers and games consoles rather than from music synthesizers. (Although I do play music keyboards, I don’t know a great deal about the technical side of them, especially not the earlier ones).

In fact, before I even start on that, I’m going to go even further back to basics and talk about how sound recording works in this first entry. (If it’s not immediately clear how that’s relevant to synthesis, I hope it will become clearer by the end).

Recording Sound

Sound is vibrations in the air that we can hear when they are picked up by our ear drums. To record sound and be able to play it back later, we need some means of capturing and storing the shape of those vibrations as they happen – and also a means of turning the stored shapes back into vibrations again so that they can be heard.

The earliest methods of sound recording didn’t even rely on any electronics – they were entirely mechanical. A diaphragm would pick up the vibrations from the air, then a needle connected to the diaphragm would etch out a groove in some soft medium – initially wax cylinders, later flat vinyl discs. The cylinder or disc would be turned by hand or by clockwork. The groove’s shape would correspond to the shape of the sound waves as they changed over time.

This isn’t actually a mechanical gramophone, but it is the oldest one I could easily get hold of. It used to be my Granny’s.

To play back the sound, the process was reversed; the needle was made to run along the groove, transmitting its shape to the diaphragm, which would vibrate in the right way to recreate the original sound (more or less – the quality of these early systems left a lot to be desired).

It’s worth pausing for a moment to say something about how the shapes of sound wave relate to what they actually sound like to us. First of all, and maybe not surprisingly, a sound wave with bigger variations (larger peaks and troughs) sounds louder than one with smaller variations. So this:

sounds exactly the same as this:

except the first one is a lot louder.

So the height of the peaks in the wave (often called the amplitude) determines the loudness, more or less. The pitch (that is, whether the sound is high like a flute or low like a tuba) depends on how close together the peaks are. When there are a lot of peaks in quick succession like this:

the sound is high pitched. When there aren’t so many, like this:

the sound will be deeper. This is called the frequency of the sound wave. Frequency is measured in Hertz (abbreviated to Hz), which is simply the number of peaks the wave has per second. Humans can hear frequencies in the range of about 20Hz up to 20,000Hz, but are much better at hearing sounds around 1,000Hz than sounds at either of those extremes. Also, ability to hear high frequencies tends to tail off quite dramatically with age, so it’s unlikely adults will be able to hear all the way up to 20,000Hz (20kHz).

Real sound waves (such as speech, music, everyday noises) are usually more complex than the ones I’ve shown above and are made up of a whole mixture of different frequencies and amplitudes, which also vary over time. This makes things more interesting from the perspective of synthesising sounds.

Electronic Recording

The simple mechanical recording system was improved with the advent of electronics. Electronic recording was more complex but resulted in much better sound quality. In the electronic system, a microphone is used to turn the sound vibrations into electrical signals whose voltage varies over time in the same shape as the sound waves. Having the sound in electronic form opens up lots more possibilities – for example, it can be boosted by electronic amplifiers, allowing a stronger signal to be stored, and the sound to be played back at much louder volumes. It can also be much more easily mixed with other sound signals, very useful for editing recordings.

The first electronic systems still stored the sound as a groove cut in a vinyl disc, just as the original mechanical systems had. And as in the mechanical systems, the groove was the same shape as the original sound waves – there was no fancy encoding or conversion going on. Later, sound was also stored as a varying magnetic field on magnetic tape. The variations in magnetic field strength, like the shape of the grooves, corresponded exactly to the shape of the sound being recorded. This is known as analogue recording.

Tune in next time for lots of information about the next big innovation in sound recording: digital recording!

Projects Update

Posted on August 2, 2012 by gcat

(This post is mainly an attempt to give myself a gentle kick up the bum towards doing something about all this stuff).

So… it’s nearly 3 months since I posted about my personal projects, so it must be time for an update. Generally I haven’t got as much done on them as I’d hoped; travelling the world and playing with geek-toys has taken up a lot of my time over the past few weeks. But looking down my list and thinking about what I’ve achieved, I can see that it hasn’t been quite as bleak as I feared. And now I have Luna and a whole month (well, nearly) of not travelling anywhere at my disposal, I should be able to make some more progress.

Projects Bubble, Everything and Chippy are not really my responsibility to keep on track. There was a tentative plan to do something on Chippy back in June, but it was scuppered by a very full schedule and a hair dye disaster. It would be nice if more was happening on them (especially Bubble), but I’m not going to beat myself up over the fact that it hasn’t yet.

Project Hohoho: the funding campaign is now over and we raised a respectable amount :). First actual filming commences soon, though I probably shouldn’t say any more about it just now as the plans are still being kept under a certain sandwich-like food item (watch the pitch video!).

Project Noah is one of my major paid work projects. It’s coming along very nicely (apart from a slight setback involving a crucial building being full of asbestos and possibly having to be evacuated for an extended period while they get rid of it). I have an idea for a blog entry I want to post about this as I do think it’s really interesting stuff… it will take a bit of preparation though.

Project Bits: This is maybe the one I feel is most important but it seems slow to get started. I did a bit of writing and a bit of general planning work and research. It’s become more and more ambitious in my mind, which is probably a good thing in that it might help to differentiate it from anything similar that’s out there, but a bad thing in terms of making it less likely to actually get finished. I definitely need to organise it and work out what exactly I want to do.

Project Buster: not much progress. I downloaded a whole load of stuff for it onto my new computer but haven’t had time to do much with it yet. In my head it’s starting to become a bit more concrete, and form tentative links with Projects IOM and Fantasy World.

Project IOM: I was sort of hoping for some nice summer evenings as they would have given me a chance to do more of this. So far I’ve been disappointed :(. Let’s hope August and September are nicer.

Project X-Ray: haven’t done much, but it’s sort of linking up in my head with Project Fantasy World, which is going a bit better… and I have a more definite (but probably impossible) idea for it.

Project Megadroid: this one actually is going OK, after a quiet spell. Getting the new phone has helped it along rather a lot. So has something else that I may blog about separately.

Project History: making a lot of progress on this lately, again after a quiet spell. The first thing that needs to be done on it is quite a laborious task but the end is now in sight!

Projects Classical, New Leaf and Tridextrous haven’t got far. New Leaf really shouldn’t be hard to get finished but other things keep distracting me.

Project Fantasy World: this was possibly the vaguest idea of them all, but it’s taken shape in my head and started to connect with X-Ray, Buster and Bits. I’ve been playing with some software that could help with it and getting further than I expected to.

Project Bonkers: … um, yeah.

I do feel a bit more inspired now :). Hopefully next time I post about one of these it won’t be in quite such vague and meaningless terms!

New York, New York…

Posted on June 12, 2012 by gcat

… so good he wrote blog entries about it twice (as the song goes… or something) :).

Day 3

Today was the day of my talk! I also, somewhat unexpectedly, ended up on the panel for a discussion afterwards. I’m glad this was unexpected, if I’d known it was coming I would have dreaded it and probably ended up sitting there rooted to the spot and saying nothing, whereas I actually did manage to contribute something this way. It all seemed to go well and get some interest from the audience. But of course the most important thing was the gifts afterwards!

I couldn’t work out what on earth the blue thing was at first. I thought maybe a doorstop, but it seemed too flimsy for that. When I showed it to Laura back home she suggested maybe it was a phone holder. It turns out the blue stuff does indeed have a weird texture that makes things stick on it rather than falling down. Anyway, whatever it’s supposed to be, it makes a great Raspberry Pi stand.

(Unfortunately when the cables are plugged in they tend to pull it off the stand again).

The afternoon sessions were about biology, and I found two of them really fascinating, particularly the one about emulating a fruit fly’s brain including how it processes vision (really!). I’m interested by how many parallels there seem to be between computing and biology, and how nature seems to do a lot better than us at some of them. It’s an area I’d like to learn more about, though realistically I’ll probably never find the time. I also managed to finally fit in a Skype call back home (which was more difficult than it should have been, thanks to Virgin Media having managed to completely balls up our internet connection and Laura not being able to get a data signal, so I ended up having to buy Skype credit and call the landline instead).

Our evening consisted of a trip on the Staten Island Ferry (cue yet more photos)…

… followed by a nice, very duck-centric Chinese meal in Chinatown. It was interesting to see that even in Chinatown there was at least one McDonalds, with the sign written in Chinese no less, though I still preferred the very sparkly one that looked like a Broadway theatre by Times Square. It was also interesting to see that some things are the same all over the world. BMW drivers, for example. Most drivers were resigned to sitting in the big queue that stretched off round the corner and waiting for it to move. Not so BMW guy… he came haring up to the end of the queue, slammed on his brakes, leaned on the horn for a good ten seconds solid, and finally proceeded to get out of his car and nearly have a fist fight with someone who’d been walking past and took exception to the noise.

After the meal some people went off to find a pub, but tiredness had caught up with me again so I got the subway back to the hotel, wanting to leave some energy for…

Day 4

Sightseeing!

It was a nice day for it too. I’d already decided I wanted to see Central Park so after checking out of my hotel I set off up there.

It was even bigger than it had looked on the map. I found my first ever international geocache (leaving a travel bug that was on its way to Canada), and saw some of the locations from Home Alone 2. After that I did a bit of souvinir buying for people back home, then wondered what else to do. It seemed a shame to spend too long indoors on a day like this. But after flicking through my guidebook I found this…

… the High Line. An old elevated railway line that had been converted into a sort of linear park and walkway. I really liked it up there and walked further than I’d intended to. Lots of other people were out and about enjoying the sun and it was a nice atmosphere. After walking in the heat for a while I felt tired and thirsty and retired to an Irish bar to spend my last hour in New York reading the start of Game of Thrones.

So yeah… I liked New York, probably more than I expected to. I have Sweden and Berlin coming up in the next few weeks, so seems like I’m hardly going to be in the country this month!

GCat's World of Stuff

Category Archives: Work