Raw Keyboard Input in Linux

In the course of porting my console emulators to the Raspberry Pi, I found the need to get keyboard input in such a way that I could detect exactly when a key was pressed and released, and also detect multiple keys being pressed simultaneously. This is a pretty common need for games where you often want to allow the player to run and jump at the same time, for example, but it’s not always very obvious how to do it. In this case I found myself in the barren world of the Linux command line, without any friendly libraries to do it for me. (Actually, SDL was there but for some reason I couldn’t coax it into giving me any keyboard input). I eventually found out, after a fair bit of googling, how to switch the keyboard into “raw” mode – in this mode your program gets the individual scan codes from the keyboard so you can tell exactly what’s going on.

Anyway, I didn’t find any handy ready-made code snippets to do the whole process, so here’s my one. (Actually, I did find this one in assembly language, but that’s not particularly useful if you want to code in C… and even less useful if you’re on a non-Intel platform like the Raspberry Pi!)

First the code to switch into raw mode:

#include "unistd.h"
#include "linux/kd.h"
#include "termios.h"
#include "fcntl.h"
#include "sys/ioctl.h"

static struct termios tty_attr_old;
static int old_keyboard_mode;

int setupKeyboard()
    struct termios tty_attr;
    int flags;

    /* make stdin non-blocking */
    flags = fcntl(0, F_GETFL);
    flags |= O_NONBLOCK;
    fcntl(0, F_SETFL, flags);

    /* save old keyboard mode */
    if (ioctl(0, KDGKBMODE, &old_keyboard_mode) < 0) {
	return 0;

    tcgetattr(0, &tty_attr_old);

    /* turn off buffering, echo and key processing */
    tty_attr = tty_attr_old;
    tty_attr.c_lflag &= ~(ICANON | ECHO | ISIG);
    tty_attr.c_iflag &= ~(ISTRIP | INLCR | ICRNL | IGNCR | IXON | IXOFF);
    tcsetattr(0, TCSANOW, &tty_attr);

    ioctl(0, KDSKBMODE, K_RAW);
    return 1;

If the first ioctl fails, it probably means the program isn’t running under a standard console (e.g. it’s running in an xterm, or remotely over ssh), and this method won’t work, so the function returns 0.

Now the reciprocal code to switch out again. It’s very important to call this before your program exits, otherwise you leave your user’s terminal in a screwed up state where they won’t be able to type anything, or even press a magic key combination to reboot or switch terminals!

void restoreKeyboard()
    tcsetattr(0, TCSAFLUSH, &tty_attr_old);
    ioctl(0, KDSKBMODE, old_keyboard_mode);

While the keyboard’s in raw mode, you can read from the stdin file descriptor (number 0), and you will get scan codes:

void readKeyboard()
    char buf[1];
    int res;

    /* read scan code from stdin */
    res = read(0, &buf[0], 1);
    /* keep reading til there's no more*/
    while (res >= 0) {
	switch (buf[0]) {
	case 0x01:
            /* escape was pressed */
        case 0x81:
            /* escape was released */
        /* process more scan code possibilities here! */
	res = read(0, &buf[0], 1);

You can find lists of the scan codes online in various places. Some keys send multiple bytes in sequence when pressed. They may differ for different keyboard types and different countries, which is one of the reasons why it’s not a good idea to do this unless you really need to – you are basically disabling all the hard work that the Linux kernel normally does for you to standardise key codes and give them to your program in a nice form.

Have fun ūüėČ

RC-4 Encryption in Python

Sometimes it would be nice to have some basic cryptography for protecting data in Python but without having to faff about installing extra libraries or coding stuff in C. Unfortunately the standard Python library is somewhat lacking when it comes to encryption; it has various hashing functions¬†but no actual ciphers unless you install an additional library such as pycrypto. This introduces extra dependencies to your program so it’s annoying if you want to distribute it or even just run it on more than one machine.

So, I did a quick implementation of the simple RC-4 stream cipher in Python for a project I’m working on, which you can find below. It’s not going to win any speed awards, but it should be fast enough for some purposes and it avoids the need to install any less portable C libraries. I wouldn’t advise using this to secure an online banking system or anything like that, and I’m certainly not taking responsibility for its security if you do, but if you just want a reasonably secure way of protecting some data, it might be useful.

It’s very simple to use. Just call init(key) to initialise the key structure from a string. This function returns a byte array that you can then pass to the rc4 function to encrypt or decrypt a string of data.

(Note: because RC-4 is a stream cipher, you shouldn’t use the same key to encrypt more than one set of data as this could seriously compromise the security. In my application I get around this by generating a random “nonce” (just a string of 8 random bytes) by calling os.urandom(8) each time I encrypt something, adding the nonce to the key, and storing the nonce along with the encrypted data).

# do the key schedule. return a byte array for the main algorithm
# k can be a list of numbers or a string
def init(k):
    # if key is a string, convert it to array of ASCII codes
    if isinstance(k, str):
        k = map(ord, k)
    # create and initialise S
    s = bytearray(256)
    i = 0
    while i < 256:
        s[i] = i
        i = i + 1
    # process S using the key data
    j = 0
    kl = len(k)
    i = 0
    while i < 256:
        j = (j + s[i] + k[i % kl]) & 0xff
        s[i], s[j] = s[j], s[i]
        i = i + 1
    return s

# encrypt/decrypt a string using RC4
def rc4(s, val):
    l = len(val)
    buf = bytearray(l)
    i = 0
    j = 0
    idx = 0
    while idx < l:
        i = (i + 1) & 0xff
        j = (j + s[i]) & 0xff
        s[i], s[j] = s[j], s[i]
        k = s[(s[i] + s[j]) & 0xff]
        buf[idx] = (ord(val[idx])) ^ k
        idx = idx + 1
    return str(buf)

A Minimalist wxPython Password Dialogue

I think from time to time I’ll post useful little chunks of code here when they come up in a project I’m doing and might be of more general use. Here’s the first one; a very simple password input dialogue in wxPython:

# Very minimalist password entry dialogue
class PasswordDialog(wx.Dialog):
    def __init__(self, parent, id=-1, title="Enter password"):
        wx.Dialog.__init__(self, parent, id, title, size=(320, 160))
        self.mainSizer = wx.BoxSizer(wx.VERTICAL)
        self.buttonSizer = wx.BoxSizer(wx.HORIZONTAL)
        self.label = wx.StaticText(self, label="Enter password:")
        self.field = wx.TextCtrl(self, value="", size=(300, 20), style=wx.TE_PASSWORD|wx.TE_PROCESS_ENTER)
        self.okbutton = wx.Button(self, label="OK", id=wx.ID_OK)
        self.cancelbutton = wx.Button(self, label="Cancel", id=wx.ID_CANCEL)
        self.mainSizer.Add(self.label, 0, wx.ALL, 8 )
        self.mainSizer.Add(self.field, 0, wx.ALL, 8 )
        self.buttonSizer.Add(self.okbutton, 0, wx.ALL, 8 )
        self.buttonSizer.Add(self.cancelbutton, 0, wx.ALL, 8 )
        self.mainSizer.Add(self.buttonSizer, 0, wx.ALL, 0)
        self.Bind(wx.EVT_BUTTON, self.onOK, id=wx.ID_OK)
        self.Bind(wx.EVT_BUTTON, self.onCancel, id=wx.ID_CANCEL)
        self.Bind(wx.EVT_TEXT_ENTER, self.onOK)
        self.result = None

    def onOK(self, event):
        self.result = self.field.GetValue()

    def onCancel(self, event):

And here’s how to use it:

dlg = PasswordDialog(frame)
# dlg.result is now set to 'None' if the user cancelled, else to
# the password string

Enjoy ūüôā

Porting an RPG to Flash: Part II – Starting the data conversion

Last time, I finished by going through all the Etirites data files and considering how I could convert them to suitable formats for Flash to read. I’m itching to get started now. Let’s go write a graphics converter.

LSG to PNG conversion

First step: get libpng installed on my machine. (I’m using the MinGW compiler on Windows and writing the conversion utilities in C. A scripting language like Python might have been a better choice for some of them, but right now I’m more familiar with C so I can probably be just as productive with it). It depends on Zlib so compile and install that first. That was easy… everything seemed to just work.

Second step: find some example code for using libpng. I basically just want to be able to write an image in PNG format. Found some nice simple code on the web courtesy of Guillaume Cottenceau, and by modifying that I mange to write my first 32-bit PNG file. That was easy, too!

Now for the LSG loading. I seem to remember there were 2 slightly different versions of the LSG format – the later one could handle bigger graphics, more graphics, etc., but looking at the Etirites code it only seems to handle the first version so that’s all we’ll need to do. Though I might want to add v2 support later if I ever feel like converting one of my other old projects.

OK, that’s the LSG being loaded now – I can see a weird looking greyscale version of the first boss in the output PNG now! Fixing it up to use the Etirites palette, he appears in full colour. But as I suspected, very tiny looking.

Let’s scale him up, just doubling each pixel in both dimensions to begin with.

Eww! As I also suspected, far too pixellated. A quick search on Wikipedia for pixel art scaling algorithms and I’ve knocked together an EPX (AKA Scale2x or AdvMAME2x) scaler. It’s not the most advanced one but it is easy to code and seems to work quite well. Much better!

Now to make it able to handle multiple LSG frames and put them all into one PNG. That didn’t take long and now I have a converter that can convert all of the Etirites LSG files. I soon have a pile of them in PNG format. They look much nicer with the up-scaling as well… “Return of the Etirites… Now in HD!” ūüėÄ

Other graphics conversion

The Gimp can indeed open the TIFFs, but having seen what the up-scaling did to¬†the LSGs, I want to do it to the other graphics as well. Looks like I’ll need¬†to write another converter. Oh, and one for the font as well.

Those weren’t too hard to do. The map looks nice up-scaled. Not so sure about¬†the font or the swords, but they should be ok at least. So that’s all the¬†graphical data converted to Flash-friendly PNG format now. Not bad for a¬†day’s work.

Music conversion

Converting the music by playing the game in DosBox wasn’t as easy as I thought.¬†For a start, there’s no “sound test” or anything similar so getting all the¬†tunes would mean playing right to the end – even with the cheat mode that¬†would take time. Also, some of the tunes are quite long and don’t always play¬†to their full length in the game.

So what to do? There were several options…

  1. Find the original MIDI files. The tunes were originally created as MIDI before being converted to LSM.
  2. Hack the game source code to make it play any tunes we like. This would mean¬†setting up a DOS compiler in order to rebuild the game. It probably wouldn’t be too hard to get trusty old DJGPP installed again, but my custom structured assembler tool could be challenging to find and get working.
  3. Hack the game binary to play any tunes we like. This isn’t necessarily as¬†insane as it probably sounds… for example, if I could find the call that plays¬†the menu tune (which shouldn’t be too hard with access to the source code), I could probably get it to play any other tune on the menu by¬†changing one single byte.
  4. Write a quick and simple Windows LSM player program. Though given that I’ve¬†forgotten almost everything I used to know about MIDI programming, this¬†probably wouldn’t be a very quick or simple option.
  5. Write a reverse converter to convert the LSM files back to MIDI. Likely to be difficult as the LSM files are missing quite a bit of information that the MIDIs should have.

All of those apart from the first could be at least a little tricky, so naturally I was glad to find all the original MIDI files in a long-forgotten backed up directory on my external hard drive. Woo hoo! The tempos are all wrong but that should be easily fixed by playing with them in a MIDI studio.


Porting an RPG to Flash: Part I – Introduction

(Originally written on 7th April 2010, mostly)

Coming up for two years ago now, I decided to make a Flash version of one of my old games, an RPG called Return of the Etirites. And for some reason I decided to document the process in a sort of blog-like style, even though back then I didn’t have a blog to publish it on. But now I do, and it might be of interest to someone… so here it is. I’ll talk a bit about how the original game from the old DOS days was put together, and how I updated it to run in a web browser.

What is Etirites?

Return of the Etirites was my attempt at an old-school 2D RPG, very much influenced by Mystic Quest on the Gameboy (I loved that game!). I originally wrote it for my brother’s 13th birthday back in 1998.

It ran on MS-DOS. The original version still works very well in DOSBox… in fact it still ran on Windows at least up until XP.

Disclaimer: I designed it and wrote the script in maths class when I was 16 and it shows. I also drew all the graphics myself despite having no real talent for it and that shows too. Even by the not-very-high standards of ancient 8-bit-style RPGs, the plot is laughable. Some of the characters and monsters and locations are loosely based on real people and places, some not. I’ll leave you to guess which are which.

If you still want to play it despite those warnings, the Flash version (the version I’m writing about creating here) is online here. Eek, I can’t believe I just shared that with the world!

Why port it to Flash?

At the time Flash seemed like a good way of putting together fun little 2D games that anyone could run easily, just by visiting a webpage, and I liked the idea of getting into that. Flash is declining in popularity now due to the upcoming HTML 5 and the fact it doesn’t work on some modern machines (notably iPhones and iPads), but I suspect it’ll still be around a little while longer yet – last time I looked at HTML 5 it didn’t seem ready for creating moderately complex games in.

I decided to convert a game I already had to Flash rather than starting from scratch so that I wouldn’t have to grapple with thinking of an idea, designing levels, drawing graphics, all of that stuff at the same time as learning a new programming environment. Because I designed Etirites I already knew it inside out and had access to all the game content, so it was a lot easier for a first Flash project than making a brand new game.

Inside the original game

As I said, the original game ran on MS-DOS, and like most DOS games it accessed the hardware directly for most purposes rather than going through an operating system like modern games do. It ran in the old low resolution VGA mode (320×200 with 256 colours) and reached 70 frames per second on most computers with very smooth scrolling – I’m still rather proud of that. The source code consisted of about 16,000 lines of C (compiled with the DJGPP compiler) and 4,000 lines of i386 assembly (assembled with NASM, after some pre-processing by a tool of my own making). None of the original source code will be re-used in the port; the whole thing will have to be rewritten in ActionScript 3, the Flash programming language, but having access to it will be very useful for seeing exactly how things work.

(I couldn’t mention DJGPP without commenting on the fact that the website that hosts it, Delorie Software, is the only site I know of that still looks pretty much exactly the same now as it did when I started using the internet in 1997!!).

So that’s the code. The data files (assets, if you prefer) – graphics, tunes, maps, etc. – of the original game are almost all in custom formats, so one of the first things I need to do is work out how I’m going to convert them all to Flash-friendly formats. The original game packs everything into one single .DAT file (also a custom format… a bit like a Zip file but without the compression), but luckily I still have the individual files that go into it. This is what’s in there:

  • Various .LEV files. These are the tile maps for the area backgrounds, telling the game engine where there should be ground, walls, trees, water, etc. This is a custom binary format, but a very simple one: just a width and a height followed by an array of bytes specifying the tiles.
  • Various .LSG files. Most of the graphics are in this format. It’s a custom binary format that can have multiple graphics per file. It has very simple RLE compression and all the graphics are in 256 colours, in the standard VGA palette. Both movable sprites and background tiles are in this format. For the sprites, colour 0 represents transparency. I seem to remember I created these using my own graphics editor program called “VGAEdit”.
  • charset.dat. This is the game’s font, a simple 8×8 monochrome bitmap for each character. The format is the same as a Spectrum character set, and probably the same as various other old computers too.
  • etirite.pal. This is a copy of the RGB values (each ranging from 0-63) for the standard VGA palette.
  • Various .LSM files. This is the music, stored in a sort of simplified MIDI format. I seem to remember it’s mostly the same as MIDI but the timing information is simpler. The game code contains drivers that can play this on Adlib/SoundBlaster FM synths (badly!), or via general MIDI (better).
  • A couple of .TIF files. These contain the bigger graphics. One is the game map, the other is the crossed swords graphic for the menu screen. I can’t remember why these aren’t in LSG format like all the others. Possibly the LSG format had some size limitation on the graphics and they were too big.

Hmmm…. so what do we do with all those?

Data Conversion

The graphics will need to be converted to a standard format that Flash can use. PNG is probably the best one as it’s better than JPEG for storing pixel-art-type graphics. For the LSGs I will write a program to convert them to PNG, putting all the frames from each LSG into a PNG side-by-side. A simple C program using libpng should do the trick. The palette file will come in useful for translating them from 256 colour indexed mode to standard 32-bit RGBA. I will probably write a similar program to convert the character set to a PNG, since it would be nice to have the game’s original fancy font in the Flash version.

Hopefully the trusty Gimp (god how I wish they’d change that name) can open the TIFFs and save them as PNG… if not, I can look at how the original game code reads them and use that to make my own conversion program.

One thing that strikes me is that 320×200 is a very low resolution these days, if I leave the game at that size it’ll appear as a tiny box on most people’s screens. Even 640×400 would be much better, but doing a simple scaling up would look awfully pixellated. Maybe one of those nice up-scalers that a lot of old machine emulators use could help? I could try incorporating one into the graphics conversion program and see how it looks, scaling all the graphics up to maybe double their original size. Of course if it “mostly” works I could manually tweak the ones it doesn’t look so good on.

The music… somehow I need to get it into MP3 format for Flash. Probably the easiest way is just to record the output from the original game (running on DosBox using general MIDI) and then encode that into MP3… saves faffing about with the complexities of the LSM format which I don’t really fancy doing.

The .LEV files could maybe be left as they are and I could write ActionScript code to read them directly, as they’re very simple and there isn’t really a standard format (that I know of) for that sort of thing anyway. Apparently Flash can read binary data so it should work, but if it turns out to be impossible I could write a simple converter to turn them into text files, or to embed them in the source code, or something.

That just leaves the palette file which won’t be needed anymore once the game is running in glorious 32-bit true colour!

There is other data hard-coded into the source code (for example the story sequences, the area information including what objects start where, and the frame sequences for animations) but I’ll get to that in due course.

Next time: the actually nitty gritty of the data conversion!