close Warning: Error with navigation contributor "BittenChrome"
Posts in category code

Ludum Dare Compo 34 Post-Mortem

I created become a game developer in 60 seconds for the Ludum Dare compo this weekend. There were two themes, "growing," and "two button controls." I focused on "two button controls" and figured that making a game was sort of like growing a game. It became very meta.

I think the game speaks pretty well for itself.

I somehow already got a lot of awesome comments.

My changelog was pretty tame:

2015-12-12 10:26:27 Initial commit.  Here we go.
2015-12-12 10:53:06 Image editor?
2015-12-12 11:13:34 A primitive menu.
2015-12-12 11:21:59 What am I doing?
2015-12-12 12:17:19 Added game.js, so it's like it's done.
2015-12-12 12:35:54 Some input changes and image editor suggestions.
2015-12-12 13:03:54 Almost a game.
2015-12-12 13:35:44 Shooting.
2015-12-12 13:46:41 Some endgame business.
2015-12-12 14:05:14 We're going to make some sounds.
2015-12-12 14:54:51 Sound editor.
2015-12-12 15:23:44 This is almost really a game.
2015-12-12 15:35:27 Fix fix fix.
2015-12-12 16:15:06 I'm now in the 'what have I done' phase of the day.
2015-12-12 16:29:56 All of the parts are there.
2015-12-12 16:45:33 Added a timer.
2015-12-12 17:23:38 Got rid of the collectible editor.  Made sharing work.
2015-12-12 17:29:09 Title screen.
2015-12-12 17:40:18 Last minute thought.
2015-12-13 10:15:27 Added random bleeps and bloops all the time.  Toned down the volume.  Added a tiny bit more helper text.  Changed the +1 now that it's configurable.
2015-12-13 10:16:52 Oops, you should always be able to get points.
2015-12-13 10:23:12 Even quieter, just because.

Thanks to the Tech Valley Game Space for giving me a place to hang out while doing this, and the Tech Valley Center of Gravity for being that space and being located in a great place for post-jam food, the center of downtown Troy, NY.

  • Posted: 2015-12-13 13:39 (Updated: 2015-12-13 13:42)
  • Categories: code games
  • Comments (0)

Ludum Dare Compo 33 Post-Mortem

I entered Crab Boss in the Ludum Dare 33 Compo this weekend. The theme was, "you are the monster."

The idea I first started to pursue was basically to make a QWOP-style game where you control a giant crab in a boss fight.

By the time I had a reasonably interesting crab to control, I ended up simplifying the controls a bit and making it a simple action game.

The crab is made using a toy animation system I built as I went. There are no textures in the game. Everything is drawn using combinations of triangles and squares and circles made out of triangles. It probably cost me as much as four hours getting all of that working, but that was all time I did not have to spend drawing, and it lent itself to a unique visual and animation style, so I was happy to do it.

Audio was a huge hassle. For some reason, Audacity would only play back my recordings at the correct speed one in ten times. The result is that levels are all terrible, half of the sounds are made with bfxr while the other half are recorded, and some of the sounds I intended to make are just not there. This probably cost me an hour or two, and since the first two comments I have gotten complained about the volume, I think it is going to hurt my overall score.

But it was fun, and I have yet another little game to show off. I wish any of my coworkers would participate.

  • Posted: 2015-08-24 19:45 (Updated: 2015-12-13 13:22)
  • Categories: code games
  • Comments (0)

OpenID

Mini-rant/follow-up: It has almost been two years since I wrote about my issues interfacing with OpenID, and since I have recently been getting deprecation warnings from Google, I finally put in the work to optionally support password authentication. That will teach me to try to do the right thing.

Similar Projects to SandboxOS

When I started working on SandboxOS, I was somewhat in disbelief that nobody was doing this already. Since then, I have discovered a handful of projects with similar goals, but as far as I can tell, it is still unique enough to continue pursuing.

JavaScript Contenders

There are a handful of JavaScript-related projects with similarities.

node.js + io.js

Node.js, and by extension io.js, are essentially another runtime environment similar to those of Perl, Python, and Ruby.

node.js popularized using JavaScript outside of the web browser, especially web server-side, allowing web application to be built completely in JavaScript. But ultimately node.js and io.js compete in an already overpopulated niche.

SandboxOS is different, because it introduces a security model and an application model.

node-os

node-os is an operating system built from node.js running on a Linux kernel.

This seems to be the result of taking the package manager from node.js, npm, to its extreme and using it for managing all system files.

There is not really much comparison with SandboxOS. It is just another interesting projects in the same area.

Runtime.JS

Runtime.JS is an operating system kernel built using V8 and JavaScript.

It is an attempt at eliminating one layer of the stack commonly present in node.js applications.

SandboxOS, for better or worse, adds another layer.

Linux Container-based Contenders

I am not especially familiar with recent developments in Linux containerization. I've gather that it is a step beyond virtualization, allowing some amount of isolation without sacrificing performance.

SandboxOS different from all of them in that it attempts to make it easy to host web applications on all sorts of devices - anything where web browsers are found. While I have yet to measure the performance implications, the goal is to move toward having many more servers with many fewer (often just one?) users.

Docker / Rocket

Docker is the current dominating presence in this area. It appears to be tailored primarily toward sysadmins.

Rocket is an alternative being built for CoreOS.

Sandstorm

Sandstorm is the first project that actually concerned me that I might be stepping on somebody's toes. Their goals are very much aligned with mine, but they're taking a completely different approach, using Linux containerization. Interestingly, Sandstorm does a good job explaining why containers are great, but they don't solve enough of the problem.

They aim to make it trivial to run servers and make a better future for open source webapps, and I wish them luck.

Editing Apps

It may just be because programmers like strange loops, but being able to edit applications from an application which is itself editable is a fundamental assumption in SandboxOS.

SandboxOS targets the most devices by far.

OLPC Develop Activity

I think I first heard such a goal proposed for the One Laptop per Child project. Their Develop activity seemed to be down-prioritized pretty quickly, but it was a neat concept and it is a neat project.

Wiki OS

I'm not clear on the origins of Wiki OS, but it seems to be an attempt at reproducing a traditional desktop environment on the web on devices where Silverlight can be run, and it allows modifying applications with an interface that I would compare to Visual Basic.

Android AIDE

I encountered AIDE while in the middle at marveling at how seemingly unnecessarily difficult mobile development is made by the available toolchains. That project puts all of the tools necessary to build Android applications on-device.

Decentralization

One final theme I am watching for activity on is decentralization.

This recent article hints at future tech to decentralize web applications "like BitTorent does" without any information on how that can work.

Unhosted web applications are another trend of building web applications without a server component altogether.

Conclusion

There is a lot of activity in this area. So far, SandboxOS seems to be going in the right direction and not stepping on anybody's toes, so I'm going to charge forward with it!

SSPI WTF

This is a continuation of TLS WTF. I added SSPI / SChannel support to my list of supported TLS backends. It was not a pleasant experience, so I am taking another moment to document my grievances.

SSPI is a shining example of an API that is hard to use correctly.

  1. Microsoft documents that rather than linking against an export library, you must load security.dll dynamically and look up a symbol.
  2. To begin the TLS handshake, you call InitializeSecurityContext or AcceptSecurityContext, depending on whether you are the client or server. The credentials already know which direction we're going at this point, so this seems redundant.
  3. One argument to those functions is a pointer to a handle that is populated when the call succeeds, and a pointer to that handle must be passed back as a different argument on subsequent calls. But not before it succeeds once.
  4. InitializeSecurityContext, AcceptSecurityContext, EncryptMessage, and DecryptMessage all take arrays of input and output SecBuffer structs. After working with two other libraries that had much simpler mechanisms for passing data in and out, this just seemed wrong.
    1. Data is encrypted and decrypted in-place. While it might seem sort of clever, as though it's saving something or destroying no-longer-needed sensitive information, due to the way it works and the lack of any apparent guarantees, I found I needed an extra copy of the data anyway in order to properly restore the remaining unencrypted/decrypted bytes for the next pass.
    2. Each particular type of invocation has special requirements on the number and type of buffers it takes as input and returns as output. It's basically deliberately throwing away all parameter/type safety. Or conversely it's requiring the user to know quite a lot about the specifics of the protocol.
    3. One type of SecBuffer contents that is returned is the number of data bytes passed in that haven't been consumed. But not a pointer to the bytes like every other buffer.
  5. Shutting down a connection is bizarre. Call ApplyControlToken and then proceed as though you're starting a new handshake. I realize only now that I'm doing this incorrectly.
  6. I was pleased that I was pretty quickly able to load a PEM certificate and private key. And then it took me the better part of a day to be able to associate them with each other to be able to use them.
    1. A certificate context (container?) seems to be necessary. There is a way to get an anonymous one, but it doesn't seem to work for this purpose. I haven't found a way to get one if it exists and create it otherwise except by trying one and if it fails, trying the other.
    2. It is necessary to open a certificate store with the name "MY". What?
    3. A property "CERT_KEY_PROV_HANDLE_PROP_ID" on the certificate context looks exactly like what I would want to set to associate a key with a certificate, but it has no apparent effect. "CERT_KEY_PROV_INFO_PROP_ID" is the only one that apparently actually works, but it requires jumping through a bunch more hoops.
    4. All of this leaves some key/certificate data in some system location. I haven't found any way to avoid that.
    5. Most of the API worked with multi-byte or wide characters. One or two functions didn't.
    6. I've completely lost track of which things are opaque data types, handles, pointers to handles, or pointers to pointers to handles. It's out of control.

Again, I just hope someone stumbles on this or browser:projects/sandboxos/trunk/src/Tls.cpp, and is saved some of the hassle I went through.

I realize I am not done, but I really need to put this aside for now. Some future tasks include:

  • Support allowing individual certificates whose signing authority isn't otherwise respected (self-signed certificates).
  • Need to separate the certificate and key setup from the connection. I'm probably adding an awful amount of overhead for each connection by re-importing them each time.
  • Need to expose error messages. I've just been instrumenting the code as I go to discover what I've been doing wrong, but the actual errors are useful, so I need to make them available.
  • Need actual tests.
  • I want to be able to generate self-signed certificates on all platforms.

TLS WTF

This is my recollection of my misadventures in trying to support TLS for SandboxOS.

The Goal

I have a JavaScript runtime environment. I want it to be able to do these things:

  • Run a web server that supports HTTPS to secure or even just obfuscate my traffic.
  • Connect to web servers over HTTPS to post to Twitter and whatnot.
  • Connect to XMPP and IRC servers requiring secure connections to run chat clients and bots.

I want it to be able to do those things on Linux, OS X, and Windows, and I don't have a very high tolerance for complicated build steps or large dependencies.

Background

"I want to use SSL," I thought to myself. Apparently what I wanted is called TLS these days. I scratched my head and moved on, because this was the least of my problems.

Going in, I knew that OpenSSL had recent vulnerabilities, leading it to be forked into LibreSSL. I knew that GnuTLS was a thing. Outside of that, I did not have much knowledge about what was available.

I did some research and found that I should look into Mozilla's NSS as something that could potentially work on all the platforms I cared about. I also learned that if I wanted to support the most common libraries on each platform, I would need to look into SSPI on Windows. I also saw that node.js uses OpenSSL on all platforms.

I'm using libuv for doing all of my socket I/O, and I'm pretty happy with it. But it poses a challenge here, because these libraries tend to prefer being used as a wrapper on top of native BSD-style sockets and I didn't want TLS interfering with libuv's event loops at all. I chose to try to pass data around myself in order to avoid that scenario. It looks like NSS creates unnecessary intermediate sockets to abstract away that interface.

Getting Started: Build the Libraries

I believe it went like this:

  1. Look at Mozilla NSS (Network Security Services).
    • GPL-compatible license. Good.
    • Supports the platforms I care about. Good.
    • Get it and try to build it. They have a non-trivial custom build harness. Not a great sign, but OK.
    • Try to build it for win32/x64. Discover this note:

      Note: Building for a 64-bit environment/ABI is only supported on Unix/POSIX platforms.

    • Looked at the API a bit. It looks like it really wants to be bound to system sockets. That will incur unnecessary overhead the way I want to use it. Lame.
    • That was enough strikes for me. I wan't about to maintain my own builds of this for three platforms, and I certainly wasn't about to let this constrain me to Win32/x86, if that note was correct.
  2. Look at LibreSSL.
    • GPL-compatible license. Good.
    • It looks like the latest release is expected to be usable. Great.
    • It only builds on Windows through MinGW. Ugg.
    • By this time I had some OpenSSL code. I tried it on OS X and found that the OpenSSL system library on OS X was supported but has deprecated for some time. Crap.

I Wrote a TLS Wrapper

I resolved to support OpenSSL on Linux, the Secure Transport API on OS X, and SSPI on Windows, and all in code that could be easily extracted to use for other projects.

Currently the entirety of the public API looks like this, though I haven't yet tackled SSPI:

class Tls {
public:
    static Tls* create(const char* key, const char* certificate);
    virtual ~Tls() {}

    virtual void startAccept() = 0;
    virtual void startConnect() = 0;
    virtual void shutdown() = 0;

    enum HandshakeResult {
        kDone,
        kMore,
        kFailed,
    };
    virtual HandshakeResult handshake() = 0;

    enum ReadResult {
        kReadZero = -1,
        kReadFailed = -2,
    };
    virtual int readPlain(char* buffer, size_t bytes) = 0;
    virtual int writePlain(const char* buffer, size_t bytes) = 0;

    virtual int readEncrypted(char* buffer, size_t bytes) = 0;
    virtual int writeEncrypted(const char* buffer, size_t bytes) = 0;

    virtual void setHostname(const char* hostname) = 0;
};

Implementation Rants

I don't know where to begin.

  1. There are not enough good examples. If you are an engineer at Apple who worked on the Security framework, where did you put your ~100 line C file test case that fetches https://www.apple.com/ and fails if you try to reach the same server by a different name?
    • This was the best I could find for OpenSSL. It's alright but 12 years old.
    • This is why I haven't attempted SSPI yet. I think SSPI will be manageable, but that example is insane.
  2. Do not pretend TLS connections are BSD-style sockets. TLS is a terribly leaky abstraction. Socket calls that ought not to block might need to block so that TLS handshaking can finish. New errors can occur at every turn. A TLS session can close without the network connection going away. Stop pretending it's not a separate layer.
  3. Verify hostnames. TLS without hostname verification isn't secure. OpenSSL did not make verifying hostnames easy. The headers I have on debian jessie required me to extract the names from the certificate and do my own pattern matching to account for wildcard certificates. I had several implementations where it appeared the default verification would check names, but only testing showed that it wasn't happening. Apple made it a fair bit easier, but it still didn't happen by default.
  4. So far I have not yet found how to load my certificate and private key from PEM files on OS X without calling this private function from the Security framework:
    extern "C" SecIdentityRef SecIdentityCreate(CFAllocatorRef allocator, SecCertificateRef certificate, SecKeyRef privateKey);
    
    All other attempts I've made at getting a SecIdentityRef from my key and certificate have failed. I could keep them in the login keychain, but I want to support PEM on all platforms for consistency.

Results

So SandboxOS does what I need for TLS for now. It took me five times longer to write than I had hoped, and doesn't support TLS on Windows yet.

It can connect to secure servers. It verifies hostnames when you do that. I can run a secure web server with it. It might be fun to support client certificates, but that is about as far as I want this to go, and that can happen later.

I'm hoping somebody searching for some of these words will stumble upon this and either show me how stupid I've been or benefit from browser:projects/sandboxos/trunk/src/Tls.cpp.

Update

I added SSPI support.

SandboxOS

Several months ago I got a notion for a potentially enormous programming project I wanted to undertake. I've spent bits of time on and off implementing it, and it has come quite far. The last thing I want to do is stop all of the exciting work to document it, but after some reflection, that is the thing that it most obviously needs right now.

What follows might be crazy-town, but it still seems pretty neat to me.

What Is SandboxOS?

I struggle to find a comparison that does it justice, but I keep trying:

SandboxOS is...

Nuts and Bolts

  • It runs applications on Google's V8 JavaScript engine.
  • It uses libuv, from node.js, to interact with the OS when possible.
  • Each application runs as a separate process, as a first step toward implementing Google Chrome's Sandbox architecture.
  • Applications can interact with other applications by calling exported functions, provided that they have requested and been granted permission to do so.

The Last Thing I Wanted

I tried everything I could think of to meet the goals if this project without resorting to using JavaScript.

I ended up settling on JavaScript for a few key reasons:

  • Web browsers are by far the most widespread use of a sandboxed runtime that I could find. Plainly, I want to force an opt-in decision whether an application can invoke any individual thing outside of the application.
    • I looked at a few attempts at sandboxes for languages like Perl, Python, and Ruby, but it was clear that sandboxed execution was so far from the goal of those runtimes that it was a lost cause. I even passed up on node.js for the same reason - it was easier to start from scratch than to audit and secure everything there. The few approaches that looked sound were not portable or incomplete.
    • I know that Java and .NET make security claims, but I ruled them out early, because the security models seemed far from anything I could leverage for my specific needs, and the licensing and portability concerns are still pretty huge.
    • Web browsers run untrusted code all of the time. They crash tabs and eat memory, but at the end of the day, they are the one example of sandboxing I could find that are essentially globally relied upon.
  • Even if JavaScript isn't perfect, it has features that give it huge potential for my needs.

Circles and Arrows

Below is the bigger picture that I see this all fitting into. The interesting thing to me is not what I specifically imagine this thing doing but what little effort in implementing a few small parts it has taken to enable these sorts of things.

GraphViz image

Resources

I intend to follow this up with related posts on more focused topics. If nothing else, this project has forced me to learn at least a little bit more about a lot of things I wouldn't have otherwise interacted much with:

  • V8
  • libuv
  • TLS
  • JavaScript
  • C++11

For now, some SandboxOS links:

Ludum Dare Compo 30 Post-Mortem

Recently I took the opportunity to participate in the Ludum Dare 48-hour Game Jam Compo #30.

I was overall pretty happy with my last entry, so I took the same basic approach with JavaScript and WebGL. When I learned that the theme was "connected worlds," I decided that Zelda games embodied that theme best in my mind, and I should do my best to riff off of that.

I wanted to explore multiple worlds connected in multiple ways. In the end this meant two worlds which you could take rockets between that needed some sort of network connection between then uncovered and repaired. I had meant to build up the dungeon, which is what I called the area starting with the rats, into its own world, and I meant to have one or two Mario-style warp pipes as another means of connecting areas, but I ran out of steam to set up all of that.

Play my entry here.

My changelog this time was only slightly less distraught than last time:

2014-08-23 08:08:09 Connected worlds.
2014-08-23 10:50:46 What am I doing?
2014-08-23 11:30:02 Hrm.  Tiles?
2014-08-23 12:42:41 I am the worst at collision.
2014-08-23 12:43:13 Forgotten file.
2014-08-23 14:10:51 Bah, collision.h
2014-08-23 16:14:52 This is...something?
2014-08-23 16:19:49 Fix the end of the line.
2014-08-23 17:24:26 Augh, doors.
2014-08-23 18:37:19 Something about rockets?
2014-08-23 19:21:48 Stubs for lots of levels.
2014-08-23 21:03:13 Fonts and shovels?
2014-08-23 22:10:12 Push the push blocks.
2014-08-24 07:49:19 Minor fixes.
2014-08-24 09:04:29 Rats.
2014-08-24 09:30:28 Yeah, cats.
2014-08-24 09:44:12 It's almost like a puzzle.
2014-08-24 09:57:19 More puzzly.  What's up with the wire now?
2014-08-24 10:22:44 Fixes.
2014-08-24 10:57:59 More bombs.
2014-08-24 11:28:55 Something about blcoks.  Restarting levels.
2014-08-24 11:29:04 Forgotten file.
2014-08-24 11:29:16 Forgotten files.
2014-08-24 12:02:56 I think everything is wired up?
2014-08-24 12:56:26 More rocket.
2014-08-24 13:28:03 This is really something.
2014-08-24 13:58:28 Finally, the worlds are different.
2014-08-24 14:24:19 Ugg, digg animation.
2014-08-24 14:28:52 Oh man oh man.
2014-08-24 15:32:33 Content content content.  Fix fix fix.
2014-08-24 16:00:29 Cats...
2014-08-24 16:20:36 WWW HTTP WWW HTTP
2014-08-24 16:25:57 Faster digging.  Wire fix.
2014-08-24 17:28:19 What have I done?
2014-08-24 17:42:30 Some fixes.
2014-08-24 18:05:56 I'm some kind of monster.
2014-08-25 20:49:46 Ported to my arcade cabinet?
2014-08-26 18:40:39 Hide cursor for the arcade cabinet.
2014-08-29 09:11:29 Some optimizations so that I can run this thing on the arcade cabinet better.
2014-08-29 09:38:37 Faster still.

The "some kind of monster" comment referred to adding in title music.

Results

Overall I was pretty pleased with how things turned out. I stopped a few hours early, just because I was tired of staring at it, and I didn't think there was anything dramatic I could change at that point without breaking something.

Things that Made Me Happy

Music and sound
I set out to use a ukulele to make all of the sound effects and music for the game. As anticipated, this was one of the last things I did, but I felt like I had enough time to give it an acceptable treatment. I ended up having to tap on my desk for more effect than I expected.
Scale
It takes me about ten minutes to play through the game. It's admittedly rather tedious, and it won't help me with ratings among the 2500+ other entries, but I'm happy that I made something big enough that it can't be fully understood in 15 seconds.
Base code
This time I reused some of the base code from my last entry. This might have easily saved me half a day.

Things that Made Me Sad

Collision response
I have written collision response code many times and have reasonable awareness of the problems that come up, but it always trips me up. Every time. This time I went down a path that wasn't working and pretty quickly switched to something naive enough to work. I could use an existing solution next time, but I'd like to come to terms with this, so I will probably continue to try doing it myself.
Fun
I knew the scope of work for this project was going to be significant, but I naively left "making it fun" and "making interesting puzzles" to near the very end (or never). I'm glad I challenged myself the way I did, but it didn't work out like I had hoped it would.
Font
At the end of the first night, I found myself implementing textured font rendering. This seemed like a mistake. It cost me a fair amount of time getting the math right, and I could have spent that time making something else better if I had just use HTML to display text.
Balance
I said I would pay more attention to balance this time, but one of the last things I did was adjust digging speed, and I think this made the game way more tedious than I intended.

Conclusion

10/10 would participate again.

In the mosaics, my title screen appears in the middle of the parrot's tail and somewhere around Turkey.

  • Posted: 2014-09-02 19:24 (Updated: 2014-09-02 20:12)
  • Categories: code games
  • Comments (0)

Ludum Dare Compo 29 Post-Mortem

http://www.ludumdare.com/compo/wp-content/compo2/342546/34198-shot1.gif I took the opportunity to participate in the Ludum Dare 48-hour Game Jam competition #29 over the weekend. I have participated in a small handful of game jams previously, but this one was special to me for several reasons:

  • I had permission from my employer to release what I created under reasonable conditions. Most things I work on, including game jams projects, have had to stay inside work.
  • This was a solo competition. I typically would work with a team.
  • I do not actually remember the last time I made a game. It has been too long.

Play my entry here.

I was honestly planning on following the lead set by notch during the last Ludum Dare and was going to use Dart to make my entry. I toyed around with it on the plane ride back from GDC and decided to give it a pass based on the fact that I could not disable bracket auto-completion in the editor.

So at the last minute, I went in deciding to use JavaScript and WebGL basically from scratch. Considering what little I started with, I am extremely pleased with the result, even though looking back, there is plenty I would change.

One final constraint I had given myself was to make something that would be suitable for my arcade cabinet.

How it Went Down

I do not know how better to describe how the process went than my commit messages, reproduced below.

I did make it a point to eat and sleep well. Saturday afternoon there is a big gap in productivity where I ran to the grocery store so that I could make a pizza. The pizza came out worse than the game. I used some sausage that did not turn out as good as I had hoped.

I have made several games before that used the same general approach for building the level out of triangles. It is of course directly inspired by Soldat. The good thing is that I can get something pretty convincing together pretty quickly, with only brief excursions to references on Wolfram Mathworld and Wikipedia. The bad thing about it is that it took me a fair amount of time to get an editor together, and it was pretty clunky even in the end. And then throughout the entire weekend I found myself falling through the world in different places and attempting fixes to the collision code to prevent that from happening. Some pre-made physics code would have saved me a lot of time and trouble here, but where is the fun in that? In the end, I think it is pretty difficult to find holes in the collision, if there exist any. I would have built the levels somewhat differently if I trusted the collision more. Ceiling especially would have made the underground feeling much better, as well as encouraging flying around the levels using weapon kickback. Enemies are mostly relegated to flat terrain to avoid collision problems. I never properly sorted out the math for going up hills, but it is fun to fly off them so quickly, so I did not see that as an issue.

The heart/meat/weapon pickups are inspired by health/armor/weapon upgrades in Earth Defense Force 2017. Except that instead of individually creating over a hundred weapons, I rigged things so that each weapon is generally better than the last with variations on a several basic properties. I think the main failure here was that I should have given the weapons corny names instead of explicitly listing out all of their properties, and I also did not spend enough time balancing the game.

As far as balance goes, it has some big problems. I changed some things and fixed some bugs shortly before submitting which changed how the game played significantly. I should have spent more time playing toward the end. Creating the levels from first to last also left me bored toward levels three and four. The intent was to make the game go on forever with each replay becoming successively harder, but I either forgot or became so lazy and concerned with bigger issues that I did not bother to implement that.

For a weekend, from scratch, I am pretty pleased with what I ended up with.

2014-04-26 09:42:50 First.
2014-04-26 10:12:01 Shaderssss.
2014-04-26 10:35:20 First triangle.
2014-04-26 10:40:28 First texture.
2014-04-26 11:06:46 Ehh.
2014-04-26 11:35:34 Moving...something.  Augh.
2014-04-26 13:08:51 Augh, collision detection.
2014-04-26 14:47:24 More collision.  I'm not going to be fiddling with this forever.  Nope.
2014-04-26 15:09:09 Yup.
2014-04-26 16:31:42 Sigh.
2014-04-26 17:03:27 Smooth movement, but...I don't know.
2014-04-26 20:16:25 Adding triangles.
2014-04-26 20:26:47 Moving camera.
2014-04-26 20:46:30 Edit more.  Adding triangles.
2014-04-26 20:59:21 Moving verts.
2014-04-26 21:21:13 Level editor persistence...eyyy.
2014-04-26 21:54:10 Finally, shooting.
2014-04-26 22:09:27 Fix triangle winding.  Snap triangles to grid.
2014-04-26 22:13:52 Some friction and slip.
2014-04-26 22:22:48 First thing killed.  Murder blood veins in my teeth.
2014-04-26 22:30:28 Breakable -> Goomba.
2014-04-26 22:42:24 omfg.  An ant.
2014-04-26 22:52:12 This is basically done.
2014-04-26 23:04:10 Ant hill.
2014-04-26 23:23:45 Forgetten images.  Dealing with death.
2014-04-26 23:57:45 I wanted to mess with a weird vertex shader instead of sleeping?
2014-04-27 08:36:06 Finally, a thing beneath another thing.
2014-04-27 08:42:11 Made it easier to save/load multiple levels.
2014-04-27 08:53:33 Panning.  Ant hill texture.
2014-04-27 10:13:28 Projectile stuff.
2014-04-27 10:57:43 Entity placement editing.  Whew.
2014-04-27 11:14:40 Um, there's a groundhog?
2014-04-27 11:46:25 Player warping between levels.  Also made stubs for all levels.
2014-04-27 12:10:41 Some textures.
2014-04-27 13:00:31 More textures and something about health and armor.
2014-04-27 13:08:09 Drops.
2014-04-27 13:11:48 Made some more things fall to the ground.
2014-04-27 13:14:11 Loop levels forever.
2014-04-27 13:53:09 More bricky  bricks.  Actually lob projectiles.
2014-04-27 14:19:00 Working on second level.
2014-04-27 14:23:32 More level2.
2014-04-27 14:25:57 Tweak some damage.
2014-04-27 14:44:05 Projectiles oh my.
2014-04-27 15:07:23 Augh, fighting with collision response.
2014-04-27 15:20:20 Fix death.  Yep.
2014-04-27 15:58:03 It's a last level.
2014-04-27 16:13:28 What I don't even.
2014-04-27 16:18:01 Well it is a title.
2014-04-27 16:33:08 Fixed next level and cleaned up things that fell through the world? :/
2014-04-27 16:40:18 Fix texture wrap modes and black boxes.
2014-04-27 16:55:32 Weapons stats and tweaked weapons.
2014-04-27 17:17:24 Gibs.
2014-04-27 17:22:39 Kickback.
2014-04-27 17:35:26 Ehhh.  It's a background.
2014-04-27 17:43:50 Different dude.
2014-04-27 18:03:25 Sounds!
2014-04-27 18:06:58 Removed debug stuff.
2014-04-27 18:32:59 Added debug settings for starting level and weapon.  Fixed reload times for enemies.
2014-04-27 18:44:56 Last minute collision tweak.  Why  not?
2014-04-27 18:45:13 Oops, debug.
2014-04-27 18:53:37 Reeled in kickback a little bit.
2014-04-27 19:01:38 Eh?
2014-04-27 19:10:16 Added directions.

Oh, and there is already a YouTube video of somebody playing it, so it was a great success, no matter how I get ranked.

  • Posted: 2014-04-29 21:21 (Updated: 2014-04-29 21:26)
  • Categories: code games
  • Comments (0)

Good Riddance to Google Reader

I am a little bit surprised at the response I have seen to Google's announcement that they are shutting down Google Reader.

For me, they killed Google Reader in October of 2011 when they broke sharing. A little over a year later, I wrote my own replacement just to get proper sharing again.

  1. I don't want to hear about any silly petition to bring Google Reader back. It was already a husk of what it should have been. If it somehow returned by popular demand, it would not be run and maintained with the care it deserved. Let it go. Find an alternative.
  1. To the multiple projects that claimed you were going to produce a replacement following the crippling sharing, you are the worst kind of projects for giving hope and not following through. Releasing a bad alternative would have been better. Saying "oops, never mind" would have been better.
  1. There is no reason why something like an RSS reader needs to be centralized and run by a big company like Google. Let's see some competition.

Applying OpenID

I have written a fair number of webapps. They are virtually all quick and dirty things mostly for myself and immediate friends, but there are more than a few. I am pretty sick of implementing authentication.

I have two previous go-to approaches to authentication:

  1. Collect a username, password, and email address. The email address is used for password recovery/resets.
  2. Avoid collecting any user data. Generate some random value and stick it on the URL. If someone wants to return as the same person, they need to save that URL.

For my last project, I decided to take OpenID for a spin. There is already plenty written on the failings of OpenID.

But my experience was slightly difference from what I have read, so I want to elaborate.

Background

I wanted to make my webapp, but I knew I wanted to try something like !OpenID for authentication. I say "something like," because going into this, I had no preconceptions about whether OpenID, OAuth, or some other proprietary thing I had never heard of was what I wanted. That probably should have been enough to stop me.

I decided I would be using OpenID via python-openid. As far as I can tell, the included examples are the only documentation of that library. The django example did not help me much, as I was not using django and am not very familiar with it. I focused mostly on consumer.py, but it has a number of configurable parameters that only confused me, and I did not find it helpful at all that it only functioned as a standalone web server.

I fought with it for a while and eventually ended up with something which purported to authenticate me. I also found and hooked up a reasonably nice frontend for encouraging people to use well-known providers.

Surprises

I eventually went on to work on actually implementing my web app and started showing it to people. Several things happened. These things surprised me. Authentication is not a place where I want to be surprised.

  1. People associate authenticating with third parties with leaking information from those third parties.
    • My app deals with RSS feeds. People authenticated with their Google accounts. People were disappointed that their Google Reader RSS feeds did not show up.
    • People do not like leaking information.

      I'm not sure why, but whenever I see "login with Facebook" or similar stuff, I tend to close the tab and move on.

  1. I misinterpreted what guarantees !OpenID was making. I may still be doing so. The biggest example of this is that Google returned a different ID depending on the realm I specified.
    • I initially had generated the realm from the HTTP request, which meant that the domain people used to get to the site and whether they included a trailing slash on the URL would cause them to authenticate as different users.
    • I experimented with moving the site to a different host, but that would have required changing the realm and therefore somehow re-authenticating everybody.

Results

This little adventure was not all failure. It is still in operation, and people use it.

Probably the coolest thing to me is that I am storing so little personal identifying information. In general, I would say that people should care about this more than they do. I store OpenIDs. I store no email addresses. Names are optional. If someone gets access to the database, they get a list of Google OpenID URLs that are specific to my site and therefore pretty worthless.

Google tends to remember that I have logged in, so I can get into my site just by clicking on the Google provider logo.

But OpenID has not really solved any problems I have, and it has been a big hassle to integrate correctly.

Passing on Message Passing

I built a small web app recently, and one particular aspect wasted my time more than I anticipated. It may just be that I'm poorly informed of current trends, but I found all of the documentation and available advice to be poor, and the solution I eventually arrived at is probably close to the best I can get.

The general idea was to have a web front-end that could start and monitor jobs. Jobs would be arbitrary commands run on separate worker machines, which would stream output back to a central location for immediate viewing on the web site.

dbserver

In several instances in the past where I wanted to have web sites with instant communication, I think I read the wikipedia articles for AJAX and Comet, and then I wrote some python scripts centered around a thing I called dbserver.py. The code is all on this site, but it's in such a poor state that I'd rather not reference it.

The general structure looked like this:

  1. The web front-end uses jQuery to issue asynchronous requests to the web server, which are handled by a python script, api.py.
  2. api.py establishes a socket connection to an always-running background daemon, server.py.
  3. server.py forwards messages between clients. In some iterations, I had it always broadcast every message to all clients. In some iterations, I registered a handful of persistent clients to which all messages went and who could direct messages at transient clients.
  4. One of the clients was often dbserver.py, which mostly interacted with an sqlite database and notified server.py to notify clients when things in the database changed.

I would say my biggest problems in this setup were in the database-side of things. I generally failed at having a good way of querying the database from the web frontend that was descriptive, safe, and fast. And I had to not just query the database but register for notification of certain types of updates.

I also some problems with duplicate messages (or perhaps the same message going to two server.py clients which turned out to be different requests from the same web client). I have some ideas on how I could have simplified things and been more rigorous.

But really it was just a lot to maintain, and I couldn't believe that there wasn't something already out there for this.

Research

I have difficulty finding things when I don't know what exactly it is that I want.

It seems that the "real-time" web is the New Thing&tm; that matches the original problem I was trying to solve.

Many of the things I found centered around Twisted, Tornado, Orbited, and Node.js. To be honest, I didn't really give any of them a fair chance. I'm not looking for a different web server - just a thing to connect my existing webapp with other things. None of them really jumped out as being capable in the ways I needed.

A common theme seemed to be "message passing." Specifically, I found a number of frameworks built around RabbitMQ. When I arrived at this point, I was feeling good. I routinely undermine libraries I'm given to be able to work at a more reliable, manageable lower level, and RabbitMQ seemed to be a common tool in this business.

RabbitMQ has a number of tutorials using a python library called pika, so I started with that. The first few tutorials were pretty good, but after that I found the documentation to be pretty awful. pika deferred to RabbitMQ. RabbitMQ documentation was absent or tautological ("exchange-name exchange Name of the exchange to bind to."). I muddled around and got a test sort of working. I got fed up when things inexplicably hung or failed to clean up after themselves as I requested. It was never clear when I needed to pump what API for things to work and which things would block.

Next I tried Kombu, which was admittedly more Pythonic but seems to be unmaintained. I ran into the same issues as with pika.

I finally went with py-ampqlib, which was a lot like pika, but somehow easier for me to manage. Or at this point I had learned enough that I was just able to manage it better than my first attempt. Regardless, I got my simple test case basically working.

  1. I clicked a button on a web site. The web site sent a request to some script.
  2. That script set up a new "response" exchange for itself, bound a queue to it, and sent a message to a well-known RabbitMQ exchange informing it of a "job" and the exchange to which to respond.
  3. There was a script listening to a queue bound to that exchange. It pulled the job off the queue and feigned doing some work.
  4. The script sent a series of messages back to the "response" exchange.
  5. The web site hit a script which hit RabbitMQ to stream back the messages output from the "response" queue, blocking when appropriate to wait for the next message.

At this point, I appreciated that there was something of a standard for this business. There was no apparent chance of a message going the wrong client, needlessly broadcasting messages to all clients, or bookkeeping on my part to make sure that I only process each message once.

But at the same time, this was super fragile. I had some of the same problems as with pika and some new ones. Ensuring that exchanges were declared at the right time and had queues bound to them so that a worker could start feeding messages in before the web client was ready to start reading them felt as thought it were as complicated as just doing what I did in dbserver. And I hadn't even done the second part of this, which was going to involve creating another process which listened to all exchanges to log into a database, so that I could view logs after the jobs were finished.

Operating Systems

My takeaway was that I didn't want queues to be centralized. At least in my use case, I wanted to have one authoritative copy of the streaming output, and I wanted each client to simply keep track of how much of the output it had seen. I never had to worry about cleaning up anything like queues or exchanges, because the only state that was needed was an offset on the client.

The solution was incredibly simple.

  1. I clicked a button on a web site. It started a "job" process immediately.
  2. The worker wrote to a file.
  3. The web site made a request to a script which read from the file, starting from an offset provided by the client.
    • If there was more data there, it returned it.
    • If there was no data and the worker was still running, it called select(), which blocked until there was more data available.

So after all of that, I had written a web based /usr/bin/tail.

Conclusion

I'm not certain what I learned from this.

And in the end, I think I'm aborting the project that caused me to go down this path.

I expect in the future I will probably find use for all of these approaches.

BlackBerry Apps

I have two free apps in the BlackBerry app world for BlackBerry PlayBooks, now:

"Ukulele Chords" was just posted today. I applied what little music theory I learned from my previous ukulele chord webapp to what little I could learn in a day about QML.

I added both to my project list.

BlackBerry PlayBook Development

Several months ago, a coworker alerted me to a promotion Research in Motion was having. I believe the original description was essentially, "port an Android app to BlackBerry and receive a free PlayBook tablet." I spent about a day setting up the development environment and configuring Winter Simulator to run on the PlayBook simulator and submitted it for approval. My app was approved quickly, but I had basically given up on the promotion until about a month later, when I received the request for a shipping address which preceded me receiving free hardware.

There are two particular points I find interesting that I want to explore here. BlackBerry development is sort of in an unfortunate transitional state right now, but it looks as though it is quickly improving. Qt Quick, which is the foundation for BlackBerry's Cascades, seems fantastic.

The State of BlackBerry Development

The following is all based on my personal experience in going from zero knowledge of what BlackBerry development is like to picking a path and releasing a few apps. Things are changing quickly, and that is part of the point.

There Are Many Options

The BlackBerry development site highlights choice, in big letters, right up front. My assessment of the choices follows.

C/C++ Native SDK
This is how I ported Winter Simulator. The C API I assume is adopted from previous BlackBerry platforms. This is entirely adequate except for the fact that it provides no facilities for UI. It is delightful to, unlike on Android or iOS, write a simple C program without any surprises.
C++/Qt Cascades
More on Cascades below. This looks like a great approach to UI, but it's not ready to be used until BB10 is released.
HTML5 WebWorks
In combination with something like Apache Cordova, this seems like the best bet for actually making cross-platform iOS+Android+BlackBerry UI-heavy applications. I have CSS nightmares and so I shied away from it.
ActionScript Adobe AIR
I don't know anything about this and don't really care.
Java Android Runtime
The BlackBerry tablet OS includes a runtime for Java Android applications. Many applications can run after just being repackaged. At the time, I only had native applications which this doesn't support. Applications run this way currently run in a shell application which makes them appear slightly as second-class citizens, but the next OS release will apparently improve this situation.
blackberry-pythonpackager
This isn't really documented, but the bbndk includes a script for packaging python scripts as applications. I didn't try more than a "hello world" application, but with the appropriate wrappers, either of BlackBerry's C API or Qt, this would be wonderful.
C++/Qt
Qt already includes BlackBerry support. I tried making a C++ Qt PlayBook application, but it seemed like it would take substantial work to make it look and feel right on a tablet.

Conclusion

  • If you're making a game, use the native SDK.
  • If you're making a UI-heavy application, either use WebWorks or wait for BB10.
  • If you already have Android applications, why haven't you already repackaged and published them?

I ended up using QML in Qt, and it make me excited for Cascades. I think the main downside is that I've had to include the Qt libs, which amount to 36M, while my application is honestly just a few k of QML. When BB10 is released, I expect I will be able to convert and republish my app, and its size will shrink appropriately.

Qt Quick

A few months ago I wrote my first real pyqt application, for work, mostly on a whim. It's all procedural code, because that's how I knew to make and manage UI, and I was already embracing enough risk by using Python, Qt, and writing the app I was writing. The first time I got frustrated with the mess I was making, I started to look at my options, got confused, and decided it was correct to return to the path I was taking.

It seems Qt presents three options.

Generate UI in code
It's really not a bad option. For simple UI, I like having things in one place, where "things" include UI layout, declarations and appearance of widgets, application logic, and event handling. This only becomes a mess if there are a lot of things.
Qt Designer
This looks like a great WYSIWYG GUI editor. I didn't see the need for that at the time, so I skipped it. I may go back and convert some UIs to encourage other people to meddle with them visually.
Qt Quick
This appears to be intended for mobile UIs. The most concrete indication of this is that it lacks standard widgets.

When I wanted to make a UI-heavy BlackBerry application, I tried out Qt Quick.

I found that the answer to my first concern, that there are no standard widgets, is mitigated several ways:

  1. BlackBerry provides standard widgets in Cascades.
  2. Many mobile applications prefer to use their own widgets. This annoys me, but it is the way things are.
  3. Qt Quick provides enough to create basic widget types without too much hassle. Scrolling lists are a fundamental type.

A few more thoughts:

  • QML is a declarative language. More things should be declarative. I don't know how put this plainly. It is the Right WayTM.
  • QML integrates JavaScript. This makes it easy to write imperative code for the times that anything else would be too cumbersome. It also is used to support powerful property bindings.
  • QML is part of Qt. Qt signals, types, and the ability to interact seamlessly with C++ code seal the deal.

Much to my surprise, I ended up writing my whole application in QML, rather than following my original plan of writing most of the logic in C++ and only presenting the UI in Qt Quick.

So All Is Great?

Well, Research in Motion has to find a way to be relevant again, but I like their development tools. :)

Tank Xing

I was observing someone make a tiled terrain system earlier in the week, and I was reminded of an old project of mine: Tank Xing.

Much to my shock, I didn't have a build of it handy, and I couldn't so much as find any screenshots. I remedied this today.

A brief write-up as well as binaries are available here.

The relevant part to my original recollection is the function Terrain::generateTexture, which takes a height map and picks textures for each tile, inserting a border of hand-crafted transition textures between particular different heights.

The result looks something like this. Uh, disregard the texture sampling issues in between tiles.

It doesn't look like much. I never made any better art than some colored checkerboard textures to test it. It was nifty, to me, at the time.

  • Posted: 2012-01-07 18:31 (Updated: 2012-01-07 18:35)
  • Categories: code games
  • Comments (0)

Computer Mediated Communication and Me

For almost as long as I have had an interest in programming, I have found myself working on tools for computer mediated communication. Which is usually to say, "chat clients." This is an attempt to chronicle some of that work.

  1. Yoink! Newsgrabber

This was a news scraper. I configured it with templates for lots of sites I found myself checking routinely and ran it periodically to produce a concise static web page with headlines and links to articles.

The last version I seem to have posted happened on 6/23/2000, but I'm sure I worked on it for a significant time before then. There were at least two completely different implementations, first in Visual Basic and later in C++. I made a small amount of money by selling keys to unlock some features as shareware.

I am entertained to find that the RSS specification was only released in March of 1999, and I've only seen RSS readers in common use by people in recent years, so I'm pretty pleased at how ahead of the curve this was. There was other similar software at the time which certainly inspired me, but my implementation was fairly novel.

Today I use Google Reader to fill the exact same need.

  1. vbIRC

This was an IRC client. There's not much more to it than that. I probably started on it to avoid paying for a mIRC license while learning about both IRC and Visual Basic (4?).

I remember nightmares of hand-writing terrible string manipulation code to parse the protocol. I like to believe I know better, now.

I'm not exactly sure why I stopped work on this. I think there were just a number of cooler IRC clients out there that I knew I didn't have the time to overtake.

Today I use irssi for my IRC needs.

  1. PPNC

I wrote a weird little tray icon IM client for Windows at one point. I think I might have run it on a few computers at home purely as a stopgap to having to run up and down stairs to talk to people in the house.

I remember nothing about the implementation except that it was very basic.

Today I use Pidgin for all of my IM needs.

  1. ElectricMonk

Sometime in college I wrote an IRC bot with a number of features. Most notably, it believes things for me, storing simple words and definitions in a dictionary for later lookup.

He's still running, and I still use him from time to time.

  1. naim (very small change)

naim is a console client for a number of chat protocols. While riding in a car with a number of geeks on a WiFi network connected to a bunch of geeks in another car, someone suggested that I add support for autocomplete for aliases. I did just that before we arrived in New York City, coming from Troy.

I haven't used naim in years, but it's still around.

  1. EB-Lite (XMPP support)

After getting the itch at another point in college, I found the multi-protocol client everybuddy and consequentlly EB-Lite. I found the codebase to be fairly clean and understandable, so I spent some time with it. I recall making some IRC modifications as well as adding support for XMPP.

I remember my implementation involved using expat to build a custom data structure of the XMPP stanzas and then using a super dumbed-down XPath-like syntax that I made to examine them in order to do the appropriate handling. I also remember XMPP being very complicated to work with despite being a fairly simple idea in concept.

EB-Lite seems to have died. Pidgin, though I cringe every time I look at the codebase, has won in this arena.

  1. pycmc

I made my own frontent for EB-Lite and used it for a fair amount of time. I believe the main feature was everything being in one window, making it easy to manage many chat rooms and private conversations in a fairly compact UI.

I forget why I stopped using this. It may have been missing features like file transfers or certain protocol support that drove me away.

  1. unmoo

I started to work on a MOO/MUD at one point. It didn't get far.

  1. Mixed Messages

I made a DS game version of the paper game paper telephone. It was released years later as a DSiWare game. It is also available in the 3DS eStore.

  1. paste

I made a paste bin. This pissed Tom off, because he also wanted to make one. There's really not much notable about it. I think I used genshi and pygments, which both did their job well and made this pretty easy.

I still run this at work, but no one uses it.

  1. scraper

This was a webapp that scraped histories of web comics for me, for some time. A full write-up of it is here.

This did its job well, but in the end it scaled poorly (the database grew huge and slow), and I tired of maintaing it.

  1. lurksome

Mark expressed a disinterest in all things ajax, and as a result I tried making a static web page of interesting thing to me. This amounted to recent twitter messages, a summary of Google Reader unread times, and upcoming things on my calendar all on one page.

This is not unlike something I would have gotten from using iGoogle, but mine, being a static page that was regenerated periodically, loaded much faster. It also (deliberately) prevented me from polling frequently for updates, because it would only provide updates on the predetermined schedule.

I think I stopped maintaining it when twitter and Google Reader both independently had issues or made changes that required work from me to fix things.

  1. pester

pester was really my first use of ajax. I drew some diagrams of an architecture involving a web app, a daemon running in the background, and protocol plugins, and tried to implement it. I think I only ever had IRC support. The interface was a webapp interpretation of the interface for the Lily CMC, which nobody reading this will know about.

I got fed up with this when I tried to incorporate SSL on my connections between protocol plugins and the daemon, and I realized that there was a handful of valid-looking implementations in Python that all had serious shortcomings. At least, they were all incompatible with my preference of using blocking sockets with select/epoll. I had also reached a point where things were getting difficult to maintain and test.

  1. Live Graffiti

This is an Android application where people can draw things and those things later show up as the phone wallpaper of other people using the application.

I'm still using it. I mean to put this in the Android Market at some point, but not before doing some things to protect the server from abuse.

  1. Pester+

This is a very recent attempt to resurrect pester while adopting a slightly Facebook/Google+-like user interface. It's a work in progress. I've skipped the SSL aspect, since everything can run on one machine. One of my goals is to make this easily demo-able, so that other people might be encouraged to help me.

I put in a lot of effort in order to not have to speak to people directly. I will try to update this with more screenshots and links as I find them.

  • Posted: 2011-08-18 19:53 (Updated: 2011-08-18 20:01)
  • Categories: code
  • Comments (0)

Kayaking: Summer #2

Given that Friday was my last Friday off for at least a month, I decided that the weather was nice enough to bust out the kayak for the first time this summer. I washed my car, mounted the roof rack, and added a new second set of kayak racks. Then I strapped my kayak on top of all that and headed to Round Lake, which was basically the first place I started last summer.

I paddled around a bit, getting fairly tired by the time I finished circling Round Lake and Little Round Lake. It was a bit cold and windy, but both of those were manageable.

About fifteen minutes after I started, I saw this off in the trees:

I can't believe how many birds around here are balding.

As usual I logged lots of GPS data using My Tracks on my phone. I am trying to do something different and am building up a Trac macro to visualize some data about where I go. A CSV file of the raw data is attached to this post, and some kind of map and summary should show up below. Depending on how much time I find, this will improve or go away.

Map

Speed Graph

Time from Start (minutes)

  • Posted: 2011-04-16 22:07 (Updated: 2011-04-16 22:23)
  • Categories: kayak code
  • Comments (2)

Things I Have Made

As I previously mentioned, I have been dusting off some of my old projects, making them build again, and organizing the source code and executables I've packaged for different platforms.

The ultimate result is this list of links for a pile of my projects. I don't expect anyone cares about these things but me, but I am making it as easy as I can for people to try them.

In addition, I made a list of projects in which I've been involved in any capacity at work. It's more than I remember. Even still, I'm not sure it's complete, yet. I hope 2011 and the future bring good things.

So that's me talking about my code.

Winter Simulator 1.1

We had a snow day from work today, which prompted me to update Winter Simulator. Plus I can link to it in the Android Market now that it is finally web-accessible. Changes were minor, adding on-screen controls and icons. I suspect some people tried the previous version, could not get it to do anything cool, and immediately removed it from their devices. Now it's pretty obvious what it can do as long as people poke at the visible icons.

I would also like to elaborate on the porting process, having brought several of my old OpenGL 1.x/GLUT demos to Android NDK.

Getting a skeleton application working went relatively smoothly, working from the documentation and hello-jni example. The source to Winter Simulator can be explored here. The Java portion does minimal work to set up an OpenGL viewport and then forwards GLUT-like events to the C++ part. I reused essentially the same Java code for several projects, now. In Gingerbread I will supposedly be able to create C++-only applications, but I don't want to limit distribution to only phones with the latest OS, and this was easy enough.

My biggest problem was the difference between the version of OpenGL I had been using and OpenGLES. It was some relief when I realized OpenGLES 1.0 existed, as it shielded me from having to completely redo all of my code to use shaders and my own matrix code.

Things that failed to build included:

  • All of the drawing code. I didn't have GLUT solids. I didn't have GLU quadrics. I used both heavily.
  • Everything STL-related. I didn't have STL.
  • I had only glColor4f, not glColor3f. Fog was different. There were a handful of other things that I just had to change slightly.

I addressed the simple things and did the more complicated things one at a time.

For all drawing code, I had previously hard-coded immediate-mode commands to do most of what I wanted, often using GLU and GLUT primitives to assemble complex shapes like trees and cabins out of cylinders and cubes. I looked into either libraries available for Android or grabbing GLU and GLUT code to run on the phone. Neither appealed to me, so I ended doing something quite strange.

I pasted all of my drawing code into a python script and started implementing missing methods. The syntax use between my C++ code and Python was close enough that I didn't have to make many changes to get it executing. The most annoying part was stripping the trailing f off of floating point constants. I then naively implemented all of the functions I needed to support my drawing code. I made all of the immediate mode GL commands generate vertex buffer data and output a C++ source file with big blobs of data and the code to draw that data. I implemented the basic solids I needed by drawing them out on paper and making slow Python code to generate the triangles. The code didn't need to execute efficiently. It was a one-time process and generated relatively efficient vertex buffers.

That all took some effort, but it was quite easy to approach incrementally, and the end result is just what I wanted.

Parallel Port Box

Back in college I acquired some old DJ light controlling equipment. This included one box with switches and buttons to turn eight channels on and off, two things that looked like big power strips where each outlet was switched by a relay, and two cables, each about fifty feet long to attach the switches and buttons to the relays and outlets.

A few years back I made a box to control the relays from a parallel port. This is used at work to control as many as eight lights that indicate the status of various builds.

I wanted my own to play with the same setup at home, so I made another one. (read: Phone-controlled Christmas lights?)

It looks like this:

/share/parallelportbox.jpg

It's just eight transistors, resistors, LEDs, and diodes with the necessary connectors in a little box.

It went a lot faster this time. I did all of the soldering and most of the customization of the project box in one night. I think the previous time it took me the better part of a day.

The only thing that prevented it from working the first time I tried it out was that the Centronics connector pinout I was relying on labelled more pins as ground than were actually connected to anything. I moved the ground wire to a different pin and it worked perfectly.

I couldn't find the right kind of Centronics connector, so I ended up soldering to a connector that was supposed to be PCB-mounted. It's ugly, but it's safely tucked away inside the box and works fine.

And finally, when I just about had it ready to test, I realized I didn't have a printer cable. Fortunately, AIT computers provided me with this annoyingly somewhat hard to find cable for a perfectly reasonable price.

In other news, the Python parallel library is about as nice and simple as it gets:

import parallel
import time
p = parallel.Parallel()
p.setData(0b101010)
time.sleep(1.0)
p.setData(0b010101)

Now I need to find something to do with a box of switches and buttons.

Continuous Integration

The end of the year is for reminiscing, right? I recently discovered I have more old projects sitting around gathering dust than I remembered. A handful are documented on this page, but quite a few are missing. I also discovered that almost all of them fail to build with recent compilers.

First step: continuous integration.

I opted for Bitten for Trac for a few reasons. Trac has hardly ever let me down. Bitten seems to have been developed in the same vein with a similar set of developers, so it's more likely to just work in my environment and have surprisingly cool integration with other parts of this web site. We use Zutubi Pulse at work, but not only is it overkill, but I don't want to deal with the license, and I'm not thrilled with everything about how it works. In the end, my needs are pretty minimal, so pretty much anything will do.

My next step was getting things building. This was for the most part pretty easy. Problems across all projects have been pretty consistent so far. I had some heinous invalid code that previous versions of gcc let slide but recent versions rejected. I'm using SCons 2.0.1 instead of 0.9.6, which is what I used previously. I also had a ton of missing #includes. This change demonstrates a little of all of that.

My next step was to expand into some of the non-Linux platforms that I had previously supported. This was kind of tricky, because I don't own any computers running Windows or Mac OS X. A quick poll of friends directed me toward VirtualBox, and it wasn't too long before I had Windows 7 Home Premium 64 Bit and Mac OS X Server / Snow Leopard running.

My first confusion came from documentation not lining up with what I was seeing. It looks like new versions of VirtualBox use RDP to give access to otherwise headless systems. The version I started with used VNC. Not a big deal, but when I ran into other problems, the uncertainty of whether I was doing anything right didn't help.

I ended up upgrading to the latest 4.0 build, and things seemed to be working until I noticed that both of my machines were hanging from time to time. I didn't resolve this before realizing that I hadn't gained anything but instability since 3.2.8, so I downgraded, juggled some files to get it to use the virtual machine I had created in the newer version, and now things seem to be working better.

Remaining issues with the virtual machines:

  • I haven't found the right VNC graphical quality balance between sluggishness and being able to see the screen without terrible artifacts.
  • The mouse in OS X is badly behaved. It doesn't follow my cursor in the VNC client.
  • The OS X virtual machine won't boot off of its disk. I need to use a boot ISO which I also needed to use to kick off the installation.
  • I haven't yet rigged them to start themselves automatically when the host machine boots and to log in to the client OS and start the bitten slave script.

Probably some other things, too. But my intention is to just let them be for a while.

I got multiple branches of Notebook Ninja building on Windows. I found Flag Fu was the project with the most interesting configuration that I could get running on all three platforms the most easily. It builds a client, server, and editor, depending on what libraries are available, and now I have that working again on Linux, OS X, and Windows.

This is still far short of what I want to do:

  • Everything I'm building should be packaged up and shared in binary form.
  • I'm still missing a bunch of projects.
  • I'm still missing easily supportable platforms for a bunch of projects.
  • I should have documented all of the setup I had to do on the build machines to get each project building. At the very least, list required libraries.
  • I haven't done much runtime testing of anything on Windows or OS X.
  • I don't have any sort of nightly or otherwise regular builds, so this won't really prevent bit rot, but bitten developers are aware of the desire for that feature.

Summary: There's a Build Status link at the top right there. It's neat!

Winter Simulator

Seven years ago, in a computer graphics class at RPI, I tried to make my last project winter-themed and highly demo-able. I think I wanted to submit it for a NeHe contest, but I never did. I took some time recently and ported it to Android.

The original project lives here. Source and executables are attached to that page.

Here is a link to it in the Android Market:

/share/winter-qrcode.png

Source lives here.

Note: The controls are terrible. It's pretty ugly. But I did it in a short time in college and ported it in just a few days.

  • Posted: 2010-12-21 21:26 (Updated: 2010-12-21 21:39)
  • Categories: code
  • Comments (2)

Vicarious Visions Game Jam

At work, we had our first ever game jam last Saturday.

It's hard to share my team's resulting game, so here's a video:

After the 14 hours, I was exhausted, and it seemed like everything we had worked so hard on had fallen apart. Minutes before the judging we were able to quickly fix some bugs, and I'm pretty happy with the results. It may not look it, but it was a substantial effort to arrive at this from what we started with (not much).

The lesson I took away from this is that the ideal length of time for making a game is less than a year and more than a day. I propose game jams of every length in between to find the sweet spot.

Pin Map

I got it in my head that I wanted a map with all of the places I've gone kayaking. I wanted not just a Google Maps map but a physical map on my wall where I could mark off places and see the big picture in person.

I got a cork board and the biggest state map I could easily find. Upon closer inspection, the map I got was not at all what I wanted, and I couldn't find any better.

I know the USGS makes lots of map data available, but after playing with their web site for some time, I just couldn't figure out how to get what I wanted, and they kept wanting me to put things in shopping carts to download them. Bleh.

I ended up using Google Maps. I remember reading that Google gets their topographical data from USGS anyway, so I didn't feel bad about stealing it. Here's how it went.

I went to Google Maps, zoomed into the rough area I was interested in at the zoom level I thought I wanted, and opened resource tracking in Google Chrome to get the URL for one of the map tiles. The parameters in it were obvious enough.

I wrote this code:

from urllib2 import urlopen
import os
from PIL import Image
from StringIO import StringIO

def get_tile(x, y, zoom=12):
        path = os.path.join('tiles', '%d,%d,%d.jpg' % (x, y, zoom))
        if os.path.exists(path):
                tile = open(path).read()
        else:
                url = 'http://mt1.google.com/vt/lyrs=t@125,r@132&hl=en&x=%d&y=%d&z=%d&s=Ga' % (x, y, zoom)
                print 'Fetching', url
                tile = urlopen(url).read()
                open(path, 'wb').write(tile)
        return Image.open(StringIO(tile))

tl = (1203, 1500)
br = (1218, 1510)

tile_width = 256
tile_height = 256

tiles_wide = br[0] - tl[0] + 1
tiles_high = br[1] - tl[1] + 1

width = tile_width * tiles_wide
height = tile_height * tiles_high

mosaic = Image.new('RGBA', (width, height))

for x in range(tl[0], br[0] + 1):
        for y in range(tl[1], br[1] + 1):
                tile = get_tile(x, y)
                mosaic.paste(tile, ((x - tl[0]) * tile_width, (y - tl[1]) * tile_height))

mosaic.save('mosaic.jpg')

I fetched a block of tiles, being sure to cache them on disk so I wasn't hammering Google every time I reran my script.

With my starting tile set, it was a quick matter of trial and error to get the exact area I wanted.

It didn't take much code with PIL to combine the tiles into one big JPEG.

Next I needed to print the big image on a bunch of sheets of paper as big as I could for my cork board. I used PosteRazor to accomplish this without much hassle, though there are a handful of other similar tools available.

I did some arts and crafts, cutting out the prints and taping them together.

Then I started throwing in some pins, using big red pins to mark my launch points and small black pins to mark roughly the maximum distances I went from the launch points.

The end result:

/share/kayak/pinmap.jpg

Not too shabby. Needs more pins in it!

  • Posted: 2010-09-05 00:15 (Updated: 2010-09-05 00:15)
  • Categories: kayak code
  • Comments (1)