Tuesday, July 29, 2008

Tachikoma 3D Model?

Dear lazy web,

I'm hoping someone might have some handy links to a (preferably free) 3D CG model of a tachikoma that I'd be able to use in a s3kr3t side-project of mine (oops, I guess it's not so secret anymore, huh?). Since I'm planning on animating the tachikoma, it would preferably have hooks for this or be componentized or something (not sure how 3D CG models are "built" to allow animation, so... excuse my ignorance).

Failing that, a technical blueprint would be greatly helpful if I am forced to model my own.

Credit will be given to the modeler (or draftsman) in my project if it ever reaches the point where I release it to the public (which I'll do if/when it becomes more than a pipedream... currently just in the planning stages and the project is likely beyond my abilities, but we'll see).

Thursday, July 24, 2008

Moonlight 2.0 Hackathon

Starting this past Monday, we've begun a 2-week Moonlight 2.0 Hackathon to try and get as much of the 2.0 API implemented as we can (1.0 is basically done, but could still use some performance optimizations).

I've been reworking our c++ Collection implementation to support the Silverlight 2.0 collections of doubles and Points (Silverlight 1.0 only allowed collections of DependencyObjects) and I'll then move on to updating the c++-side code that used to use generic double and Point arrays to use my new DoubleCollection and PointCollection classes respectively.

I've also taken the liberty to implement some of the additional functionality that Silverlight 2.0 has added to the base Collection class (aka PresentationFrameworkCollection` on the .NET-side of things) like Contains() and IndexOf() (we used to get these 2 for free with my c++ List implementation, but I'm no longer using that as the base) as well as changing the way collection change-notification is done.

Once that is finished, I'll be binding them in managed land.

Meanwhile, Chris Toshok and Rolf have been refactoring the way DependencyProperties get registered since Silverlight 2.0 allows the programmer to create new DependencyObject derivatives and register new DependencyProperty's on those objects.

They'll also be making the type-system per-AppDomain rather than global like it is right now and will be looking into allowing each AppDomain to have multiple Surfaces so that applications like LunarEclipse will be able to work.

Friday, July 18, 2008

PulseAudio: My Last Post On The Topic?

I wasn't going to post anymore on PulseAudio now that Lennart has fixed the core issue that was breaking audio for me on my system (there was a deadlock condition in PulseAudio - my libesd and libgnome fixes were merely a fix to make those libs handle the PA deadlock more gracefully).

But, since Lennart has now attacked me, I guess I should respond:

A few comments:

1. Not directly related to PulseAudio itself. Also, finding errors in code that is related to esd is not exactly the most difficult thing in the world.

Considering that the GNOME desktop and many other applications speak to pulseaudio via libesd, it is still relevant. Until all applications are ported to use PulseAudio directly, libesd is still a part of the "PulseAudio stack".

OK, Jeffrey found a real bug, but I wouldn't say this is really enough to make all the fuss about. Or is it?

Actually, I believe you mean three in the PulseAudio code itself (granted, the pavucontrol bug was already fixed and in a public release - shame on my distro for shipping 0.9.5 instead of 0.9.6, but I don't remember even flaming you about this one - I only reported it as a bug to try and help you make PA better). Yes, the deadlock one was fixed a while ago (assuming the deadlock you fixed is the same one I am experiencing, which it probably is) but it is not in any released version of PA. Nor is the "won't play short sound clips" bug fix.

Hence, not mine nor my distro's fault.

You also completely fail to grasp how frustrating that deadlock bug (which was really the core of my issues) was for anyone using GNOME on a system where that deadlock occurs. Because when that deadlock happens, the entire system locks up! You cannot launch firefox, you cannot start any new GNOME apps, etc. You can't even close gnome-terminal.

Now, I will grant you that part of the blame certainly lies in libesd (hence why I fixed it), but to say it is completely unrelated is bologna. My ESD fixes were not random fixes so I could point fingers, they were fixes so that my system didn't hang whenever PA did.

5. A valid bug, but not really in PulseAudio. Mostly caused because the ALSA API and PA API don't really match 100%.

This seems like a design failure - if the plan is to make applications that use ALSA directly "Just Work(tm)" through PA, then shouldn't PA have been designed to map better to ALSA?

I still consider this to be a PA "bug" even if the fix was in ALSA(?).

Many people (like Jeffrey) wonder why have software mixing at all if you have hardware mixing?

Actually, that's not at all what I was wondering. I had wondered why PA was needed at all because ALSA already did mixing for me.

Now, I will admit to being naieve about the full list of goals that PA is trying to achieve (which are worthy goals), but I think they all take a backseat to having sound actually work reliably in a basic setup (apps playing audio through local speakers).

Jeffrey thinks that audio mixing is nothing for userspace. Which is basically what OSS4 tries to do: mixing in kernel space. However, the future of PCM audio is floating points. Mixing them in kernel space is problematic because (at least on Linux) FP in kernel space is a no-no. Also, the kernel people made clear more than once that maths/decoding/encoding like this should happen in userspace. Quite honestly, doing the mixing in kernel space is probably one of the primary reasons why I think that OSS4 is a bad idea. The fancier your mixing gets (i.e. including resampling, upmixing, downmixing, DRC, ...) the more difficulties you will have to move such a complex, time-intensive code into the kernel.

Fair enough.

I still don't like it (but I guess there's nothing you can do about it if the kernel folk are holding you back), but so long as it Just Works(tm), in the end I don't care.

Not every time your audio breaks it is alone PulseAudio's fault. For example, the original flame of Jeffrey's was about the low volume that he experienced when running PA. This is mostly due to the suckish way we initialize the default volumes of ALSA sound cards. Most distributions have simple scripts that initialize ALSA sound card volumes to fixed values like 75% of the available range, without understanding what the range or the controls actually mean. This is actually a very bad thing to do. Integrated USB speakers for example tend export the full amplification range via the mixer controls. 75% for them is incredibly loud. For other hardware (like apparently Jeffrey's) it is too low in volume. How to fix this has been discussed on the ALSA mailing list, but no final solution has been presented yet. Nonetheless, the fact that the volume was too low, is completely unrelated to PulseAudio.

This is actually partly also openSUSE's fault in 2 ways:

1. the volume control in the panel launches pavucontrol instead of gnome-volume-control. Since I need to use gnome-volume-control in order to adjust the master volume of the ALSA device, and the fact that the volume shown in pavucontrol was set to MAX, led me to be confused.

2. the keyboard volume controls do not properly adjust the master volume (I think I may have wrongly blamed this on PA in one of my posts, but don't feel like re-reading my posts to confirm - if I did, apologies).

OTOH Ubuntu didn't exactly do a stellar job. They didn't do their homework. Adopting PA in a distribution is a fair amount of work, given that it interfaces with so many different things at so many different places. The integration with other systems is crucial. The information was all out there, communicated on the wiki, the mailing lists and on the PA IRC channel. But if you join and hang around on neither, then you won't get the memo.

If distros are getting this wrong, then maybe there needs to be better communication so that things like what happened to Ubuntu don't keep happening.

FWIW, I have personally read over the "The Perfect PulseAudio Setup" wiki page and afaict, openSUSE 11 followed the recommended setup but there were still problems (obviously).

(Although the problems I experienced were bugs in code rather than bugs in setup...mostly)

[2] In fact, Flash 9 can not be made fully working on PulseAudio. This is because the way Flash destructs it's driver backends is racy. Unfixably racy, from external code. Jeffrey complained about Flash instability in his second post. This is unfair to PulseAudio, because I cannot fix this. This is like complaining that X crashes when you use binary-only fglrx.

Sort of like it's unfair that Evolution users complain when Evolution doesn't work with some broken server. But users don't care that it's a broken server if other clients work with it, so it's still on the Evolution developer's shoulders to work around those bugs.

Sort of like how historically, hardware makers haven't been giving Linux kernel devs the specifications to make their hardware work, and so it is left up to F/LOSS developers to bear the burden of making it work anyway.

No one ever said life was fair and unfortunately, a LOT of users are going to expect Flash to work. If/when it doesn't, they will complain and in a manner of speaking, it is PulseAudio's fault because it did work pre-PA.

Wednesday, July 16, 2008

Gtk+ 3.0: The Things That Make You Go "Hmmm?"

Lots of discussion about Gtk+ 3.0 lately, unfortunately there's no real discussion going on (heh).

There's the camp that is excited about API/ABI breakage and there's the camp that isn't so enthused.

I have to take the side of the "not so enthused", alongside Miguel de Icaza[1][2], Havoc Pennington and Morten Welinder: it's crazy insane to go breaking API/ABI for the sake of cleaning up the API without adding new functionality that could not be achieved by simply extending the current 2.x series.

There are definitely things that can be done to the current gtk code base without having to break the current API/ABI to achieve them, so in my humble opinion that's what should be done. Granted, eventually there will have to be a Gtk+ 3.0 which breaks API/ABI in order to add some new functionality that isn't possible to implement in 2.x, but that road can be crossed when we get there. No sense breaking it now if it isn't absolutely needed.

Even more worrying is the lack of promise to not break API/ABI again after 3.0. In fact, I've read on one of the mailing lists or somewhere that they plan on breaking API/ABI every 4-5 years after 3.0, ripping out any API's they marked as deprecated in the meantime. This is crazy. One-Flew-Over-the-Cukoo's-Nest crazy (or, if you prefer, I'm-Cukoo-For-Cocoa-Puffs crazy).

For large applications (Gnumeric, Evolution, Mozilla, Eclipse, etc), porting from 1.2 to 2.0 was a huge PITA but we all did it because (besides being young and stupid) 2.0 made a long-term commitment to maintain API/ABI compatibility going forward in addition to the fact that it added a ton of much-needed functionality (accessibility, UTF-8, model/view, and really good text layout w/ support for RTL and CJK).

If 2.0 was just 1.2 + sealed structs and the removal of some deprecated APIs, no one would have bothered with 2.0.

To respond to a point on Tim Janik's blog:

3.0 will ABI-incompatibly remove all deprecated and private APIs. Of course, the above described deprecation scheme only scales well if deprecated APIs are really removed from the code base at some point.

One reason that ISV's are typically on-board with Microsoft is that Microsoft has gone to great lengths to maintain backward compatibility with their old APIs. Joel Spolsky talks about it a bit in How Microsoft Lost the API War (specifically, the section The Two Forces at Microsoft).

The Windows testing team is huge and one of their most important responsibilities is guaranteeing that everyone can safely upgrade their operating system, no matter what applications they have installed, and those applications will continue to run, even if those applications do bad things or use undocumented functions or rely on buggy behavior that happens to be buggy in Windows n but is no longer buggy in Windows n+1.

I'm not going to argue that Gtk/GNOME should go to these extensive lengths to make sure buggy apps continue to work, but there's no reason that G*_DISABLE_DEPRECATED can't simply mean "we're not going to bugfix these anymore, what you see is what you get". You don't have to ever remove them, especially since it likely doesn't require much maintenance to keep them around. The port to Gtk+ 3.0 would require some work to make sure all the internals of things like GtkCList use accessor methods to anything that they use internally, but after that the job should be done and over with.

(ABI might not be a bit hard, but you could maintain API - thus meaning a simple recompile would work)

On the *giggle* *giggle* You should be using Mono/Gtk# *giggle* *giggle*-side, any application built against Gtk# 2.x will still work with Gtk# 3.x w/o any changes :-)

Oh, and Gtk# has DataBinding support (which, btw, does not require API/ABI breakage to Gtk+ nor sealed structs like some people seem to think on this morning's #gnome-hacker's discussion *cough* *cough*).

PulseAudio: I Told You So

To those who told me that my PulseAudio problems were my fault and/or my distro's fault, you were wrong[1][2][3][4][5]*. I told you so.

The first 2 bugs were fixed by me last week (libgnome and libesd) and simply worked around the 4th bug linked above. On that note, I released esound-0.2.39 with my fixes (along with a number of other patches that had been sitting in bugzilla god-knows how long collecting dust).

The 3rd bug is just a crash in the pavucontrol which is apparently fixed in a new version (fairly minor annoyance since by the time I tried to use pavucontrol, pulseaudio was already deadlocked iirc).

The 4th bug (which was the most serious of the ones I've been experiencing) was a deadlock in the pulseaudio daemon (it is supposedly fixed now, but it is not in any public release yet). Just because you didn't experience it, doesn't mean the bug was my fault or my distro's fault ;-)

The 5th bug was found by Geoff Norton, Rolf Kvinge, and myself today while trying to get moonlight to work well with PulseAudio after much head scratching wondering why sound wasn't playing for certain short mp3's (turns out that it's reproducible with any mp3 player if you try to play a short enough audio stream where the decoded stream is less than ~22 kilobytes iirc, but we didn't notice that until just after filing the bug).

So the good news is that the PulseAudio devs have fixed the more serious issues I've found so far, but I'm still annoyed by the attitude of the people pushing PulseAudio of "well, we are forcing people to use PulseAudio so that bugs get found by the users".

For normal applications, this is less annoying... but when it is something as low-level/fundamental as PulseAudio (or heaven forbid, the linux kernel), it should be QA'd thoroughly before dumping it on the users.

Anywho, hopefully openSUSE/Ubuntu/Fedora will push my patches and the PA dev's patches soonish, so that should really help minimize more unnecessary user suffering.

* there are other bugs that I filed as well, but I'm too lazy to go digging for them since the "My Bugs" link on the PulseAudio Trac doesn't actually work.

Sunday, July 13, 2008

Good News on the Evolution Front

Novell has just recently (end of last week) announced that Evolution will no longer require copyright assignment. This is awesome news that I've been hoping would happen for a quite a while now.

Evolution's license is also changing to be dual-licensed under the LGPLv2 and LGPLv3.

Thanks to Michael Meeks and Srini for making this happen!

Thursday, July 10, 2008

PulseAudio Again...

Just had another exciting PulseAudio crash where my sound card got continually spammed until my ears bled and then it spammed the sound card some more.

Seriously though, my coworkers were a bit annoyed by my laptop blaring really loud and obnoxious noise for the past hour (made worse because my keyboard volume control keys do not actually work with pulseaudio which is apparently a known issue with Lenovo T61 laptops I'm told).

However, this did provide me with a most excellent opportunity to test the patches I posted in my previous blog entry about my pain and suffering at the hands of PulseAudio.

Yet again, I was unable to launch any new apps - though instead of blocking in esd_open_sound(), they blocked in esd_send_auth(), so I went and patched that up to use non-blocking IO, build new packages, installed them, and tested and HALLELUJAH! they work!

Yes, ladies and gentlemen, I can now actually use my system even when PulseAudio takes a proverbial dump on my desktop.

I've submitted my patches upstream:

  • bug #542391: libesd should not block if the daemon dies/hangs/whatever
  • bug #542296: gnome_sound needs to handle pa hangs more gracefully

Unfortunately, these patches do not solve the root problem (pulseaudio suckage), but at least they allow me to get work done when pulseaudio craps out on me (which only happens every few hours if I'm lucky).

Oh, and if you are on 32bit x86 openSUSE 11 systems, you can grab some packages I made up at http://www.gnome.org/~fejj/pulseaudio/. I've also uploaded the raw patches in case anyone wants to push them in their own distro packages.

Sex With Dead People Deemed Illegal

Ok, so as I'm riding the MBTA in to work this morning reading my Metro newspaper, I come across the article "Court: Sex with corpses is illegal"

Apparently until yesterday's ruling, sex with dead people was technically legal in the state of Wisconsin.

The last sentence of the article was, in my humble opinion, the most hilarious sentence I've ever read in my life:

In yesterday's 5-2 decision, the high court said Wisconsin law makes sex acts with dead people illegal because they are unable to give consent.

Yes folks, it was a 5-2 ruling. So 2 people ruled in favor of allowing sex with dead people. W. T. F.

But to make it even more hilarious, the reason they decided it was not kosher was that the dead people could not give consent. L. O. L.

We live in a funny world...

Wednesday, July 9, 2008

More PulseAudio Problems

I reinstalled PulseAudio to try and resolve the plethora of problems I've been having with it (that and openSUSE 11 doesn't actually ship th esound daemon anymore, so if I want full audio support I have no choice but to get the pulse-esound-compat package working short of forking the distro and maintaining my own set of esound/gnome/etc packages which is not my idea of fun).

Unfortunately, this has resulted in countless hours of frustration. After having read and followed the instructions at http://www.pulseaudio.org/wiki/PerfectSetup, it turns out that is exactly how my system was already configured. So no help there.

I don't know what the deal is, but I constantly have problems with PulseAudio.

For example, earlier today I was having an IM conversation with a friend in Pidgin and it locked up. I restarted Pidgin, but I wasn't getting audio events anymore. *shrug* Oh well.

At some point later I navigated my firefox window over to youtube to link a friend to some video. No surprise here, I get no sound and in fact, the flash video pauses as soon as the sound was supposed to start (which is a few seconds into the video).

At this point I close firefox and decide to reload it, thinking maybe that will solve it. Nope, I was wrong. Instead, firefox hangs before any windows pop up. Great.

Since I got flamed last time I blogged about PulseAudio for issuing a `killall -9 pulseaudio` command (btw, pulseaudio appears to have a --kill command-line option), I figured I'd log out and log back in again instead... but this was not to be. Instead, my desktop locks up (presumably because of the logout.wav).

Ok... so I Ctrl+Alt+F1 to the console, login as root, `init 3; init 5`. Since my system is configured to auto-login, when X came back up it tried to log me in. Instead, I get a hang.

Good grief, I guess I have no option but to reboot so that's what I do.

In the interest of saving other people from this catastrophe, I grabbed libgnome out of svn and wrote up the following patch to prevent it from hanging whenever PulseAudio decides to misbehave (which it seems quite prone to do).

Index: libgnome/gnome-sound.c
--- libgnome/gnome-sound.c (revision 3750)
+++ libgnome/gnome-sound.c (working copy)
@@ -30,8 +30,12 @@
 #include <stdio.h>
 #include <stdlib.h>
+#include <sys/types.h>
 #include <unistd.h>
+#include <fcntl.h>
+#include <errno.h>
 #include <time.h>
+#include <poll.h>
 #ifdef HAVE_ESD
 #include <esd.h>
@@ -413,6 +417,55 @@
+#ifdef HAVE_ESD
+static int
+send_all (int fd, const char *buf, size_t buflen)
+ struct pollfd pfd[1];
+ size_t nwritten = 0;
+ int flags, rv;
+ ssize_t n;
+ if ((flags = fcntl (fd, F_GETFL)) == -1)
+  return -1;
+ fcntl (fd, F_SETFL, flags | O_NONBLOCK);
+ pfd[0].events = POLLOUT;
+ pfd[0].fd = fd;
+ do {
+  do {
+   pfd[0].revents = 0;
+   rv = poll (pfd, 1, 100);
+  } while (rv == -1 && errno == EINTR);
+  if (pfd[0].revents & POLLOUT) {
+   /* socket is ready for writing */
+   do {
+    n = write (fd, buf + nwritten, buflen - nwritten);
+   } while (n == -1 && errno == EINTR);
+   if (n > 0)
+    nwritten += n;
+  } else if (pfd[0].revents & (POLLERR | POLLHUP)) {
+   /* we /just/ lost the esd connection */
+   esd_close (fd);
+   fd = -1;
+   break;
+  } else if (rv == -1 && errno == EBADF) {
+   /* socket is bad */
+   fd = -1;
+   break;
+  }
+ } while (nwritten < buflen);
+ if (fd != -1 && flags != -1)
+  fcntl (fd, F_SETFL, flags);
+ return fd;
  * gnome_sound_sample_load:
@@ -469,21 +522,23 @@
     * file, or event type, for later identification */
    s->id = esd_sample_cache (gnome_sound_connection, s->format, s->rate,
         size, (char *)sample_name);
-   write (gnome_sound_connection, s->data, size);
-   confirm = esd_confirm_sample_cache (gnome_sound_connection);
+   gnome_sound_connection = send_all (gnome_sound_connection, (const char *) s->data, size);
+   if (gnome_sound_connection != -1)
+     confirm = esd_confirm_sample_cache (gnome_sound_connection);
    if (s->id <= 0 || confirm != s->id)
        g_warning ("error caching sample <%d>!\n", s->id);
        s->id = 0;
-   g_free (s->data);
-   s->data = NULL;
   sample_id = s->id;
-  g_free(s->data); g_free(s);
+  g_free(s->data);
+  g_free(s);
   return sample_id;
@@ -521,9 +576,12 @@
   sample = gnome_sound_sample_load (buf, filename);
-  esd_sample_play(gnome_sound_connection, sample);
-  fsync (gnome_sound_connection);
-  esd_sample_free(gnome_sound_connection, sample);
+  if (gnome_sound_connection != -1 && sample > 0)
+    {
+      esd_sample_play(gnome_sound_connection, sample);
+      fsync (gnome_sound_connection);
+      esd_sample_free(gnome_sound_connection, sample);
+    }

This patch should hopefully help prevent the typical gnome apps from hanging when trying to play audio, but I don't really have any surefire way of testing that it actually works.

Update 2008-07-10 10:42am I've now also written the following patch for libesd so that it doesn't hang for god knows how long when trying to open a connection to the sound server (in my case, pulse-esound-compat):

diff -up esound-0.2.38.orig/esdlib.c esound-0.2.38/esdlib.c
--- esound-0.2.38.orig/esdlib.c 2008-07-10 10:10:31.000000000 -0400
+++ esound-0.2.38/esdlib.c 2008-07-10 10:32:23.000000000 -0400
@@ -20,9 +20,11 @@
 #include <arpa/inet.h>
 #include <errno.h>
 #include <sys/wait.h>
+#include <poll.h>
 #include <sys/un.h>
 #ifndef SUN_LEN
 #define SUN_LEN(ptr) ((size_t) (((struct sockaddr_un *) 0)->sun_path)  \
                      + strlen ((ptr)->sun_path))
@@ -408,6 +410,43 @@ int esd_resume( int esd )
     return ok;
+static int
+connect_timeout (int sockfd, const struct sockaddr *serv_addr, size_t addrlen, int timeout)
+ struct pollfd pfd[1];
+ int flags, rv;
+ if ((flags = fcntl (sockfd, F_GETFL)) == -1)
+  return -1;
+ fcntl (sockfd, F_SETFL, flags | O_NONBLOCK);
+ if (connect (sockfd, serv_addr, addrlen) == 0) {
+  fcntl (sockfd, F_SETFL, flags);
+  return 0;
+ }
+ if (errno != EINPROGRESS)
+  return -1;
+ pfd[0].fd = sockfd;
+ pfd[0].events = POLLIN | POLLOUT;
+ do {
+  pfd[0].revents = 0;
+  rv = poll (pfd, 1, timeout);
+ } while (rv == -1 && errno == EINTR);
+ if (pfd[0].revents & (POLLIN | POLLOUT)) {
+  /* success, we are now connected */
+  fcntl (sockfd, F_SETFL, flags);
+  return 0;
+ }
+ /* took too long / error connecting, either way: epic fail */
+ return -1;
  * esd_connect_tcpip: make a TCPIP connection to ESD
  * @host: specifies hostname and port to connect to as "hostname:port"
@@ -512,7 +551,7 @@ esd_connect_tcpip(const char *host)
            goto error_out;
-         if ( connect( socket_out, res->ai_addr, res->ai_addrlen ) != -1 ) 
+         if ( connect_timeout ( socket_out, res->ai_addr, res->ai_addrlen, 1000 ) != -1 ) 
          close ( socket_out );
@@ -596,9 +635,9 @@ esd_connect_tcpip(const char *host)
     socket_addr.sin_family = AF_INET;
     socket_addr.sin_port = htons( port );
-    if ( connect( socket_out,
-    (struct sockaddr *) &socket_addr,
-    sizeof(struct sockaddr_in) ) < 0 )
+    if ( connect_timeout ( socket_out,
+      (struct sockaddr *) &socket_addr,
+      sizeof(struct sockaddr_in), 1000 ) < 0 )
  goto error_out;
@@ -650,8 +689,7 @@ esd_connect_unix(void)
     socket_unix.sun_family = AF_UNIX;
     strncpy(socket_unix.sun_path, ESD_UNIX_SOCKET_NAME, sizeof(socket_unix.sun_path));
-    if ( connect( socket_out,
-    (struct sockaddr *) &socket_unix, SUN_LEN(&socket_unix) ) < 0 )
+    if ( connect_timeout ( socket_out, (struct sockaddr *) &socket_unix, SUN_LEN(&socket_unix), 100 ) < 0 )
  goto error_out;
     return socket_out;

Hopefully after these 2 patches get applied, my system at least won't become unusably hung when pulseaudio decides to crap out on me, but these patches doesn't solve the root of the problem :(

Monday, July 7, 2008

4th of July Fireworks from VMWare Office

I watched the 4th of July fireworks from the 11th Floor offices of VMWare in Cambridge this year with some friends.

dscn0265.jpg, originally uploaded by jstedfast.

dscn0343.jpg, originally uploaded by jstedfast.

dscn0349.jpg, originally uploaded by jstedfast.

dscn0400.jpg, originally uploaded by jstedfast.

dscn0401.jpg, originally uploaded by jstedfast.

Update: I should note that these photos were made possible thanks to Alan McGovern's 256 MB CF card he mailed me last week. So thanks, Alan! ;-)

Code Snippet Licensing

All code posted to this blog is licensed under the MIT/X11 license unless otherwise stated in the post itself.