We're still playing with 2D barcodes and testing out how well they work with various Android devices.  One question that came up from our last test was how well they dealt with errors in the barcode, that is dirt, rips or parts missing entirely in the way.

The answer is pretty darn well.  Here's a barcode that still runs despite me scribbling all over it:

But there are vulnerabilities.  Specifically on QR codes, they are very sensitive to having one of the position detection patterns destroyed.  This QR code, while in much better shape than the one above, refused to read, just because the top left block was filled in.

Note of course that all these results are in the context of using the Barcode Scanner application on an Android phone.  There are almost certainly weaknesses in the scanning algorithm, but that's the state of the art for Java based QR Code reading, so it is what is relevant for us.

Also, we tried the same tests with DM codes with similar results.  Newer DM codes support the very same Reed Solomon error correction algorithm, and just like QR codes allow for up to 30% of the data being corrupted before failing entirely.  Again though, if you start messing with the position markers for DM codes, which are the left and right edges, you run into trouble more quickly.

We're a huge fan of the Huawie Ideos phones.  They are ridiculously cheap, $100 unlocked, and can be bought over the counter at Safaricom in Kenya or about $120 from NewEgg in the states.  That is a crazy pricepoint, totally blowing away any other Android phone and really making J2ME handsets mostly irrelevant as contenders for the kind of ICT4D projects Nyaruka undertakes.

Recently we've been working with a client and have been considering using these as a way of managing and tracking inventory.  After all, the cheapest barcode scanners are around the same price, and you don't get a built in touch screen, GSM connectivity, GPS and the ability to build the entire application on the device.  That makes the Ideos mighty tempting indeed.

One thing we were curious about though is just how well they would perform with high resolution 2D barcodes.  In our case, we'd like to have the option to encode quite a bit of data in the barcodes, so we were curious how the inexpensive Ideos optics would fare.

Methodology

We created and printed 2D bar codes encoding 23, 50, 100, 200, 500 and 1000 characters, then tested them at different sizes.  The codes were all generated at Invx.com, which seemed to do a decent job, even for our monster 1000 character barcode.

We printed each using a laser printer and scaled them from roughly 4"s to 2"s and finally 1" squared.  This was under normal lighting conditions using the most excellent "Barcode Scanner" app in the Android App store which uses the open source ZXing library.  We also tried things out with a Nexus S, and a Sidekick 4G.

Camera Specs

The big knock to the Ideos is that it is a fixed focus camera, despite being 3.15 megapixels.  What that means in this application is that you always need to hold the Ideos at a fixed distance from the code in order to get a sharp picture, which means that you start running into the limitations of the sensor resolution for the higher data QR codes.

The Sidekick4G and Nexus on the other hand, have autofocus cameras, which help greatly.  The Sidekick also has a 3 megapixel camera, but it is able to focus so fares much better.  The Nexus S has a 5 megapixel camera with auto-focus, so should be the best yet.

Scoring

For each phone and barcode, we ran a series of tests and ranked the performance as Fail, Poor, Good or Excellent.  Fail means that we either couldn't get the phone to read the barcode at all, or it was very, very difficult.  Ok means that it took a good bit of searching to read the barcode, but it did so consistently.  Good meant the barcode was easily read when well aligned and Excellent meant the barcode was instantly read even when misaligned or at an oblique angle.  Excellent readings have that magical quality to them where you just wave the phone in the general direction of the code and it picks it up.

Results

So what did we find?

Not surprisingly, auto focus made a huge difference.  Since the Ideos has a set focus point, you essentially always have to hold it roughly the same distance from the code.  That is reasonably easy to do and you get a feel for it, but it means that if the QR code is small, then the sensor won't be able to make out the details.  Here's a quick example that makes it clear:

That's the Ideos quite easily reading the 100 character 2" barcode and struggling on the 1" version.  You can see that the 2" code takes up a greater size so reads easily, but the smaller 1" code doesn't.  We can't move closer to the 1" code to make it fill the screen as then it becomes out of focus.

One interesting learning there is that you can work around that resolution if you have the freedom to make larger QR codes.  The 'optimum' size for a code for an Ideos is probably around 3"'s square, that would fill the frame at the exact distance where it is focused.  We didn't try this out, but I bet you could get a 200 character code reading excellently.

The benefit of the autofocus is obvious when you compare the Ideos results to the Sidekick, which has a similar resolution, though much better optics and autofocus.  The Sidekick has no problem reading even 500 character barcodes at 1".

For all practical purposes the Nexus doesn't do any better, which probably indicates that we are reaching the limits of the optics rather than the limits of the sensor sizes.

Practical Limits

So what are the practical limits?  On an Ideos, I would stick to ~100 characters or less on a 2" bar code, and ~50 characters or less on a 1" barcode.  At those sizes things work brilliantly, instant recognition at virtually any angle.

On an autofocusing device, you have a lot more flexibility.  Even at 1" you can easily decode codes with up to 500 characters.  For QR codes that seems to be the practical limit, though if you are willing to use a DataMatrix code you might be able to squeeze a bit more in.

QR Codes vs DataMatrix Codes

The two biggest 2D barcode standards for this kind of application are QR Codes and DataMatrix codes.  DataMatrix is actually the older standard, but due to leaving out support for Kanji characters, QR Codes were created in Japan and have been taking over.

There are some advantages and disadvantages to each.  QR Codes are more tolerant to being read at oblique or rotated angles.  The guide blocks in the corners seem to help tremendously here and I definitely noticed faster and more consistent reading for QR codes.

However, those blocks come at the cost of data density.  Here are the two 1000 character codes used in these tests, the QR code at left and the DM code at right.  You can see that the DM code requires less resolution to read, so by and large the DM codes tended to fail last, at least when good alignment was used.

Error Correction

One interesting feature of QR codes is that they are a bit more tolerant to errors.  You can cover up the bottom right corner of a QR code and it will usually still read, the DM codes are more sensitive here.  This might be an overstated benefit though, as I didn't have as much luck covering up the top right, left, or bottom left corners of QR codes, where the guide blocks are, so the practical cases where the error correction would come in seem minimal.

Error correction is something you can tune in QR codes, some generators lets you define more redundancy in the generated code.  However, this will presumably come with lower data density, so is probably very specific to your application.

Troublesome QR Code

Strangely, the lowest resolution QR code generated, of 23 characters, actually was troublesome to read in many cases.  Looking at it closely it looks like it contains one less control block than the 50 character code.  You can see that below, the 23 character on the left vs the 50 character on the right.  

This might just be a limitation or bug in the scanning software itself, but does point out that real world testing is still really important.  The DM code didn't display any such issues.

Full Results

Finally, here's the full results if you want to see the nitty gritty details:

The Nyaruka team has been at Pivot25 for the past few days, presenting
Bizito as one of the finalists. It has been a fun time, it is great
to see what the technology scene is like here in Kenya. There is no
doubt that that Nairobi is much farther along than Kigali, but I view
that as encouragement as to what is yet to come for us in Rwanda.

But one thing that is the same, is the constant ask for access to
capital by various companies. A few presentations over the past few
days have had asks for six figures in funding, which gives me serious
pause. It seems like there is a belief that funding is the goal as
opposed to a necessary evil. The truth is just the opposite, as
software engineers, we have the unique advantage that we can build
systems to earn us money without doing physical work. Add to that
that all we really need is an internet connection, some ramen noodles
and an ample supply of coffee and the funding asks really make me
scratch my head.

One interesting exercise is to look at YCombinator as a comparison.
YCombinator is arguably the leading seed funder in the Silicon Valley
today. Companies compete in a similar way as we have here at Pivot25
to be one of the companies to be taken under their wing. And you know
how much they give? $20,000. And this is only the most promising,
most experienced, most qualified companies in the world. Their
graduates include the likes of DropBox and Loopt among countless other
visionaries.

So my advice to some of the less experienced teams. Forget about the
funding and go build your idea, after work if you need to, using a
loan from your friends and family if necessary. If you don't believe
in your idea enough to suffer through eating ramen, or to sell your
family on it, then maybe you don't believe in your idea as much as you
say you do.

My point is the only expensive thing about starting a software startup
is paying engineers, and if you are one, then you should be able to do
it without raising money, and you'll be all the better for it.