tag:blogger.com,1999:blog-33658902227838707312024-03-13T03:40:05.725+01:00The TypethinkerTrying to make senseThomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.comBlogger83125tag:blogger.com,1999:blog-3365890222783870731.post-58829610874051009822015-03-16T21:28:00.001+01:002015-03-16T21:28:28.682+01:00QR codes on the BBC Micro, Epilogue: KryoFlux to the rescue<div dir="ltr" style="text-align: left;" trbidi="on">
<i>In which I once more revel in nostalgia and still fiddle with bits, but do not talk about QR codes at all.</i><br />
<br />
So with the BBC Micro quite dead, but my floppy disks and <i>possibly</i> the disk drive still intact, I looked around on the internet what other options I had. I found a few products which promised to control old floppy drives and convert the result to something more modern. In the end, I decided to buy the <a href="http://typethinker.blogspot.com/2015/01/qr-codes-on-bbc-micro-part-6-of-6.html" target="_blank">KryoFlux</a> package, mostly because of its extensive documentation and active support forums. The KryoFlux is a small circuit board with a floppy drive cable connector on one side and a USB connector on the other.<br />
<br />
But I also needed another disk drive, because the Beeb's drive didn't have the right plug. Or so I thought. So I found a list of drives known to work with the KryoFlux and searched for them on eBay. These turned out to be really easy to find, and on the second auction I was the highest bidder at the shocking price of €16,50 (including shipping).<br />
<br />
A few weeks later, both the KryoFlux and the disk drive were sitting on my desk, ready to be hooked up. I followed the instructions to connect everything up, and installed the somewhat arcane command line tool. Then I typed <span style="font-family: Courier New, Courier, monospace;">dtc -c2</span> to detect how many tracks the drive can access. The disk drive made a noise, its head moved around, and eventually the tool printed <span style="font-family: Courier New, Courier, monospace;">maxtrack=83</span>. That's quite a common number, so everything seemed to work so far.<br />
<br />
Fortunately, when my dad brought the BBC Micro over, he also included all of <i>his</i> old floppy disks. They mostly contained teaching material that nobody cared about anymore, so I had plenty of disks to experiment with and possibly ruin. I popped one of these into the drive and figured out the command to make a raw image.<br />
<br />
The KryoFlux was explicitly designed for software archival and forensics, and is able to read floppy disks at a much lower level than most hardware. What you get out of it isn't bytes, it isn't even bits, but it's the raw magnetic fluxes coming directly from the disk surface. This is great, because even if you can't salvage all of your data right away, you can still save off everything that's on the disk before it degrades even more, and do your rescue operation offline. It's also possible to decode the data on the fly, so that unreadable sectors can be retried automatically.<br />
<br />
And it comes with a nice GUI tool to visualize the spectrum coming off the disk:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjrElrPQqPSRnl9giPuqt73VnSMSG01ERDl-x6RYh-ENM8P0BAtcLnjppCKgIajf4k5bd1jKCPLTeh1H9EoW-SZwq6SoWShIA2znc7LGDXK12zo1VY3m1XZBVFqahHRzAlJJEeeA-RXgs/s1600/kryoflux_spectrum.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjrElrPQqPSRnl9giPuqt73VnSMSG01ERDl-x6RYh-ENM8P0BAtcLnjppCKgIajf4k5bd1jKCPLTeh1H9EoW-SZwq6SoWShIA2znc7LGDXK12zo1VY3m1XZBVFqahHRzAlJJEeeA-RXgs/s1600/kryoflux_spectrum.png" height="217" width="400" /></a></div>
<br />
In the process, I learned a thing or two about how floppy disks work. You probably know that the disk surface contains magnetic patterns. The data is arranged in 80 (or 84) tracks, which are laid out as concentric rings, track 0 on the outside, track 79 (or 83) on the inside. The disk drive positions the head above the track it wants to read, spins around the disk at 300 rpm, and the resulting fluctuations in the magnetic field result in an electrical current in the drive's head. Essentially, the output of the floppy drive is just an analog waveform, and it's up to the controller to make sense of it.<br />
<br />
There are different ways to encode data in such a wave. The disks I was looking at use FM, which – indeed – is Frequency Modulation. It's really simple: one frequency is used to encode a zero, another frequency is used to encode a one. To allow the drive to figure out where one bit ends and the next starts, <i>clock bits</i> are inserted into the stream: each other bit is a clock bit, and they are all set to 1. So, for example, the bit sequence <b>0110</b> would be encoded as 1<b>0</b>1<b>1</b>1<b>1</b>1<b>0</b>. (The problem with this is that you might lose synchronization if you get a long run of just 1s. That's why <a href="https://en.wikipedia.org/wiki/Modified_Frequency_Modulation" target="_blank">MFM</a> was invented.)<br />
<br />
Now you can probably make sense of the spectrum shown in the screenshot above. The 4.0 µs band shows the zeros, the 8.0 µs band shows the ones. When you zoom in, you can even somewhat make out the individual bits:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEJFL67nyffmrem_5RZM_gvt1NlnFNfmJoTJGp65NYLOLBXM6H8c4IDByiZbuvA3enzVzCr62Kk1LYCziEi95pfYSOVLOc6hcog3_Gw1FiAglc_Di2PM4tT7-2FU7BQMVYROrxw172DCI/s1600/kryoflux_bits.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEJFL67nyffmrem_5RZM_gvt1NlnFNfmJoTJGp65NYLOLBXM6H8c4IDByiZbuvA3enzVzCr62Kk1LYCziEi95pfYSOVLOc6hcog3_Gw1FiAglc_Di2PM4tT7-2FU7BQMVYROrxw172DCI/s1600/kryoflux_bits.png" height="217" width="400" /></a></div>
<br />
Fun fact: The magnetic flux was sometimes also used to write special copy protection patterns to the disk, which couldn't be reproduced by normal byte-based disk copy tools. <a href="http://www.softpres.org/kryoflux:ui:stream-plot" target="_blank">This article</a> shows a gorgeous example near the end.<br />
<br />
So with the help of the KryoFlux tool, I was able to create a series of files (one per track) that contained these raw waves straight off the disk. But the tool doesn't stop there; it also lets you decode the FM (or other) format into bytes, in a proper disk image. And it gives you some indication of how well it succeeded.<br />
<br />
As it turned out, not very well. Decoding the FM stream would consistently get me <span style="font-family: Courier New, Courier, monospace;"><error></error></span> for the first dozen or so tracks on the disk. From there on out, it got slightly better, with the occasional track showing <span style="font-family: Courier New, Courier, monospace;">OK*</span>, and near the end of the disk all tracks were supposedly <span style="font-family: Courier New, Courier, monospace;">OK*</span>. Except they weren't. When running <span style="font-family: Courier New, Courier, monospace;">strings</span> on the resulting disk image, I got stuff like this:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">This test consists`of two perts. Part One is about the works you have read</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">in class with yo}r teakher. Part Two is about the books you have read on</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">your own. You only have to answer questionw0</span><br />
<br />
It got better near the end of the disk, but still not good enough. To preserve actually working software, we need every last bit to be correct.<br />
<br />
I tried with two more disks, including some that did work in the BBC Micro only weeks before, but both showed the same pattern. I suspected that the drive I bought on eBay might be faulty. Before buying yet another drive, I <a href="http://forum.kryoflux.com/viewtopic.php?f=3&t=941&p=8337" target="_blank">asked on the support forums</a> if anyone had a clue what might be wrong, but didn't get much useful info.<br />
<br />
I left the project alone for a few weeks, and eventually decided to modify the BBC Micro's original drive so I could connect it to the KryoFlux. The data cable looked like a regular old flatcable, but the power connector was all weird. I opened up the drive case so I could see how it was wired up, but lo and behold! On the inside, the weird power cable ended in a standard molex plug that connected to the drive!<br />
<br />
Encouraged, I went through the setup procedure again with this drive. Would it still work, or did it break along with the Beeb itself? I connected the data cable, and nothing interesting happened. I connected the power cable, and no smoke came out. I sniffed the drive, but it didn't emit that typical fried-electronics smell. So far so good.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPwz9xbWse3F7doDvRZBspDMMhiIiBeS94Clnm9tbTUFyOewxMmraK2kmsbIUCM2kGrcZ3aDeCaFfYj8F2qs8C6WA3psiz323ZANb21ejal5ECPcM2EOplFHQl2hkQmSFziPZ2kqMSReM/s1600/IMG_20150314_142104.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPwz9xbWse3F7doDvRZBspDMMhiIiBeS94Clnm9tbTUFyOewxMmraK2kmsbIUCM2kGrcZ3aDeCaFfYj8F2qs8C6WA3psiz323ZANb21ejal5ECPcM2EOplFHQl2hkQmSFziPZ2kqMSReM/s1600/IMG_20150314_142104.jpg" height="300" width="400" /></a></div>
<br />
<br />
Then I ran the KryoFlux calibration tool again. The drive head moved! It seemed to bump up against something near the end, but the tool still reported <span style="font-family: Courier New, Courier, monospace;">maxtrack=83</span>. Oh well, it's probably fine. Let's read a disk.<br />
<br />
And yes! Yes! The tool printed a slew of <span style="font-family: Courier New, Courier, monospace;">OK</span>s! Let's look at the image content:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">This test consists of two parts. Part One is about the works you have read</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">in class with your teacher. Part Two is about the books you have read on</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">your own. You only have to answer questions about the books you have read.</span><br />
<div>
<br /></div>
<div>
Now <i>that</i> is proper English!</div>
<div>
<br /></div>
<div>
Would the entire disk be okay? In particular, would we be able to read the catalogue that stores where files reside on the disk? To check that, we have <a href="http://www.g7jjf.com/dfs_explorer.htm" target="_blank">DFS Explorer</a>. And it looks like this:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHC4HQTvSdbgw7TdUQ3kPuZ154y9cDL5NNfwP-z5dQ9iC0mmV8xId4ijRj-C9f3ejSKZhLE8Isd370ezF68EGVX3AfbUrphS2Xd07H-tZ8rftU5i-e0Ef1yWuEGdSGZiLyGHiTR-etXac/s1600/dfs_explorer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHC4HQTvSdbgw7TdUQ3kPuZ154y9cDL5NNfwP-z5dQ9iC0mmV8xId4ijRj-C9f3ejSKZhLE8Isd370ezF68EGVX3AfbUrphS2Xd07H-tZ8rftU5i-e0Ef1yWuEGdSGZiLyGHiTR-etXac/s1600/dfs_explorer.png" height="307" width="320" /></a></div>
<div>
<br /></div>
<div>
Oooh yeah. Actual files! Readable ones, too. Just to make sure that everything was working properly, I made another image of the same disk and compared the two. Some bad sectors seemed to throw it off, but the good sectors did not seem to have any differences.</div>
<div>
<br /></div>
<div>
I could also load the disk into the emulater <a href="http://www.mkw.me.uk/beebem/" target="_blank">BeebEm</a>, but that seemed to lack the <a href="https://en.wikipedia.org/wiki/Wordwise" target="_blank">Wordwise</a> word processing ROM, so I couldn't view any of the files. Time to try with a disk containing actual programs; mostly games I used to play (but didn't write).</div>
<div>
<br /></div>
<div>
Track 0: <span style="font-family: Courier New, Courier, monospace;">OK</span>. Track 1: <span style="font-family: Courier New, Courier, monospace;">OK</span>. Track 2: <span style="font-family: Courier New, Courier, monospace;">OK</span>. And so on… this disk seemed <i>absolutely fine</i>! Excited, I loaded it into BeebEm, and got a scary message: "WARNING - Incorrect disc type selected? This disc file looks like a double sided disc image. Check files before copying them." Indeed, I could get a disk catalogue but everything I tried to load gave me <span style="font-family: Courier New, Courier, monospace;">Bad program</span>.</div>
<div>
<br /></div>
<div>
Eventually, I figured out that naming it <span style="font-family: Courier New, Courier, monospace;">spel.dsd</span> (for double sided disk, as opposed to <span style="font-family: Courier New, Courier, monospace;">spel.ssd</span> or <span style="font-family: Courier New, Courier, monospace;">spel.img</span>) fixed things. I was in business! Look at all the cool stuff on this disk!</div>
<div>
<br /></div>
<div>
The CATALOG:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPoD4PuLHaTxnoT7jlYg7fGtL0QxTUmbLjILppuvyJ8TXStpcgg_1a4QRlewJkL_PG6VJXaxM6hHhLgQopJ4LjzBHiAqcmTDNDVF4Qi9OpXdVHo9HOWDsFpOYKSlCxiCbrz-ewX_onZ58/s1600/catalog.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPoD4PuLHaTxnoT7jlYg7fGtL0QxTUmbLjILppuvyJ8TXStpcgg_1a4QRlewJkL_PG6VJXaxM6hHhLgQopJ4LjzBHiAqcmTDNDVF4Qi9OpXdVHo9HOWDsFpOYKSlCxiCbrz-ewX_onZ58/s1600/catalog.png" height="256" width="320" /></a></div>
<div>
<br /></div>
<div>
The exciting game MUNCHING MOUSE, where you run from left to right in the shortest time possible, collecting the cheese:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuO_IihYxtugXv4dTQSjYko8XcGdQwpKfZstPE8AZ9aARJiIjr7C1-TAlzF4aCxe_3MQeqLnLqVHd0ei9hM_Wuvlf_FiBVz_qI6oUtT-kC2HQlGpd4hTlFZIO5wWrhiWzsath1VS6PIAs/s1600/mouse.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuO_IihYxtugXv4dTQSjYko8XcGdQwpKfZstPE8AZ9aARJiIjr7C1-TAlzF4aCxe_3MQeqLnLqVHd0ei9hM_Wuvlf_FiBVz_qI6oUtT-kC2HQlGpd4hTlFZIO5wWrhiWzsath1VS6PIAs/s1600/mouse.png" height="256" width="320" /></a></div>
<div>
<br /></div>
<div>
The even more exciting game BAT'N'BALL, which… well, you guessed it:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidKwXwszcpj0V9l3I1141BgUJJZNL1FnurNK73FcIp48HV0_to5WTF5NEhvcatZxSJ1PsYe_nNApgoobtqpkZVaxfqmqN-XYhSvK_UaxdKX1vkB4xgZhnNu5MxjguC3QJ54dX16M6XUa4/s1600/batnball.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidKwXwszcpj0V9l3I1141BgUJJZNL1FnurNK73FcIp48HV0_to5WTF5NEhvcatZxSJ1PsYe_nNApgoobtqpkZVaxfqmqN-XYhSvK_UaxdKX1vkB4xgZhnNu5MxjguC3QJ54dX16M6XUa4/s1600/batnball.png" height="256" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The <i>actually</i> somewhat interesting YELLOW RIVER KINGDOM, where you assign your villagers to (a) defend the dyke to prevent flooding, (b) work in the fields to grow rice, or (c) protect the villages from thieves:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGJs4ZrXmjILp2GcbpTlWF8h-eJj1sGNfW2Q9ofDhynEn9Ld4JEK9jNbfpxICQpmhP6Yox6rHitM6UvWziXdEGprFZlwZ8cXS7slpz1jAHwbka263Eua0eV_LI3rliOQfpWaIqwVVMYDc/s1600/kingdom.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGJs4ZrXmjILp2GcbpTlWF8h-eJj1sGNfW2Q9ofDhynEn9Ld4JEK9jNbfpxICQpmhP6Yox6rHitM6UvWziXdEGprFZlwZ8cXS7slpz1jAHwbka263Eua0eV_LI3rliOQfpWaIqwVVMYDc/s1600/kingdom.png" height="256" width="320" /></a></div>
<div>
<br /></div>
<div>
And on the other side of the disk, I rediscovered one of my childhood classics: CASTLE OF NIGHTMARES. Damn, this game is <i>hard</i>. I can't seem to be able to record uncorrupted video with BeebEm, but here's the music:</div>
<div>
<br /></div>
<div>
<iframe frameborder="no" height="300" scrolling="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/195847307&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=true" width="100%"></iframe>
<br />
<br />
Hmm, maybe a bit repetitive? How about this great tune instead? The soundtrack of my childhood!<br />
<br />
<iframe frameborder="no" height="300" scrolling="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/195866946&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=true" width="100%"></iframe><br />
<br />
<div style="text-align: left;">
Another great program I recovered was "Speech!", a wonderfully retro speech synthesizer that sounds like this:<br />
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/t8wyUsaDAyI" width="420"></iframe><br /></div>
<br />
Now I really want to use that in a game some day.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Speaking of games, what about this '80s version of Kerbal Space Program?<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbcFBr35FA8duyt8zU012OA7x7dqXRlcAds1zkyjPVEEnmMnZaaCMX1xg9IY-_pgQ9-bPF4W3cTBzuhhFwEZme_0qVh9BM_EkJ9Pzb15UfMcr-N5kI3X6qdfGEa_BOzZBkBHcUJQTMhYs/s1600/maanlander.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbcFBr35FA8duyt8zU012OA7x7dqXRlcAds1zkyjPVEEnmMnZaaCMX1xg9IY-_pgQ9-bPF4W3cTBzuhhFwEZme_0qVh9BM_EkJ9Pzb15UfMcr-N5kI3X6qdfGEa_BOzZBkBHcUJQTMhYs/s1600/maanlander.png" height="320" width="400" /></a></div>
<br />
I once wrote a PC clone of that in QBasic when I was 13. And a sequel to that clone. And a mouse-driven GUI level editor for the sequel.<br />
<br />
There were also a few files I couldn't readily identify. They were neither text files nor BASIC programs. Looking at the hex dump, I noticed there were some repeating patterns: a streak of 0xff bytes, then some other bytes, then another streak of 0xff bytes, and so on. Could these files be… images? And how would you store an image on the BBC Micro, if not by writing the contents of the video memory directly to disk? We can reverse that with a simple <span style="font-family: Courier New, Courier, monospace;">*LOAD</span> command!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqSTqGp6y46TsbU0Ga48Ramo2AUzKG8kBrI8OwyR3aH-KStBbQzsfN2xpbZ1cjovFNXZ7qgwflvyBE6GnXV4bGaVcz8s8uFmZlpF6vTSjSGoIHYqKHgOFIarW5CvoBm1PSfcS0KNrz5d4/s1600/zeeschorpioen.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqSTqGp6y46TsbU0Ga48Ramo2AUzKG8kBrI8OwyR3aH-KStBbQzsfN2xpbZ1cjovFNXZ7qgwflvyBE6GnXV4bGaVcz8s8uFmZlpF6vTSjSGoIHYqKHgOFIarW5CvoBm1PSfcS0KNrz5d4/s1600/zeeschorpioen.png" height="320" width="400" /></a></div>
<br />
Copyright © yours truly, year unknown. Notice how IKON, the drawing program, didn't even bother to remove its own UI elements from the saved file.<br />
<br />
So far, I'd only converted compilation disks, with programs probably typed in from magazines and such. Most of the old disks were still 100% readable, and although some had bad sectors, even they were largely still usable. Eventually, this gave me the courage to put in a disk with my <i>own</i> programs. Apparently, I once started work on a Breakout clone, imaginatively named BREAKOUT. The ball bounces and the paddle moves, but blocks you hit don't disappear and the game crashes when you die.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpBulJdyjE9D8vmcT_rqhicjjhHtdFugNfTAwRPoCqE6YpsStIH-NoPfJzCo2-q9_d5c28nS8j6O3EaXuGdiLXtvR05VqG10FeyJNiZb9EOvOT0kt-Ui46WypVbORcV9xic49WHyJgRXQ/s1600/breakout.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpBulJdyjE9D8vmcT_rqhicjjhHtdFugNfTAwRPoCqE6YpsStIH-NoPfJzCo2-q9_d5c28nS8j6O3EaXuGdiLXtvR05VqG10FeyJNiZb9EOvOT0kt-Ui46WypVbORcV9xic49WHyJgRXQ/s1600/breakout.png" height="320" width="400" /></a></div>
<br />
There was also a Mastermind game that I was sure I couldn't have written, until I encountered my name in the comments. Go me for commenting stuff! But it appeared to give me contradictory clues, so maybe that wasn't finished either.<br />
<br />
I even dabbled in cryptography! Can you figure out the algorithm?<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_RcUVRd55DVvpEwXdYXJaAkuAB-oNXlmNYLYDwCcpt3uiqxdio_92bys4BTgXb6Ne3R-F8uOKGxXrgEYHi3brQmQRSQiWH8UzGD6sj5I8CSvy2BqWK1pZL9mAIVIhAnhHoB09qkA4BE8/s1600/crypto.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_RcUVRd55DVvpEwXdYXJaAkuAB-oNXlmNYLYDwCcpt3uiqxdio_92bys4BTgXb6Ne3R-F8uOKGxXrgEYHi3brQmQRSQiWH8UzGD6sj5I8CSvy2BqWK1pZL9mAIVIhAnhHoB09qkA4BE8/s1600/crypto.png" height="320" width="400" /></a></div>
<br />
And apparently I had some database program cunningly disguised as MARIO, which was password-protected; look at how I obfuscated the password (didImentionthatthingaboutspacesbeingoptional?):<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">170IFLEFT$(T$,9)=CHR$(74)+CHR$(5*13)+CHR$(79-1)+CHR$(68)+CHR$(80-1)+CHR$(59+10)+CHR$(70-2)+CHR$(70-1)+CHR$(76)THENPRINTTAB(12,13)"Wachtwoord correct":T=TIME:REPEATUNTILTIME-T>200:GOTO240</span><br />
<br />
Never mind that if somebody could read this code, they could also just replace it by <span style="font-family: Courier New, Courier, monospace;">GOTO240</span> and have instant access – but it might have kept out non-programmers who could still type <span style="font-family: Courier New, Courier, monospace;">LIST</span> (i.e. my brother). I don't think I ever even put any data into this…<br />
<br />
I did have some trouble with bad sectors during the salvage operation, but eventually, bumping it up to a hundred retries or so, every last bit came off of the disks I cared about. Even then, one disk gave me trouble: ADFS Explorer would read it just fine, but the BeebEm emulator reported <span style="font-family: Courier New, Courier, monospace;">Bad directory</span> instead of giving me a listing. Diving into the <a href="http://mdfs.net/Docs/Comp/Disk/Format/ADFS" target="_blank">internals of the ADFS filesystem</a> showed why: the directory's sequence counter is written to disk twice, once in the directory header and once in the footer, and the two must match. A one-byte change in a hex editor later, and this last disk image too was perfectly usable. I don't know if it was bitrot, or if the disk had been corrupted previously during a write operation.<br />
<br />
I'd like to put all my old stuff up on GitHub at some point, but the disk images contain some private information (e.g. address stickers, letters I typed up for other people) so I'll have to curate them first. But the bits are safe now, which is what counts!</div>
</div>
</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com3tag:blogger.com,1999:blog-3365890222783870731.post-78007872575529005532015-01-23T15:23:00.000+01:002015-01-23T15:23:00.485+01:00QR codes on the BBC Micro, Part 6 of 6: An unexpected ending<div dir="ltr" style="text-align: left;" trbidi="on">
<i>In which I convince myself that this plan would have worked.</i><br />
<br />
So, with all the code written, and a pixel-perfect QR code on the screen, the question was once again: will it scan?<br />
<br />
Disappointment. The ZXing Android app wouldn't recognize the code, even though it had little problems with its twin on the flat TFT screen of my PC.<br />
<br />
Since the pixels were, as far as I could tell, identical, I figured this must be due to image quality. The code was displayed on an old fish-bowl style CRT monitor, which means it's bulging a little. Perhaps this was throwing off alignment of the reader. Another hypothesis was blooming: white pixels tend to “glow” around the edges, possibly bleeding into black pixels and making them be registered as white by the reader.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7MECQzEi5C2rr7SRT8t8637JODByOABaxlUrJzcSww8sxAlkPr3E8BXa2J2lSGuIu2MNkJBWtSsAdUe_aJDQPoW16lU2SVb_2FgDVpF4Wrwc0H7r0XYeS9kxUJQ53LU9H14Mmzov4AEE/s1600/IMG_0488.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7MECQzEi5C2rr7SRT8t8637JODByOABaxlUrJzcSww8sxAlkPr3E8BXa2J2lSGuIu2MNkJBWtSsAdUe_aJDQPoW16lU2SVb_2FgDVpF4Wrwc0H7r0XYeS9kxUJQ53LU9H14Mmzov4AEE/s1600/IMG_0488.JPG" height="240" width="320" /></a></div>
<br />
I tried to remedy both of these by doing some postprocessing. I started by taking a photo of the screen using my Canon PowerShot S90 digital camera, which is a tad better than the camera in my Nexus 5 phone. Then I opened that in Gimp, sharpened it, and applied the Lens Distortion filter to make a mostly straight-looking copy.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRVbKqGscXdVufXwlrWc2XWAA44SIw7Hip23ECaQduuGr9oqPi3_rajA5aGhifGISFQeP1YCGc1XlnD7uyRBoUPVL0SOIYiJjf2Wzt5QV0MEWR5vLdr9PJsbHIeQwoiLPQx5tBLpAk5Vk/s1600/IMG_0488.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRVbKqGscXdVufXwlrWc2XWAA44SIw7Hip23ECaQduuGr9oqPi3_rajA5aGhifGISFQeP1YCGc1XlnD7uyRBoUPVL0SOIYiJjf2Wzt5QV0MEWR5vLdr9PJsbHIeQwoiLPQx5tBLpAk5Vk/s1600/IMG_0488.png" height="240" width="320" /></a></div>
<br />
Then I tried reading from this image using <a href="http://zbar.sourceforge.net/" target="_blank">Zbar</a>, a command line barcode reader. It wouldn't recognize it. I also tried libdecodeqr following <a href="http://askubuntu.com/questions/22871/software-to-read-a-qr-code" target="_blank">this AskUbuntu question</a>, but no dice. Time to hack up my own.<br />
<br />
Based on previous success with the app, I grabbed the <a href="https://github.com/zxing/zxing/" target="_blank">ZXing source code</a> from GitHub and wrote a Java program to load the PNG file and feed it to the library. The idea was that this would let me debug things and figure out <i>why</i> it wasn't recognizing the QR code.<br />
<br />
The first problem was that ZXing was throwing a NotFoundException without a stack trace. It turns out that the library is optimized for mobile, memory-constrained devices; there is only one instance of the exception that gets reused every time, and stack traces are suppressed. This was easily fixed by hacking up the library code. Now at least I got an answer: it was failing to detect the alignment patterns, the three big squares on the corners.<br />
<br />
Diving into the code, I added some print statements to output the location of potential alignment patterns detected in the image. It found four: the three correct ones, and a spurious one in the bottom right corner. This candidate was chosen as the correct one for reasons that eluded me, but I worked around it by removing any candidate found in the bottom right quadrant of the image. Remember, I wasn't trying to build a general-purpose QR code reader; I just wanted to see if <i>this particular</i> code could be scanned before I put all my eggs into its basket.<br />
<br />
With the correct alignment patterns, the library got a lot further: it correctly read the version and format info, and subsequently threw a ChecksumException. I printed the data bytes that it had read, expecting mostly <span style="font-family: Courier New, Courier, monospace;">ff</span> bytes with the occasional error. Instead, I got this sort of garbage:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">99 99 00 66 85 00 00 00 60 00 7f fd 00 00 9d 1e 40 ff 1b 00 49 00 02 ff ff 00 00 02 60 40 ff 00 f7 ff 00 4f 00 00 00 ff 99 00 f6 00 00 00 01 00 00 9a 00 00 00 00 9d 00 ff 37 ff 00 ff ff 00 ff 00 ff 00 ff ff 00 ff 40 ff 40 ff ff 00 ff ..</span><br />
<br />
Clearly, no amount of error correction could have fixed that mess. I figured that, in order to make this thing robust and reliable, I would have to resort to larger pixels: representing one module in the QR code as 2×2 pixels on the screen of the Beeb. Doing the math, I worked out that I could use at most a Version 24 QR code, able to contain slightly over 1 kB of data. Not as much as the 3 kB of the Version 40, but still good enough.<br />
<br />
Unfortunately, I had been writing much of my code under the assumption that the version would not need to be changed, so there were hard-coded numbers all over the place. I turned back to the BBC Micro and started replacing them by variables. First, with the correct values for Version 40 so I could verify that the output didn't change; then I would replace them by the appropriate values for a Version 24 code, and change the pixel-plotting routine to output 2×2 pixels instead. I was halfway through this process when disaster struck.<br />
<br />
When saving the program to tape as a matter of routine, the computer unexpectedly printed:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">>SAVE"QR"</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">RECORD then RETURN</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">QR 0A</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Index</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">></span><br />
<br />
“Index”? Excuse me? I rewound the tape, wanting to try again, but found that the machine was no longer responding to my keystrokes. Somewhat nervous, I pressed the BREAK key, essentially rebooting the OS, and typed <span style="font-family: Courier New, Courier, monospace;">OLD</span> to recover the program from its original memory location. Phew, that worked, it was still intact. I tried saving again, but this time, the counter didn't even start running. I really didn't want to hard reset the machine, because I assumed that the failed save would have corrupted the tape, so the computer's RAM was the only place where the QR program still lived.<br />
<br />
A few more tries, a few more reboots, nothing helped. Until, suddenly during a <span style="font-family: Courier New, Courier, monospace;">SAVE</span> command, the screen got garbled and seemingly random pixels scrolled across the screen. I pressed BREAK again, expecting to end up at the BASIC prompt again, but instead was dumped into the <a href="https://en.wikipedia.org/wiki/ViewSheet" target="_blank">ViewSheet</a> spreadsheet program. I had no idea that this machine even had that ROM chip installed. The fact that I booted into it seemed like a severe case of memory corruption. I guessed, correctly, that <span style="font-family: Courier New, Courier, monospace;">*BASIC</span> would get me back to BASIC mode, but by now <span style="font-family: Courier New, Courier, monospace;">OLD</span> didn't work anymore, and my program was definitively gone from the computer's memory.<br />
<br />
Now what? Let's cold-boot the machine and see what can be recovered from tape. But it seemed that the poor old thing was too far gone for that… instead of booting, it gave a continuous tone from its speaker, didn't show anything on the screen, and didn't respond to the BREAK key. I tried powercycling it literally hundreds of times, each time getting a slightly different tone and sometimes getting the keyboard LEDs to light up, but I couldn't get the machine to boot anymore. It seemed that my trusty old pal had finally given up the ghost.<br />
<br />
I had hoped to make the QR code program send itself across to the PC as a maiden voyage. I'm quite confident that (perhaps with even larger pixels) I could have made it work. Then I could post a YouTube video of a data transfer in progress, to help explain to my family what I've been up to in the past few weeks. I could publish the code on GitHub, annotate it, explain it in detail on this blog. Maybe it would even be useful to someone, although I didn't expect it to. I don't think any of that is going to happen anymore.<br />
<br />
But of course, this QR code program was just a means to an end. The point of this project was to transfer all the old programs I wrote as a kid. Those programs are still there, sitting on a stack of 5 1/4" floppy disks on my desk, so close and yet so far away. If the machine would have worked again, I would have typed a simpler program, too small to need saving, to try and transfer them via sound signals. That, too, isn't going to happen anymore. Even if I could fix the computer itself, the disk drive also seems to be broken since the machine won't even boot when it's plugged in.<br />
<br />
Fortunately, there is another way: a controller board like the <a href="http://www.kryoflux.com/" target="_blank">KryoFlux</a> can be used in conjunction with a PC style 5 1/4" drive to rescue data off of virtually any type of disk. I just ordered a KryoFlux board, and placed a bid on eBay on a compatible floppy drive. Fortunately, working drives for PCs seem to grow on trees still. The game is not over.<br />
<br />
But for my old BBC Microcomputer, it is. The very device that launched my career as a software engineer will never run again. When I'm done mourning it, I'll probably take it to an electronics recycling facility. May it rest in peace.</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com6tag:blogger.com,1999:blog-3365890222783870731.post-42531249650072591082015-01-22T15:20:00.000+01:002015-01-22T15:20:00.584+01:00QR codes on the BBC Micro, Part 5 of 6: Developing on the Beeb<div dir="ltr" style="text-align: left;" trbidi="on">
<i>In which I marvel at how programming has changed in the past 30 years and how it has remained the same, and fret over spurious hardware failures.</i><br />
<br />
After having written a program on the PC to generate QR codes, it was time to do the same thing on the BBC Micro. During a long train journey, I had already written most of the assembly code on paper, so it would be a simple matter of typing it in, and writing the surrounding glue code in BASIC.<br />
<br />
I switched on the machine and… nothing happened. Just a slight “pop” from its speaker. A few more tries got me the “boooo” tone normally followed by a cheerful “beep!”. Another few tries, and it went “boooo-beep!” as normal, showed me a prompt and responded to my keystrokes, then froze after a few seconds. Another reboot, and it worked happily for hours, until I turned it off and went to bed.<br />
<br />
This was a repeating pattern over the next few days, as I wrote and debugged my program. Every time I wanted to turn the machine on, it took more attempts to get it to work. But every time, it eventually worked. On the last day, it took so long to get the machine working that I just left it on overnight.<br />
<br />
But when it worked, it worked well. I found that I loved the predictability of this machine. There is 32 kB of RAM in a single address space, no virtual memory, no processes, no threads. Just your program and the OS – which mostly gets out of the way while your program is running, so you are in full control of the machine.<br />
<br />
Compared to modern machine architectures, this is wonderfully simple. There is no distinction between the interpreter, the shell and the editor; the BASIC prompt serves all three purposes just fine. It acts as a REPL, so you can quickly try out oneliners and see if they have the desired effect. If you have broken out of your program with the Escape key or the <span style="font-family: Courier New, Courier, monospace;">END</span> statement, all variables remain intact for your inspection, essentially giving you non-resumable breakpoints. The prompt also serves as a shell, so you can save and load programs, list and manage files, and so on.<br />
<br />
But the most interesting thing compared to present-day systems is that there is no standalone editor; the prompt <i>is</i> your editor. To make this work, every line in a BASIC program has a number. When you type a line that starts with a number, it gets added to the program in the appropriate place. When you use a line number that already exists, it replaces that line. The <span style="font-family: Courier New, Courier, monospace;">LIST</span> command (abbreviated <span style="font-family: Courier New, Courier, monospace;">L.</span>) is used to show the current program, or a range of line numbers that you pass to the command.<br />
<br />
But what if you don't want to add or replace an entire line, but just make some change to an existing line? This is what the COPY key is for. Using the arrow keys, you can control a secondary “copy source” cursor, which can be positioned anywhere on the screen. When you press COPY, the character under the copy cursor is copied as if you typed it on the keyboard, and both the regular and the copy cursor are moved to the right. This enables you to copy lines from a listing while changing them. It also enables interactive development: you try running a line of code, and when you got it to work, you prefix it with a line number and COPY it into your pogram. Your screen essentially doubles as the equivalent of the clipboard in modern operating systems. But when something scrolls off the top of the screen, it's gone forever, so this mechanism isn't all-powerful.<br />
<br />
So far, I've talked about BASIC. But I couldn't write the entire program in BASIC, because it's too slow. It takes about 15 seconds to simply invert all screen pixels, something a machine code program can do in much less than a second – say, 100 times faster. My back-of-the-envelope calculations made it very clear that a 100× slowdown would be unacceptable. But how do we write and edit machine code?<br />
<br />
It turns out that BASIC fulfils yet another purpose on this machine: it acts as an assembler. When you write assembly instructions between the characters <span style="font-family: Courier New, Courier, monospace;">[</span> and <span style="font-family: Courier New, Courier, monospace;">]</span>, upon interpreting these, BASIC generates the appropriate machine code instructions at the address indicated by the variable <span style="font-family: Courier New, Courier, monospace;">P%</span>. The great thing is that it's still “just BASIC”, and you retain the ability to use BASIC expressions to compute values inside the assembly code. This makes for a really powerful system where you get the best of both worlds: the speed of assembly, but the expressive power of BASIC as a kind of preprocessor.<br />
<br />
All in all, I was pleasantly surprised by how easy and fast it was to develop directly on the BBC Microcomputer. Of course the development cycle isn't as fast as on a modern system, but it comes surprisingly close. It took me just a few evenings to type in and debug most my hand-written assembly routines. After writing each routine, I wrote a few simple unit tests to make sure it was working, but I threw them away after they passed to save space.<br />
<br />
I started by drawing the structural elements of the QR code using BASIC <span style="font-family: Courier New, Courier, monospace;">MOVE</span> and <span style="font-family: Courier New, Courier, monospace;">DRAW</span> commands, making the parts white where no data would go. Then I turned to assembly to “snake” the data through it. Then some more BASIC to cut out the black pixels of the structural elements, and finally a simple assembly routine to invert all pixels on the screen, because when drawing a black-on-white QR code on the white-on-black screen, I found it easier to think “inverted”. Then some more assembly code to prepare the data codewords by shifting each data byte down by half a byte.<br />
<br />
Then, one evening, when I just needed to implement the error correction calculations to complete the program, the computer wouldn't turn on at all anymore. No “boooo” sound, no “pop”, not even the usual slight hiss, no keyboard LEDs, nothing. Retrying did not help. I started eliminating possible causes. First I unplugged the monitor. No dice. Then I unplugged the tape deck. No dice.<br />
<br />
Then I unplugged the disk drive. “Pop”, the machine said. A few more tries, and it was running again. Turned it off, plugged the disk drive's power cable (but not its data cable) back in, turned it on, nothing. So it seemed that the power cable was somehow messing with the boot process. Maybe a short circuit in the cable, or in the drive?<br />
<br />
I decided to worry about this later. First I wanted to see if I could make a scannable QR code containing some useful data. I put the final pieces of code in place and rendered a QR code containing the byte 255, repeated 2953 times. I did the same on the PC and checked by hand that the pixels looked about right.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6mjyQM3tslxnThna-sjha6Z0LVeZbkpLzJLWnihSENoWfNaXSxT_pVPW0nzKecVEH8kNryRHlmjoEgHI3h79zX5PuBOwq2o2SIJGHiF6FPA407WQB5DEcV-qLnpyKMrYZD0YxviD6HVc/s1600/IMG_20150109_183020.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6mjyQM3tslxnThna-sjha6Z0LVeZbkpLzJLWnihSENoWfNaXSxT_pVPW0nzKecVEH8kNryRHlmjoEgHI3h79zX5PuBOwq2o2SIJGHiF6FPA407WQB5DEcV-qLnpyKMrYZD0YxviD6HVc/s1600/IMG_20150109_183020.jpg" height="240" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A bunch of 0xff bytes in a QR code on the Beeb's screen</td></tr>
</tbody></table>
Once more, the question was: will it scan?</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com0tag:blogger.com,1999:blog-3365890222783870731.post-8615720076884433462015-01-21T15:11:00.000+01:002015-01-21T15:11:00.058+01:00QR codes on the BBC Micro, Part 4 of 6: Prototyping in C<div dir="ltr" style="text-align: left;" trbidi="on">
<i>In which I wrap my head around algebra and write code for myself, not the machine.</i><br />
<i><br /></i>
The preliminary feasibility study turned out positive, the hardware was sufficiently cooperating, so I was cautiously optimistic that this thing was possible. Time to learn what QR codes are all about.<br />
<br />
The QR code standard is behind a paywall at ISO, but it's trivial to find online elsewhere, and before long I had a 124-page document on my screen. I began to read. For an introduction that's easier to digest, I recommend <a href="http://www.thonky.com/qr-code-tutorial/" target="_blank">this guide at Thonky.com</a>.<br />
<br />
So, there are various sizes of QR code. I already knew that. The standard talked about various alphabets, including numeric, alphanumeric and Kanji, but fortunately, raw bytes are also among the options. Then it describes in detail where all the standard parts go: the big corner squares, the smaller inset squares, and the barely noticeable timing patterns along the top and left. So far so good, I can code that.<br />
<br />
Then it starts getting complicated. The stream of codeword bits has to be snaked through the matrix and in between the other parts in a very particular way: left-up-right-up until you hit the top, then left-down-right-down until you get back to the bottom. Sometimes not all these modules are part of the timing or alignment patterns, and you need to skip them. At the leftmost timing pattern, we skip the entire column. A lot of fiddly edge cases, but doable.<br />
<br />
As a final step, we have to apply a XOR mask to all data bits, to make sure no patterns appear that look like the timing and alignment patterns and might throw off the QR code reader. The standard specifies eight different masks, and a QR encoder is supposed to try them all. Then it should choose the one that minimizes some penalty computed from patterns that appear in the output. I decided to just go with a single type of mask first, the one that was easiest to compute, and hope for the best.<br />
<br />
But the really hard part is actually generating the codeword stream. The data is broken up into blocks, and for each block we have to compute a set of error correction codewords. In the case of the 40-L QR code, there are 19 blocks of 118 codewords, and 6 blocks of 119 codewords each; for each of these blocks, we have to compute 30 error correction codewords. The error correction codewords are simply the coefficients of the remainder polynomial when we divide the data polynomial by a particular 30th-degree generator polynomial with coefficients in the GF(256) Galois field. <i>Wait, what?</i> The words reminded me of a class on error correction codes I took in university, but I had all but forgotten the details. How do you do polynomial long division? What is a generator polynomial? What is a Galois field?<br />
<br />
I won't explain it all in detail here, but again, <a href="http://www.thonky.com/qr-code-tutorial/error-correction-coding/" target="_blank">Thonky.com has you covered</a>. To summarize: polynomial long division is just like regular long division; if you divide by a polynomial of degree <i>n</i>, you get a remainder polynomial of degree <i>n </i>– 1, at most. The generator polynomial can be calculated somehow, but I didn't need to know, because it was listed in an appendix. This particular Galois field is a field with 256 elements, and if we represent those by 8-bit numbers, addition can be done by the XOR (exclusive or) operation. For multiplication, we have a log and antilog table, which let us do multiplication as addition, preceded and followed by table lookups. Nice!<br />
<br />
So, time for some code. I decided that this stuff was complicated enough that I wouldn't be able to learn it, re-learn 6502 assembly, wrestle with memory limits, and debug the code at the same time, so I wanted to build a prototype on the PC in C99 instead. It was a simple matter of programming (ahem) that I spent some free evenings and weekends on. I wrote it very carefully, in the same way I'd have to do on the Beeb, checking output manually at every step. This would have been perfect for TDD (test-driven development), but I wouldn't have that luxury on the Beeb due to memory constraints, so I refrained from adding too many tests.<br />
<br />
A few weeks later I had about 350 lines of C code that took on its standard input the ASCII text of the book I'm currently reading, and produced a PNG file that looked like this:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWU2gY-GnpqZ5XUYpBfr5-BGB7fTOfU6Q8-Zu813lqoHpxcn2VAVk6nDDqa_h_UZtyYNijROtZyreBYUq4ygi6MIYJCBKKBNBo2wKbyobWpz9pMJxlsrc-RPQG2BhDr8Yy9QXlTbL_yPs/s1600/test.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWU2gY-GnpqZ5XUYpBfr5-BGB7fTOfU6Q8-Zu813lqoHpxcn2VAVk6nDDqa_h_UZtyYNijROtZyreBYUq4ygi6MIYJCBKKBNBo2wKbyobWpz9pMJxlsrc-RPQG2BhDr8Yy9QXlTbL_yPs/s1600/test.png" height="320" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Output of the C program</td></tr>
</tbody></table>
The code is <a href="http://github.com/ttencate/beebqr" target="_blank">up on GitHub</a> if you want to take a look. You can see some repeating patterns; this is because it's ASCII text, where not all bits are equally well-trodden. But now again that big question… <i>will it scan?</i> Let's see!<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXbCSRadgRkAP0I-YAbIi4XsoJKQxD7nQyr7lQ1-JL-X6UXazUrp4_ACJuUB8KzpdooFBlJuYBkLs66nNsOUwlwgrfrpFSIh5wG5Nw0HEl48hSaZ5SN_MkA5oG2h5BWiVFv_ejy-bZBl8/s1600/Screenshot_2014-12-17-20-59-14.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXbCSRadgRkAP0I-YAbIi4XsoJKQxD7nQyr7lQ1-JL-X6UXazUrp4_ACJuUB8KzpdooFBlJuYBkLs66nNsOUwlwgrfrpFSIh5wG5Nw0HEl48hSaZ5SN_MkA5oG2h5BWiVFv_ejy-bZBl8/s1600/Screenshot_2014-12-17-20-59-14.png" height="180" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">ZXing Barcode Scanner recognizing the above QR code</td></tr>
</tbody></table>
Oooh yeah! Hole in one! Victory!<br />
<br />
So, how big is this code? Remember that the BBC Micro has only 32 kB of RAM, about 16 of which we can actually use in the screen mode we're targeting. I estimated that I'd need about 5 kB of scratch memory to generate the QR code. The C source weighs in at 11.5 kB, almost exactly the space we'd have left on the Beeb, but of course that doesn't mean much. The binary program on the Beeb might be larger because of the limited instruction set and lack of standard library, or smaller because it doesn't contain semicolons and whitespace.<br />
<br />
A better comparison would be the compiled C code; the .o file is only 8.7 kB. But it would also contain headers and such, a symbol table, maybe even debugging symbols; how much executable code is actually in there? We can find out using <span style="font-family: Courier New, Courier, monospace;">objdump </span><span style="font-family: Courier New, Courier, monospace;">-d</span>, which gives a nice disassembly of the executable code, next to the raw bytes so I could count them with a simple shell pipeline:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">$ objdump -d qr.o|egrep '^ *[a-f0-9]*:'|cut -c11-32|wc -w</span><br />
<span style="font-family: Courier New, Courier, monospace;">4762</span><br />
<div>
<br /></div>
<div>
So that's 4.7 kB of x86-64 code, with its weird variable-length instruction set. On average, these instructions were 3.7 bytes long, whereas the 6502 instructions on the Beeb are 1-3 bytes each (instruction plus up to 16 bits of operand), so on average probably about half the size. That was the good news. The other good news was that gcc probably didn't produce the smallest code possible. The bad news was that an x86 processor can do 32-bit arithmetic, whereas on the Beeb you have to hand-roll everything that's more complicated than 8-bit addition. Even 16-bit addition and multiplication are not supported by the hardware. So the comparison could go either way.</div>
<div>
<br /></div>
<div>
So why not cross-compile this C code to target the 6502 processor, and type it into the Beeb? Maybe I could have. But I felt it would be more trouble than it's worth; one typo and the program is ruined, but if I hadn't written the assembly myself, there'd be no way I'd be able to debug it. Also, I wasn't sure that a compiler targeting such an old CPU would be any good; the <a href="https://github.com/cc65/cc65" target="_blank">only one I tried</a> didn't even get past the parsing phase, and choked on my C99-style variable declarations. I also had <a href="http://github.com/ttencate/tis" target="_blank">some experience</a> with writing tiny code and felt up to the challenge.</div>
</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com0tag:blogger.com,1999:blog-3365890222783870731.post-78573146933412026532015-01-20T15:03:00.000+01:002015-01-20T15:03:00.380+01:00QR codes on the BBC Micro, Part 3 of 6: Storing progress<div dir="ltr" style="text-align: left;" trbidi="on">
<i>In which I fight an uphill battle with barely functioning hardware so I don't lose my precious bytes.</i><br />
<br />
As I started looking into the standard for QR codes, and browsed some tutorials (<a href="http://www.thonky.com/qr-code-tutorial/" target="_blank">this one is excellent</a>), it dawned on me that generating QR codes is by no means simple. Putting in all the fixed elements (like the three corner squares and smaller internal squares) is easy enough, but the data stream needs to be snaked through this in a very particular (and peculiar) manner, and even coming up with the data stream requires a fair amount of algebra. This wasn't going to be a Sunday afternoon project.<br />
<br />
That meant I needed some way to save my code on the Beeb while I was working on it. Remember, it has no internal hard drive; typical storage options are 5¼" floppies and cassette tapes. Tapes are a bit of a pain to work with: your programs are stored much in the same way as songs, so you need to remember or write down where on the tape they are stored, and fast forward/rewind to the right location. It's also pretty easy to accidentally overwrite (part of) an existing program.<br />
<br />
So let's look at the disk drive first. I knew that it worked for reading floppies, otherwise this whole project wouldn't have gotten off the ground, but I realized I hadn't actually tested writing. I dug up a disk with some unused data on it, and tried to format it. Formatting will also check whether the disk is still okay, so I ran less risk of losing my work.<br />
<br />
This turned out to be a no-go. The <span style="font-family: Courier New, Courier, monospace;">*F80</span> and <span style="font-family: Courier New, Courier, monospace;">*F40</span> commands found in some old manual both resulted in the error “Bad command”. The <a href="http://chrisacorns.computinghistory.org.uk/docs/Acorn/Manuals/Acorn_DiscSystemUGI2.pdf" target="_blank">DFS Manual</a> I found online offered <span style="font-family: Courier New, Courier, monospace;">*FORM</span>, but it didn't work for me either. Maybe the disk ROM installed in this particular machine is of a different breed, I don't know. (Yep, that's how you installed “device drivers” lacking an internal disk drive. You would use your fat fingers to open up your £1000 computer and poke an extra chip into it. Those were the days.)<br />
<br />
Anyway, lacking a format command, I figured I'd just delete the existing files. How hard can it be? <span style="font-family: Courier New, Courier, monospace;">*DELETE oldfile</span> should work, right? Nope, <span style="font-family: Courier New, Courier, monospace;">Disk read only</span>. Is it? These old floppies have a small notch on the side, and you can cover it with a sticker to make the disk write-protected. There was no such sticker on this one, but I tried another one just to be sure. <span style="font-family: Courier New, Courier, monospace;">Disk read only</span>. Another? Same problem. I tried adding and removing stickers to make sure I hadn't gotten the logic backwards, but that did no good either. According to the computer, every disk I threw at it was unwriteable.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNsYbRYV4JhggiNhrxZaMusvoas8xtUaMGJ08V_fV-PG1P1M_uHLc9XX34t8tB755gwLX8BRcun2Ts-1gcf63s-6ZOL0X3HqUg_IAiPlUj_wCxiflHHj0527zaq7_Ap67pYemp8g8QRNA/s1600/IMG_20141202_171009.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNsYbRYV4JhggiNhrxZaMusvoas8xtUaMGJ08V_fV-PG1P1M_uHLc9XX34t8tB755gwLX8BRcun2Ts-1gcf63s-6ZOL0X3HqUg_IAiPlUj_wCxiflHHj0527zaq7_Ap67pYemp8g8QRNA/s1600/IMG_20141202_171009.jpg" height="240" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The write-protect indentation in a 5¼" floppy disk.</td></tr>
</tbody></table>
At this point, I wasn't sure where the problem was, but I had a few hypotheses. How would you build a device to detect whether the notch is covered? I'd either have a little spring-loaded lever that moves into the indentation when the disk gets inserted, or I'd have an optical system with a light and a photoresistor to detect whether there is line of sight between the top and the bottom. In the former case, the spring might have become too weak over the years to overcome the friction. In the latter case, there might be dust inside the drive, blocking the light's path.<br />
<br />
Carefully, because this drive was critical to the entire project, I unscrewed the cover. There was a little black block of plastic around the notch, but without any mechanical parts. At the back, I could see tiny slits where I presumed the light would go through. I couldn't see any obvious dust, but I blew into them anyway. It made no difference.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizqrX8rcQCwPQEBOXZPCq4ROgg8tcZgoxsKqaN3bvGvlqnf-8v2FdjjBlDy58Bu236ryoNjdQ_r4444oILYmFUJW_XYgxSrb5wFd33P2BnJKJWp9Of58lCAe_NGfgKyxuR-6czrxPXG5k/s1600/IMG_20141130_160854.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizqrX8rcQCwPQEBOXZPCq4ROgg8tcZgoxsKqaN3bvGvlqnf-8v2FdjjBlDy58Bu236ryoNjdQ_r4444oILYmFUJW_XYgxSrb5wFd33P2BnJKJWp9Of58lCAe_NGfgKyxuR-6czrxPXG5k/s1600/IMG_20141130_160854.jpg" height="320" width="240" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The optical sensor that detects write-protected stickers on the disk.</td></tr>
</tbody></table>
The drive identified itself as a Canon MDD-221. Online, I found a couple of maintenance manuals and even <a href="http://www.vintagecomputer.net/fjkraan/comp/trs80/doc/Canon_MDD211_221.pdf" target="_blank">circuit diagrams</a>. There were three wires on the flatcable that went to the sensor, labelled G, 5 and WA. I guessed that G would be the ground and 5 would be the +5V line to power the LED. That left WA for detecting the photoresistor's resistance. This was on the PCB on the bottom of the drive, whereas most action takes place on the top one; a green wire also marked WA went from the bottom to the top. Armed with this knowledge, I found the right lines in the circuit diagram, and confirmed that there were indeed an LED and photoresistor inside the black box. The WA line went into pin 9 of an IC labelled J4, and from another pin of that IC came a line marked W. PROTECT. It took a while to realize that the other end of <i>that</i> line just went straight to the flatcable between the disk drive and the computer. In other words, if I could wire the right pin to be either high or low voltage, I could override the write protection mechanism.<br />
<br />
I'm not an electrical engineer. I can unscrew things and look at them, maybe even poke at them with a multimeter, but deep understanding of how such circuits work is beyond me. Besides, I didn't want to experiment and risk breaking my last and only disk drive. So at this point I chickened out, put the drive back together, and dug up the old tape drive and an unused cassette tape. I had used the tape drive before, and it worked fine for loading in most of my old games, like Hopper and Asteroids.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgasNnmV17z0zWJXhuPf4j0bvEyBtYeG7kvXbcXqpsbkjXNWnFacPHR0KW6rsTqHxno7j5rxV67pMU0282AoT9cCBMOgPzrX7FwYD_1PXQLfIHZYoojSWGYNTVj56PUC1ZZ6PsUyxk0Zzk/s1600/IMG_20141202_173426.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgasNnmV17z0zWJXhuPf4j0bvEyBtYeG7kvXbcXqpsbkjXNWnFacPHR0KW6rsTqHxno7j5rxV67pMU0282AoT9cCBMOgPzrX7FwYD_1PXQLfIHZYoojSWGYNTVj56PUC1ZZ6PsUyxk0Zzk/s1600/IMG_20141202_173426.jpg" height="240" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Attempt to make the tape drive read back what it just wrote.</td></tr>
</tbody></table>
But somehow, the tape drive also thwarted me: it was as though the data was never written. Neither <span style="font-family: Courier New, Courier, monospace;">LOAD</span> nor <span style="font-family: Courier New, Courier, monospace;">*CAT</span> (abbreviated <span style="font-family: Courier New, Courier, monospace;">*.</span>) would show it to me, even after waiting long enough for three copies of my tiny test program to have passed by on the tape.<br />
<br />
I flipped the tape around to try the other side. It contained my Elite save game (I think I made it to Dangerous level back in my youth), but fast-forwarding beyond that, I managed to successfully save my test program and load it back in again. Hooray!<br />
<br />
I also noticed something else: this time, during saving, the RECORD light of the tape drive lit up. I didn't recall seeing that before. Maybe I didn't press the RECORD button hard enough together with PLAY? I flipped the tape again to keep my Elite savegame safe, rewound, and tried saving again. This time, it worked like a charm, so I decided to use this side exclusively from now on.<br />
<br />
So, armed with a read-only disk drive and a single side of a tape to store my code in progress, I set out to discover the secrets of the QR code.</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com0tag:blogger.com,1999:blog-3365890222783870731.post-16747259869581410562015-01-19T14:05:00.000+01:002015-01-19T14:05:00.140+01:00QR codes on the BBC Micro, Part 2 of 6: Proof of concept<div dir="ltr" style="text-align: left;" trbidi="on">
<i>In which I learn about weird video memory conventions and run a small-scale experiment.</i><br />
<br />
Before diving in and writing assembly code to generate QR codes on the BBC Microcomputer, it seemed wise to check whether those could be read from the screen at all. After all, it's an old monochrome 50 Hz interlaced CRT monitor, and although the pixels are quite large by modern standards, they are far from stable, crisp squares. The picture flickers, and bright pixels tend to bleed into darker areas. Would my phone's camera and QR app even be able to recognize the code?<br />
<br />
So I took the example QR code from <a href="http://www.thonky.com/qr-code-tutorial/format-version-information/" target="_blank">this excellent guide</a>, and started to convert it to a bunch of bytes. This is a 1-Q code, the smallest variant, easy enough to type in by hand once I've got the raw bytes printed on my PC screen. It contains the text “HELLO WORLD” in alphanumeric encoding.<br />
<br />
Using the Gimp to downscale the image, remove the border and convert it to a PGM file was easy enough. PGM is an image format that supports an ASCII-based variant, so it's trivial to read such a file in a C++ program.<br />
<br />
The idea was to have the C++ program pack the pixels into bytes, as a monochrome image at 1 bit per pixel. The Beeb's display memory is a part of its main memory, so we can put things onto the screen simply by writing the bytes into this memory.<br />
<br />
So where exactly is this display memory? This depends on the screen mode. The Beeb has 8 screen modes, 0 through 7, each with its own characteristics: resolution, colour depth, and memory usage. The largest possible QR code is 177×177 modules (“pixels”), so I'd need at least that much resolution, but I didn't care about colours and would be fine with monochrome (1 bit). Moreover, the pixels must be more or less square; mode 2 in particular has very squashed pixels. It seemed like mode 4 would be my best bet: 2 colours (black and white), at 320×256 pixels, taking up 10 kB of the computer's 32 kB of RAM. These 10 kB are mapped into the addresses <span style="font-family: Courier New, Courier, monospace;">&5800</span>…<span style="font-family: Courier New, Courier, monospace;">&7FFF</span>. (On the Beeb, <span style="font-family: Courier New, Courier, monospace;">&</span> is used to denote hexadecimal numbers).<br />
<br />
But here the fun begins. You'd expect the pixels to be stored linearly in memory: the first byte contains the 8 leftmost pixels in the top row, the next byte contains the 8 pixels right to those, and so on. But on the BBC Micro, things work differently: the first 8 bytes go down to form the first character cell, then we jump back up and to the right to the second character cell, and so on:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSBoZFRyejW02V901ObSkETNifPfoWSst1UF_w2yAP2hCMcNoHu2f2_FEjYH_QeZJ29tr01yQ6OdturjLbClwipoyU8tWRPlzbw9vrDXFk1u-MUoxCKHFpSZg_yGR8QRhiZZCIL9hvHNg/s1600/IMG_20141201_202135.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSBoZFRyejW02V901ObSkETNifPfoWSst1UF_w2yAP2hCMcNoHu2f2_FEjYH_QeZJ29tr01yQ6OdturjLbClwipoyU8tWRPlzbw9vrDXFk1u-MUoxCKHFpSZg_yGR8QRhiZZCIL9hvHNg/s1600/IMG_20141201_202135.jpg" height="320" width="240" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The memory layout for graphics mode 4.</td></tr>
</tbody></table>
I guess it works this way to make displaying characters faster, because they can be written into 8 contiguous bytes of memory. This mapping will probably cause me headaches down the road, but once you know the system it's easy enough to code for. After a while I had a BASIC program to poke the right bytes into the right place in memory:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">10MODE4</span><br />
<span style="font-family: Courier New, Courier, monospace;">20!&5A90=&BABA82FE</span><br />
<span style="font-family: Courier New, Courier, monospace;">30!&5A94=&00FE82BA</span><br />
<span style="font-family: Courier New, Courier, monospace;">40!&5A98=&FA5ACA13</span><br />
<span style="font-family: Courier New, Courier, monospace;">50!&5A9C=&D8AB4AD2</span><br />
<span style="font-family: Courier New, Courier, monospace;">60!&5AA0=&E8E808F8</span><br />
<span style="font-family: Courier New, Courier, monospace;">70!&5AA4=&00F808E8</span><br />
<span style="font-family: Courier New, Courier, monospace;">80!&5BD0=&B42BBDDE</span><br />
<span style="font-family: Courier New, Courier, monospace;">90!&5BD4=&82FE00DF</span><br />
<span style="font-family: Courier New, Courier, monospace;">100!&5BD8=&58130FCE</span><br />
<span style="font-family: Courier New, Courier, monospace;">110!&5BDC=&A46689EE</span><br />
<span style="font-family: Courier New, Courier, monospace;">120!&5BE0=&C00070D0</span><br />
<span style="font-family: Courier New, Courier, monospace;">130!&5BE4=&B87840F8</span><br />
<span style="font-family: Courier New, Courier, monospace;">140!&5D10=&82BABABA</span><br />
<span style="font-family: Courier New, Courier, monospace;">150!&5D14=&000000FE</span><br />
<span style="font-family: Courier New, Courier, monospace;">160!&5D18=&E742B8D2</span><br />
<span style="font-family: Courier New, Courier, monospace;">170!&5D1C=&00000050</span><br />
<span style="font-family: Courier New, Courier, monospace;">180!&5D20=&3018A038</span><br />
<span style="font-family: Courier New, Courier, monospace;">190!&5D24=&00000010</span><br />
<div>
<br /></div>
<div>
I typed it in, fixed the typos, and tada! A QR code! On 30 year old hardware!</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTEtMYEFSQbwTnPO6RummbVR2-hdkl1vJKlfeFP4LCp9zaprobFF-bf88NPSgyj96mryz0t4NmaPSMK9eXNieCx9jp1WwBGx7o7jnzR3E0pGUytarUitw3gX527-ltQPaDhsIGvw8sEvA/s1600/IMG_20141201_200220.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTEtMYEFSQbwTnPO6RummbVR2-hdkl1vJKlfeFP4LCp9zaprobFF-bf88NPSgyj96mryz0t4NmaPSMK9eXNieCx9jp1WwBGx7o7jnzR3E0pGUytarUitw3gX527-ltQPaDhsIGvw8sEvA/s1600/IMG_20141201_200220.jpg" height="240" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A QR code displayed on the Beeb's monitor. Maybe the first ever?</td></tr>
</tbody></table>
<div>
Now the big question… <i>will it scan?</i> I launched the ZXing Barcode Scanner app, pointed it at the screen, waited for the camera to focus, and… nothing. I fiddled with the brightness and contrast knobs on the CRT, but to no avail. The scanner showed brief blips of partial recognition everywhere but in the actual code.</div>
<div>
<br /></div>
<div>
Would this project be doomed to fail before it even got properly started? Let's try one more thing. The QR code specification says that it's legal to invert the colours on a code, like we've done here: what is normally dark is here shown as light, and vice versa. However, this is an addition to the 2005 edition of the spec. Perhaps it's not implemented? So let's invert the screen and find out:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">200FORI%=&5800TO&7FFF:?I%=?I% EOR &FF:NEXTI%</span></div>
<div>
<br /></div>
<div>
A quick explanation for the uninitiated is probably in order. It's just a for loop from <span style="font-family: Courier New, Courier, monospace;">&5800</span> to <span style="font-family: Courier New, Courier, monospace;">&7FFF</span>, inclusive. The loop counter is <span style="font-family: Courier New, Courier, monospace;">I%</span>, one of 26 resident integer variables, presumably a bit faster than a plain <span style="font-family: Courier New, Courier, monospace;">I</span>. Spaces are optional in most of BBC BASIC, which means you can write stuff like <span style="font-family: Courier New, Courier, monospace;">FORI</span> and it will recognize the <span style="font-family: Courier New, Courier, monospace;">FOR</span> keyword just fine. (This also means that you are in for a surprise if you name your variable <span style="font-family: Courier New, Courier, monospace;">ANDY</span>, <span style="font-family: Courier New, Courier, monospace;">DIVA</span> or <span style="font-family: Courier New, Courier, monospace;">COST</span>.) For each of these addresses, the <span style="font-family: Courier New, Courier, monospace;">?</span> operator is used to query the byte (“peek”), <span style="font-family: Courier New, Courier, monospace;">EOR</span> is used invert it by xor'ing it with <span style="font-family: Courier New, Courier, monospace;">&FF</span>, and <span style="font-family: Courier New, Courier, monospace;">?</span> is used again to store it back.</div>
<div>
<br /></div>
<div>
So I waited for some 15 seconds while this loop ran and inverted the screen, and ended up with a properly uninverted (outverted?) QR code.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXcSM9k_vMCQ4LYCz_jnJgq_m9-Ba43C94Ovn1CaC3pviRR05c6mqEY7Ns_v4swuZKx_1ieCJN3tl-3DGPOETeqQq8B5V3dCzJss7hg89aMW5NtV6kUn_YhKoWgE4slqo7xGdcCiOcBsI/s1600/IMG_20141201_200149.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXcSM9k_vMCQ4LYCz_jnJgq_m9-Ba43C94Ovn1CaC3pviRR05c6mqEY7Ns_v4swuZKx_1ieCJN3tl-3DGPOETeqQq8B5V3dCzJss7hg89aMW5NtV6kUn_YhKoWgE4slqo7xGdcCiOcBsI/s1600/IMG_20141201_200149.jpg" height="240" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">An uninverted QR code displayed on the Beeb's monitor.</td></tr>
</tbody></table>
<div>
The big question again… <i>will it scan?</i> And after lowering the brightness on the monitor to reduce blooming, the answer is a resounding YES! The code was recognized almost instantly, and the text HELLO WORLD appeared on my phone's screen.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgy75ZeUZ_r2DH7Ld6BXIzkvHbimlIZztG6MN7xIGE6v7Fq3IhnmEYdDmaiRo5UdkufOG5gXOPiEArB2Xu63M22zp78ehVgsHhuArYjGCrT9iPaoPMBMsvXiRdezcRN0AtjxteRr-x95rw/s1600/Screenshot_2014-12-01-20-41-05.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgy75ZeUZ_r2DH7Ld6BXIzkvHbimlIZztG6MN7xIGE6v7Fq3IhnmEYdDmaiRo5UdkufOG5gXOPiEArB2Xu63M22zp78ehVgsHhuArYjGCrT9iPaoPMBMsvXiRdezcRN0AtjxteRr-x95rw/s1600/Screenshot_2014-12-01-20-41-05.png" height="180" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">ZXing Barcode Scanner correctly recognizing the above code.</td></tr>
</tbody></table>
<div>
Even at a distance sufficient to scan a screen-size QR code, it still worked quickly and reliably. So far, so good!</div>
</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com3tag:blogger.com,1999:blog-3365890222783870731.post-3050209956529067062015-01-18T15:23:00.001+01:002015-01-18T15:23:47.419+01:00QR codes on the BBC Micro, Part 1 of 6: The beginning<div dir="ltr" style="text-align: left;" trbidi="on">
<i>In which I revel in nostalgia and explain what this series is all about.</i><br />
<br />
On a shelf in my dad's study, there is an old photograph, made some time in the late '80s. In it is my dad, with 3-year-old me sitting in his lap, looking at this very device:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9-_LN-HwGhKKUFNmhagqM_cN_kOZjfAhZaPOGsZ_Mh3OWXBA6fHv1WzKP1-0k7fiFRtHNR9f4OpgDT2gMVcyLDQuCZ0rMphsHDCqJITufdaWoYRpq1K6CLkOn2h8UlsNXAjCCr2kCpew/s1600/IMG_20141202_175121.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9-_LN-HwGhKKUFNmhagqM_cN_kOZjfAhZaPOGsZ_Mh3OWXBA6fHv1WzKP1-0k7fiFRtHNR9f4OpgDT2gMVcyLDQuCZ0rMphsHDCqJITufdaWoYRpq1K6CLkOn2h8UlsNXAjCCr2kCpew/s1600/IMG_20141202_175121.jpg" height="320" width="240" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My first computer, the BBC Micro.</td></tr>
</tbody></table>
Some of you might recognize this beauty as a BBC Microcomputer. One of the best things to come out of the '80s, the BBC Micro (or “Beeb” for short) became hugely popular in the UK, and to a lesser extent in the rest of the world. They were used at the school where my dad worked, and that's how this specimen ended up in our house. I'm not exaggerating if I say it has been a defining influence on the rest of my life.<br />
<br />
But at some point, the PC took over the world, and our house. At first the Beeb was moved to my bedroom, but it eventually ended up in a box in the attic when I got a PC of my own. The reason it's now sitting on my desk again is that my parents are moving house, and would rather have thrown it away than move it to a different, much smaller attic. I couldn't let that happen. Having moved all my possessions between four countries I'm not one to get attached to inanimate objects, but this little machine is special.<br />
<br />
On the other hand, in the digital world, I'm a bit of a hoarder. My fileserver still contains my university coursework, the Word documents that I wrote for school assignments at the age of 14, and the source code of the games I made on my first PC and tried to sell to people on floppy disks. But the record becomes harder to access beyond that: my very first programs were written on the BBC Micro, and are stored on these old 5¼" floppy disks, which don't fit into my PC's USB 3.0 port or its BluRay drive.<br />
<br />
Sadly, these old floppy disks don't have eternal life. Nor does the Beeb. Nor its disk drive; these drives are notoriously fragile, and a replacement might be hard to find. So I set out to salvage the earliest work in my programming career and back it up onto more modern media. It would be the completion of the earliest records of my digital life.<br />
<br />
From the low-level software point of view, a floppy disk contains just a series of bytes. If we transfer all the bytes and put them into a file on the PC, we have created an <i>image</i> of the floppy: an exact replica, which can be loaded into an emulator like <a href="http://www.mkw.me.uk/beebem/" target="_blank">BeebEm</a> and used as if it were the real thing. And since the image is just a file, it can be backed up on any of the many storage media available today and in the future.<br />
<br />
But as I mentioned, my PC does not have a floppy drive. The PC I had before this one didn't either. The one before <i>that</i> might have had a drive for the “modern” 3½" not-so-floppy disks, not the 5¼" that the BBC Micro uses. Old IBM hardware, like 286 and 386 PCs, typically did use these floppies, but I think they were in an incompatible format. Even if not, the flatcable that connects the computer to the drive is definitely different.<br />
<br />
So my best bet seemed to be to let the BBC Micro itself read the disks into memory, and somehow transfer the data to the PC. Easier said than done. The Beeb predates the Wifi era and even the Ethernet era. It does come with its own type of networking interface, called Econet, but of course no modern machine has any clue how it works.<br />
<br />
Another option would be the 5V analogue I/O port. I once used this to transfer data between the Beeb and my TI-83 graphical calculator, so I knew it was possible. Most PCs built in the 90's came with an RS-232 serial port, but the voltage levels are different. I would need to build some kind of adapter, but I'm not an electrical engineer, and I was terrified of blowing up my only remaining Beeb by accident. Moreover, my PC lacks such a serial port to begin with.<br />
<br />
So I started considering more arcane options. The first thing I thought of was sound. The BBC Micro comes with a pretty cool 4-channel audio chip, and even from BASIC you can make it play 256 different pitches. That's one tone to transfer one byte – great! On the other end, we could do an FFT to detect the pitch, and map it back into a byte. However, the minimum duration of such a tone is 1/20th of a second, so the maximum transfer rate would be 20 bytes/s. At that rate, copying a single 180 kB floppy disk would take over 2.5 hours of listening to noisy bleeping and blooping. Not fun, but definitely doable! I was encouraged.<br />
<br />
However, there is a drawback. We need an exact copy of the signal. Misinterpret a single byte, and the program ceases to work. Worse, if we lose or gain a byte halfway the stream, the entire alignment of the data is thrown off and the floppy image becomes unreadable.<br />
<br />
So at the very least, we need a checksum. CRC32 is <a href="http://golang.org/src/pkg/hash/crc32/crc32.go" target="_blank">relatively easy to implement</a> and might work well enough to detect errors. But checksums are just a class of error-<i>detecting</i> codes, so while they can tell me that the transfer went wrong, they cannot tell me where or how, and that would be 2.5 hours of bleeping and blooping down the drain.<br />
<br />
What we really need here is an error-<i>correcting</i> code. The most well-known is the Reed-Solomon code, used for data transfers from the Voyager space craft, and more recently also on CDs, DVDs and Blu-ray discs. But there's another place where RS codes are used.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7zEpSTSim2kvoCx7xGhEc8A-OiBSEaG8QXIOYh6bBZKZGCd1pw8-F-9l7YkJzC2rDS4lOhQqAno-MmnqhUoORR8petEriTuUBvU5BCillRNSoS9UP7F6RxSiOvaTQlR1RwZ3PoJBl7k8/s1600/hello-world-final.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7zEpSTSim2kvoCx7xGhEc8A-OiBSEaG8QXIOYh6bBZKZGCd1pw8-F-9l7YkJzC2rDS4lOhQqAno-MmnqhUoORR8petEriTuUBvU5BCillRNSoS9UP7F6RxSiOvaTQlR1RwZ3PoJBl7k8/s1600/hello-world-final.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A QR code containing the text “HELLO WORLD”.</td></tr>
</tbody></table>
The ubiquitous QR code, used on posters, milk cartons, websites <a href="https://www.google.co.uk/search?q=qr+code+tattoo&es_sm=93&tbm=isch&tbo=u&source=univ&sa=X&ei=ePV9VJngDITgyQPjjoCwBA&ved=0CCIQsAQ&biw=1184&bih=1068" target="_blank">and even tattoos</a>, is little more than a bunch of 1s and 0s encoded as black and white squares, with Reed-Solomon error correction codes on top, and some markers thrown in to help the scanner. Would it be possible to display a floppy's contents on the screen as a series of QR codes, scan each of them with a smartphone, and assemble them at the other end into a disk image?<br />
<br />
Let's see. Most QR codes are fairly small, as they contain just a web address or some such. But they can actually grow to the “Level 40” monster code of 177×177 squares, which (at the lowest error correction setting) can hold 2953 bytes of arbitrary binary data. That's about 60 QR codes to transfer the contents of one floppy. At a conservative rate of one QR code per minute, I'd be able to reliably transfer an entire floppy in an hour.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiL4vuWXaEevL0GGH6avPP8BFZjEoDTlGdtT46y6RQAExdsVP7v9Ub1orslpxBFozpz3QQHqjTtgemp0ESFNWWtPCsxUOJqDNsiiQkeYCzLE2Bc1gq9h1wwnf6Z2Vla3Z60gCtMkaAEuew/s1600/Qr-code-ver-40.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiL4vuWXaEevL0GGH6avPP8BFZjEoDTlGdtT46y6RQAExdsVP7v9Ub1orslpxBFozpz3QQHqjTtgemp0ESFNWWtPCsxUOJqDNsiiQkeYCzLE2Bc1gq9h1wwnf6Z2Vla3Z60gCtMkaAEuew/s1600/Qr-code-ver-40.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A level 40 QR code from Wikipedia. I haven't checked whether it contains anything NSFW.</td></tr>
</tbody></table>
Encouraged, I did some back-of-the-envelope time and memory calculations. About 4 kB for the QR code data bits, and maybe another kB for scratch data. In screen mode 4, we have about 16 kB to play with, so that leaves 11 kB for the BASIC program and its assembly output. A tight squeeze, perhaps, but not impossible.<br />
<br />
On to calculating time. Maybe it takes 1000 clock cycles to generate one pixel, so at the 2 MHz that the Beeb runs at, we should be able to generate a QR code in 16 seconds. Even if I'm off by an order of magnitude, that's still fine.<br />
<br />
So let's do this!</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com0tag:blogger.com,1999:blog-3365890222783870731.post-37231177944164389432014-10-31T13:58:00.001+01:002014-10-31T13:58:16.707+01:00Market prices in Acornsoft Elite<p>I wouldn't be exaggerating if I said that <a href="http://en.wikipedia.org/wiki/Elite_(video_game)" target="_blank">the original Elite game</a> has had a huge impact on my life. Whilst I did play other games on my father's BBC Microcomputer, like Hopper and Pinball, I think Elite may still be top of the list in terms of hours played over my lifetime. Moreover, it was Elite that motivated me to badger my dad into teaching me the BASICs of programming, and from there, everything else followed naturally.</p>
<p>So, of course, I'm really excited about the upcoming sequel, <a href="http://elitedangerous.com" target="_blank">Elite: Dangerous</a>. I bought the beta yesterday, but haven't had time to properly play yet, so my verdict is still outstanding. First impressions are good, though.</p>
<p>But the best thing about Dangerous is that you get a copy of the original BBC Micro version of Elite for free. (You can also get it for free if you don't buy Dangerous, by the way.) So while the gigabytes of Dangerous were trickling down my narrow tubes, I fired up the BeebEm emulator and started playing my old childhood classic.</p>
<p>Back when I was a kid, I played very conservatively. Liquor/wines from Lave to Leesti, computers from Leesti back to Lave. Over and over again. Now, of course, I want a little more adventure – not in the least because loading and saving is so much faster. So I started looking deeply into market prices at planets of various types (Poor Industrial, Rich Agricultural), and noticed something weird. Almost everything is more expensive almost everywhere than the “average prices” printed in the Elite manual. What is going on?</p>
<p>Fortunately, back near the turn of the milennium, C.J. Pinder ported the original Elite 6502 assembly code to C, resulting in <a href="http://www.christianpinder.com/games/" target="_blank">Elite: The New Kind</a>. A while later, David Braben got wind of this, and ordered it to be taken down. Fortunately, I still had a copy of the ZIP file, which I religiously held on to. It seems Braben has softened since, or got some more sense, or wanted some free publicity, because The New Kind is back online again.</p>
<p>Anyway, I went into the source code of Elite: The New Kind and discovered the shocking truth of these market prices.</p>
<p>First off, no fractional credits exist in memory. Everything is stored in integer values of decicredits, 0.1 Cr. There is a table that contains all the items, together with some metadata about them. Here is the formula for computing an item's price:<br>
<code>item.price = ((item.base_price + (market_rnd & item.mask) + planet.economy * item.eco_adjust)) & 0xFF * 4;</code><br>
The meaning of each of these fields:</p>
<ul>
<li><code>item.base_price</code> is a constant read from the table of items. It's 19 for Food up to 235 for Narcotics, with the highest value for a legal substance being Luxuries at 196.</li>
<li><code>market_rnd</code> is a random byte, 0 to 255 inclusive, generated whenever we do a hyperspace jump.</li>
<li><code>item.mask</code> is also constant, varying from <code>0x01</code> for Food to <code>0x1F</code> for Slaves, Alloys and Platinum. An outlier is Narcotics at <code>0x78</code>, the only value that isn't a series of lower bits.</li>
<li><code>planet.economy</code> is set by the galaxy generator to 3 bits of the planet's random seed, so between 0 and 7, inclusive. Its meaning:
<ul>
<li>Bit 2 is the type, 0 for Industrial, 1 for Agricultural.</li>
<li>Bits 0 and 1 are the wealth: Rich, Average, Poor, Mainly. The order depends on whether the world is Industrial or Agricultural, and Mainly is in the middle: a Mainly Industrial world is leaning towards Agricultural and vice versa. That's that cleared up!</li>
</ul>
</li>
<li><code>item.eco_adjust</code> is a signed number, ranging from <code>-9</code> for Furs to <code>29</code> for Narcotics; the highest value for a legal, buyable item is <code>14</code> for Computers.</li>
</ul>
<p>So how to make sense of all this? How to find out if the manual was wrong, or just my impressions? Let's turn to a tool I feel is undervalued by most programmers: the humble spreadsheet. <a href="https://docs.google.com/spreadsheets/d/1qSvAZBbFJ3vdBzF9eLTMVxnDDQZUgClf1KLdytd_L04/edit?usp=sharing" target="_blank">This Google spreadsheet</a> shows my results. A few things stand out:</p>
<ul>
<li>I was right, the manual was wrong. Prices are on average significantly higher than printed. Furs are almost 26% more expensive, and Minerals even 50%! Computers and Luxuries, on the other hand, are slightly cheaper.</li>
<li>The prices in the manual are taken from the minimum possible price on a Mainly Agricultural world. Except for Narcotics, Minerals and Alien Items, the two are identical.</li>
<li>My calculations are incorrect for Narcotics because it overflows its byte and wraps around. Calculating how to get the real maximum would be a bit more work, but it must be around 102 credits. With its odd bit mask, Narcotics are the biggest gamble with the biggest potential payoff.</li>
<li>Planet classification doesn't have two axes (Industrial/Agricultural, Rich/Poor); it has just one. The range is from Rich Industrial to Poor Agricultural.</li>
<li>The planet's government type doesn't play into it at all. You might think that it would pay better to ship Firearms to an Anarchy world than to a Democracy, but you'd be wrong.</li>
<li>Prices on a single world are linked; they rise and fall together.</li>
<li>Profit margins are obviously highest on illegal substances. The best you can do legally is ship Computers from a Rich Industrial world to a Poor Agricultural one, and bring Liquor/Wines back. (Food has an even higher profit margin, but is so cheap that your cargo bay becomes the limiting factor quickly.)
</ul>
<p>Now I'm back to playing Elite. The old or the new? It's hard to decide!</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com0tag:blogger.com,1999:blog-3365890222783870731.post-14563221348662349022014-04-21T14:41:00.000+02:002014-04-21T14:41:56.739+02:00Password security hall of shame<div dir="ltr" style="text-align: left;" trbidi="on">
With the recent <a href="http://heartbleed.com/" target="_blank">Heartbleed vulnerability</a> (wait, why am I even making that a link), I figured it was time to upgrade my password security. Like all proper nerds, I have dozens of accounts all over the web. And like most humans, I am unable to remember distinct passwords for all of them. I'll admit, I have sinned, and reused passwords all over the place. Today, I repent.<br />
<br />
So far, my strategy has been to have three tiers of passwords:<br />
<ul style="text-align: left;">
<li>a very strong one for my email and banking</li>
<li>a moderately strong one for accounts that I somewhat care about</li>
<li>a laughably weak one for throwaway accounts</li>
</ul>
<div>
The reason that these aren't all strong passwords is that I started using increasingly strong passwords over the years, but never bothered to update lesser-used accounts.</div>
<div>
<br /></div>
<div>
After some research and asking around, my new strategy will be random unique passwords, stored in the password manager <a href="https://lastpass.com/" target="_blank">LastPass</a>. There are several other password managers, but LastPass is the only one that ticks all but one of the boxes that I care about:</div>
<div>
<ul style="text-align: left;">
<li>Easy to use.</li>
<li>Passwords are encrypted locally and never leave your computer.</li>
<li>It's available as a browser plugin.</li>
<li>It's available on Linux, OS X and Windows.</li>
<li>It's available on mobile devices.</li>
</ul>
<div>
The one box it doesn't tick is being open source, which makes me a bit sad. Open source alternatives would be <a href="http://keepass.info/" target="_blank">KeePass</a> or <a href="http://passwordsafe.sourceforge.net/" target="_blank">Password Safe</a> (the latter by security expert <a href="https://www.schneier.com/passsafe.html" target="_blank">Bruce Schneier</a>) but neither seems to have great usability.</div>
</div>
<div>
<br /></div>
<div>
So I spent a few hours today changing passwords on over 40 websites, and got a nice overview of how different sites approach password security. Here is my bottom-3 of the worst ones.</div>
<h3 style="text-align: left;">
#3: Steam</h3>
<div>
You can log in to your account on the web, view purchases, chat with friends, and do many more of the things that nobody ever uses Steam for... but you cannot change your password on the web. You have to use the client for that. (Also, why haven't they bought steam.com yet?)</div>
<h3 style="text-align: left;">
#2: HSBC</h3>
<div>
HSBC, the large multinational bank, have a very... interesting approach to login. Some banks use a username/password combination for their online banking. Most will then require two-factor authentication to actually do transactions. Some banks use username/OTP for logging in, where you sometimes need your bank card to generate the OTP.</div>
<div>
<br /></div>
<div>
HSBC does none of these things. Their login flow uses a username, a memorable question, and a 6-digit OTP. The admissible memorable questions are:</div>
<div>
<ul style="text-align: left;">
<li>What is your eldest child's middle name? (I don't have any children.)</li>
<li>Who was your first employer? (Easy to find on LinkedIn.)</li>
<li>What is your father's middle name? (Somewhat harder to find, probably not impossible.)</li>
<li>What is the name of the street you grew up on? (Easy to guess if you know where I went to primary school.)</li>
<li>What was the name of your first school? (And the name of that school is probably on the untrimmed version of my CV.)</li>
<li>Name a memorable character from a film or book or TV (I wouldn't remember what I'd answered to this one.)</li>
<li>What is the make and model of your first car? (I never owned a car. And if I still had it, it'd be parked outside my house for all the world to identify. Maybe even on Street View.)</li>
<li>Name a memorable meal (What?!)</li>
<li>Name a memorable restaurant (Actually, this one might work for me. But for close friends, my answer would be easy to guess.)</li>
<li>What is your memorable answer?</li>
</ul>
<div>
Yes, really. The self-referentiality almost makes the entire banking world disappear in a puff of logic. But of course I chose this option, and just entered a password. I can only imagine how this option came about.</div>
</div>
<blockquote class="tr_bq">
Management: "Passwords are too hard. I forgot the password to my email for the third time this month. IT people, make our site use a secret question instead."<br />
IT person: "But they are less secure. They're easy to guess for outsiders. We're a bank, right? We manage people's money."<br />
Management: "I have to rush off to a meeting. You know what to do."<br />
IT person: "Just one more question – what secret questions should we use?"<br />
Management: "You're the expert. Figure something out."<br />
IT person (trying to suppress a grin): "OK, will do."</blockquote>
<h3 style="text-align: left;">
#1: Charles Schwab</h3>
<div>
Schwab is a big investment firm that manages stock portfolios, so you'd expect them to be secure, right? Let's take a look at their password policy:</div>
<div>
<ul style="text-align: left;">
<li>The password must be at least 6 characters. So far, so good.</li>
<li>The password must contain at least one digit. OK.</li>
<li>The digit must be between the first and last characters. What?</li>
<li>The password must be at most 8 characters. What?!</li>
<li>The password may not contain any symbols. WHAT?!</li>
<li>The password is case insensitive. <i>WHAT?!</i></li>
</ul>
<div>
Even LastPass had some trouble coming up with a password for <i>that</i>.</div>
</div>
<div>
<br /></div>
<div>
Well, at least they have recently added the ability to change your username, so I can stop being THOMAS3722. Or was it THOMAS9560? I forgot.</div>
</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com6tag:blogger.com,1999:blog-3365890222783870731.post-3064628226541885472013-06-30T06:24:00.000+02:002013-06-30T06:24:06.090+02:00Hodor qualification<p>In the world of <a href="http://www.codinghorror.com/blog/2012/07/new-programming-jargon.html">creative programming jargon</a>, we already knew about Yoda conditions (“if three the size is”):</p>
<pre>if (3 == size) {
...
}</pre>
<p>And about Pokémon exception handling (“gotta catch 'em all!”):</p>
<pre>try {
...
}
catch (Exception ex) {
...
}</pre>
<p>Now I recently ran into a new antipattern, and decided to name it <strong>Hodor qualification</strong> after the character from Game of Thrones who can only say his own name.</p>
<p><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi15knU-daNzwArxo7zpcZOSqtFr07XI79_neH5H5yZuBPlAuoAGV3vmpq6GOhZzQvHKVRlrpijetv6nRxdusyQe9YDww7XWHAkpfb9UNjQQ8ky8Y2iPkqa3yD7H5XmdNHlskRsgUmhj3A/s310/hodor.jpg"/></p>
<p>Hodor qualification is to needlessly prefix calls to static methods by their class name, even from within the same class:</p>
<pre>
class Hodor {
private static int foo(int x) { ... }
private static void bar() {
return Hodor.foo(1) + Hodor.foo(2) + Hodor.foo(3);
}
}
</pre>
<p>Hodor!</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com0tag:blogger.com,1999:blog-3365890222783870731.post-57996361589447683942013-02-19T19:47:00.002+01:002013-02-19T19:47:17.445+01:00Uninstalling old kernels in Ubuntu<p>One would think that Ubuntu would be smart enough to remove obsolete kernel packages by itself. One would be wrong.</p>
<p>I think the Computer Janitor can clean them up but you still have to run that manually. So I prefer the following command:</p>
<pre>uname -r; sudo apt-get remove $(dpkg --get-selections 'linux-image-*' | grep '\binstall' | head -n-3 | cut -f1)
</pre>
<p>This removes all kernels except the latest two. It prints the currently running kernel version so you can easily check that it's not removing that one.</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com1tag:blogger.com,1999:blog-3365890222783870731.post-42882517602537723712012-10-21T11:42:00.001+02:002012-10-21T11:42:25.729+02:00Turning off the screen saver in Ubuntu<p>TL;DR: <code>xset s off</code></p>
<p>“That's easy,” I could hear you think when you read the title. Well, it took me hours to figure it out. It drove me crazy that I'd have to get up and move the mouse every 10 minutes while watching a movie¹, and I couldn't figure out how to disable this behaviour. And when something takes that much effort, I blog about it so that other people might find the solution more easily.</p>
<p>Of course you can easily disable the screen saver through the control panel thing. But I'm not using the Gnome panel, so I never know how to get to that GUI in the first place. Also, you may find, like me, that it just <em>does not work</em> and your screen will still turn black after 10 minutes.</p>
<p>From the console, you can <a href="http://askubuntu.com/questions/67355/how-do-i-completely-turn-off-screensaver-and-power-management">disable the screen saver</a> with:</p>
<pre>$ gsettings set org.gnome.desktop.screensaver idle-activation-enabled false</pre>
<p>This probably does the same thing as the check box in the GUI. But for me, it wasn't enough: the screen would <em>still</em> blank after 10 minutes!</p>
<p>As it turns out, this is a default built into Xorg itself. Try this:</p>
<pre>$ xset q
...
Screen Saver:
prefer blanking: yes allow exposures: yes
timeout: 600 cycle: 600
...</pre>
<p>Yep, that's the problem. Once you've found it, the solution is simple:</p>
<pre>$ xset s off
$ xset q
...
Screen Saver:
prefer blanking: yes allow exposures: yes
timeout: 0 cycle: 600
...
</pre>
<p>This only works until the X server is restarted; put <code>xset s off</code> in your <code>.xsession</code> file to make this change permanent.</p>
<hr>
<p>¹ At least for some movies. <code>mplayer</code> disables the screensaver automatically, but Flash doesn't.</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com2tag:blogger.com,1999:blog-3365890222783870731.post-17294928540323069832012-07-15T14:55:00.002+02:002012-07-15T15:20:49.991+02:00Switching default PulseAudio device when USB microphone is plugged in<p>I have a PlayStation Eye for a USB webcam and microphone, which is supposed to be supported by Linux. Unfortunately the microphone won't work until I unplug it and plug it in again after boot.
<p>But the unplugging causes PulseAudio to change its default device back to my on-board sound card, which doesn't have a microphone plugged into it. It won't change the default back to the Eye on its own. Here's a workaround, which took me several hours to develop, and isn't for the faint of heart. But it works now, dammit.
<p>All we need to do is nudge PulseAudio a little via a udev event. First, install the daemon package:
<p><code>sudo apt-get install daemon</code>
<p>Then create a new file <code>/etc/udev/rules.d/99-pseye.rules</code> with the following content (all on one line!):
<p><code>SUBSYSTEMS=="usb", ATTR{idVendor}=="1415", ATTR{idProduct}=="2000", RUN+="/usr/bin/daemon -- /bin/su thomas -c 'sleep 1 && /usr/bin/pacmd set-default-source alsa_input.usb-OmniVision_Technologies__Inc._USB_Camera-B4.04.27.1-01-CameraB404271.input-4-channels'"</code>
<p>OmniVision is apparently the manufacturer of this camera. Reports around the internet have slightly different version numbers; type <code>pacmd list-sources</code> and use the name of your particular device (which shows up as <code>name: <...></code>). You also want to replace <code>thomas</code> by your own username; this is used to find the running <code>pulseaudio</code> daemon.
<p>There shouldn't be any need to restart the udev daemon, but if you find otherwise, do <code>sudo restart udev</code>.
<p>The <code>daemon</code> command is needed because we need to delay the <code>pacmd</code> command a bit; if we run it right away, PulseAudio hasn't picked up the new device yet. The trouble is that udev tries very hard to wait for completion of all child processes of the <code>RUN</code> command before firing its events into userland, so PulseAudio will always get its event only after our script has already finished and failed. Even various combinations of <code>&</code> and <code>nohup</code> wouldn't convince udev not to wait, but <code>daemon</code> does the trick.
<p>If this doesn't work, here are some debugging methods that I used. Try adding <code>-o/tmp/daemon.log</code> after <code>/usr/bin/daemon</code> and inspect the output. To watch udev events happen in real time, use <code>udevadm monitor</code>. Another good source of information is <code>/var/log/syslog</code>; do <code>tail -f /var/log/syslog | grep pacmd</code> to filter it. Also check the output of <code>pacmd dump</code> to see whether the <code>set-default-source</code> command has taken hold.
<p>Oh, and the upstream PulseAudio people in all their wisdom decided (<a href="https://bugs.freedesktop.org/show_bug.cgi?id=39664" target="_blank">bug report</a>) that 4-channel microphones like this one don't deserve to have a default profile, resulting in the message <code>Failed to find a working profile</code> in <code>/var/log/syslog</code>. The result is that PulseAudio will retry loading, and retry, and retry … causing brief and hard to debug system freezes once every minute or so. To make the microphone work at all in Ubuntu 12.04 (<a href="https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug/886449" target="_blank">bug report</a>), you also have to add the following at the end of <code>/usr/share/pulseaudio/alsa-mixer/profile-sets/default.conf</code>:
<pre>[Mapping input-4-channels]
device-strings = hw:%f
channel-map = front-left,front-right,rear-left,rear-right
description = 4 Channels Input
direction = input
priority = 5
[Profile input:mic-array]
description = Microphone Array
input-mappings = input-4-channels
priority = 2
skip-probe = yes</pre>
<p>Then type <code>pulseaudio -k</code> to reload the daemon (it will be restarted on demand). Yes, this will be overwritten on upgrades. I've found no way to work around that yet.Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com6tag:blogger.com,1999:blog-3365890222783870731.post-91475164862400787892011-08-14T22:06:00.001+02:002011-08-14T22:07:16.142+02:00Heathrow runway alternation in Google Calendar<p>I moved to London recently, and unfortunately I found out too late that my new flat is <i>right</i> in the flight path of Heathrow's northern runway, causing quite a bit of overhead noise. However, I noticed that some periods were very quiet (including when I was viewing the flat). Was there some method to this madness?</p><p>Turns out, there is. The <a href="http://www.heathrowairport.com/portal/page/Heathrow+noise%5EGeneral%5EWhat+we+do+about+it%5EMeasures+already+in+place%5ERunway+use%5ERunway+alternation/bb67ff6b59b53210VgnVCM10000036821c0a____/448c6a4c7f1b0010VgnVCM200000357e120a____/">Heathrow runway alternation schedule</a> is nicely documented on the Heathrow website. Basically: one week has noisy mornings, the next week has noisy afternoons/evenings.</p><p>But it's a PDF file, which is only useful for printing and sticking on walls, which is sooooo 2010. So I turned it into a public Google Calendar, for use by myself and anyone else who might be interested. But alas, Google <a href="http://www.google.com/support/forum/p/Calendar/thread?tid=55bc5319154cc24e&hl=en">removed</a> the ability to search for public calendars back in 2009, which forces me to post about it here to make it findable.</p><p>Here's the ID of this calendar:<br />
<code>r1846kc94hj3ekk24ueb69151g@group.calendar.google.com </code><br />
Copy this, head over to Google Calendar, and paste it into the box labelled “Add a friend’s calendar”. Simple as that.</p><p>That's only for the northern runway, known as 27R (when landing/departing towards the west) or 09L (when landing/departing towards the east). Since landing and departing probably make the same amount of noise, I lumped them all together. If anyone would like a similar thing for the southern runway, let me know and I'll put one together.</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com2tag:blogger.com,1999:blog-3365890222783870731.post-42915275162299705442011-02-18T11:57:00.001+01:002011-02-18T11:59:01.527+01:00Exporting slices from Inkscape, part 2<p>I <a href="http://typethinker.blogspot.com/2010/07/exporting-slices-from-inkscape.html">previously wrote</a> about how to export slices from an Inkscape file, using a bash script. Here's a better script, using Python, which doesn't require you to give your slices any special label.</p>
<p>The new process is as follows:</p>
<ul>
<li>First, draw your image as usual.</li>
<li>Then, add a layer that will hold the slices; name it <code>slices</code> (this is important). Set the layer's opacity to about 50% to be able to see what you're doing. Enable the grid, and make sure it is set to pixels: you want your slices to align with pixel boundaries.</li>
<li>Draw your slice rectangles onto the <code>slices</code> layer, aligned to the grid. Ensure that the rectangles have no border; the fill is irrelevant (I use red).</li>
<li>Right-click a slice, and choose Object Properties. Change Id field to the name of the eventual PNG file, without the extension. The area defined by a rectangle named <code>foo</code> will be saved to <code>foo.png</code>. Repeat this for all slices.</li>
<li>Hide the slices layer. If you forget this, the script will print a warning.</li>
<li>Save your image. Let's say you called it <code>layout.svg</code>.</li>
<li>Run the script as follows:<br/>
<code>./export.py layout.svg</code><br/>
You should see each of your slices being exported to a PNG file in the same directory.</li>
</ul>
<p>Here's the code for the script. Save this to a file <code>export.py</code> and make it executable.</p>
<pre>
#!/usr/bin/env python
import sys
import os
from xml.dom import minidom
if len(sys.argv) < 2:
print 'Usage: %s filename.svg' % sys.argv[0]
sys.exit(0)
input_file = sys.argv[1]
dom = minidom.parse(input_file)
groups = dom.getElementsByTagName('g')
for group in groups:
if group.getAttribute('inkscape:groupmode') == 'layer' and group.getAttribute('inkscape:label') == 'slices':
if 'display:none' not in group.getAttribute('style'):
print 'Warning: slices layer might still be visible'
for element in group.getElementsByTagName('rect'):
export_id = element.getAttribute('id')
filename = '%s.png' % export_id
print 'Exporting %s...' % filename
os.system('inkscape --export-id="%s" --export-png="%s" --file="%s"' % (export_id, filename, input_file))
break
else:
print 'No layer named "slices" found; not exporting anything'
sys.exit(1)
</pre>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com3tag:blogger.com,1999:blog-3365890222783870731.post-43551659351776683802011-02-02T21:35:00.000+01:002011-02-02T21:35:12.579+01:00Editor's Prick?<p>Some time ago, I received an e-mail concerning my little open source program <a href="http://taekwindow.sf.net/">Taekwindow</a>:</p>
<blockquote>
<p><em>Monday, Jan 17, 2011, 10:27<br/>From: F.R.<br/>To: me</em></p>
<p>My name is F., Customer Manager of WareSeeker.com, a professional software directory. I would like to propose a solution to promote your software on our site.</p>
<p>We would like to take this opportunity to introduce ourselves. Wareseeker is one of top 5 professional software directories in the world and currently serving about 2 million page views per day. Many renowned software publishers rely on us as their primary media of promoting their products, thus increasing the sales or download dramatically.</p>
<p>We have tested thoroughly and guarantee that Taekwindow 0.3.1
is 100% SAFE TO INSTALL, which means it does not contain any form of malware: spyware, viruses, trojans and dialers.</p>
<p>Taekwindow 0.3.1
has been received Editor’s Pick Award from us.</p>
<p>We hope that you will gain more benefit through this Award. Moreover, we would like to request you add our link and Editor's Pick icon on your Awards category at http://taekwindow.sourceforge.net/download.html</p></blockquote>
<p>It's not the first time Taekwindow receives such an award, and they generally don't mean very much. But still, it's a nice gesture, and I've done other download sites the same favour, so I replied in the positive.</p>
<p>Since the Taekwindow site is largely built by an automated build system that only runs on Windows, and I am spending all my time in Linux lately, I did not make any promises when I would get round to it. It would probably have been later that week.</p>
<p>Two days later, I received two e-mails in quick succession.</p>
<blockquote>
<p><em>Wed, Jan 19, 2011, 02:40<br/>From: F.R.<br/>To: me</em></p>
<p>How about our request of our link and Editor's Pick icon at http://taekwindow.sourceforge.net/download.html</p>
<p>You will do that, ok? but when</p>
<p>Keep contact me</p>
</blockquote>
<blockquote>
<p><em>Wed, Jan 19, 2011, 09:46<br/>From: F.R.<br/>To: me</em></p>
<p>How long you got message of mine? I will remind you in final time for our link and Editor's Pick icon on your site.</p>
<p>Give me the respond when you reached this email</p>
</blockquote>
<p>Okay, so he expects me to answer to e-mail within 7 hours and 6 minutes, while I'm asleep. And he's getting pushy. But at least I could tell him that I received his mails.</p>
<blockquote>
<p><em>Wed, Jan 19, 2011, 12:04<br/>From: me<br/>To: F.R.</em></p>
<p>I got your messages alright. I'm a busy person, and I do not
appreciate your impatience about me doing you a favour. Please stop
pushing me. I'll get to it as soon as I have time.</p>
</blockquote>
<p>I might still have done it later that week, but I didn't get round to it. Today, another e-mail appeared.</p>
<blockquote>
<p><em>Tue, Jan 25, 2011, 02:42<br/>From: F.R.<br/>To: me</em></p>
<p>I don't know how are you busy, I'm also a busy man with a huge business volume each day but I still spend some minutes to write for me to remind you about our request.</p>
<p>Today If you don't reply my mail, I will remove our award for your product on my site. Contact me immediately when you reached this mail</p>
</blockquote>
<p>Is that how you treat people when you want them to do you a favour? Not appreciated.</p>
<blockquote>
<p><em>Tue, Jan 25, 2011, 11:45<br/>From: me<br/>To: F.R.</em></p>
<p>Dear Mr. R.,</p>
<p>The usual nature of awards is that they are given without requesting
anything in return. In spite of that, I graciously agreed to return
the favour, although due to other obligations I couldn't say when.
After a few days, a gentle reminder might certainly have been
appropriate. However, two reminders within eight hours (during which I
was asleep, I might add), of an increasingly unfriendly nature, did
not increase the likelihood that I would answer to your request. Your
further pushy behaviour did not improve the situation, and now you
request has turned into a demand, with a corresponding punishment
should I not comply.</p>
<p>I noticed that Taekwindow has received 13 downloads through your site,
and this number has not changed in the last few days. Compared to 149
downloads at Softoxi and 1121 downloads at Softpedia, I doubt there is
much value for Taekwindow to be hosted on your site. Given the
apparent attitude of the site's employees, it might even do more harm
than good.</p>
<p>If it must be like this, I would prefer not to have anything to do
with you or your site anymore. I have removed the staged modifications
to the Taekwindow website. I will now proceed to write a post about
our exchange on my blog, which receives a moderate amount of traffic.
Good luck with your future endeavours.</p>
<p>Sincerely,</p>
<p>Thomas</p>
<p>P.S. Incidentally, the download page for Taekwindow on WareSeeker
reads: "Software piracy is theft, Using crack, password, serial
numbers, registration codes, key generators, cd key, hacks is illegal
and prevent future development of Taekwindow 0.3.1 Edition." None of
these statements are true. Software piracy may be considered a crime
in most countries, but it is distinct from theft. The use of
password[s], serial numbers, registration codes and/or cd keys is
often a part of software installation required by the software's
manufacturer, without which the software will not function, so this
can hardly be considered illegal. Finally, the use of any of these
will not impact the future development of Taekwindow 0.3.1 in the
slightest.</p>
</blockquote>
<p>Here's that blog post. See? I do keep my promises — mostly.</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com6tag:blogger.com,1999:blog-3365890222783870731.post-79062905639397667772010-10-09T20:42:00.003+02:002010-10-15T19:56:24.957+02:00Installing Rails 3 in Ubuntu 10.4<p>For the record, and for those of my friends who are going to participate in the <a href="http://railsrumble.com/">Rails Rumble</a> next weekend: here's how you install Rails 3 on your Ubuntu Karmic system. (10.10, known as Maverick, is supposed to arrive tomorrow and might make this post obsolete. But I won't be risking an upgrade before the weekend.)</p>
<ol>
<li>Install Ruby; apt contains the required version:
<pre>sudo apt-get install ruby</pre>
<li>Uninstall RubyGems, because we'll install a more recent version from source:
<pre>sudo apt-get remove rubygems</pre></li>
<li>Install RubyGems from source:<br/>
<pre>cd /tmp
wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz
tar xzf rubygems-1.3.7.tgz
cd rubygems-1.3.7
sudo ruby setup.rb
sudo ln -s gem1.8 /usr/bin/gem</pre></li>
<li>Install Rails:
<pre>sudo gem install rails</pre></li>
</ol>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com3tag:blogger.com,1999:blog-3365890222783870731.post-10090460995954448352010-07-18T15:39:00.006+02:002011-02-18T11:58:28.281+01:00Exporting slices from Inkscape<p><strong>If you have a Python interpreter available, you can use the slightly better alternative from <a href="http://typethinker.blogspot.com/2011/02/exporting-slices-from-inkscape-part-2.html">Part 2</a>.</strong></p>
<p>In Photoshop, it's possible to “slice” an image into pieces, name the individual pieces, and export each to a separate file. This is really useful for web design: you can do the whole design in one image, then export the bits and pieces for later reassembling in HTML/CSS. Can we do the same slicing trick with Inkscape?</p>
<p>It turns out that we can; it's even on the <a href="http://www.inkscape.org/doc/tips/tutorial-tips.html">Inkscape Tips and Tricks page</a>. However, the exporting itself remains manual labour. I wrote a bash script to automate that; the code is given below.</p>
<p>The process is as follows:</p>
<ul>
<li>First, draw your image as usual.</li>
<li>Then, add a layer that will hold the slices; name it, for example, <code>slices</code>. Set the layer's opacity to about 50% to be able to see what you're doing. Enable the grid, and make sure it is set to pixels: you want your slices to align with pixel boundaries.</li>
<li>Draw your slice rectangles onto the <code>slices</code> layer, aligned to the grid. Ensure that the rectangles have no border; the fill is irrelevant (I use red).</li>
<li>Right-click a slice, and choose Object Properties. Change Id field to <code>EXP-something</code> and hit Enter. The tag <code>EXP-</code> indicates that this rectangle is intended for export; <code>something</code> will become the filename (<code>something.png</code>). Repeat this for all slices.</li>
<li>Hide the slices layer, and save your image. Let's say you called it <code>layout.svg</code>.</li>
<li>Run the script as follows:<br/>
<code>./export.sh layout.svg</code><br/>
You should see each of your slices being exported to a PNG file in the same directory.</li>
</ul>
<p>Here's the code for the script. Save this to a file <code>export.sh</code> and make it executable.</p>
<pre>#!/bin/bash
if [[ -z $1 ]] ; then
echo "Usage: $0 [FILE]"
exit 0
else
FILENAME=$1
fi
PREFIX=EXP-
for ID in `grep -o "id=\"$PREFIX.*\"" $FILENAME | cut -d\" -f2` ; do
OUTPUT=${ID#$PREFIX}.png
echo "Exporting area $ID to $OUTPUT..."
inkscape --export-id=$ID --export-png=$OUTPUT --file=$FILENAME
done</pre>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com1tag:blogger.com,1999:blog-3365890222783870731.post-841196744622728042010-05-20T13:00:00.008+02:002010-05-20T14:08:07.261+02:00Are C++ iostreams really slow?<p>In the C++ world, one often hears the statement “<code>iostream</code>s are slow, you should use <code>printf</code> instead”. Is this true?</p>
<p>It is possible that this story results from abuse of <code>endl</code>. Beginner's C++ books often recommend <code>endl</code> over <code>'\n'</code> for writing a newline. The observable result is often the same, but <code>endl</code> results in extra overhead: it flushes the stream. This results in a system call, slowing things down while often unnecessary.</p>
<p>Using my newly written <a href="http://typethinker.blogspot.com/2010/05/timing-section-of-code-in-c.html"><code>TimedSection</code> class</a>, I tested whether the story is actually true, or just a myth. I tried three methods of printing:</p>
<ul><li><code>cout</code> with <code>endl</code> for line endings
<li><code>cout</code> with <code>'\n'</code> for line endings
<li><code>printf</code> (also with <code>'\n'</code> for line endings, so no forced flushing)</li></ul>
<p>Each of these was tested in the following scenarios:</p>
<ul><li>printing an empty line</li>
<li>printing a line containing a string literal</li>
<li>printing a line containing a string variable, a string literal, and an integer variable</li></ul>
<p>All strings used were 20 characters long; I figured this would be a typical average length for printing messages. Newline characters were absorbed into string constants wherever possible.</p>
<p>Each of these was run 10 million times, redirecting the output to <code>/dev/null</code>. The testing machine is an Intel i7 920, running Ubuntu 10.4, Linux 2.6.32 and gcc 4.4.3.</p>
<p>Here are the results:</p>
<pre>cout with only endl 1461.310252 ms
cout with only '\n' 343.080217 ms
printf with only '\n' 90.295948 ms
cout with string constant and endl 1892.975381 ms
cout with string constant and '\n' 416.123446 ms
printf with string constant and '\n' 472.073070 ms
cout with some stuff and endl 3496.489748 ms
cout with some stuff and '\n' 2638.272046 ms
printf with some stuff and '\n' 2520.318314 ms</pre>
<p>Surprise! Yes, for printing newlines, <code>printf</code> greatly outperforms <code>cout</code>. We can also see the huge slowdown of using <code>endl</code> inappropriately, something which is clearly to be avoided.</p>
<p>However, even for printing a moderately-sized string constant, <code>cout</code> outperforms <code>printf</code>. This effect becomes more pronounced as the string gets longer, probably because <code>printf</code> has to do more processing per character.</p>
<p>Finally, for some slightly more complicated formatting, the difference is quite small. For longer string constants, it becomes even smaller.</p>
<p>We can conclude that <code>iostream</code>s are definitely not always slower than C-style <code>printf</code>. It depends on the particular use, and as the formatting becomes more complicated, the difference drops to zero. Part of the myth may be ascribed to inappropriate use of <code>endl</code>; maybe another part is due to unoptimized implementations of the standard library. Either way: myth busted.</p>
<p>Update: For the curious, here is the full source code. You may need to link with <code>-lrt</code> to get <code>clock_gettime</code>.</p>
<pre style="height:20em;overflow:scroll">#include <stdio.h>
#include <iostream>
#include <ctime>
class TimedSection {
char const *d_name;
timespec d_start;
public:
TimedSection(char const *name) :
d_name(name)
{
clock_gettime(CLOCK_REALTIME, &d_start);
}
~TimedSection() {
timespec end;
clock_gettime(CLOCK_REALTIME, &end);
double duration = 1e3 * (end.tv_sec - d_start.tv_sec) +
1e-6 * (end.tv_nsec - d_start.tv_nsec);
std::cerr << d_name << '\t' << std::fixed << duration << " ms\n";
}
};
int main() {
const int iters = 10000000;
char const *text = "01234567890123456789";
{
TimedSection s("cout with only endl");
for (int i = 0; i < iters; ++i)
std::cout << std::endl;
}
{
TimedSection s("cout with only '\\n'");
for (int i = 0; i < iters; ++i)
std::cout << '\n';
}
{
TimedSection s("printf with only '\\n'");
for (int i = 0; i < iters; ++i)
printf("\n");
}
{
TimedSection s("cout with string constant and endl");
for (int i = 0; i < iters; ++i)
std::cout << "01234567890123456789" << std::endl;
}
{
TimedSection s("cout with string constant and '\\n'");
for (int i = 0; i < iters; ++i)
std::cout << "01234567890123456789\n";
}
{
TimedSection s("printf with string constant and '\\n'");
for (int i = 0; i < iters; ++i)
printf("01234567890123456789\n");
}
{
TimedSection s("cout with some stuff and endl");
for (int i = 0; i < iters; ++i)
std::cout << text << "01234567890123456789" << i << std::endl;
}
{
TimedSection s("cout with some stuff and '\\n'");
for (int i = 0; i < iters; ++i)
std::cout << text << "01234567890123456789" << i << '\n';
}
{
TimedSection s("printf with some stuff and '\\n'");
for (int i = 0; i < iters; ++i)
printf("%s01234567890123456789%i\n", text, i);
}
}</pre>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com12tag:blogger.com,1999:blog-3365890222783870731.post-46599427873525477962010-05-20T12:49:00.004+02:002010-05-20T13:00:14.632+02:00Timing a section of code in C++<p>Sometimes it's useful to measure how long a certain part of your C++ code takes to execute. Here's a little utility class that makes this extremely easy. It can be used like this:</p>
<pre>void foo() {
TimedSection s("foo()");
// do some long operation...
}</pre>
<p>Calling <code>foo()</code> will print something like this to stderr:</p>
<pre>foo() 336.856127 ms</pre>
<p>As you may have guessed, a <code>TimedSection</code> records the time between its construction and its destruction. Since C++ has no garbage collection, destructors are called at a deterministic moment: when the local object (called <code>s</code> above) goes out of scope. If you want to time a section that is not a scope by itself, you can introduce a pair of braces to scope the object. If the section is even more complex (e.g. spanning multiple functions), you can use <code>new</code> and <code>delete</code>, but that destroys the elegant simplicity of <code>TimedSection</code>'s usage.</p>
<p>The implementation is rather simple as well:</p>
<pre>#include <iostream>
#include <ctime>
class TimedSection {
char const *d_name;
timespec d_start;
public:
TimedSection(char const *name) :
d_name(name)
{
clock_gettime(CLOCK_REALTIME, &d_start);
}
~TimedSection() {
timespec end;
clock_gettime(CLOCK_REALTIME, &end);
double duration = 1e3 * (end.tv_sec - d_start.tv_sec) +
1e-6 * (end.tv_nsec - d_start.tv_nsec);
std::cerr << d_name << '\t' << std::fixed << duration << " ms\n";
}
};</pre>
I hereby release this little code snippet into the public domain. Enjoy!Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com2tag:blogger.com,1999:blog-3365890222783870731.post-4263935860285771472010-04-05T12:10:00.005+02:002010-04-05T12:56:00.770+02:00So long, and thanks for all the glass<p>When I first installed Windows 7 after nearly a decade of using XP, I decided I'd not customize it. I'd just leave the settings at their default values and see what the wise men at Microsoft had decided for me. After all, there are thousands of hours of usability testing behind those decisions, so it seemed a good idea to give them a chance.</p>
<p>Now, half a year later, I've gotten used to most of Windows 7. Although I've tweaked the occasional thing here and there (show me the damn file extensions already!), I think most of it is pretty reasonable. But there's one thing I just can't get used to: the Aero theme.</p>
<p>It's not that it's semi-transparent. It's not that it fades and zooms and zips and zaps. It's just the minor little thing that <em>I can't see the difference anymore between active and inactive windows</em>.</p>
<p>When you activate a window, the title bar and borders become a little bit darker, the shadow becomes a little bit blacker, and the close button changes its colour from glassy to red. You can see the changes happen while you're switching focus, but telling the difference at a glance from a static image is hard. And because the title bar is transparent, the colour depends on the things behind it, making it even harder. There's just no single, consistent look for active and inactive title bars. I kept having to look at the close button to tell which window was active.</p>
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkdAHlPRTqGnIOGDf1XwZoM9pHHLfODcpWrtshCG5VwH8lIBuunjbiaExQVVSSUv_NAR8S6tPg4BlGXClQipSVIjpC__KBBcVmd12o56alxex1f51zf1e8qfsJ8TfQX5UAxLdnX-va-ls/s1600/aero-focus.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 400px; height: 129px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkdAHlPRTqGnIOGDf1XwZoM9pHHLfODcpWrtshCG5VwH8lIBuunjbiaExQVVSSUv_NAR8S6tPg4BlGXClQipSVIjpC__KBBcVmd12o56alxex1f51zf1e8qfsJ8TfQX5UAxLdnX-va-ls/s400/aero-focus.png" border="0" alt="The subtle difference between an inactive and active title bar in Windows Aero" id="BLOGGER_PHOTO_ID_5456602714472088722" /></a>
<p>So what am I to do? Am I to magically know which window is active? When I've just clicked it, or when I'm typing into it, sure, I'll know. But being the Alt+Tab guy that I am, I shift focus without clicking all the time, and not always to the correct window right away. And what if some unexpected window suddenly pops up that might have stolen my focus, like an auto-updater? How can I tell if I can continue typing, or that I'd better yank my hands away from the keyboard in case I inadvertedly activate the Launch Nukes button?</p>
<p>Unfortunately, there is no way to change the look. The wise men at Microsoft have decided that a “theme” will be nothing more than a single colour for the glass, along with a set of wallpapers. There's no way to select different colours for active and inactive windows, like there is for the “classic” themes. There's also no way to install a clearer Aero theme without resorting to commercial third-party software like <a href="http://www.stardock.com/products/windowblinds/">WindowBlinds</a>.</p>
<p>So in the end, after my millionth focus fuckup, I switched back to the classic Windows 2000 lookalike theme. Blue versus gray seems like a distinction my eyes can handle.</p>
<p>Why do I have to sacrifice good looks to gain usability? Why can't Microsoft just make them get along?</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com1tag:blogger.com,1999:blog-3365890222783870731.post-70009449460220036222010-03-20T23:02:00.004+01:002017-01-22T11:13:11.970+01:00A harmonograph in JavaScript<div dir="ltr" style="text-align: left;" trbidi="on">
What's a harmonograph, you ask? Well, it's something like a Spirograph for grown-ups. A harmonograph uses a construction with pendulums to draw pretty patterns.<br />
<br />
One particular incarnation uses two pendulums of nearly equal length, each about two metres. To the right pendulum, a slowly revolving disk is attached, which is driven by a small electrical motor. The drawing paper is taped onto the disk. To the left pendulum, an arm is attached with the pen at the end. When the pressure and thus the friction of the pen is sufficiently low, the pendulums can keep swinging for tens of minutes.<br />
<br />
For learning and fun, I replicated this harmonograph in JavaScript with the HTML5 canvas element. The pendulum motion is approximated by a sine function, and friction by an exponential, but apart from that it should be pretty physically accurate.<br />
<br />
Without further ado, I present to you: <a href="https://ttencate.github.io/harmonograph/">Harmonograph in JavaScript</a>. Usage should be self-evident. Although some input validation is done, you will be able to break it with strange values.<br />
<br />
With the link at the top, you can save and bookmark your creations or send them to your friends. Enjoy!<br />
<br />
<i>Update, 2017-01-22</i>: Hosting is now done <a href="https://ttencate.github.io/harmonograph/">here on GitHub Pages</a>. The old location redirects you. I also published the <a href="https://github.com/ttencate/harmonograph">source code</a> (“View Source” still works just fine, but it's now officially MIT licensed as well).</div>
Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com8tag:blogger.com,1999:blog-3365890222783870731.post-12028537872176715062010-03-14T14:29:00.007+01:002010-03-14T14:57:32.286+01:00Capitalizing MP3s from the command line<p>For my own future reference and your enjoyment, here's a neat little oneliner to transform <tt>01 - some song.mp3</tt> into <tt>01 - Some Song.mp3</tt>:</p>
<p><code>rename "s/([^._'])\b(\w)(\w*)/\$1\U\$2\L\$3/g" *.mp3</code></p>
<p>It needs the <code>prename</code> utility from Perl, which on Ubuntu is available by default under the name <code>rename</code>.</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com3tag:blogger.com,1999:blog-3365890222783870731.post-69287977254882643052010-02-05T18:30:00.008+01:002010-08-04T19:34:14.268+02:00Ubuntu fails to load nvidia kernel module<p>As much a note to myself, as a warning to others…</p>
<p>After a kernel upgrade, my Ubuntu Karmic started the X server in low-resolution mode. My Xorg.0.log said:</p>
<pre>(II) LoadModule: "nvidia"
(II) Loading /usr/lib/xorg/modules/drivers//nvidia_drv.so
(II) Module nvidia: vendor="NVIDIA Corporation"
compiled for 4.0.2, module version = 1.0.0
Module class: X.Org Video Driver
(EE) NVIDIA: Failed to load the NVIDIA kernel module. Please check your
(EE) NVIDIA: system's kernel log for additional error messages.
(II) UnloadModule: "nvidia"
(II) Unloading /usr/lib/xorg/modules/drivers//nvidia_drv.so
(EE) Failed to load module "nvidia" (module-specific error, 0)
(EE) No drivers available.</pre>
<p>I tried <code>modprobe nvidia</code> and such, but it seemed that the module actually did not exist. This module should be installed by the package <code>nvidia-185-kernel-source</code>, which was present on my system. However, it turns out that the kernel module is compiled on-the-fly by a program called <code>jockey</code> which controls DKMS, the Dynamic Kernel Module Support.</p>
<p>It is possible to force a recompile using <code>dpkg-reconfigure</code>:</p>
<pre>$ sudo dpkg-reconfigure nvidia-185-kernel-source
Removing all DKMS Modules
Done.
Loading new nvidia-185.18.36 DKMS files...
Building for architecture x86_64
Module build for the currently running kernel was skipped since the
kernel source for this kernel does not seem to be installed.</pre>
<p>I need the kernel source, eh? Why the hell is that not a dependency, if the driver package is useless without it? Anyway, let's install the kernel source then:</p>
<pre>$ uname -r
2.6.31-19-generic
$ sudo apt-get install linux-source-2.6.31</pre>
<p>Installs fine, but makes no difference. Turns out that <code>dpkg-reconfigure</code> was lying: I just need the headers. Here we go:</p>
<pre>$ sudo apt-get install linux-headers-2.6.31-19-generic
...
$ sudo dpkg-reconfigure nvidia-185-kernel-source
Removing all DKMS Modules
Done.
Loading new nvidia-185.18.36 DKMS files...
Building for architecture x86_64
Building initial module for 2.6.31-19-generic
Done.
nvidia.ko:
Running module version sanity check.
- Original module
- No original module exists within this kernel
- Installation
- Installing to /lib/modules/2.6.31-19-generic/updates/dkms/
depmod......
DKMS: install Completed.
$ modprobe nvidia
$
</pre>
<p>That's better.</p>
<p>Several bug reports indicate similar problems, but the current way this is handled is terribly inadequate. The driver package should pull in the kernel headers if it needs them. There was no warning about this when the kernel was upgraded. There was no warning when the module failed to compile on boot. A fix for a problem with the same symptoms was released back in December; another one is in the upcoming Lucid release.</p>
<p>Oh yeah, I ended up rebooting my system. Whatever happened to Ctrl+Alt+Backspace? (<a href="https://wiki.ubuntu.com/XorgCtrlAltBackspace">Answer</a>.)</p>
<p><strong>Update, 2010-08-04</strong>: After another kernel upgrade, my display driver was hosed again. After hours of tinkering, I typed <code>sudo dpkg-reconfigure nvidia-current</code> and was greeted with the message <code>gzip: stdout: No space left on device</code>. Apparently, my <code>/boot</code> partition was full (of abandoned kernels). Something to check, for whoever runs into similar problems! Also, the kernel module appears to have been renamed from <code>nvidia</code> to <code>nvidia-current</code>.</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com23tag:blogger.com,1999:blog-3365890222783870731.post-84054872760450970702010-01-24T14:58:00.017+01:002010-01-24T17:27:02.125+01:00Review of free C++ IDEs for Linux<p>For my new project, I started looking into IDEs for Linux for the C++ language. And since I'm testing them anyway, I might as well write some quick reviews. ‘Quick’, because I'll mostly be going on first impressions, and some things I say might be quite wrong. Feel free to correct me in the comments!</p>
<p>My previous IDE experience includes Visual Studio for C++ and for C#, Eclipse for Java. I will sometimes compare to these.</p>
<h2>Code::Blocks</h2>
<p>I am greeted with very nice wizards, and it's easy to get up to speed. There is even a template for GLFW projects, which I select. But when trying to compile (using shortcuts nothing like any other IDE I know) I get a linker error; some XF86 library is missing, and even <code>apt-file</code> cannot tell me where to get it.</p>
<p>Hunting through the build settings, I notice many other problems. The settings are nested three levels deep: a bar with huge icons on the left, <em>scrolling</em> tabs (the horror) on the right, and <em>nested</em> tabs below those. Many options are unclear. When I press Esc to cancel my changes and dismiss the dialog, nothing happens.</p>
<p>Some parts of the Code::Blocks UI are really good, other parts are… not so good. I wouldn't want to work in this for a long time.</p>
<h2>KDevelop</h2>
<p>It starts up with a blank screen, so I have to ask it to create a new project myself. Not a big deal. The New Project wizard is reminiscent of Visual Studio, so I feel quickly at home. It allows me to create a CMake project; I like CMake. The wizard gives me a Hello World program, but I cannot run it: I have to create a Launch Configuration first. Then I run my program, I think, but where does the output go? The Debug Area (‘Perspective’) is empty. Turns out that building failed, because I moved the project in the mean time, and the project file contains absolute paths. Bad!</p>
<p>Support for external tools seems good. I already mentioned CMake; the Valgrind profiler and Git versioning system can also be used right from the IDE.</p>
<p>The IDE itself actually seems like a blend of Eclipse (borrowing much of its concepts and terminology), Visual Studio (some wizards and the layout of dialogs) and Kate (the general KDE stuff). Not bad at all. Still, I had some trouble getting everything set up, but once you've taken the time to do that, KDevelop might be a decent IDE.</p>
<h2>Eclipse</h2>
<p>Big, bloated, but feature-rich and quite user-friendly once you've warmed up to it. That's been my impression of Eclipse for Java, which I'm quite familiar with. For Java, it's easily the best IDE around; the refactoring options alone make it worthwhile. Let's see how Eclipse with the CDT plugin performs for C++.</p>
<p>I install Eclipse, grab the CDT plugin through its update site and restart Eclipse. A few clicks later, I have an empty C++ project. I add a main Hello World file, and after some strange trouble with the Debug Configuration, it runs. I add a library to link with, finding the option without any difficulty. Not being unhappy about this, I continue to write some more code, to try the completion and refactoring features.</p>
<p>Completion pleasantly surprises me. This stuff is better than Visual Studio's. It's fast, it's accurate, it gets the contents of a newly included header file right after I saved… nice! Smart insert mode adds closing paretheses, braces and quotes, but if I type them myself as well, they are overwritten.</p>
<p>Error highlighting is equally fancy. Unlike Visual Studio, syntax errors are highlighted in real time, without even having to save the file.</p>
<p>I ask it to create a new class, and get a header and source file with a nice skeleton. It's not exactly the way I would write it, but I'm sure it's customizable.</p>
<p>Refactoring, unfortunately, is not as fancy as it is for Java. It is limited to renaming identifiers, and doesn't rename the source and header file accordingly. It can also extract selected expressions/statements into a separate function, but it does not create this as a member function. There are no “quick fix” options for adding include directives, like with Java. Yet, it's better than what Visual Studio offers on this front.</p>
<p>Eclipse with the CDT plugin is not perfect, but it's good. I could live with it.</p>
<h2>Anjuta</h2>
<p>I am greeted with a welcome screen, which I ask to create a new project. I am asked for all kinds of needless details, like the license type and my e-mail address. No wonder: it creates an entire autotools project for me. I tried to add a library to the linker command, but couldn't find the option anywhere. Do I really have to edit the Makefile.am?</p>
<p>I've only had a brief look, and Anjuta seems like a nice IDE if you want what it wants, but at first sight doesn't seem flexible enough to cope with everything.</p>
<h2>Qt Creator</h2>
<p>Would this work for non-Qt applications as well? It wouldn't seem so at a first glance, but I imported an empty directory as a Makefile project, which seemed to work. I had to write a Makefile, of course, which failed because the IDE quietly changed my tab to four spaces. After fixing that, and some directory issues, I could build the program. Running it required similar trickery.</p>
<p>Code completion works very well, and identifiers from a newly included header are available quickly. The entire IDE looks and feels smooth and polished, and is reminiscent of Visual Studio in many details.</p>
<p>The build system remains weird, though. It created an autotools project for me, with about a dozen files, and not offering me any choice in the matter. There is a reference to CMake in the Preferences, but no clue how to create a CMake project.</p>
<p>Qt Creator is a slick, clean, usable IDE. I like it. Still, it is very Qt-centric, and its build system seems inflexible. Too bad.</p>
<h2>NetBeans</h2>
<p>Yes, another Java IDE that can be used for C++. I have actually never used NetBeans for Java before, so I have no preconceptions about how it should work.</p>
<p>I apt-get the IDE and install the C/C++ plugin; this is very straightforward. I create a new C++ project, and a Makefile gets created automatically. I run the empty main function, and the program pops up in an external terminal.</p>
<p>Then I start editing the code, and I'm pleasantly surprised by the editor. Not just syntax errors, but unknown identifiers get highlighted in real time. Function signatures pop up in a large, but helpful tooltip. Even the filenames of include files are autocompleted! The contents of the include file become available without even having to save the file.</p>
<p>Yet, not all compile errors are highlighted, even though you can click to them from the compiler output window. I could not manage to get the function signature to reappear while I was typing arguments. Refactoring options are all grayed out; I assume those work for Java only.</p>
<p>The project settings are very much like Visual Studio, but allow for little customization. It's just enough to add a <code>-l</code> flag to the linker command line to link in an external library. There is also Git integration (if you install the plugin) but I didn't try it.</p>
<p>I checked out NetBeans as an afterthought, but that seems to have been a mistake. Of all the IDEs I tried, this is actually one of the nicest. It doesn't have many bells and whistles, and its build system might be inflexible, but the code editor makes up for it.</p>
<h2>Conclusion</h2>
<p>Are IDEs for C++ worth the trouble? I'm not sure. The value of an IDE lies in fast code editing, refactoring options, easy building, and error highlighting, and integrated debugging. None of the IDEs I tried score highly on all these fronts. I can understand: a flexible, usable IDE for C++ is almost impossible to create. Maybe I'll just go back to my trusted old Kate and gnome-terminal…</p>Thomas ten Catehttp://www.blogger.com/profile/02609144861191873031noreply@blogger.com3