OOXML: defective, but don't exaggerate

In a recent article titled Microsoft Office XML Formats? Defective by design, self-proclaimed file format expert Stéphane Rodriguez explains 13 reasons why Microsoft's Office Open XML (OOXML) format should not become an ISO standard. Although I completely agree with his conclusion, Rodriguez seemingly got carried away by his rant, uttering nonsense in some places. Still, most arguments make sense, so I'll cover only those that don't, below.

1) Self-exploding spreadsheets. Here, he modifies an Excel file manually, and is surprised that even this “simple” change breaks the file. He does not once refer to the specification to see whether the thing he changes may be dependent on things elsewhere in the file. So, probably, the file he created is not at all according to spec. Is it strange, then, that Excel goes boom? An Office document is a lot more complex than you would say at first sight, and the storage format is bound to reflect this.

2) Entered versus stored values. Clearly, the values get processed internally as (binary) floating-point numbers, which explains numbers like 1234.1233999999999 cropping up when converted back to the decimal storage needed for XML. If an implementation simply uses the IEEE floating point format, like pretty much any CPU does, no problem. Besides, the fact that Excel writes out (very slightly) inaccurate numbers does not mean that the OOXML standard is flawed, only Excel's implementation thereof.

6) International, but US English first and foremost. Another complaint of Rodriguez is that the numbers get stored in US English locale format (1,234.56), and not the localised format (e.g. 1 234,56). Also, formulas always use English function names, like SUM. Rodriguez claims that this canonicalisation makes processing more complex. Wait, what? You want us to go store things differently depending on how some people would like to view them on the screen? You think a file will be easier to process when we introduce diversity in locales?

12) Document backwards compatibility subject to neutrino radioactivity. So, Excel 2007 cannot properly import graphs from earlier versions. What else is new? Anyway, I do not see how that can be regarded as a flaw in OOXML; the quality of the Microsoft Office suite is something quite different from the standardisation of the OOXML format.

Unrelated to Rodriguez's rant, there's one gem I would not want to keep from you. The folks at Google discovered [PDF link], buried in the over 6000 pages of the OOXML spec, 51 pages of stuff like the following:


If anyone dares to say these specs aren't bloated, point them to part 4, section 2.18.4, pages 1632–1682. I honestly don't know whether to laugh or cry.

If you like, you can have a look at the standard yourself. It is known as ECMA-376. The specs can be downloaded in either zipped PDF format or in a mysterious format called DOCX. For the latter, alas, no fully functional implementation appears to exist.

Blink

Shortly after my previous post, Trust you gut, I read about the book Blink: The Power of Thinking Without Thinking by Malcolm Gladwell. Both my conscious and my subconscious mind told me to purchase it. It took me only two days to finish the book.

Gladwell makes essentially the same point as I did in my previous post. In his words, much of our thinking happens behind a “locked door”: you can only sense the outcome, but not where it came from. If you try to figure out why you took that particular decision, only nonsense will come out. If you try to follow the reasoning while your subconscious is deciding, your conscious thoughts appear to interfere with it, and the subconscious is essentially disabled.

This subconscious reasoning is not magic. It relies on subtle clues that are too numerous for your conscious mind to process, and on tacit knowledge that comes from previous experience. For example, I am quite good at troubleshooting computer problems, but only if I'm there to witness them. If somebody tells me “my computer is doing this-and-this, do you know how to fix it?” I can give some general pointers, but it's only at the keyboard that I really get the insights. Not only the text of an error message is meaningful to me, but also its looks, its responses and the precise thing I was doing at the moment it popped up. It's not magic, it's simply lots of experience. And no, I will not fix your computer.

Of course Gladwell took much more time for his research than I did for my blog post, and he also gives examples of situations where you shouldn't trust what your subconscious tells you. For example, when you are in a dangerous situation, your mind goes into a state that psychologists call “arousal”. It shuts down parts of the brain that are deemed non-essential at that moment, including the one that recognises emotions on human faces. This explains why police officers shot a man because they thought he had a gun, while in fact he thought he was being robbed and reached for his wallet in mortal fear.

Whether you like it or not, a lot of our thinking happens subconsciously. I think that we are not aware of over 99% of our own thinking, our consciousness being just a flimsy layer on top of that. I pulled that number straight out of my arse, of course. But my arse may well be a lot smarter than you'd think …

Trust your gut

Note: I'm not a psychologist. The following is based on personal experience only, probably including generalizations and skewed perceptions.

With my recent moving to another place, I've had to make many decisions, both big and small. Most people think that, once you've gathered enough information, you make your decisions as follows:

1. think about the pros and cons of each option;
2. make decision based on the balance between the pros and cons.

But I noticed that the process is usually more like this:

1. make decision;
2. pretend to think about the pros and cons;
3. reinforce decision by stressing the pros and diminshing the cons.

The catch is that the making of the decision happens unconciously: you have often already decided before you're even aware of it. All thinking after that point only serves to rationalize it for yourself or others.

I also noticed that my unconscious decisions very often turn out to be right. If I overrule them with some rational argument, it often turns out that the rational argumentation was overlooking some important point. Apparently, more thinking happens “behind the scenes” than you know. And once you've decided, how important is the reasoning anyway?

So, I try to keep an eye on my subconscious. When I notice that it has already made the decision, I often go with it. It saves a lot of time and effort spent on needless thinking.

God does play dice

Einstein was wrong. God does play dice. I heard them rolling tonight.

It was nice enough weather for most of the day. A bit warm and clammy, but not much to complain about. Until, at half past nine, I looked out the window and saw it getting at least three stops darker over the course of less than eight minutes.

A quick check of the radar pictures confirmed that a pretty heavy thunderstorm was rapidly coming my way.

I had tried to photograph lightning the day before, but that had been in the afternoon. I couldn't use a long enough exposure time because the daylight was still too bright. Out of hundreds of pictures I only got one with a small lightning bolt on it. This time, it was rapidly getting darker, and not only because evening was falling. I must be able to do better.

I grabbed my camera and tripod and ran up the stairs to the walkway on the 7th floor. Out there the wind was getting stronger, and I had trouble keeping the camera steady on my light tripod. I removed the cord from the camera to make it catch less wind.

Varying the exposure time between 1/4th of a second and 15 seconds, I snapped many pictures in a row. The camera was a bit too slow for the task, so I didn't want to waste time looking whether I got any decent pictures. I took care to heavily underexpose them, so that when lightning arrived, it would not overexpose the picture.

Thunder rolled time and time again. It was as if someone up there was rolling his massive dice over the clouds each time the light flashed. I wondered whether even God would know the outcome of His roll in advance—if so, why was He even playing?

In the meantime it had started to rain, and the walkway on one side of the flat was pretty open to it. I took back a few steps to avoid getting drops on the lens and ruining my pictures. Many awesome lightning bolts flashed before my eyes, but often right in between two exposures. It seemed like the dice were loaded in my disadvantage.

When the best part of the storm seemed over, I folded up the tripod and turned off the camera. I smiled as I saw, right before the automatic lens cap closed, that the lens was dry.

As soon as I was back in my apartment, I pulled the 684 pictures off the card and started flicking through them. Many of the pictures had a light, pink sky, but no directly visible lightning. But I had gotten lucky a few times.

Then, while sorting out the pictures, I saw more flashes out of the corner of my eye. It wasn't over yet! So I mounted the camera on the tripod again and, not taking the time for all the stairs, went out onto the balcony. I could hear from the many shouts and exclamations that I was not the only one watching the show.

Suddenly, there was a flash brighter than all the others had been. Even before the light had died out, a cracking, deafening thunderclap sounded. “Fucking hell!” I exclaimed. The impact couldn't have been much over a hundred metres away.

Then I figured that, after this close call and my profanity, I'd better get back inside. God does play dice. I heard them rolling tonight. And I wouldn't want them to come down on my head.

JVC Everio GZ-MG575 review

Last week, I bought another camera: the JVC Everio GZ-MG575. It set me back € 995. I'll be returning it today, and I do not have time to write a full review like I did with the Panasonic NV-GS320. But I can give you an impression. Short story: Do not buy this camera. Ever.

Contents of the package

Apart from the camera, the most important part is the dock, which connects to the through a single connection at the bottom. There is the mandatory remote control (including battery). The Dutch manual is translated quite badly and there was no English version in my package. There's an USB cable, an A/V cable, a power adaptor and a shoulder strap.

Picture quality

The picture quality is reasonable, but not outstanding. Widescreen is fake, simply chopping off the top and bottom of the picture. Instead of making the view angle wider, it is actually made less tall.

The picture looks a bit greenish, and there's no setting to correct for this. It is not a matter of white balance.

The (digital) image stabilizer only works if your picture is already nearly still, and even then results in a motion that feels choppy. Once you've worked with an optical, physical stabilization system, you realize how much better that works.

Autofocus is very slow. It focuses solely on the centre, and does not pay attention to other factors, so apart from slow it is also stupid. As is to be expected without an optical viewfinder and a lens ring, manual focus sucks. Your only option is to point where you want to focus, wait for autofocus to kick in, then switch to manual focus to lock the focus.

Auto white balance detection is reasonable, but… it is not smooth. When walking from inside to outside, for example, it will instantly switch the white balance from indoors to outdoors. Something you definitely do not want.

Low-light performance

I haven't tested the low-light performance, but I expect that it does a pretty good job here, judging from the large lens opening.

Photographs

In a well-lit room, with big windows and a lot of outside light, setting the aperture to F3.5, the camera tried to use a shutter speed of… 1/5 second. In case you're not into photography: it will be impossible to get a sharp picture at that speed, and even cheap and crappy compact cameras like mine do a lot better. I did not bother trying to take any more photos after this insanity.

Audio

I haven't tested the internal microphone thoroughly. If it turns out to be crap, you at least have the option to plug in an external microphone.

Recording medium

The camera records to a hard disk. Compressed like a dvd. Even on the maximum quality setting. You're probably going to export to dvd sooner or later anyway, so it may not be a big problem, but I like to start with the highest possible quality of material.

There's also an SD card slot that I haven't tested. Apparently you can record photos as well as video to this card.

LCD

The LCD has a pretty limited view angle, especially in the vertical direction. I haven't tested it in direct sunlight, but if it fails there's no viewfinder to fall back on.

Usability

The user interface is downright horrible. Let me give you an example. When set to manual mode, with the little wheel at the top, you can switch to manual focus by pressing the joystick downwards. Now switch to, say, S mode where you can set the shutter speed. Manual focus is retained. But you set the shutter speed—you guessed it—by pressing the joystick up and down: you can no longer turn off manual focus!

The menu wraps around like a cilinder, so you don't know when you've seen all options. All menus are animated, and rather slow and annoying too.

All buttons seem to be in just the wrong places, and there's no way you can control this camera with one hand. Many buttons have different functions depending on the mode. All of it just fails to make sense.

There's a “drop detection” that will shut down the camera when it detects a falling motion, to protect the hard disk. It also shuts down if you simply lower it a bit too quickly. So I turned the feature off; the best way of keeping your hard drive intact is still simply not dropping it. But at startup, the camera keeps warning me that drop detection is off, and an annoying blue icon keeps flashing on the screen all the time.

Battery life

I'm not sure how this compares to other hard disk cameras, but in itself, I found the life of the accompanying battery quite limited. Apparently there's a good reason that the box (that you see even before you buy the thing) already states: “Don't Forget A Back Up Battery!”

Capturing and editing

Here's a point where I do like this camera. It comes with a dock, to which you can connect a power supply, USB, FireWire, S-Video and AV. A welcome change from having to plug in two or three cables each time.

The camera shows itself to Windows as the hard disk it contains. You can simply copy off the .mod files. These are, according to the manual, a “proprietary format”, but actually they are simply .vob files like on a dvd. After renaming, Premiere has no problem importing them. No external software is needed.

I had expected editing to go less smooth with compressed material, but I haven't noticed anything of the kind. It all works just as well as with raw DV videos, as far as my (admittedly quick and short) editing session could tell.

Conclusion

Horrible usability, and the picture quality does not make up for it. Do not buy. Definitely not worth your money. I'll be going back to my good old Panasonic, external microphone input or no.

Side note: during the entire time I'm writing this, the camera is erasing its hard disk. Although it says “formatting”, I hope it is actually overwriting all the bits seven times, because nothing else accounts for half an hour of formatting time…

Equality checking on interrelated objects

Most, if not all, object-oriented programming languages allow you to define what it means for two objects to be “equal”. In C# and Java, this is done by overriding the Equals or equals function, respectively.

But what to test when comparing two objects? You'll obviously want to compare (most) instance fields, for example the name of a person or the number of wheels of a car. But it's less obvious when an object is (conceptually) part of another object, or dependent on it in some other way. I ran into such a situation today.

Consider the class Pig. A Pig is associated with a Farmer, the owner. Each farmer has, in turn, a list of ownedPigs. (I use Java capitalization conventions here to make a clear distinction between types and fields/methods.)

Suppose we want to check whether two Pigs are equal. (Do not just return true. Some animals are more equal than others.) Clearly, two Pigs are not equal if they are owned by a different Farmer, so we need to compare their owner fields. Reference equality on this field is not enough: if we can have multiple Pig objects representing the same pig, then the owner fields may well refer to different objects representing the same farmer. So we do a value comparison of the owner fields, by calling owner.equals.

Of course, two Farmers are not equal unless they own the same Pigs. We have to call Pig.equals for each owned pig. This, in turn, calls Farmer.equals, which calls Pig.equals … and so on ad infinitum (which is Latin for “stack overflow”).

How to solve this problem? The key is, when equality of type A depends on equality of A's fields of type B and vice versa, to check not the entire B objects for equality, but only compare the parts we're interested in.

For example, a Pig couldn't care less what the social security number of its owner is. It does, however, care where he lives: a pig on the South Pole will have very different life circumstances from one in the Sahara. In Pig.equals we would then only compare the addresses of the owners, and not the entire Farmer objects.

With this modification, the cycle is already broken. We could also tackle the problem from the other side: the Farmer cares only about how fat his pigs are, but not about their political preference. When comparing two Farmers we could look only at the weight of all their ownedPigs, disregarding their vote field.

I admit this example is a little contrived. I'm not programming an animal farm. But if you ever do, and your Pig.equals method gets called with a Farmer object, remember to return false.

My goal in life

The standard impossible-to-answer philosophical question, which has become very much a cliché, is “What is the meaning of life?” I've answered this for myself ages ago: life has no meaning by itself. Everybody can create as much or as little meaning for his or her own life as they please.

It helps to have an ultimate goal in life. For every decision you need to make, you can check whether it aligns with your life goal. Having a goal also gives your life purpose and motivate you to keep going.

When I read a post by Steve Pavlina a couple of months back, I was reminded of this. What was my goal in life? Steve suggests to write down potential goals until one makes you cry — that is the goal you're looking for. I discovered I didn't need this: the answer popped right into my head, and had probably been there for a long time.

All that remains is to write it down. My goal in life is to continuously keep improving myself, and thereby the world around me.

I'm a pretty altruistic person. Yes, I want to improve the world. But I also have a selfish part. I want to grow, to learn, to become a better person in every sense. These goals don't have to be in conflict. A “good” me also cares about his surroundings, helps other people, tries to make a difference for the better. That's the person I want to be.