Walking around Blind

For me, one of the most difficult aspects of running a business is that you will never have good information to take decisions. You will walk around blind most of the time.

Of course, I keep track of many metrics, the obvious ones (new customers per day, monthly revenue) and the domain-specific ones (like diagnostic accuracy or repair success rates split by camera models).

But you only can track what users do, and the most important information is about what people doesn’t do and why, from people that will not even become users, or from users that give up because your product sucks. But at the end, there is no easy way for you to know why your product doesn’t get noticed or sucks.

Therefore, the information that you collect tends to be heavily biased towards confirming what you already know, rather than telling you what you ignore or what needs to be fixed.

One example: Localization
Our Repair Service is currently only available in English. In Aero Quartet we have also native French and Spanish people, so we would like to make the service available in those languages, and also in German (we’ll need to hire for that). Now I need to make numbers and figure out how much growth I can expect from localization in FR, ES and DE.

I know that exactly 7.4% of my customers are in french-speaking countries or regions. Easy.
But the really interesting figure is how many would-be customers are turned away because the service is not in French. And that figure is not something I can measure, because people searching for “réparer fichier XDCAM” will never find us or contact us in the first place.

Therefore, to take the right decisions, I will try to look at it from different perspectives, and take calculated risks, just like a blind person compensating with other senses.

Footage Worth Crushing a Camera or two

A trend that we are seeing in Video Repair is the use of video cameras for high-risk or destructive takes, for example:

• Mount a camera on a truck that will fall off a cliff
• Integrate a camera into a space rocket and shoot video until battery dies
• Mount a camera on a RC plane

Nowadays, it’s easy to find an inexpensive camera with good enough image quality and appropriate robustness. Even if the camera is crushed, the footage is safe because it is stored in the camera SD card, a small and robust component with no moving parts.

Just take the SD card out of the crushed camera and read the files!

Of course, the movies whose recording did not end “gracefully” are corrupt and have to go through a repair (and we know who can do this job…) but at the end you can have an impressive and valuable piece of footage that you couldn’t produce by “conventional ways”.

Disposable Video Cameras

Actually, the camera makers are starting to see a market for disposable video cameras, and it comes naturally as an extension of action cameras market, with popular models like GoPro HERO, Vholdr Contour or Drift HD. Established brands like Olympus have also shown interest, going as far as to develop a specific “Tough” series of shock-resistant and waterproof video cameras.

Don’t Try This at Home

If you want to record high-risk or destructive takes, you should plan it carefully:

• Fasten the camera to a strong support
• Try to avoid direct exposure of camera to shock or water. Pick position carefully, and use protective padding if possible
• Use strong tape to prevent camera doors from opening. Battery, SD card can be ejected under a violent acceleration and jeopardize the take
• And the most important: Never expect to capture the last second of life of a camera.

The Missing Last Second

Why? It hasn’t been written to the card yet when the camera dies.

Short Explanation:

In your camera, the recording process is more or less as follows: raw data from the sensor go to a memory “buffer”. Once the buffer is full (which corresponds maybe to one second of action), the encoding chip make his job and writes the compressed video in a second memory “buffer”. Once the second buffer is full (containing one to a few seconds of action), the data is written on the card. Only at this moment the data is persistent. (because the two memory “buffer” are volatile memory).

When the camera death occurs, the information in the two buffers (corresponding to last seconds of action) are in volatile memory. Not on the card.

Therefore, you should plan your take so that your camera stays alive for 3 to 5 seconds after the moment you want to capture.
If it’s not possible, a live recording system should be used, but that’s another story.

Does FW1.2.5 Fix Canon 7D Reliability Problem?

Canon EOS cameras offer great things to videographers. But since video recording was added to 5D Mark II and then to 7D, there has also been a big number of reliability problems.

Canon EOS is “producing” an order of magnitude more repairs than other consumer cameras that outsell Canon 7D in the market.

Our job is to repair video files, so we are in a position to say something about it. Over the last two years, Canon EOS repairs have sky-rocketed, quickly becoming our main “product”. In most cases, a few clips shot during an event are corrupt, despite having recorded good clips before and after. The media data is present and is consistent (a condition to make the repair possible), but the movie index is missing.

It’s as if the camera had stopped working in the middle of the recording, or could not finish its job of adding the index just after recording ends. This smells like a firmware bug or system limitation. In any case, this is something that Canon can only hope to solve through a firmware upgrade — or — a new camera design.

(There are other failure modes found in Canon EOS cameras, like use of low performance CF cards, accidental deletion, battery issue … but those are the “usual suspects” that we find in all cameras. Here we focus on top Canon-specific failure mode: corrupt file due to firmware bug or system limitation).

FW1.2.5: Canon Acknowledges the Problem

In April 2011, Canon has released a new firmware, FW1.2.5, for Canon 7D.
According to Canon, Firmware 1.2.5 should eradicate corrupt movies:
“Fixed a rare phenomenon in which there were rare cases where movie files taken could not be opened when repeatedly shooting movies with specific CF cards.”

Strange wording. “Rare cases”, “specific CF cards”, look like words forced by PR or Marketing department, and in my opinion, try to masquerade the fact that Canon has only partially solved the problem (or the problems) and that you should not be surprised if failures keep occurring…

So I decided to dig into our repair history and figure out whether FW1.2.5 has changed anything over the last 6 months.

First, I notice that after FW1.2.5 release, our Canon repairs drop by 15%, then they start growing again, but at a slower pace than before.

How Firmware Roll-Overs Work

When a new firmware is released, all new cameras are manufactured with it. But it can take a few months until those cameras get into customers hands. If distribution and sales channel is slow, if inventory doesn’t receive the upgrade, the old firmware will continue growing in installed based for many months until new one first appears, then becomes significant.

Firmware upgrade is also possible on older cameras. But my guess is that only a small percentage of people is upgrading its camera firmware. One exception: if they are confronted to a problem, or receive advice to do so by Canon support, in a newsletter or in an Internet forum, they will upgrade.

FW1.2.3 Roll-Over as a Baseline

FW1.2.3 is transparent: It doesn’t claim to fix any failure mode causing corrupt clips.
That’s why it’s a good baseline to see how firmware roll-overs happen in Canon EOS world.

We look at the picture 6 months after the release to give it time to significantly affect the installed base through new sales and manual upgrades. The chart shows how repair cases are distributed by firmware versions:

• After 6 months, the latest firmware accounts for 18% of repairs.
• FW1.2.2, the last one, is now the most represented. This is due to inertia of installed based and sales channel.

Now with FW1.2.5, a Different Story

The big change with FW1.2.5 is that it claims to solve clip corruption issues. So all other things being equal, we will evaluate the efficiency of the firmware fix by comparing the occurrence of repair cases.

If FW1.2.5 were not fixing the bug, then 6 months after release, the distribution of repair cases by firmware releases would look exactly like the FW1.2.3 chart (see above).

It turns out that it’s NOT the same:

The significant differences and their possible explanation are:

• Volume of repairs decreases by 15% (as noted in first chart). Sounds like there is improvement1
• FW1.2.5 represents only 12% of total, which is significantly lower than 18% of FW1.2.3 six months earlier.
• Considering growth of installed base (with important FW1.2.3 inertia and FW1.2.5 in most recent sales), FW1.2.3 should be the most frequent version, but its volume is almost the same as six month earlier!
• We see an important and unexpecte decrease in FW1.2.2.

I can only understand those results by imagining that customers have upgraded en masse from FW1.2.2 and FW1.2.3 to new FW1.2.5. This is probably due to Canon pushing FW1.2.5 as a “problems killer”, and due to Canon users advocating for upgrading firmware in Internet discussions.

But then, FW1.2.5 should now the most frequent in the Repair Cases. And actually, it’s the less frequent one!

This can only be because FW1.2.5 is producing significantly less corrupt movies than the previous versions.

Conclusion

To all Canon 7D users: Please upgrade to FW 1.2.5 and do it fast or you will have to use our Movie Repair Service!

Tribute to Steve Jobs

When I was 12 years old, in the mid-80s, in a french village dedicated to wine production, my passion was computing!

And in my kid’s mind, all those people who were creating this “magic”, all those distant places in America like Cupertino, were sort of legendary and I was avidly reading about them through the computers magazines of the day, like “L’Ordinateur Individuel” (to my surprise it’s still alive!).

He was not even 30 years old, and he was already THE legend.
Not a mainstream person, like today, but for people who cared about computers, he was already the child prodigy.

Your humble Aero Quartet CEO will never have a chance to meet Steve Jobs, but owes him a lot.

I have started Aero Quartet because I knew that an individual can make significant contributions. And I knew that from him.

When you feel you can’t make a difference because you are at the wrong place doing the wrong thing, then you must make hard decisions and switch gears.
At my low level, my “jobsian” moment was in 2008 when I decided to leave HP and go full-time on Aero Quartet, and I don’t regret it!

Just watch Jobs Stanford commencement speech (YouTube) if you haven’t done it already, and thank him for the pearls of wisdom.

RIP Steve