Survival Bias

During World War II, researchers from the Center for Naval Analyses conducted a study of the damage done to aircraft that had returned from missions, and had recommended that armor be added to the areas that showed the most damage.

Shortly after, a mathematician called Abraham Wald noted that the study only considered the aircraft that had survived their missions—the bombers that had been shot down were not present for the damage assessment. The holes in the returning aircraft, then, represented areas where a bomber could take damage and still return home safely.

Wald proposed that the Navy instead reinforce the areas where the returning aircraft were unscathed, since those were the areas that, if hit, would cause the plane to be lost.

And by following this advice, the Navy managed to minimize aircraft loss in the last years of World War II.

Focusing on “survivors” when analyzing some process leads to false conclusions, and I was about to make this exact mistake when I started writing this post.

A Nine Years Survival

A few weeks ago my business turned 9, and I asked myself what was the biggest mistake I had done during those years.

But when I started making a ranking of mistakes, I realized that this wouldn’t be very useful to publish such the list without first talking about the mistakes I didn’t make.

Just as an aircraft returning to base after a WW2 mission, my business had avoided the seminal mistakes that kill so many short-lived ventures.
Therefore, for a reader willing to take my advice, the pitfalls that I had managed to avoid would be the most important to consider.

I’m leaving this ranking of mistakes for my next post and I’m focusing in this post on things I got right.

1. Make Something People Want

The main reason I got a few things right when I created Treasured is that it wasn’t my first take at creating a product. In the previous years I had published a couple of “hobby products” and learnt a few lessons the hard way.

The first lesson is that without a painful, unsolved problem, that you can solve and charge money for, there is no future.

And that’s really counter-intuitive, because media coverage is always about twenty years old drop-outs that create an app that does a few trivial things, become an overnight planetary success, make no money but end up selling to Google or Facebook for a few hundred millions after a couple of years.

There’s no doubt that this is the story people wants to read, the success that startup founders want to experience. But for founders that’s like playing lottery and picking the winning ticket. For each such success they will be thousands of failures that nobody will talk about.

On the opposite, I prefer to bet on low risk ventures where my chances to win become acceptable. Granted, I won’t become super-rich, low risk brings lower reward. But the fact is that most innovative businesses that stick around for nine years have the same DNA: building a low key, niche, uncool product, but that solves a real, painful problem that people is willing to pay money for.

That’s true in the Internet age, and even now when free mobile apps are all the rage.

Now, how do you discover a painful, unsolved problem? Easy:
Just hang around in places where people with problems go, and ask them!

Forums in specialized websites are a perfect place to start hunting for a business idea.

One day in 2007 in a video forum, I stumbled upon a school teacher that needed to repair a surveillance video to figure out who had stolen the classroom computer. Luckily, the computer was recording live video to a hard disk in a locked case, that the thief had left behind. Classroom footage was available on the disk, but the MOV file been corrupted when the computer was taken by the thief.

That day, I repaired my first video (but unfortunately the thief was wearing a hoody and the teacher could not identify him) and decided that this could be a business idea worth investigating. The whole episode was captured in this blog a few days laters and is still a fresh read!

Once I knew that some people badly wanted to repair corrupt MOV files, the next step was to discover what the market would look like.
Through a market survey? No, by offering directly my expertise (which at this point was all but non-existent)

During a few weeks I requested corrupt MOV videos through my website, claiming that I would repair them for free.

I was contacted by around 20 persons, and discovered a surprising diversity of customers, situations, video formats and technical problems to solve. I managed to decently repair most of the videos, and the feedback was great.

This gave me confidence about feasibility and helped me take the seminal decisions about product and marketing, those that you can’t get wrong:

  • Target Video Professionals, on Mac platform
  • Deliver as a Service

Creating a “universal video repair program” was immediately ruled out: Diversity of situations, of formats, and quality required by professionals require specialized solutions and personalized support. Only a service can provide that.

The moment of truth is when I started to charge for the service.
Everyone is enthusiastic when you give away free repairs, but when there is money in the equation, everything changes.
You suddenly become liable. You quickly understand that unless you build trust and give solid proofs of repair, no customer will send you money.

Therefore, the service was shaped first and foremost to build confidence:

  • Providing a diagnostics and price quote free of charge, using a sample of the damaged video provided by the customer
  • Providing a short sample of repaired video
  • By developing a customized Repair Kit that the customer can use on his Mac, to produce watermarked video out of the damaged MOV file and take the purchase decision
  • Finally, give a refund guarantee in case expectations are not met

The first payment soon arrived. For sure, for the first months the service was still 100% craftsmanship, every customer being a new challenge, but the business was born. A minimal product (Treasured was still one year away) but a viable one.

To be continued…

#1 Lumix GH4

You can almost sense when technologies are about to be disrupted.

Take the internal combustion engine used in automobiles.

For about a century the petrol and diesel motors in our cars have experienced incredible improvements, we can talk about 10x increase in energy efficiency, and don’t get me started with reliability, noise and air pollution.

Even in the last 15 years we have seen dramatic changes, like turbocharged diesel engines in sport cars, common rail direct fuel injection, with electronics and firmware keeping the fuel consumption always optimized.

The 1955 Mercedes-Benz 300SL, the first production sports car to use fuel injection, is remembered first and foremost for its gull wing doors.

Diminishing Returns

Yet it seems that reducing fuel consumption further, even a measly 5% or 10%, will require enormous efforts.

In high-end cars we start to see byzantine solutions, like reducing floor clearance at highway speeds to improve aerodynamics, closing partially the front grille to reduce drag while having enough engine cooling, “smart” management of pumps and air conditioning, and so on…

We are clearly in “diminishing returns” phase of the innovation cycle. Actually the engine system itself is already fully optimized; today fuel efficiency improvements come from the rest of car.

We are getting close to the hard limit. Thermodynamic cycle of combustion engines allow for a fuel efficiency of 25% to 30% at most. End of story.

Electric cars, on the other side, will soon be mainstream starting at 80% efficiency. (Producing electricity efficiently is another story, but let’s stay focused on our topic)

The State of Art for H.264 video in DSLR Cameras

I can almost sense that H.264 video encoding is at the same point in its technological cycle as combustion engine.

Today we are pulling almost 100% of its potential.

Just as combustion engine efficiency is limited by the laws of thermodynamics, H.264 encoding efficiency is limited by a standard approved in 2003. It has received amendments over the last decade, but the efficiency of the codec can’t improve significantly unless you redesign it.

First H.264 encoding chips didn’t have all the bells and whistles that Advanced Video Coding standard allows for.
For example, CABAC entropy encoding is about 10% more efficient than CAVLC, but makes chip design more complex.

This improvement in H.264 is the equivalent of common rail direct injection.

Panasonic Lumix GH4
source: ephotozine – Joshua Waller

Then some manufacturers started to use different H.264 encoding settings depending on light, for example 3 possible settings for low light, interior and outdoor.

Canon introduced this optimization in cameras with DIGIC 5+ chip, starting with EOS 5D Mark III in 2012.

It consists in offering 3 different H.264 encoder configurations (called Picture Parameters Sets or PPS), each with a different value for a parameter called pic_init_qp_minus26.

The H.264 specification doesn’t shed too much of a light on what this parameters stands for:

pic_init_qp_minus26 specifies the initial value minus 26 of SliceQPY for each slice. The initial value is modified at the slice layer when a non-zero value of slice_qp_delta is decoded, and is modified further when a non-zero value of mb_qp_delta is decoded at the macroblock layer. The value of pic_init_qp_minus26 shall be in the range of -(26 + QpBdOffsetY ) to +25, inclusive.

In layman terms, this tells the encoder to encode with more detail the shades of color that dominate the picture, and thus avoid color banding: Image quality is slightly improved while keeping same bitrate.

This is like fine tuning the automatic transmission to optimize fuel efficiency.

Lumix GH4 seeking optimal H.264 video encoding

But there’s still a problem: at the time you start recording, the camera has to pick one of the possible H.264 encoding settings and stick with it until you push STOP. This is not optimal if you walk from interior to outdoor while camera is recording!

This is where Panasonic engineers made a smart contribution:
GH4 cameras also use several PPS, but they dynamically change the active PPS frame by frame to always use the settings that will yield optimal image quality.

Not only that: instead of playing with pic_init_qp_minus26, they use a more fine-grained method to get optimal quality called Picture Scaling Matrix.

Picture Scaling Matrix gives you fine-grained control

pic_init_qp_minus26 is more like a dial

Lumix GH4 engineers have used all the tricks in the H.264 encoding playbook to achieve optimal video quality.
There’s no much room for improvement for future H.264 cameras.

The Lumix GH4 is a serious contender to be remembered as a top DSLR camera of the H.264 era, just as the Leica M6 for rangeviewer film cameras.

Disruption Ahead: H.265

HEVC (H.265), the new standard for video encoding, promises a 50% economy in bitrate at constant quality over H.264.
Let’s see how the transition plays out in the next months. The whole industry has yet to start adopting the new standard.

If this is any indication of what is coming, last month we have received our first corrupt H.265 video from a Samsung Gear 360 camera.

Our tools are not yet ready to detect and repair H.265 routinely, but we already have some prototypes running in-house to address such repair requests and we will be ready for when the H.265 deluge starts.

For the moment, H.265 is only present in specialized markets (surveillance, IP cameras) and in new Samsung products (Gear 360, NX series).

#2 DJI drones

During this 2015 we did a lot of drone footage repairs. A lot. Different drone models, brands, settings, but all of them share some common points. A worried customer and some non working files.
Crashed Drone
Back in 2013 and 2014, when all the drones fever started, we used to receive a lot of repair requests with, more or less, same content. Rude flight footage ending with a hard landing.

Fortunately, seems that our 5 Tips to Avoid Crashing your Copter on the First Day were taking into account and the skills of all of those pilots improved during this time.

But not only man skills were improved, but also the drone capabilities (Worry-free autopilot, Built-in practice flight simulator, etc). Now the copters are able to flight longer, faster, higher, farther and, what’s more interesting, they’re able to broadcast the live footage that is being recorded (Live HD view).

[UPDATE]: Seems DJI removed human pilot skills from the equation in their latest released model, the DJI Phantom 4.
Those factors changed our view about the drone’s damaged files. From a basic repair approach we evolved to a recovery one. Now the customer finds that sometimes their files are simply gone. Only a blank, usually mistakenly, formatted card is left behind. Data should be found and afterwards repaired.

That’s where our magic begins…

What could be done when a whole card, containing the aerial takes of your Thailand trip, is formatted before all clips had been backed up on your computer’s hard drive? Reshoot seems not an option but thanks to Treasured and it’s DeepMediaScan feature we can bring back the footage.

Thailand Aerial View

This particular case was involving a DJI Phantom 3 drone and 1080p at 50 fps video only (no Audio track). Despite DeepMediaScan results were promising, there was something weird within the data that caused any Video player closes when trying to play the clips. Seemed clear we must go the extra mile.

After some data struggling, we finally find the clue. There were some blocks of alien data (garbage data from any other source but not the footage we’re trying to repair) of 0x100000 bytes each every 4 seconds of Video.

That required a custom tailored cleaner to accurately remove them and, finally, repair the real footage data. Those blocks of data, as after some more study revealed, were in fact, parts of the proxy video that is being broadcasted from the drone to the remote control.

Looks similar to the good known Kayak problem with the GoPro cameras, but this time, those blocks of data are formatted in raw H264 instead of LRV (Low Resolution Video). Lesson learnt and repair flow updated.

Nothing compares with the satisfaction of the positive feedback from the customers when they get back their clips repaired. That make us feel that the effort put in every request is worthy.

Please fly your drone in the safest way possible, improve your pilot skills, and use top quality memory cards to record your footage. And obviously, have fun!

When everything else fails, we’ll be here ready to help you.


Thanks to the success of Sony A7 series and Sony AX series, XAVC-S is today one of the most common format that we are requested to repair, and so it deserves its number 3 ranking in my list of Tops and Flops of 2015.

In Sony XAVC family, the “S” flavor is the mainstream one, used by a wide range of devices, from DSLR cameras to cheap consumer camcorders.

In its novel “The Leopard”, Tandredi has the famous quote:

If we want things to stay as they are, things will have to change.

This illustrates perfectly what XAVC-S is in essence: A new name for our usual suspect, AVC/H.264

Back in 2013 when XAVC was unveiled, there wasn’t a lot of choice as far as video compression is concerned:
Once discarded the future technology (HAVC/H.265) that wasn’t ready, Sony could only pick either 20-years old MPEG-2 or 10-years old AVC/H.264.

H.264 scales to 4K, has better compression and quality, so the technical decision was easy.
But the marketing guys were not happy with that, because

you know, everybody else is doing H.264, we are Sony and for our new 4K products we need differentiation. We will use H.264 but it must be with another fancy new name.

OK let’s call it XAVC.

And the marketing guy to add: “and we need different brand names for cinematography-grade, professional and consumer H.264” and so were invented the names XAVC-I, XAVC-L and XAVC-S.

Therefore, XAVC-S is nothing more than regular AVC/H.264 wrapped in a standard MP4 container.

Change Management

Let me tell you my first contact with XAVC-S in spring 2014.
Ironically, this was a corrupt recording of a conference on “Change Management”…

I was expecting this to be business as usual, a boring video (change management, anybody?) recorded in standard H.264 in MP4 container.
Apparently nothing new under the sun.. but I was wrong.

What first grabbed my attention was Treasured insistence in reporting “MXF”. In a video allegedly wrapped in MP4, the last thing you expect to detect is precisely MXF.

Treasured sometimes has “false positives”, so I fired my hex editor to check what was going on.
Upon searching for the MXF header (060e 2b34), I quickly found lots of matches inside the file.

00000c0: 0008 0100 00ad 034b 060e 2b34 0253 0101 .......K..+4.S..
00000d0: 0c02 0101 0101 0000 8300 000c 8000 0002 ................
00000e0: b3e3 8001 0002 51ad 060e 2b34 0253 0101 ......Q...+4.S..
00000f0: 0c02 0101 0201 0000 8300 0036 8100 0010 ...........6....
0000100: 060e 2b34 0401 010b 0510 0101 0101 0000 ..+4............
0000110: 8101 0001 0181 0900 0800 0000 0100 0000 ................
0000120: 6481 0a00 0200 0081 0c00 0200 6481 0d00 d...........d...
0000130: 0101 060e 2b34 0253 0101 0c02 0101 7f01 ....+4.S........
0000140: 0000 8300 002f e000 0010 9669 0800 4678 ...../.....i..Fx
0000150: 031c 2051 0000 f0c0 1181 e300 0001 00e3 .. Q............
0000160: 0200 0100 e303 0001 ffe3 0400 0844 2015 .............D .
0000170: 0625 0321 5100 0000 0000 0000 0000 0000 .%.!Q...........

Obviously Treasured was right, the Sony engineers had inserted MXF stuff inside their XAVC-S MP4 file. WTF?

I asked Mike (my “Change Management” customer) to send me a playable XAVC-S file, so I could investigate this in depth.

What were they smoking?

What I discovered was even more puzzling: MXF data is indeed part of a bigger, more byzantine thing.

The bits of MXF are inside samples of a real time metadata track of the MP4.
To every video frame corresponds one of such metadata sample of 1024 bytes.

Metadata samples look like the diagram below, and are padded with 00 bytes until filling the 1024 bytes size.
Others samples also include a mysterious a “kkad” atom at the end (more on this below).

Header was easy to understand:
0008 0100 00ad 034b
It reports the length in bytes of the components of the sample:

  • 8 bytes for the header itself
  • 0xAD bytes for the MXF KLV (Key Length Value)
  • 0x34B bytes for padding or kkad data.

And of course, the sum is 1024.

MXF KLV itself contains several items, but most are encoded in a proprietary format, and I haven’t managed to understand everything.
In any case, it is not very important and I can prove it:

This metadata is not even necessary.
For two years, I have been repairing XAVC-S without bothering to include the metadata inside the repaired videos, and no customer has reported any problem!

Therefore, Sony engineers have managed a real tour de force:
They have created a video format whose unique “new” feature (besides the shiny XAVC-S name) is to stuff some metadata that is not even being used!

To really appreciate the feat, let’s take a look at this mysterious kkad stuff:

“kkad” blocks include tables with addresses, lengths, and time code information of every video sample.

And with this we close the circle of stupidity:
This information would be useful if it wasn’t already included in the standard MP4 video track.

Yes, Sony engineers have added a metadata track with a proprietary and byzantine format that not only isn’t used, but contains information already present elsewhere in the MP4 container.

And there’s more strange stuff throughout our XAVC-S file…
For example, inside the H.264 stream we have NAL objects of type 6 with more metadata “smoke”.
Or the unnecessary file header below:

0000000: 0000 001c 6674 7970 5841 5643 0100 1fff ....ftypXAVC....
0000010: 5841 5643 6d70 3432 6973 6f32 0000 0094 XAVCmp42iso2....
0000020: 7575 6964 5052 4f46 21d2 4fce bb88 695c uuidPROF!.O...i\
0000030: fac9 c740 0000 0000 0000 0003 0000 0014 ...@............
0000040: 4650 5246 0000 0000 2000 0000 0000 0000 FPRF.... .......
0000050: 0000 002c 4150 5246 0000 0000 0000 0002 ...,APRF........
0000060: 7477 6f73 0000 0000 0000 0000 0000 0600 twos............
0000070: 0000 0600 0000 bb80 0000 0002 0000 0034 ...............4
0000080: 5650 5246 0000 0000 0000 0001 6176 6331 VPRF........avc1
0000090: 0164 002a 0003 0002 0000 c350 0000 c350 .d.*.......P...P
00000a0: 0032 0000 0032 0000 0780 0438 0001 0001 .2...2.....8....

Besides the funny “XAVC”, we have here a description of the tracks:
Audio “twos” at 48000Hz (0xbb80)
Video “avc1” with resolution 1920 x 720 (0x780 0x438)

Again, all this information was already present in the MP4 tracks, so why duplicate it in other places?

XAVC-S: Engineered with love by Sony

If you are in software or firmware development and you have been working in a large organization, the reasons behind all this oddity should now be clear to you.

This is what I call “Frankenstein Engineering”, and it works like this:

  • Marketing decides that a new product with features A, B and C shall be developed
  • Management green lights the project, and gives the R&D departments a short schedule and a low budget
  • Every department has no choice but picking existing, off-the-shelf components and shoe-horn them into the product
  • Extra points if the different departments are spread across continents, have incompatible road maps, and don’t talk to each other anyway
  • When you put hardware, electronics, firmware and software together it’s a disaster, but it can be fixed
  • Eventually the company manages to ship a decent product, but internally the scar tissue is still visible

Therefore, all the odd things I have discovered inside those XAVC-S files was just that:

Scar tissue of the rushed development.

But let’s give credit to Sony: XAVC-S is very successful, and it delivers what Sony promised despite being a Frankenstein monster made with parts that barely fit together and ignore each others.

And the last word about odd XAVC-S again to Tancredi: “A house of which one knew every room wasn’t worth living in.”