Once upon a time, there was an action camera called GoPro. It quickly became the top seller of its category, with 9 millions cameras shipped since 2009, and made its CEO a mad billionaire.
And since action cameras have a rough life, with no surprise did we start receiving several GoPro repair requests every day.

GoPro corrupt footage was routinely repaired, using deep parsing and re-indexing techniques.

One Day…

One day, we received a damaged GoPro file that couldn’t be cleanly repaired. Re-indexing seemed to work fine, and repaired videos were playable, but they were full of glitches, and audio was repeated every couple of seconds, like a mad echo effect.

At first we didn’t pay a lot of attention to this, we just said to the customer that the video could not be cleanly repaired due to presence of “alien data” inside the file. Alien data is material that should not be in the damaged file, and that makes re-indexing more difficult or even impossible. This is a bit like waste recycling that can only be efficient if waste has been properly triaged beforehand.

We were not wrong about “alien data” contamination, but not for the expected reasons… A few days later we received one, then two more damaged videos from different customers that were having the exact same symptoms:

• Damaged files preview perfectly with Treasured.
• However, after repair, it was a mix of clean and garbled video, with audio repeating (like echo) every two or three seconds.

At this point I considered having discovered a new failure mode, and I guessed that it was probably a side effect of how new GoPro cameras operate.

Cracking New GoPro

The most intriguing fact about this failure mode is that audio repeats, so we started our investigation with audio.

After close examination, I noticed that not only audio was repeating, but video as well! The only difference is that secondary video wasn’t directly visible, indeed the glitches inside repaired video are our secondary video stream. So we have a camera that records two streams at once, both with same audio but incompatible video.

Does it sound familiar?

It didn’t for me at first, but when I read the list of changes in new GoPro firmware and cameras, everything became clear…

LRV for Low Res Version

It turns out that GoPro camera now records an LRV clip at the same time as the HD clip. The LRV file is a lower resolution version of the video recording used for preview purposes on the phone and tablet GoPro applications (less data to transfer wirelessly to the device running the application, and less decoding raw power required as well).

During recording, your GoPro camera produces two streams of videos, one high resolution (represented by big orange images on illustration, and one low resolution, whose volume is one fourth to one tenth, hence the small blue images.

The Locker Room

Now let’s figure out what happens when the camera writes the two streams on the SD card. In computers, but also in phones and cameras, a program called “file system” is in charge of storing data and keep track of where it is stored to be able to read it later.

You can think of the SD card as a locker room, with thousands of lockers, all of the same size. People who want to store their stuff give it the person in charge, the file system, who will put it in one or several lockers and keep track of what goes where.

During recording, every second the GoPro guy comes to the locker room with new boxes that he hands over to the file system: 4 orange boxes, that fill 4 lockers, and 4 small blue boxes, that fit in one locker. To go faster, the file system fills the lockers in rows, so orange and blue boxes are interleaved.

Disaster Strikes

Unfortunately, before our file system could write in the registry where boxes have been stored, fire alarms suddenly start shrieking, and everyone leaves the building. A few minutes later, people is authorized to go back to work, but our file system no longer remember where video has been stored. Our GoPro is desperate… and calls SoS!

Cleaning up this Mess

Finding footage is easy, just open all the lockers, put all items together, and pray. This is what Data Recovery utilities do. Unfortunately, this won’t work because besides fragmentation, we have LRV data interleaved with HD video, that will cause glitches and echoing audio once fixed.

As you have guessed, we now have two problems:


• Separating LRV from HD video
• Re-indexing video

Triage

There’s no such thing as orange and blue data. Both streams contain H.264 video and AAC audio and are nearly indistinguishable. Yet we need to surgically separate them. Cutting just one byte off would generate glitches in several frames.

Fortunately, two things makes triage possible:

• Streams are mixed in locker-sized chunks. Cuts can only happen every 131072 bytes exactly, at discrete locations.
• Every H.264 frame starts with a pattern that informs of its exact bytes length, so we can verify whether it’s in one piece – or – whether 131072 bytes of “blue data” are contaminating the frame.

Combining those two hints, I have managed to remove LRV from the raw data. Now we are back to the normal situation of a corrupt video in need of re-indexing.

Amazing: Kayak Footage Restored!

The final step is to re-index the cleaned – yet still corrupt – raw file. Needless to say, when you have cracked a new failure mode like this one, this is a lot of excitement the first time your repair program finally cranks out clean video.

The first video that I have managed to fix shows some guys paddling in kayaks, so I have decided to call this feat: to pull a kayak

Frame from repaired clip, courtesy of Gordon Scott

If ever you face a corrupt video problem with your GoPro, even if it’s not a “kayak”, just send us a request through Treasured and we will do the paddling for you.

I have examined the damaged footage: video is actually encoded in Motion JPEG, 4096 x 2160 and audio is in Linear PCM 16 bits.
At almost 3 MB per frame, the camera has to write its 4K/24p footage at an astonishing speed of 500 Mbits/s. Needless to say, the fastest CompactFlash cards in the market can barely sustain this write speed, and actually the camera uses a Dual CF set-up to cope with that.

Freeze, Zoom, Focus … Read the Newspaper!

4K videos recorded by Canon EOS-1D C produces images so detailed and sharp that you can crop 1% of a frame and still read the newspaper! A Blade Runner-ish performance!

Highlighted image represents just 1% of the original video frame!

Repairing Corrupt 4K Videos

Such demanding bandwidth and storage specs will inevitably cause problems. Corrupt videos will likely be produced by “usual suspects”:

• lack of space in card
• lack of speed of card
• camera failure in hot conditions

At this point there is no evidence that firmware 1.0.2 is responsible of the failure. We will track the failures by firmware versions as we did during Canon 7D epidemic outbreak.

The damaged footage that I have examined comes in a QuickTime MOV file, but without the moov atom at the end since recording did not end gracefully. This missing moov atom should contain the media tables and index that are needed for the video to be playable.

Fortunately, Motion JPEG media is easy to parse, thanks to the markers used to encode the image, in particular FFD8 (start of image) and FFD9 (end of image). With a standard re-indexing technique, we have managed to make the video playable, restoring original quality.

Therefore, as of February 2013…

• Treasured cannot detect and preview corrupt Canon EOS-1D C videos yet.
  we will try to update Treasured in March with Canon 4K detection and preview.
• But we can repair Canon 4K videos if you send us a repair request with Treasured.

After having burnt days trying several tools and spent money on the most promising one, their recovered video files are still corrupt and they finally come to us because only our diagnostics tools Treasured (or MP4repair.org) can preview them and give them some hope.

Why does it work for my PDFs but not for my videos?

Data recovery tools are usually good at undeleting small files, like PDFs, Word and Excel documents, or low-res images. But as file size increases, the chances of recovery plummet.
Even for JPEG pictures in the 2 – 5 MB range, about 15% of files appear missing or corrupted.

For video files, whose size is usually in the hundreds of MB, and even in the tens of gigabytes for post-production formats, the chances of recovery are almost zero.

The reason is FRAGMENTATION: In your disk, everything is stored in fixed-sized “blocks”, like the shelves in the picture below. But if the file is big, it will not fit in one shelf.

Video files occupy many shelves. If the file is really big, let say 20 shelves, it’s not even possible to store it in contiguous shelves. In picture below, it’s obvious that really big files will end up spread over different walls.

Why Fragmentation Kills Recovery

Fragmentated files are split into several pieces, each piece being stored at a different place. Data is no longer contiguous.

This is not a problem for the operating system, that knows where those pieces are stored. But after a file deletion or a disk failure, this “map” also disappears and the file is now lost in the middle of thousands of shelves.

All data recovery tools (except high-end forensics tools, more about that below) use simple file carving techniques to recover deleted data.

Most common technique is Header/Footer carving. It consists in identifying the beginning of a file of a certain type, using a signature or pattern. AVI files, for example, start with RIFF word. MP4 files have a movie atom at the end of the file (footer) that can easily be detected using pattern matching.

Once a header and a footer are found, the data recovery program assumes that all data in between is part of the video file. This will only be true is the file was stored in one piece, without fragmentation.

This rarely happens. Even in disks with plenty of space available, fragmentation often happens as a side effect of video recording. For example, XDCAM cameras write as the same time several files (MP4, BIN, PPN) and this causes a systematic fragmentation of the MP4 video file.

Alien Data

Therefore, any fragmented video file recovered will contain “alien data”.
This will cause the following problems, from less serious to more serious:

• Glitches in video and audio.
• Possible crashes during playback
• Missing footage, footage from different clip inserted in video
• File no longer opens due to inconsistency of video container
• File not even recovered if software doesn’t find footer

Recovery is not Repair

Data recovery tools are very good at restoring small files of many different formats, by identifying superficial features of the data (header signatures, footer patterns) and assuming that the file is not fragmented.

But this assumption is not true for video files. The only effective technique that can be used to recover fragmented files is called “deep carving”.

Tools implementing this technique are very few, very specialized, more expensive, slower, and are based on probabilistic algorithms specific to the video and audio formats to recover.

• Recovery tools achieve good results on small files, but not on videos, due to fragmentation.
• Recovery tools can’t actually repair fragments of video files, because they can only identify generic and superficial file features
• Video Repair tools are specialized. They are built to repair one type of videos only,for example Canon EOS 1080p24 or XDCAM 720p30.
• Video Repair tools identify video frames and audio blocks deep inside the file and re-index them into a playable video.

This is the reason why our repair service works when other generic recovery tools don’t: We build specialized tools that can assemble the puzzle, fit the parts of the disk together to restore a functional video file.

Users have started reporting problems with the camera in November 2012: One of the most annoying failure mode is the dreaded “lock up during video recording” that leaves you with a corrupt video. We are now receiving new damaged clips to fix every day.

Affected Firmware Versions

Lock-up problems can be traced back to firmware versions that include Protune features. All GoPro models with Protune are affected.

As of January 2013:

• All GoPro HERO 3 cameras except White model (because it doesn’t include Protune)
• GoPro HERO 2 cameras with Firmware upgrade 8.12.198 (October 2012) that enables Protune

Two important points:

• Lock ups happen even if Protune mode is disabled.
• GoPro HERO 2 firmware can be downgraded to version 8.12.124, without Protune.

Tip: Save $20 on GoPro Repairs with Treasured and MP4repair.org

We have been busy in the last months catching up with new GoPro cameras. Current releases of Treasured (Mac app) and MP4repair.org (web app) can detect and preview corrupt GoPro videos, even from new models and formats.

However, GoPro special pricing is not always correctly applied. When video is identified as Media: H.264 instead of Brand: GoPro HERO, price quote will be higher. We will fix that by end of February.

In the meantime, just add a comment in your repair request or send us an email to notify that it’s GoPro footage and we will apply the correct price to your repair.

Whether you’re a professional using Apple’s Logic Studio or AVID Pro Tools, an extreme biker torturing the last GoPro camera or a video aficionado editing the last family gathering with Windows Movie Maker, you can end up fighting with a corrupt WAV, M4A, AIF or QuickTime MOV audio file.

Your fight is really unfair: Even if you are a skilled computer guy, you probably will spend hours on that, to no avail. Professional video and audio repair services exist for a reason. We have spent the last 5 years developing recovery tools for Mac and a web video repair app to give you predictable results.

You’ve been warned.

Do It Yourself: Linear PCM Audio Repair

Note that the techniques below only work for Linear PCM format and for containers mentioned below. We won’t be able to repair an .m4a file containing AAC with those methods, for example.

• How to repair a corrupt MOV file containing Linear PCM audio using “Transplant” technique
  Note that if you have a good audio file with exactly same settings than damaged one, and whose size is bigger, you can use it as placeholder and skip step 1.
• How to repair a corrupt AIFF file using “Container Correction” technique
• How to repair a corrupt WAV file using “Container Correction” technique

Unfortunately, repairing compressed audio formats like AAC or ALAC (Apple Lossless) is much more complex and we’ll leave it for future posts.

Note: With Treasured, our diagnostics app , you can preview those corrupt files and then repair them with through our Repair Service. This post is not about repairing videos, but about understanding why we see this pattern.

Let’s do a post-mortem of an unplayable video and figure out why.

Playing Detective

With the help of an hexadecimal editor, we look at the beginning of the file. Trained eyes will see the normal header of a QuickTime .MOV movie. It contains a moov atom, which usually is at the end of the file. Strange…

0000000: 0000 0018 6674 7970 7174 2020 2007 0900  ....ftypqt   ...
0000010: 7174 2020 4341 4550 0001 7fe8 6d6f 6f76  qt  CAEP....moov
0000020: 0001 004a 7564 7461 0000 0026 434e 4356  ...Judta...&CNCV
0000030: 4361 6e6f 6e41 5643 3030 3130 2f30 332e  CanonAVC0010/03.
0000040: 3030 2e30 302f 3030 2e30 302e 3030 0000  00.00/00.00.00..
0000050: 000c 434e 444d ffd8 ffd9 0001 0010 434e  ..CNDM........CN
0000060: 5448 0000 36e3 434e 4441 ffd8 ffe1 0e26  TH..6.CNDA.....&
0000070: 4578 6966 0000 4949 2a00 0800 0000 0900  Exif..II*.......

If we have a moov atom, let’s check if it is consistent and what is inside it. For this we use Dumpster or Atom Inspector, two free apps that you can download from developer.apple.com (free developer account required).

Surprise, our moov atom is a valid! Complete with 3 tracks (video, audio, timecode) and metadata.
This explains why the corrupt video file opens: the valid moov atom at the beginning of the file contains all the information required to configure the movie and the tracks, so QuickTime Player initializes and can start decoding media.

One Second of Audio

Now let’s look at our audio track. It corresponds to the second ‘trak’ atom. Tables ’stco’ and ’stsc’ tell us where audio media is fetched. First entry of tables gives us: at address 98312, 48048 samples.

Now let’s look at what we have at this address. Addr 98312 (hexadecimal 0×18008) is just after ‘mdat’ atom, and Linear PCM audio data starts here as expected.. Everything happens like in a valid movie file.

0018000: 0bd1 afd4 6d64 6174 0b0c 0b0c bc0b bc0b  ....mdat........
0018010: 780b 780b 9b0b 9b0b 390b 390b 190a 190a  x.x.....9.9.....
0018020: 0109 0109 0d08 0d08 3507 3507 e106 e106  ........5.5.....
0018030: 3507 3507 4d07 4d07 a506 a506 b006 b006  5.5.M.M.........
0018040: 8007 8007 9607 9607 f706 f706 9306 9306  ................
0018050: 4b06 4b06 8b05 8b05 cf04 cf04 3a04 3a04  K.K.........:.:.

Black Video

Now let’s looks at first video entry: Address is correct, but length is not. This explain the black image: H.264 cannot be correctly decoded. If you dig deeper, you see that audio and video tables don’t correspond to actual data inside the file. Maybe those tables refer to the last recorded video.

If you look at more corrupt videos, the same facts hold true. Everything can finally be explained:

• A fixed-length ‘moov’ atom is written at the beginning of the file. This is why it can be open.
• Audio and Video tables are wrong. This is why video is corrupt.
• But since first Audio block is always at the same address, and has same length, it’s correctly reproduced.

What it tells about Canon 5D Mark III firmware

Canon Mk III firmware writes a placeholder ‘moov’ atom at the beginning at the file, then writes media data (video and audio) while camera is recording. This placeholder is not empty, it could actually be the same ‘moov’ as in the last recorded video.

When recording ends, the camera overwrites the placeholder with real ‘moov’ atom containing good media tables.

If this last operation doesn’t complete, the video is corrupt.

Cameras usually write the ‘moov’ atom at the end of the file: Data is written in the same order as it is produced. But Canon 5D Mk3 firmware does it different as we have seen. This is possible because camera limitations dictate a maximum ‘moov’ atom size, so the corresponding space can be “booked” at the beginning of the file (placeholder) and then overwritten.

I can only see one advantage to this: Since ‘moov’ atom is at the beginning of the file, the file can be streamed. We talk of fast-start, streamable videos. This is important for Internet applications, but honestly I don’t see how this matters for Canon 5D workflows.

How the “magic” works is explained here.

The Catch

But if you read Apple’s technical note, you will see that this doesn’t work great for high-speed video capture.

My understanding is that high-speed, high-volume writing operations have a serious bottleneck, because when the 4 GB SSD write cache is full, it has to be moved to the hard disk, and this causes some latency that probably drops frames during video capture.

Video Repair?

How does it affect video repair?

Video capture limitations will probably keep video professionals away from those Fusion Drives, at least for video capture. High-throughput Thunderbolt drives are a better fit.

But Fusion Drives will become mainstream among casual users. I fear that this will make our video repair job harder, in particular for DeepMediaScan jobs. (DeepMediaScan is used when footage is not found inside files, but scanned directly on disk)

With traditional disks, footage is written sequentially as the disk fills up, so it’s possible to for DeepMediaScan to extract it consistently.
In a Fusion Drive, it’s more complex. Footage is written to an SSD cache and then, maybe it’s transferred to the hard disk, or maybe not. In the end, there is more fragmentation, and footage recovery is more challenging.

As soon as I get get my hands on a Mac with a Fusion Drive, I will carry out repair tests and provide advice about Fusion Drive usage.

First surprise: 40% of Treasured downloads are from Windows users, who will obviously never be able to run our Mac video recovery app.

Despite clearly stating on the website that this is a Mac app, a lot of Windows people is reaching the big Download button. This is not cluelessness, just the standard practice in a “we don’t read, we scan” world.

Second surprise: Among Mac people with a fresh copy of Treasured just downloaded, only 60% perform a diagnostics on a damaged video. In other words, 40% never get any value from the software.

I could give more examples. Every time I have measured a new metric related to our repair service use, I have discovered that it was grossly inefficient.

The most amazing thing: Those problems are usually very easy to fix. You can just add a warning for Windows guys trying to download Treasured and redirect them to MP4repair.org, for example.

This is a real eye-opener. Don’t start working on hard problems until you get the basic stuff right.

The Secret: Measuring

This summer has been long and hot as usual, but we managed to keep the boat afloat (despite a small breach during last week end).

Video Repair Backlog

We have about 30 cases pending. This is roughly the same volume as last year, and it took us around 10 days to go back to normal.

This is now our priority. Our apologies to people who are still waiting.

Treasured 2.7

New Treasured 2.7 has been released in early August:

• Preview for Canon EOS 5D Mark III corrupt videos
• Fixed small problems with OS X Mountain Lion
• Improved support for iPhone 4S, REDCODE, AVCHD, and IMX video formats

Projects

We also want to finish important stuff before Xmas:

• Our video repair service is being localized in 2 new languages
• SimpleMovieX update for Mountain Lion

This problem is affecting on a daily basis thousands of people still using QuickTime for mission-critical video tasks, but was not considered important enough by Apple to be solved during 10.7.4 development.

One year after Final Cut X getting free of its QuickTime roots, does this mean the Apple no longer cares about QuickTime legacy? Sure, QuickTime 7 stills runs on Mac OS X 10.8, but it’s safe to bet that future OS X releases will not support 32-bit technologies, and this will automatically put the nail on the QuickTime API coffin.

As a vendor relying heavily on QuickTime API, this is a top concern. We have already taken the first steps to cut our dependency on the 21 years old technology that brought multimedia to computers. Our services and apps are being rewritten to be platform-agnostic and we think that by 2013 we will be ready.