Back to School Update

All Aero Quartet crew members are now back to work!

This summer has been long and hot as usual, but we managed to keep the boat afloat (despite a small breach during last week end).

Video Repair Backlog

We have about 30 cases pending. This is roughly the same volume as last year, and it took us around 10 days to go back to normal.

This is now our priority. Our apologies to people who are still waiting.

Treasured 2.7

New Treasured 2.7 has been released in early August:

  • Preview for Canon EOS 5D Mark III corrupt videos
  • Fixed small problems with OS X Mountain Lion
  • Improved support for iPhone 4S, REDCODE, AVCHD, and IMX video formats

Projects

We also want to finish important stuff before Xmas:

  • Our video repair service is being localized in 2 new languages
  • SimpleMovieX update for Mountain Lion

The Slow Agony of QuickTime 7

With Mac OS X 10.7.4 update, QuickTime 7 (and all apps using good old QuickTime API) is showing green screen on paused H264 videos. QuickTime X behaves correctly.

QuickTime 7 problems no longer fixed

This problem is affecting on a daily basis thousands of people still using QuickTime for mission-critical video tasks, but was not considered important enough by Apple to be solved during 10.7.4 development.

One year after Final Cut X getting free of its QuickTime roots, does this mean the Apple no longer cares about QuickTime legacy? Sure, QuickTime 7 stills runs on Mac OS X 10.8, but it’s safe to bet that future OS X releases will not support 32-bit technologies, and this will automatically put the nail on the QuickTime API coffin.

As a vendor relying heavily on QuickTime API, this is a top concern. We have already taken the first steps to cut our dependency on the 21 years old technology that brought multimedia to computers. Our services and apps are being rewritten to be platform-agnostic and we think that by 2013 we will be ready.

Canon 5D Mark III (update)

Repair cases for Canon 5D Mark III have started to flow in. It seems that Canon’s firmware team is up to its reputation…

New Model, New Failure Mode

It seems that when you record a video that ends up filling or almost filling the card, video recording stops but video is lost.
Fortunately, we have devised a method to recover it and we are currently fine-tuning it with a few customers. It involves using DeepMediaScan for extraction and a Repair Kit for recovery.

Mk III now (unofficially) supported by Treasured

We have added Canon 5D Mark III detection to Treasured.
You can download this unofficial Treasured release (temporary link – v2.7 will include it officially.)

  • Preview of “All I-frame modes” without need of good file
  • Other modes require a good file

How to copy a file with I/O errors?

Customers sometimes have problems using our Treasured diagnostics tool or our repair tools. Among usual suspects, we have the dreaded I/O error causing a crash of the video application.

How to Detect an I/O Error

It’s easy to verify whether a file has I/O problems: Just try to copy it to a different location. If the operating system reports an error (Finder error -36 for Mac) then we have an I/O error.

error -36 Finder I/O error

I/O error means that the storage media (card, disk, …) cannot be read at a certain location. Think of it as a scratch on a DVD or a “bad sector” on a card or a disk. The bad news is that this “rough spot” is in the middle of your file.

If the “rough spot” is big (a big scratch on your DVD, hundreds or thousands of “bad sectors” on your disk), then the chances of recovery will be low.

But in any case, the first step is to generate a clean, error-free copy of the damaged file. Since some segments of the file are not readable, the best that can be done is to skip them. We will explain later how to do this.

With a clean copy of the file, parts of the original data are missing, but we no longer have I/O errors causing our diagnostics and repair tools to choke and crash.

How to Copy a File with I/O Errors

To salvage all readable data, we will use a fault-tolerant copy utility called dd.

dd stands for “disk duplication”. This is a command-line utility bundled with Mac OS X. A free version for Windows is also available for download.

Be extremely careful, even if you are familiar with command line, because dd can cause unrecoverable damage to your data if you don’t specify correctly input and output. You have been warned.

using dd to copy file with I/O errors

The arguments of dd are straight-forward:

  • if=path_of_file_with_IO_errors specifies input path
  • of=path_of_clean_copy_to_create specifies output path
  • conv=noerror,sync tells dd to be fault-tolerant

Your output path should not be on the same disk as the damaged file. Actually, you shouldn’t continue using the card or disk with I/O errors, because it’s likely to cause more problems in the future. After recovery the data, you should scrap it or at most use it to store unimportant stuff.

Using the Clean Copy

Due to I/O errors, dd can take more time to create the clean copy than a normal copy would take.

But it’s really worth the wait, because now we have a clean file on which we can use our arsenal of diagnostics, preview and video recovery tools.

I/O errors are a serious business, and scrapping the card after recovering the videos is probably the RIGHT THING TO DO. You should also consider that some amount of footage won’t be recovered, or with a less-than-stellar quality.

But that’s a whole lot more than what you have now, right?

Unmanned Aerial Vehicles for Video capture. A Bad Good Idea?

A new gadget category is born this year at NAB2012: Unmanned Aerial Vehicles used for photo and video capture. Will it stand the test of time?

Picture from coverage by Adam Wilt at PROVIDEO COALITION:

Octocopter carrying a Canon camera

Obvious questions that immediately come to mind:

  • What is the life expectancy of an UAV? Unless it’s controlled by a trained RC guy, my guess is that it will crash within minutes of use.
  • Insurance costs, anybody? Beyond destruction of valuable hardware, there is also a risk in injury for persons. Will even you get coverage?

More problems

  • Vibrations can make photo and video capture unusable. Your UAV needs proper dampening and resonance cancellation features. I fear that cheaper UAVs don’t have such niceties. DYI solutions are not for amateurs, unless you have a structural engineering grade.
  • In addition to controlling your UAV, you will need to remotely control your camera (zoom, aperture, …), so you probably need two people to operate it.

Aero Quartet Advice

Since our business is to repair corrupt video footage, we are seeing an increasing number of cases coming from UAVs users. Here is our advice about how to minimize risks and costs:

  • Use cheap cameras. Image quality will likely be limited by vibrations and by your RC skills, rather than by camera. Crashes will be less painful.
  • Use strong tape to prevent camera doors from opening. Battery, SD card can be ejected under a violent acceleration and jeopardize the take.
  • We can always recover corrupt footage due to a “hard landing”, but the last few seconds cannot be repaired because they have not yet been written to the persistent memory card.

We have also covered “Destructive or High-Risk Video Capture” in a previous blog post.

Conclusion

I have no doubt that UAVs will become very common in the next few years, not just for video capture, but for many other activities. Today, this is still an emerging technology that is difficult to master, that presents some risks, and doesn’t have a legal framework. It’s not ready for prime-time, but as it opens amazing new possibilities, I’m convinced within a few years it will become a standard practice, just like use of camera cranes or steadycam today.

Corrupt videos from new Canon 5D Mk III

We have received our first damaged video files from Canon 5D mark III users.

The good news is that we have managed to repair them with the same level of quality as damaged videos from the rest of Canon EOS family.

Corrupt videos Canon 5D Mark III

At bitstream level, there are a few differences: H264 video is now also using bidirectional frames (B frames), whereas Canon 5D Mk II and Canon 7D videos were only using keyframes and progressive frames (type I and P).

Audio encoding is the same, except for a 96 bytes padding found between audio and video. We haven´t found any convincing reason for the presence of the padding, but maybe Canon will use it in future models or firmware releases for advanced features.

The new API economy

A side effect of every world economic crisis is to invent new forms of economy and wealth creation. When everything looks grim, green buds will inevitably appear at the fringes of the system.

Jobs, as we have known them since the industrial revolution, are now under several threats: Low wage countries, of course, with the lever of global trade and technology.

But global trade and technology is indeed what will create huge opportunities in the next years, even in industrialized countries:
Imagine a small company that want to expand internationally. Today you have two options: create subsidaries, or close deals with local companies to distribute your products or services. None of them is adapted to modern economy that requires high speed and efficiency.

APIs are building blocks for new businesses

Here comes the API economy: What if we expose our business also through APIs? This will allow people all over the world to efficiently provide our services to their customers. For example, a person in Israel could provide our repair service in Hebrew, and a new business is born:
– we sell our service to people that we would never reach (albeit with a smaller margin)
– our affiliate in Israel builds a local customer base, and make extra money
– we keep things simple, in the end an “affiliate user” just needs to accept our terms of service and we provide him with tools

And what makes this kind of wealth creation possible is an API at the intersection of a company’s know how and an affiliate’s market reach.

In API economy takes off, we will see less traditional jobs and more people engaged in one or several API businesses. Implications on society and economy can be huge.

Walking around Blind

For me, one of the most difficult aspects of running a business is that you will never have good information to take decisions. You will walk around blind most of the time.

Of course, I keep track of many metrics, the obvious ones (new customers per day, monthly revenue) and the domain-specific ones (like diagnostic accuracy or repair success rates split by camera models).

But you only can track what users do, and the most important information is about what people doesn’t do and why, from people that will not even become users, or from users that give up because your product sucks. But at the end, there is no easy way for you to know why your product doesn’t get noticed or sucks.

Therefore, the information that you collect tends to be heavily biased towards confirming what you already know, rather than telling you what you ignore or what needs to be fixed.

One example: Localization
Our Repair Service is currently only available in English. In Aero Quartet we have also native French and Spanish people, so we would like to make the service available in those languages, and also in German (we’ll need to hire for that). Now I need to make numbers and figure out how much growth I can expect from localization in FR, ES and DE.

I know that exactly 7.4% of my customers are in french-speaking countries or regions. Easy.
But the really interesting figure is how many would-be customers are turned away because the service is not in French. And that figure is not something I can measure, because people searching for “réparer fichier XDCAM” will never find us or contact us in the first place.

Therefore, to take the right decisions, I will try to look at it from different perspectives, and take calculated risks, just like a blind person compensating with other senses.

Footage Worth Crushing a Camera or two

A trend that we are seeing in Video Repair is the use of video cameras for high-risk or destructive takes, for example:

  • Mount a camera on a truck that will fall off a cliff
  • Integrate a camera into a space rocket and shoot video until battery dies
  • Mount a camera on a RC plane

Nowadays, it’s easy to find an inexpensive camera with good enough image quality and appropriate robustness. Even if the camera is crushed, the footage is safe because it is stored in the camera SD card, a small and robust component with no moving parts.

Just take the SD card out of the crushed camera and read the files!

Of course, the movies whose recording did not end “gracefully” are corrupt and have to go through a repair (and we know who can do this job…) but at the end you can have an impressive and valuable piece of footage that you couldn’t produce by “conventional ways”.

Disposable Video Cameras

Actually, the camera makers are starting to see a market for disposable video cameras, and it comes naturally as an extension of action cameras market, with popular models like GoPro HERO, Vholdr Contour or Drift HD. Established brands like Olympus have also shown interest, going as far as to develop a specific “Tough” series of shock-resistant and waterproof video cameras.

Don’t Try This at Home

If you want to record high-risk or destructive takes, you should plan it carefully:

  • Fasten the camera to a strong support
  • Try to avoid direct exposure of camera to shock or water. Pick position carefully, and use protective padding if possible
  • Use strong tape to prevent camera doors from opening. Battery, SD card can be ejected under a violent acceleration and jeopardize the take
  • And the most important: Never expect to capture the last second of life of a camera.

The Missing Last Second

Why? It hasn’t been written to the card yet when the camera dies.

Short Explanation:

In your camera, the recording process is more or less as follows: raw data from the sensor go to a memory “buffer”. Once the buffer is full (which corresponds maybe to one second of action), the encoding chip make his job and writes the compressed video in a second memory “buffer”. Once the second buffer is full (containing one to a few seconds of action), the data is written on the card. Only at this moment the data is persistent. (because the two memory “buffer” are volatile memory).

When the camera death occurs, the information in the two buffers (corresponding to last seconds of action) are in volatile memory. Not on the card.

Therefore, you should plan your take so that your camera stays alive for 3 to 5 seconds after the moment you want to capture.
If it’s not possible, a live recording system should be used, but that’s another story.

Does FW1.2.5 Fix Canon 7D Reliability Problem?

Canon EOS cameras offer great things to videographers. But since video recording was added to 5D Mark II and then to 7D, there has also been a big number of reliability problems.

Canon EOS is “producing” an order of magnitude more repairs than other consumer cameras that outsell Canon 7D in the market.

Our job is to repair video files, so we are in a position to say something about it. Over the last two years, Canon EOS repairs have sky-rocketed, quickly becoming our main “product”. In most cases, a few clips shot during an event are corrupt, despite having recorded good clips before and after. The media data is present and is consistent (a condition to make the repair possible), but the movie index is missing.

It’s as if the camera had stopped working in the middle of the recording, or could not finish its job of adding the index just after recording ends. This smells like a firmware bug or system limitation. In any case, this is something that Canon can only hope to solve through a firmware upgrade — or — a new camera design.

(There are other failure modes found in Canon EOS cameras, like use of low performance CF cards, accidental deletion, battery issue … but those are the “usual suspects” that we find in all cameras. Here we focus on top Canon-specific failure mode: corrupt file due to firmware bug or system limitation).

FW1.2.5: Canon Acknowledges the Problem

In April 2011, Canon has released a new firmware, FW1.2.5, for Canon 7D.
According to Canon, Firmware 1.2.5 should eradicate corrupt movies:
“Fixed a rare phenomenon in which there were rare cases where movie files taken could not be opened when repeatedly shooting movies with specific CF cards.”

Strange wording. “Rare cases”, “specific CF cards”, look like words forced by PR or Marketing department, and in my opinion, try to masquerade the fact that Canon has only partially solved the problem (or the problems) and that you should not be surprised if failures keep occurring…

So I decided to dig into our repair history and figure out whether FW1.2.5 has changed anything over the last 6 months.

First, I notice that after FW1.2.5 release, our Canon repairs drop by 15%, then they start growing again, but at a slower pace than before.

How Firmware Roll-Overs Work

When a new firmware is released, all new cameras are manufactured with it. But it can take a few months until those cameras get into customers hands. If distribution and sales channel is slow, if inventory doesn’t receive the upgrade, the old firmware will continue growing in installed based for many months until new one first appears, then becomes significant.

Firmware upgrade is also possible on older cameras. But my guess is that only a small percentage of people is upgrading its camera firmware. One exception: if they are confronted to a problem, or receive advice to do so by Canon support, in a newsletter or in an Internet forum, they will upgrade.

FW1.2.3 Roll-Over as a Baseline

FW1.2.3 is transparent: It doesn’t claim to fix any failure mode causing corrupt clips.
That’s why it’s a good baseline to see how firmware roll-overs happen in Canon EOS world.

We look at the picture 6 months after the release to give it time to significantly affect the installed base through new sales and manual upgrades. The chart shows how repair cases are distributed by firmware versions:

  • After 6 months, the latest firmware accounts for 18% of repairs.
  • FW1.2.2, the last one, is now the most represented. This is due to inertia of installed based and sales channel.

Now with FW1.2.5, a Different Story

The big change with FW1.2.5 is that it claims to solve clip corruption issues. So all other things being equal, we will evaluate the efficiency of the firmware fix by comparing the occurrence of repair cases.

If FW1.2.5 were not fixing the bug, then 6 months after release, the distribution of repair cases by firmware releases would look exactly like the FW1.2.3 chart (see above).

It turns out that it’s NOT the same:

The significant differences and their possible explanation are:

  • Volume of repairs decreases by 15% (as noted in first chart). Sounds like there is improvement!
  • FW1.2.5 represents only 12% of total, which is significantly lower than 18% of FW1.2.3 six months earlier.
  • Considering growth of installed base (with important FW1.2.3 inertia and FW1.2.5 in most recent sales), FW1.2.3 should be the most frequent version, but its volume is almost the same as six month earlier!
  • We see an important and unexpecte decrease in FW1.2.2.

I can only understand those results by imagining that customers have upgraded en masse from FW1.2.2 and FW1.2.3 to new FW1.2.5. This is probably due to Canon pushing FW1.2.5 as a “problems killer”, and due to Canon users advocating for upgrading firmware in Internet discussions.

But then, FW1.2.5 should now the most frequent in the Repair Cases. And actually, it’s the less frequent one!

This can only be because FW1.2.5 is producing significantly less corrupt movies than the previous versions.

Conclusion

To all Canon 7D users: Please upgrade to FW 1.2.5 and do it fast or you will have to use our Movie Repair Service!

Tribute to Steve Jobs

When I was 12 years old, in the mid-80s, in a french village dedicated to wine production, my passion was computing!

And in my kid’s mind, all those people who were creating this “magic”, all those distant places in America like Cupertino, were sort of legendary and I was avidly reading about them through the computers magazines of the day, like “L’Ordinateur Individuel” (to my surprise it’s still alive!).

He was not even 30 years old, and he was already THE legend.
Not a mainstream person, like today, but for people who cared about computers, he was already the child prodigy.

Your humble Aero Quartet CEO will never have a chance to meet Steve Jobs, but owes him a lot.

I have started Aero Quartet because I knew that an individual can make significant contributions. And I knew that from him.

When you feel you can’t make a difference because you are at the wrong place doing the wrong thing, then you must make hard decisions and switch gears.
At my low level, my “jobsian” moment was in 2008 when I decided to leave HP and go full-time on Aero Quartet, and I don’t regret it!

Just watch Jobs Stanford commencement speech (YouTube) if you haven’t done it already, and thank him for the pearls of wisdom.

RIP Steve

Where is my footage? (part III)

Last week, I have explained how DeepMediaScan helps you visually map the footage on the disk and make repair operations much easier, faster and accurate.

But a few questions remain unanswered:

  • How do you tell your repair kits what regions of the disk map should be repaired?
  • What happens with disputed cells like A6?
  • Which repair kit use?

DeepMediaScan “Clips”

Your Disk Map is a cell-based representation of the contents of the disk. Now we want to group cells together into “Clips”.

from cells to clips

Looking at the Disk Map images, it’s easy to determine that biking footage occupies cells A1 to A6.
Similarly, birthday footage is in A7-A8 and B6-B7. As A6 and B5 are disputed, let’s include them also here: A5-A8 and B5-B7.
And we do the same for party and concert images:

defining clips in DeepMediaScan

Therefore, you just have to define 5 clips. For each clip, just select the images and drag them into Treasured.

Birthday footage is split over Clip2-A and Clip2-B, because the file was recorded fragmented. We cannot define it in one clip, because Treasured wants the images inside a clip to be consecutive. We will see later how this affects repair.

Repair Kits

Once clips are defined, we want to repair them.
As explained in part II, the clips have been recorded in 2 formats and need 2 different repair kits.

We know that Clip1 and Clip3 are in HD format, because the corresponding Disk Map cells have an HD resolution of 1280×720. The other clips are in SD, their display resolution in 640×480.

This means that we know exactly what kit to apply to what clip.

The diagram below shows that this time, we can recover all the footage.

  • Once repaired with HD kit, Clip1 and Clip3 contain 100% of the original footage for biking and party.
  • Once repaired, Clip2-A and Clip2-B contain 100% of the original footage for birthday. The two files can be merged together with a video editor.
  • 100% or concert footage is recovered from Clip4 with SD kit. Note that it’s done in one shot, the “line-break” in the Disk Map does not divide it into two clips.
from cells to clips

Conclusion

We have tools to recover every bit of footage present in the present, even if there is no visible file, or if files have funny names and end up not containing what is expected.

If DeepMediaScan doesn’t find it, it doesn’t exist.
If DeepMediaScan finds it, we can repair it.

Where is my footage? (part II)

Last week, I have explained why recovered video files are often not playable, and why even once repaired, they do not always contain all the footage that you expected.

This week, we will see how we can improve and try to salvage all the footage anyway.

SD and HD

Let’s go back to our disk “disaster”: We have used a data recovery utility and we have files but they do not contain what we expect:

recovered files not matching original media

On top of that, the files were recorded in two formats:

  • Biking and Party in high definition (HD) and mono-aural audio,
  • Birthday and Concert in standard definition (SD) with stereo audio.

And a different repair kit is needed for SD and for HD, but the real problem is that our recovered files can contain both formats mixed, like “Party”.

What will happen?

You will have to try each file with each kit:

  • SD kit on an HD file will produce an empty file, or garbage video.
  • HD kit on an SD file will do the same.
  • On files with mixed footage, the kits will only repair the corresponding segments of the file. “Party” will give two repaired files, an HD file with some biking and an SD file with part of the birthday.
  • “Birthday” file will give empty files with both kits
  • None of the repaired files with contain an event end-to-end, due to recovered files not matching the media data. No party footage will be recovered at all (“Party” file contains biking and birthday)
repair with two kits

Frustration

Bottom line: You will spend a lot of time repairing every file with every kit and you will end up with a mess:

  • 80% of biking but in two separate files (one of them named Party-0.mov)
  • Only 50% of the birthday recovered (in a file named Party-1.mov)
  • No party footage found
  • Only 95% of concert recovered. A couple of songs missing

Imagine if you have to repair a 2TB hard disk with 1000 files and 5 kits!

DeepMediaScan can do better than data recovery

Enter DeepMediaScan, a free feature inside Treasured.

The problem we have is to map correctly the data regions inside the disk, a task at which the data recovery utility failed.
DeepMediaScan helps you do that by creating a Disk Map.

Just like archeologists to put a grid on the ground area they are searching, DeepMediaScan puts a grid on your disk, and divides it into tens or hundreds or cells:

recovered files not matching original media

DeepMediaScan lets you specify the number of cells. Sometimes a fine-grained map is needed, in particular if disk is big and files are relatively small (needles in a hay stack).

But the idea is simple:
For every cell, DeepMediaScan will generate a preview picture, that tells you the format, resolution, and visually indicates what clip it belong (biking, partying, birthday and concert images are easily distinguishable).

The benefits are clear:

  • Now you know where you footage is stored, with the resolution of a cell
  • Thanks to format and resolution information, you know what kit is needed for a given cell or group of cells
  • On big disks with hundreds of clips, you can focus just on the project that you need to recover, you don’t have to blindly try all files on all kits as above

All in all, with the visibility brought by DeepMediaScan, a big recovery becomes manageable.

What about A6?

Some cells, like A6 or B5, contain media from 2 different clips and formats.
In the Disk Map, however, you only get one preview image, corresponding to the left-most clip/format: A6 will show biking and HD format.

To minimize loss, you can either increase the scan resolution (smaller cells), or consider the A6 cell as disputed between biking and birthday (since A7 coming just after is a birthday image), and schedule A6 to be repaired in two different clips (biking and birthday).

In the third and last part of this story, I will explain how to do it and more…

Where is my footage? (part I)

My customers often get confused when the file they repair…

  • doesn’t contain stuff corresponding to the file name or extension,
  • contains footage from several clips,
  • or contains large swaths without footage, or are completely empty

And in this case, your confusion is legitimate, because after a storage failure, we have in a twilight zone as far as data storage is concerned: Normal assumptions are no longer valid.

In this post, I will try to explain what is going on “behind the scenes” and how the repair results can be optimized.

Disk, Data and Files

Through this serie of posts, I will use the terms “disk”, “data” and “files” very often, so let’s start defining what I mean and how I will represent them in the diagrams:

disk, data and files

disk stands for physical storage, it can be your computer hard disk, an external hard disk, an SD memory card, or and USB pen drive, for example. It is represented by an area whose surface is the capacity.

data is represented by a colored area. For example, green is used for the biking video. Note that the dark green region at the right side represents the index of the video, whereas the light green is video media. Both are necessary for the video to be playable.

file is represented by a boundary and a bubble with the file name.

In a normal situation, your disk contains several files and looks like the diagram below:

  • We have recorded 3 video clips: Biking, Joke and Party
  • File boundaries match exactly the corresponding data
  • Disk is getting filled from left to right, in rows from top to bottom.
filling disk with files

Joke, not so funny

Now imagine that we don’t find the joke very funny after all, and we delete the video file.

Note that only the file has disappeared, the data is still written on the disk, but unreferenced. This is why recovering deleted files is possible in the first place:

unreferenced data after file deletion

Later we record more clips called Birthday and Concert. Let’s observe the new diagram:

  • Birthday file has overwritten Joke data
  • Birthday is one file but occupies two regions: We call that fragmentation.
  • Concert is stored in one region, because line-breaks do not count as boundary.
fragmented files

Bottom line: Unreferenced data will be overwritten when new files are added. So whenever you experience a storage failure or data loss, stop using the disk immediately. Reading doesn’t harm, but writing will end up wiping definitively your lost data. Now we can’t recover Joke because Birthday is written on top of it.

IMPORTANT: When repairing data or video from a disk, always operate from a different disk!
This avoids writing to the damaged disk.

Unplayable videos

To be playable, a video file must contain media data (video and audio), an index, and be coherent.

  • Without media data, the file is not just unplayable, it’s not repairable either.
  • A file with media but without index can be repaired.
  • A file with media and incoherent index can be repaired, but is unplayable.

The most typical corrupt videos are those that you find after a recording failure. The media data is present, but unlike a well-formed video file, the index is not here:

unplayable video

This is because the index is written at the end of recording, and if we experience a failure (battery dead, camera overheating, …) this will never happen.
Note also that those unplayable files can also contain some empty space at the end, where data was about to be written when recording stopped.

Disaster strikes!

Now the dreaded storage failure suddenly happens. We no longer see any file in the disk (or we have files with funny names, or files that are now unplayable).

The first thing to do is to quarantine the disk, to avoid writing on it. Because the data is still here, waiting to be recovered, but it is also vulnerable since it is unreferenced.

storage failure, files have disappeared but data is still here

At this point, many people use a free or commercial data recovery utility to “undelete” the files.
The result is usually disappointing, because video files are difficult to recover due to their diversity, complexity and huge size.
You often end up with a few good files, and plenty of bad ones: (see diagram below)

  • File boundaries do not match exactly the data regions
  • “Biking” contains only half of the footage and is unplayable because index is not present.
  • “Party” now contains footage from Biking and Birthday, and is unplayable for being incoherent.
  • “Concert” is unplayable, because the missing bit makes the whole file incoherent.
  • “Birthday” is empty!
data recovery doesn't do a good job

Note that I’m using quotes to differentiate the recovered files (“Party”) from the original ones (Party). Indeed data recovery often mixes file names and contents, so after recovery, file name should always be taken with a grain of salt.

File sizes, however, are always a good indicator of what you can expect. A 160 kb file is too small to contain footage. A 16 MB file can contain a few seconds (or can be empty). A 160 MB file can contain several minutes of footage, …

We can also find combinations of the examples above, like a recovered file with footage from 3 or more clips, a recovered file with large empty regions and footage, …

Repair will give limited results

Even if you invest time (and money) in repairing those files, you will probably not get a satisfactory result. As we have seen before, the recovered files do not contain all the missing footage, and can also contain footage from several clips:

  • Once repaired, “Biking” will just contain half of the original video.
  • Once repaired, “Party” will contain some biking and part of the birthday video!
  • Once repaired, “Concert” will now have the first songs.
  • “Birthday” will not repair, it contains nothing!
  • Party cannot be repaired. Parts of other clips are still missing.

But there is hope…

To achieve optimal results, we would need to recover files that match better the data regions, and then repair them.
Fortunately, this is possible, using our advanced repair tools, and I will explain it in the second part of this post.

Unveiling our new logotype: “Grulli”

Aero Quartet is now in its teens, as far as we can use the age metaphor for a software company.
Childhood was the 4 years where Aero Quartet was a one-man-shop (I was the man!)

18 months ago I quit my day job at HP and went full-time, then a few months later I hired the first employee, then the second one, and so on.
As you can imagine, teen years are busy, fast-growth years, and our identity is building accordingly.

That’s why we have designed a new logotype:

This is a crane bird during migration. It let you imagine the company values that we want to convey through it.

The logotype will appear in products in website as soon as we find time to refresh them.

Breaking blog ice

This has been a few months since I posted the last entry in this blog.
Every time I let too much time pass, it becomes increasingly difficult to post the next entry.

It’s a bit as if a layer of fresh ice appears and becomes every day thicker. After a couple of months, you start refraining yourself from posting, because what you have to say doesn’t look “important enough” to break the ice.

The longer the silence, the harder it is to break it.

So today I’m taking my little hammer and with this modest post I reopen the channel.