Hello all you Mac users out there! I am a fellow editor in DESPERATE need of some help. I have been working in a Final Cut library for a 4+ hours project, which occupied 1,4 terrab. of information. I was working on a 5k Mac, but the library was on a 6 terra Pegasus external drive (connected via thunderbolt). Yesterday, after applying stabilization to many clips in the timeline, the library grew to the point there was no more space on the Pegasus drive. We have moved some files and freed 2 terra of space (including over 500 gb of time machine backups). This morning, when we opened the mac to continue working on the library, we were in shock to see that the size of the library is no longer 1,4 terra, but 99kilo! When we open this library in Final Cut, it is empty!
What could have happened? Does this disaster relate to the deleting of the older time machine backups? And most importantly, how can we recover the library contents?
P.S. we don't have a backup of this library, due to its huge size.
Please help me with any piece of information!
1st, putting a Time Machine backup on your working drive degrades performance and is simply a bad idea. Very, very bad. Backups should be on a backup drive. Did you delete the Time Machine stuff, or just move it? If you simply moved it, you should be able to restore from it.
I am going to assume you have Background Rendering turned on, and are doing some other things that are unnecessary (undesirable) that are eating up drive space needlessly. You should be sure Background Rendering is always turned off, and occasionally select the Library in the Browser, go to the File menu, Delete Generated files, and delete all render files.
As for everything disappearing, it sounds like you imported everything with "Keep Files In Place" maybe? And then moved all those files away?
If you right-click the Library, Show Package Contents, go to the Original Content folder, can you give us a screen shot?
Basically, if your icons in there have curled arrows in the lower left corner, those are links, your original media was external of the Library.
So, what exactly do you mean "it is empty"? The Library package has absolutely nothing in it? There's nothing in the Original Media folder? Is there anything in any "...Media" folder?
When I choose "File/Open Library/from backup" in FCPX, it shows the automatic backup library, that was generated from the original library. The only problem, is that it only contains the edits, but not the clips, themselves. And we do not have a backup of the original media. We imported the media directly in this huge library, and deleted the movies from the cards, out of lack of space reasons (I know, it's a mistake that we don't have a backup of the source media, but it's a lesson well learned)..
First of all, thank you for your detailed response.
Secondly, thanks for sharing information about the Time Machine work logic, which makes sense now (too bad it didn't make sense previous of this event). We will take into consideration your thoughts about this, and will purchase a separate drive for Time Machine only.
Background rendering is turned on, yes. We will makes some tests to see if timeline scrubbing is as fluent, as without the background rendering option enabled, for long projects (3-4 hours, 1080p project, with color corrections, stabilisation, and transitions).
Regarding the disappearance of the library contents, when importing those clips, we selected the "Copy to library" option, after which we formated the cards, for the following event shootings. Therefore, the only place where the media was located, was inside the library, itself.
Unfortunately, I cannot provide a screen with the "Show Package Contents" folder structure, because the external drive is currently recovering from deleted files, with Stellar Phoenix software (we used this one, because of several recommendations from highly skilled FCP / Apple editors). But we did check the contents, and it looked like an new/empty library. It has all the folders you'd expect in a new, empty library, and there is no media / original media inside of it.
Can you suggest some other ways of solving this issue? Or maybe future tips on handling editing data / original media, in order to be safe from this kind of situations, for the following projects?
Thank you in advance!
Thank you for your responses. After running the recovery software yesterday, we weren't able to recover anything, unfortunately. We've talked to the official Apple store/service in another city, and we'll be taking the external drive & machine that was used in this entire process, for a certified recovery process. Hopefully, this solves our problem, but I really don't know what to say..
"We will makes some tests to see if timeline scrubbing is as fluent, as without the background rendering option enabled"
- Background rendering has NO effect on scrubbing at all. Only on playback. And 90 percent of the time, that is just fine, even if render lines appear in the Timeline pane.
- If you have a project that long, you make Projects for each section (chapter, scene, how ever it is organized). Edit in these smaller chunks. When done, make a Master Project and copy/paste everything into that. Pretty standard film editing.
- Recovery software and recovery services rarely recover much of anything useful. I had 20+ years as an IT engineer before I transitioned to media production full time. In all my time, i've rarely seen a drive recovery retrieve much of anything. Because hard drives have hidden “working” files written and read constantly by the OS and applications. A lot of this will overwrite drive blocks labeled as available, making data recovery of what used to be on that block impossible.
Most importantly, is how your media all went missing. It was either deleted manually, or as stated you didn't watch your Background Process window to verity the cards were finished copying. FCPX is not known to delete media on its own by accident. There’s really no mechanism for it to happen by. So it is either one of those two scenarios.
NOTE: Just because you can see the card media in the Browser and can edit with it does NOT mean the copy is completed! FCPX allows you to start editing before the full copy is completed. But even then, if you ejected a card or quit FCPX during a card copy, you should have gotten a warning of some sort.
- Did someone generate proxy media, then delete the original media, via the File menu’s Delete Generated Media function? I’ve had students to that mistakenly.
1- Always monitor Background Process window when copying media from cards.
2- ALWAYS have a backup drive to run backups on every night, PERIOD, or be prepared to lose it all (not "if", "when").
4- You won't get much of your footage back, simple fact of life. Whatever you do get recovered, feel blessed. Data recovery software and other methods outside of a clean room forensic recovery is mostly a scam, in my experience.
Crossing my fingers and hoping for the best with data recovery at the Apple Store, but advise you don't hold your breath for anything. Sucks when something like this happens. I have been in your shoes before, and know what a sick feeling it can give you. Here's hoping for the best!
Dear FCPX.guru, thank you for your response, and sorry about my delayed reply to your message. We haven't been able to send it to the Apple Store, but we have contacted some IT specialists, and they have suggested even recovery software for windows OS, which we have tried, and still waiting for it to run (it was a very in-depth scan, as we were told, and it takes a long time). Unfortunately, the situation does not look very bright, as the software sees some files (.mov files), which says can restore, but it does not look like they will be usable, because the video players don't recognise them..
Unfortunately, we do not afford a 10K dollars forensic recovery option (and I don't really think we could find such a service for public use in our country), so we still hope that we will recover at least partial footage, as it looks like the only feasible option. Also, we gave the original SD cards to be recovered from, to another IT specialist (all of the cards were formatted after downloading them to the computer, and 2 of them were used in another project to film on). As far as I understood, the cards that were formatted and left untouched afterwards will be recovered 100% (that is some bold statement from the IT specialist), but the cards that were formatted and used again to be filmed on in another project afterwards, don't have much chance of recovery. We are waiting for the results. Thank you for your kind words and suggestions! We will surely be taking them into consideration!
Please let me know how things turn out. This is a very depressing situation, but there is a lot of positive lessons to be learned. It has been a very long time, but I've been in your shoes before. This is why we have good old American sour mash whiskey for... medicinal purposes of course.
Get Disk Drill Media Recovery, an app that can salvage camera cards really well, depending on if they've been re-used or not. I got it while on a reality TV show production. As DIT manager we were offloading dozens of cards a day constantly. Dragging contents to the trash and emptying the trash right away. Fortunately we had a system to double check offloads before the cards went back out. A few times we were missing clips, and this app recovered vital data from the cards. Now, you may be asking yourself, "Why did you double-check cards before wiping them clean?" Well, when the producer hires some kids for less than minimum wage, and you're personally more on-set than in the DIT room because the producer/director is rarely around and you have to take over, well, stuff will go wrong...
Still crossing my fingers! Best of luck, and chin up!
We've tried to recover using KrollOntrack Easy recovery software, and it did find some original video files, but, unfortunately, they are corrupted, and now somebody is trying to repair them using some specialised software. Also, other bad news is related to the found files - the largest, single file, is only 7 gb in size, which is way too small, considering we've shot multiple 30-50 minutes takes, that took a lot more space, and those held the most "video content". So I guess, most of the original files are already lost..