Welcome, Guest
Username: Password: Remember me

TOPIC: Lost all files inside FCPX library!

Lost all files inside FCPX library! 26 Jul 2017 12:09 #89569

  • andreitest
  • andreitest's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
Hello all you Mac users out there! I am a fellow editor in DESPERATE need of some help. I have been working in a Final Cut library for a 4+ hours project, which occupied 1,4 terrab. of information. I was working on a 5k Mac, but the library was on a 6 terra Pegasus external drive (connected via thunderbolt). Yesterday, after applying stabilization to many clips in the timeline, the library grew to the point there was no more space on the Pegasus drive. We have moved some files and freed 2 terra of space (including over 500 gb of time machine backups). This morning, when we opened the mac to continue working on the library, we were in shock to see that the size of the library is no longer 1,4 terra, but 99kilo! When we open this library in Final Cut, it is empty!
What could have happened? Does this disaster relate to the deleting of the older time machine backups? And most importantly, how can we recover the library contents?
P.S. we don't have a backup of this library, due to its huge size.
Please help me with any piece of information!
Thank you...
The administrator has disabled public write access.

Lost all files inside FCPX library! 26 Jul 2017 23:25 #89577

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2820
  • Thank you received: 372
  • Karma: 34
1st, putting a Time Machine backup on your working drive degrades performance and is simply a bad idea. Very, very bad. Backups should be on a backup drive. Did you delete the Time Machine stuff, or just move it? If you simply moved it, you should be able to restore from it.

I am going to assume you have Background Rendering turned on, and are doing some other things that are unnecessary (undesirable) that are eating up drive space needlessly. You should be sure Background Rendering is always turned off, and occasionally select the Library in the Browser, go to the File menu, Delete Generated files, and delete all render files.

As for everything disappearing, it sounds like you imported everything with "Keep Files In Place" maybe? And then moved all those files away?

If you right-click the Library, Show Package Contents, go to the Original Content folder, can you give us a screen shot?

Basically, if your icons in there have curled arrows in the lower left corner, those are links, your original media was external of the Library.

So, what exactly do you mean "it is empty"? The Library package has absolutely nothing in it? There's nothing in the Original Media folder? Is there anything in any "...Media" folder?
The administrator has disabled public write access.

Lost all files inside FCPX library! 27 Jul 2017 04:56 #89582

to add to Ben's list:

What happens when you choose in FCPX
File/Open Library/from backup?

(those are the automatically generated 'snapshots', not the ones TImeMachine does…)


And have you tried to reset FCPX' Prefs?
The administrator has disabled public write access.

Lost all files inside FCPX library! 27 Jul 2017 06:42 #89584

  • andreitest
  • andreitest's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
When I choose "File/Open Library/from backup" in FCPX, it shows the automatic backup library, that was generated from the original library. The only problem, is that it only contains the edits, but not the clips, themselves. And we do not have a backup of the original media. We imported the media directly in this huge library, and deleted the movies from the cards, out of lack of space reasons (I know, it's a mistake that we don't have a backup of the source media, but it's a lesson well learned)..
The administrator has disabled public write access.

Lost all files inside FCPX library! 27 Jul 2017 08:15 #89586

  • andreitest
  • andreitest's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
Dear FCPX.guru,
First of all, thank you for your detailed response.
Secondly, thanks for sharing information about the Time Machine work logic, which makes sense now (too bad it didn't make sense previous of this event). We will take into consideration your thoughts about this, and will purchase a separate drive for Time Machine only.
Background rendering is turned on, yes. We will makes some tests to see if timeline scrubbing is as fluent, as without the background rendering option enabled, for long projects (3-4 hours, 1080p project, with color corrections, stabilisation, and transitions).
Regarding the disappearance of the library contents, when importing those clips, we selected the "Copy to library" option, after which we formated the cards, for the following event shootings. Therefore, the only place where the media was located, was inside the library, itself.
Unfortunately, I cannot provide a screen with the "Show Package Contents" folder structure, because the external drive is currently recovering from deleted files, with Stellar Phoenix software (we used this one, because of several recommendations from highly skilled FCP / Apple editors). But we did check the contents, and it looked like an new/empty library. It has all the folders you'd expect in a new, empty library, and there is no media / original media inside of it.
Can you suggest some other ways of solving this issue? Or maybe future tips on handling editing data / original media, in order to be safe from this kind of situations, for the following projects?
Thank you in advance!
The administrator has disabled public write access.

Lost all files inside FCPX library! 27 Jul 2017 12:43 #89596

  • Sallynx
  • Sallynx's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 14
  • Thank you received: 1
  • Karma: 0
As Karsten already suggested maybe you can try resetting the FCPX prefs? Of course it's not guaranteed that it'll work but it usually does for me.
The administrator has disabled public write access.

Lost all files inside FCPX library! 28 Jul 2017 05:12 #89599

… to sum up, what you've told us so far:
  • a +4h project
  • using a single Raid for storage of Lib and TimeMachine
  • no backups/archives of original media, nor the Lib on additional drive(s)
  • and actually using some (I never heard of) 'recovery' software to rebuilt that raid …

… if this doesn't end in some 'wonder' of that recovery-software and therefore is just promotion, I would say: make a schedule for reshoot …
The administrator has disabled public write access.

Lost all files inside FCPX library! 28 Jul 2017 06:29 #89603

  • andreitest
  • andreitest's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
Thank you for your responses. After running the recovery software yesterday, we weren't able to recover anything, unfortunately. We've talked to the official Apple store/service in another city, and we'll be taking the external drive & machine that was used in this entire process, for a certified recovery process. Hopefully, this solves our problem, but I really don't know what to say..
The administrator has disabled public write access.

Lost all files inside FCPX library! 28 Jul 2017 10:52 #89604

  • FvH
  • FvH's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 61
  • Thank you received: 10
  • Karma: 0
Just out of curiosity: How long did it take to copy the 1,4 TB to your library? Are you definitely sure that FCPX had finished the import before you removed the card?
The administrator has disabled public write access.

Lost all files inside FCPX library! 28 Jul 2017 11:53 #89610

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2820
  • Thank you received: 372
  • Karma: 34
"We will makes some tests to see if timeline scrubbing is as fluent, as without the background rendering option enabled"

- Background rendering has NO effect on scrubbing at all. Only on playback. And 90 percent of the time, that is just fine, even if render lines appear in the Timeline pane.

- If you have a project that long, you make Projects for each section (chapter, scene, how ever it is organized). Edit in these smaller chunks. When done, make a Master Project and copy/paste everything into that. Pretty standard film editing.

- Recovery software and recovery services rarely recover much of anything useful. I had 20+ years as an IT engineer before I transitioned to media production full time. In all my time, i've rarely seen a drive recovery retrieve much of anything. Because hard drives have hidden “working” files written and read constantly by the OS and applications. A lot of this will overwrite drive blocks labeled as available, making data recovery of what used to be on that block impossible.

Most importantly, is how your media all went missing. It was either deleted manually, or as stated you didn't watch your Background Process window to verity the cards were finished copying. FCPX is not known to delete media on its own by accident. There’s really no mechanism for it to happen by. So it is either one of those two scenarios.

NOTE: Just because you can see the card media in the Browser and can edit with it does NOT mean the copy is completed! FCPX allows you to start editing before the full copy is completed. But even then, if you ejected a card or quit FCPX during a card copy, you should have gotten a warning of some sort.
help.apple.com/finalcutpro/mac/10.3/#/ver64e71609

- Did someone generate proxy media, then delete the original media, via the File menu’s Delete Generated Media function? I’ve had students to that mistakenly.

Lessons Learned:
1- Always monitor Background Process window when copying media from cards.

2- ALWAYS have a backup drive to run backups on every night, PERIOD, or be prepared to lose it all (not "if", "when").

3- Rendering has no effect on skimming, ALWAYS leave Background Rendering turned off, delete render files as a part of regular maintenance.
www.finalcutprox.guru/blogs-main/techniq...troubleshooting.html

4- You won't get much of your footage back, simple fact of life. Whatever you do get recovered, feel blessed. Data recovery software and other methods outside of a clean room forensic recovery is mostly a scam, in my experience.

Crossing my fingers and hoping for the best with data recovery at the Apple Store, but advise you don't hold your breath for anything. Sucks when something like this happens. I have been in your shoes before, and know what a sick feeling it can give you. Here's hoping for the best!
Last Edit: 28 Jul 2017 12:12 by FCPX.guru.
The administrator has disabled public write access.

Lost all files inside FCPX library! 07 Aug 2017 08:07 #89736

  • andreitest
  • andreitest's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
Dear FCPX.guru, thank you for your response, and sorry about my delayed reply to your message. We haven't been able to send it to the Apple Store, but we have contacted some IT specialists, and they have suggested even recovery software for windows OS, which we have tried, and still waiting for it to run (it was a very in-depth scan, as we were told, and it takes a long time). Unfortunately, the situation does not look very bright, as the software sees some files (.mov files), which says can restore, but it does not look like they will be usable, because the video players don't recognise them..
Unfortunately, we do not afford a 10K dollars forensic recovery option (and I don't really think we could find such a service for public use in our country), so we still hope that we will recover at least partial footage, as it looks like the only feasible option. Also, we gave the original SD cards to be recovered from, to another IT specialist (all of the cards were formatted after downloading them to the computer, and 2 of them were used in another project to film on). As far as I understood, the cards that were formatted and left untouched afterwards will be recovered 100% (that is some bold statement from the IT specialist), but the cards that were formatted and used again to be filmed on in another project afterwards, don't have much chance of recovery. We are waiting for the results. Thank you for your kind words and suggestions! We will surely be taking them into consideration!
The administrator has disabled public write access.

Lost all files inside FCPX library! 07 Aug 2017 11:47 #89739

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2820
  • Thank you received: 372
  • Karma: 34
Please let me know how things turn out. This is a very depressing situation, but there is a lot of positive lessons to be learned. It has been a very long time, but I've been in your shoes before. This is why we have good old American sour mash whiskey for... medicinal purposes of course.

Get Disk Drill Media Recovery, an app that can salvage camera cards really well, depending on if they've been re-used or not. I got it while on a reality TV show production. As DIT manager we were offloading dozens of cards a day constantly. Dragging contents to the trash and emptying the trash right away. Fortunately we had a system to double check offloads before the cards went back out. A few times we were missing clips, and this app recovered vital data from the cards. Now, you may be asking yourself, "Why did you double-check cards before wiping them clean?" Well, when the producer hires some kids for less than minimum wage, and you're personally more on-set than in the DIT room because the producer/director is rarely around and you have to take over, well, stuff will go wrong...

Still crossing my fingers! Best of luck, and chin up!
Last Edit: 07 Aug 2017 11:48 by FCPX.guru.
The administrator has disabled public write access.

Lost all files inside FCPX library! 09 Aug 2017 06:26 #89791

  • andreitest
  • andreitest's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 6
  • Karma: 0
We've tried to recover using KrollOntrack Easy recovery software, and it did find some original video files, but, unfortunately, they are corrupted, and now somebody is trying to repair them using some specialised software. Also, other bad news is related to the found files - the largest, single file, is only 7 gb in size, which is way too small, considering we've shot multiple 30-50 minutes takes, that took a lot more space, and those held the most "video content". So I guess, most of the original files are already lost..
The administrator has disabled public write access.

Lost all files inside FCPX library! 09 Aug 2017 10:45 #89796

  • FCPX.guru
  • FCPX.guru's Avatar
  • OFFLINE
  • Platinum Boarder
  • bbalser.com
  • Posts: 2820
  • Thank you received: 372
  • Karma: 34
:(
The administrator has disabled public write access.