fbpx
Welcome, Guest
Username: Password: Remember me

TOPIC: How do you tell FCPX to find and delete (COMPLETELY) unused media from a project?

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 31 Mar 2018 02:48 #94914

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2
I'm making a documentary and have a very organic working process whereby I try things and have to see them on the screen in action before deciding if they work or not. Hence a logical "only import what you will need" way of doing things cannot be applied here.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 01 Apr 2018 15:27 #94931

  • joema
  • joema's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 1554
  • Karma: 27
  • Thank you received: 332

tj7 wrote: I'm making a documentary and have a very organic working process whereby I try things and have to see them on the screen in action before deciding if they work or not. Hence a logical "only import what you will need" way of doing things cannot be applied here.


Similarly, documentaries often have high shooting ratios. My doc team can shoot up to 1TB of 4k H264 per day, which equates to about 20 hr of footage. That's two multicam teams, several b-roll shooters, multiple drones, multiple action cams, etc. I can't take time to look through that before import to FCPX -- it's just too slow, plus the multicams aren't synced.

For similar situations it seems better to import everything to FCPX with "leave files in place", then do the initial evaluation, selects and rejects there. At some point you may want to trim that or leave behind the totally unused clips.

I do that myself, but there are several issues:

(1) We never delete any shot material -- it's too costly to obtain and we might use it in the future. Thus it's not really saving disk space because we must keep it around. If we want a trimmed-down more portable version for collaborative editorial, just copying the project to another library (which brings along any used clips and leaves behind all unused clips) works very well.

If you want to trim clips during the copy, not just copy entire clips where a small range is used, there's the 3rd party utility Worx4 X. I personally haven't used this yet because it would take me a long time and many tests to validate it: itunes.apple.com/us/app/worx4-x/id1195903030?ls=1&mt=12

(2) Out of concern over data safety, FCPX never deletes any external media. Even if it identified totally unused clips within a library, and even if you could delete those from the library, they would not be deleted on disk so no space is saved. Only the symlinks are deleted.

(3) Maybe FCPX could be enhanced to (a) query totally unused clips, then (b) tag those external files with a Finder tag (or use you could use FindrCat) so you could manually delete them: intelligentassistance.com/findrcat.html

That's possible but I don't see it as a top development priority when so many more urgent things are needed, and there's already a usable solution of just copying the project to another library. OTOH it seems a very small enhancement, just adding one query type. I vaguely recollect there used to be a trick using a blank smart collection and FCPX would show you totally unused clips, but I can't remember the details. I don't think it works any more.

(4) Internal or managed media can be deleted within FCPX and the file will actually be deleted -- provided no other library or app is referencing that. If a media file is referenced by any other library on that volume, it will show as deleted from the 1st managed library but the disk space will not be cleared up. The file is still hard linked by another reference somewhere. Because of high documentary shooting ratios, I think external media is more common, and in that situation there's no way to delete *any* file (not just unused clips) from within FCPX.

Probably the fastest, easiest way to free up lots of space is using Library File Manager: www.arcticwhiteness.com/finalcutlibrarymanager/

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 02 Apr 2018 20:57 #94951

  • pszilard
  • pszilard's Avatar
  • Offline
  • Gold Boarder
  • Gold Boarder
  • Posts: 220
  • Thank you received: 1
What a great thread! I was also going to suggest Final Cut Library Manager.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 21 Apr 2018 04:01 #95254

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2
Ok, still managing this.
I have not done the whole copy -project-to-new-library thing because it is going to be a major, time consuming head-f%^k doing that for me.
I've been manually deleting totally unused media by using the "show used media ranges" function.

I've just learned the hard way that this function LIES.

If your media is part of a compound clip, IT DOES NOT COME UP AS BEING USED until you ungroup the compound clip, "revealing" the "raw" media and then suddenly FCPX is all like "oh, that's right, sorry, that media IS actually being used, let me put that orange marker on it now."

I guess more than anything I am simply venting frustration that such a basic function is so problematic.

I don't want to f___ around clearing yet more HD space to create a copy of an entire library and play swap arounds just to fool FCPX into doing the clean up job which should be a basic, native function (says me, expecting the universe to revolve around me, haha)

Sigh.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 21 Apr 2018 13:29 #95256

  • Albo
  • Albo's Avatar
  • Offline
  • Gold Boarder
  • Gold Boarder
  • Posts: 166
  • Thank you received: 3
Great thread indeed.
I'm also working on a documentary.
This conversation is very usefull.
I also import everything as I change edit as I move along the edit.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 21 Apr 2018 15:20 #95258

  • VidGreg
  • VidGreg's Avatar
  • Offline
  • Platinum Boarder
  • Platinum Boarder
  • Posts: 972
  • Karma: 28
  • Thank you received: 197
Hi tj7
Personally I think the instructions that Joema outlined for creating a new library, copy/move the project to the new library, delete old library is very fast and easy. This will also avoid the problem of not seeing media in compound clips as these will move into new library. You can then move and consolidate the new Library to external drives for archiving purposes if desired. You could most likely create an Automator action for this if you are going to do this a lot.
As Joema stated, if staying on the same drives, creating new Libraries is fast and doesn't really take up very much space at all, only when you copy the media to a different drive does actual copying occur.
The various 3rd party apps that are mentioned provide additional abilities. There is also the use of XMLs and the Timeline Index as additional resources.
Be cautious deleting via Finder.
Finally, if you really won't be adding any additional footage, you can just save a Master File.
Hope this Helps, Greg

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 04:44 #95286

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2
Having followed Joema's instructions:

"If your goal is to clean up disk space consumed by library media... this can easily be done by just dragging and dropping the project to a library you create for that purpose....After the copy, click the library and in the Inspector click "Consolidate". "

Done and done.

Outcome:

1.
My library effectively shrunk from around 980GB to 140GB in size after following these steps.
Before consolidating, the library was 40GB in size.
A success in terms of "cleaning up".

2.
Problematically, the process of exporting (or "sharing" as FCPX calls it) is now observably much more CPU intensive, and takes more than ****6 times longer****. The most troubling aspect of this is an evident instability which has emerged. I just completed one more test export of the entire 90 minute feature, and FCPX was observed to hang with the "preparing media for sharing" window, which remained on screen for the entire duration of the export process, "background tasks" was greyed out, and the application would not allow me to quit. It was only by checking the output directory and observing lag in mouse movement that I noted that the export may still be happening despite this hanging window. Before I followed these steps, when my library was completely enormous and in need of a cleanup, the process of exporting the entire feature took no more than 15-20 minutes. Since the cleanup, it now takes between 90 and 180 minutes, and my mac is very sluggish whilst this is happening. Neither were an issue before.

Note that after backing up the original bloated library, and after creating the copy / consolidated library, I deleted the original bloated library and simply replaced it - on the same drive - with the new consolidated copy. So there has been no change in my hardware or process chain to explain the extreme degradation in processing time and efficiency which I have described.

Note that the way I set up my libraries is uniform, ie I always choose to leave original media in its original location. I made no changes to how I set up my original library, which bloated to 980GB after about 18 months of work on this project, as compared with the copy.

So I feel that one step forward has been taken, but two steps back. At this stage, the process of sharing is now so inefficient and worrying that I am tempted to revert back to the original bloated library and continue the stupidly "manual labour intensive" process of identifying and removing unused media, given that FCPX , as evidenced in this thread, can't handle such a simple task natively.

Again: Rhetorically and blusteringly, perhaps - why on gods earth does FCPX not allow for a simple clean up process which allows the user to immediately identify clips which are not used in their entirety, and why, for _____'s sake has it got up to its current version without addressing the diabolically unreliable "show used ranges" feature which fails to identify the used ranges of clips which have been used in compound clips? How enormously frustrating and potentially fatal for those who use a lot of compound clips.

More answerable - I hope - is my final question - what is causing this enormous change - for the worse - in the export process?

Please Log in or Create an account to join the conversation.

Last edit: by tj7.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:14 #95287

  • pszilard
  • pszilard's Avatar
  • Offline
  • Gold Boarder
  • Gold Boarder
  • Posts: 220
  • Thank you received: 1
When you reduced your Library, you effectively also removed all rendered optimised content, which is why exporting took a long time, as it had to be re-created.

I would suggest that you only do a Library reduction after you had done all your exports.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:31 #95288

  • MsJustine
  • MsJustine's Avatar
  • Offline
  • Banned
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 971
  • Karma: -76
  • Thank you received: 46
It is a great question, something I feel Apple+FCPX dev team has overlooked, back in the day if you used non tape based recording, you used a different ingest method than the tape method in final cut pro 6/7, you could part ingest, in other words range ingest, and only that part was copied to the storage location you set.

Now in X, you don't have this option, some codecs allow for range ingesting, some don't, a bit confusing, after all X relies on a database to store the information, so maybe the database does not allow for deleting a portion of a clip, as I understand it, if you use a range, "flags " are set at the clip in point and out point for that range to highlight in orange.

The clip itself, it is hard baked into the database as 1 clip, maybe I am wrong, but once it is coded with a certain codec or wrapper, it cannot be chopped up into smaller chunks inside of FCPX.

I have always wondered about this from 10.0.1, is this aspect going to change and it has not yet, and I am sure there are many that maybe cannot afford to purchase yodabytes of storage, if you shoot in R3D, and convert, or keep the original, you have storage issues.

Editing, or pre-editing in the browser of FCPX is great, you range select a usable portion of the clip, tag as favorite, keyword, it makes every sense to me, to you, and to Tim's underpants that the next step is to delete from the hard drive clips, ranges within a clip, "send to trash", when I edit, and find a portion of a clip that is garbage, instinctively I press the delete key, large as live, it says what it does right there, DELETE!!! yet in fcpx this marks as rejected...

Sorry but that is just not right, it was not right in 10.0.0 and is not right in 10.4.1, it should mark as send to trash, and when you exit out of fcpx, you are asked "Do you want to empty the trash YES NO" you select yes, and you only have the clips you need the next time you launch FCPX...

After all do we need half a yodabyte of data wasting hard drive space that will never be used??? We all shoot more than we need, back in the tape days, when tapes cost 5 for a dollar, it mattered not, you could walk/drive to the nearest wallymart and pick up a 5 pack of mini dv for $1....run the tape once to ingest, throw the tapes into a shoe box, and shoeboxes went to the basement/attic, under the bed, safe...

Now with storage costing more than the camera, we cannot afford to waste storage, come on Apple get your head out of your rear-end....Sort this garbage out....in other words be professional....

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:35 #95289

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2

pszilard wrote: I would suggest that you only do a Library reduction after you had done all your exports.


Thanks for the advice - however, for me, the problem with this suggestion is that it presumes that everyone has the same way of working, a finite deadline and no need to clean up shop during the project editing process. This project is one which is multi-stage, I do a draft, pitch that, consult etc and then redraft. There is potentially 2 years worth of legal stuff to get through before I even have final approval to use much of the footage I have. So basically, "waiting until it's done" is not practical for me, I need to be able to keep things as clean and tidy as possible, as efficient as possible - now, not later.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:38 #95290

  • MsJustine
  • MsJustine's Avatar
  • Offline
  • Banned
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 971
  • Karma: -76
  • Thank you received: 46
Then there is only I guess 1 option, flood the feedback page with the suggestion, keep sending in requests, who knows, maybe it will be read, I am sure it won't, other users here swear feedback is read, but if the feedback page itself is obsolete, then how is the feedback read???

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:39 #95291

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2

pszilard wrote: When you reduced your Library, you effectively also removed all rendered optimised content, which is why exporting took a long time, as it had to be re-created.


Which just illustrates yet another "i'll decide what you want" aspect of FCPX which drives me effing insane,
this is just diabolical.

I should be able to simply tell it "please delete the media which was imported but left completely unused up to this point." It's ridiculous that this has been overlooked.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:40 #95292

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2

MsJustine wrote: Then there is only I guess 1 option, flood the feedback page with the suggestion, keep sending in requests, who knows, maybe it will be read, I am sure it won't, other users here swear feedback is read, but if the feedback page itself is obsolete, then how is the feedback read???


Yes, diabolical!

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:47 #95293

  • MsJustine
  • MsJustine's Avatar
  • Offline
  • Banned
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 971
  • Karma: -76
  • Thank you received: 46
What really burns my bum fluffy hairs is that instead of fixing the problems already in FCPX, and there are many, they just keep adding in new shiny new things, as if that will be enough....

yay for 360...not needed by 99% of the user base, yay for captions, not needed by 99% of us...

Delete of unwanted material in the middle of a project is what 100% of users need, even if you have 2000 yodabytes of storage, dealing with garbage all the time is just a waste of time...Hide rejected is a stupid method, it solves nothing, it only in the end makes things worse.

Use the delete key as Steve intended back in the apple 11e days, delete means get rid off... We sort once and range select the garbage, it should vanish like a fart, smelled once and forgotten....But no Apple thinks they know what is best.. we do not need a nanny company looking after us, we are big enough, stupid enough to know what works...You are not qualified to know what I need/want...stop it...

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 07:01 #95294

  • pszilard
  • pszilard's Avatar
  • Offline
  • Gold Boarder
  • Gold Boarder
  • Posts: 220
  • Thank you received: 1
Well we are all the same boat, but I try to find a workflow rather than just feel upset.

Ripple training has good tutorials explaining how if you create a camera archive then you can choose only to import the segment that you use. I also use Final Cut Library Manager to reduce bulk. If you have a 2 year long project then basically you either allocate the HDD space, or wear the time to re create, maybe over-night exports?

I use a Promise Pegasus TB box for Libraries and current projects and store archived media on a slower Synology NAS. But that's just me. Hard drives are no longer that expensive, you can get a USB3 SATA cradle and buy a dedicated disk for the big project, and when not required just unmount it and stick in the cupboard.

Just some ideas...

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 07:53 #95295

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2
I also use a Promise Pegasus setup.
I have HD's coming out of my ears, no probs there.
I *am* trying to find workflow solutions.
Thanks for reading and contribiting, though, and hopefully logic will prevail as far as a future update of FCPX goes.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 04 Jun 2018 05:06 #95808

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2
Just learned the hard way that this process (see my "outcome" posts here) leads to FCPX placing everything used in a project into the "ORIGINAL MEDIA" folder rather than retaining my preference for all media files to REMAIN IN THEIR ORIGINAL LOCATION.

Bloody hell.

This actually gets worse and worse.

Please Log in or Create an account to join the conversation.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 04 Jun 2018 05:16 #95810

  • MsJustine
  • MsJustine's Avatar
  • Offline
  • Banned
  • Banned
  • I am in Durban, safe and warm...
  • Posts: 971
  • Karma: -76
  • Thank you received: 46
It is nigh middle of the year, another snooze announcement from the mothership in less than 12 hours...And still FCPX is broken...It is not that hard to see what needs to be fixed...

I agree that the number 1 issue is data management, how to delete once ingested...This makes sense...and I see the reason to feel the need to ingest 100% and reduce later..after all to reject a clip/range you use the DELETE key...epic fail on part of the dev team there...almost a sick joke!!

Does anyone have a sensible solution...the ripple twins are strangely silent on this topic, this aspect of FCPX is not covered in any tutorial I am aware of...maybe I missed something.

The only viable solution for data management is the Felipe Baez method, to compress the heck out of the material, mind you, you will need two copies of the data, 1 at 100% unproxied and 1 proxy file version...

In trumpland where data storage is cheaper than loo rolls, maybe, here data storage is super expensive, sub compact cars are cheaper than a couple of TB drives....What do we do???

This is a serious issue the mothership is not taking seriously at all... :angry:

Please Log in or Create an account to join the conversation.

Last edit: by MsJustine.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 26 Jun 2018 02:58 #96092

  • tj7
  • tj7's Avatar Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
  • Posts: 114
  • Karma: -1
  • Thank you received: 2
It's unbelievable that this basic functionality isn't provided and that it's not even formally acknowledged as an issue. I remain gobsmacked that after taking the measures described throughout this thread - workarounds for the most basic function having been inexplicably left out - the software went and copied and relinked all media files to the "original media" folder, thus wasting HD space (and creating more work - when I choose "reveal in finder", instead of being linked to the file in the location I want to find it in so I can keep myself organised, I now magically get linked to a copy of the f__king thing in the "original media" folder WHICH IS NOT WHAT I FREAKING TOLD FCPX TO DO) - requiring a further cleanup - and I can literally see right in front of me my preference for all media to stay in its original god damned mother f__ing location has been checked, never unchecked, and yet this thing evidently has a mind of its own. Furious.

Please Log in or Create an account to join the conversation.

Last edit: by tj7.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 26 Jun 2018 03:19 #96094

  • Tom Wolsky
  • Tom Wolsky's Avatar
  • Offline
  • Moderator
  • Moderator
  • Posts: 4491
  • Karma: 112
  • Thank you received: 733
This is a user to user forum. Shouting at us doesn't do anything. I assume you've used feedback.

Please Log in or Create an account to join the conversation.

Last edit: by Tom Wolsky.