Welcome, Guest
Username: Password: Remember me
  • Page:
  • 1
  • 2

TOPIC: How do you tell FCPX to find and delete (COMPLETELY) unused media from a project?

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 26 Mar 2018 07:33 #94807

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
FCPX 10.3.4

Can't find this basic function anywhere - have searched online.
Have searched here.
Evidently you can tell the browser to show only "Unused" media -
however, this is a lie because as this thread illustrates:

www.fcp.co/forum/4-final-cut-pro-x-fcpx/...showing-unused-clips

Selecting "unused" actually pulls up clips which ARE partially used, the application presuming that a user would have no conceivable reason for wanting to keep partially used clips whilst deleting completely unused clips.

I'm also aware of the show "used media ranges" function, but this does not solve the primary, basic issue - how do you tell FCPX to *only display completely unused media* for a one step clean-up process?

Or is the only way to manually click by click select media without an orange range use marker on it?

Surely not...?
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 26 Mar 2018 12:38 #94816

  • joema
  • joema's Avatar
  • NOW ONLINE
  • Platinum Boarder
  • Posts: 938
  • Thank you received: 203
  • Karma: 15
tj7 wrote:
FCPX 10.3.4...Evidently you can tell the browser to show only "Unused" media......
Selecting "unused" actually pulls up clips which ARE partially used, the application presuming that a user would have no conceivable reason for wanting to keep partially used clips whilst deleting completely unused clips...how do you tell FCPX to *only display completely unused media* for a one step clean-up process?...

Most filtering functions in FCPX are range based, not clip based. In essence there are used and unused ranges, not used and unused clips. If no portion of a clip is used, then that entire unused range is shown, which by coincidence is the clip. So it won't show you just totally unused clips so you can delete those. However there is an easy solution.

If your goal is to clean up disk space consumed by library media (internal or external) after your project is finished, thereby freeing you to delete from disk any totally unused clips, this can easily be done by just dragging and dropping the project to a library you create for that purpose. Any clips with used ranges will be copied in their entirety. Any clips with no used ranges will not be copied. You can then archive that new smaller library or use it to refine the edit . The original source library and its media are no longer needed -- provided you are certain you won't need any additional unused clips. In general I always save all media shot by all cameras on all endeavors.

Due to optimizations in FCPX, in many cases copying the project (which automatically copies any clips with used ranges) to a new library will be very fast and not take up any more disk space.

After the copy, click the library and in the Inspector click "Consolidate". That ensures the media itself is *logically* copied inside the library. If this copy operation is on the same disk volume it will be extremely fast and efficient because no data is moved. Behind the scenes FCPX uses hard links from the original files to the new library -- whether those files were originally used externally by an unmanaged library or internally by a managed library -- again, assuming they and the library were on the same disk volume.

Even if you could filter and delete just totally unused clips, they would still be there taking space on the disk (if using external, unmanaged media). If using a managed library the clips would be physically deleted -- if they are only used in one library. If they are used in multiple managed libraries on the same disk volume FCPX creates a "hard link" behind the scenes to conserve disk space. IOW the same file copied to multiple managed libraries only exists as one physical file on disk. The media file is only deleted from disk inside the managed library when the last reference in the last library to that file is deleted. So deleting a clip instance present in two different unmanaged libraries doesn't save any space, and deleting it if present in two different managed libraries on the same disk volume doesn't save any space -- in both cases there was only one file present.

Likewise copying a clip to a different unmanaged library doesn't consume any space, and copying a clip to a managed library on the same disk volume doesn't consume any space -- in both cases there was only one file present.
Last Edit: 26 Mar 2018 16:10 by joema.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 26 Mar 2018 16:15 #94828

  • VidGreg
  • VidGreg's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 890
  • Thank you received: 175
  • Karma: 26
Hi tj
What Joema is trying to say is that FCPX wants to keep all clips entirely that are used in your project. Just how the program is written. Some work arounds by creating new project in new Library, but will still maintain whole clips/media.
You might want to check out Worx4X and I think this might be what you want to do.
worx4.com/Worx4-X/worx4-x-overview.html
It has been written up here on fop.co in the past.

Hope this Helps, Greg
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 26 Mar 2018 16:57 #94830

  • joema
  • joema's Avatar
  • NOW ONLINE
  • Platinum Boarder
  • Posts: 938
  • Thank you received: 203
  • Karma: 15
VidGreg wrote:
...What Joema is trying to say is that FCPX wants to keep all clips entirely that are used in your project. Just how the program is written. Some work arounds by creating new project in new Library, but will still maintain whole clips/media.
You might want to check out Worx4X and I think this might be what you want to do.
worx4.com/Worx4-X/worx4-x-overview.html..

I think there are two separate issues here:

(1) The OP wants to delete (not trim) any totally unused clips as part of cleaning up and saving space.

(2) Though unstated so far, he might also want to trim the used clips. Worx4X is good for that.

Re #1, I don't think FCPX has a query method to isolate only clips which are totally unused. It can only show unused *ranges*. My point was just copying the project to a managed library on the same volume as the media accomplishes the same thing -- leaving behind only totally unused clips. In many cases the "copy" is very fast and takes no more appreciable space. Then the original media and library could theoretically be deleted, although I personally never delete unused media.

There are various scenarios whether the source and destination libraries are managed or unmanaged and whether they are on the same disk volume or not. But for archiving a timeline or jettisoning lots of totally unused media while retaining the project and files for the timeline, creating a managed library on the same disk volume as the media and copying the project to the new library is very fast and efficient.

The fact there is only one physical copy of each FCPX-referenced file on the volume can be verified using the terminal commands ls -l which shows the reference count of the media file and ls -i which shows the inode number. It can even appear as two separate files in Finder, e.g, one inside a managed library and another outside a different unmanaged library, but there is only one physical file occupying space.

E.g, if within FCPX a file is copied from an unmanaged library to within a managed library, from Finder it will appear as two files in two locations. However if you delete the external file using Finder, nothing on disk is really deleted. Rather the reference count is decremented and it still exists inside the managed library. This not only saves space and improves performance when FCPX is copying media on the same volume, but it reduces "missing media" problems when external files are deleted in Finder which are still used by a managed library.
Last Edit: 26 Mar 2018 20:31 by joema.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 30 Mar 2018 02:32 #94900

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
Thanks everyone.

"If your goal is to clean up disk space consumed by library media (internal or external) after your project is finished, thereby freeing you to delete from disk any totally unused clips, this can easily be done by just dragging and dropping the project to a library you create for that purpose."

This is what I will have to do. I am quite surprised that there isn't a simpler way to do this, but there you go.
My real concern now is that because I am paranoid (through hard lessons learned), I will have to first back everything up and then go through the step by step library copy process described here. A little more time consuming than should be necessary, but looks like it will sort things out for me.

It is indeed completely unused media I wish to remove, I do not want to do anything with partially used media.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 30 Mar 2018 04:59 #94902

  • Karsten Schlüter
  • Karsten Schlüter's Avatar
  • NOW ONLINE
  • Moderator
  • Posts: 3389
  • Thank you received: 590
  • Karma: 58
on risk sounding very smart-assy …
Why did you import it anyhow? :evil:

Watching others importing their material, 'nobody' uses the options to select BEFORE import, so, any 'cam down but running', any take1/10 (which is by 99% chance a no-wrap ;) ) gets added to and bloat the Lib …

And, 2nd comment, more tech based:
when you 'trash' material outside a range - what shall happen when you lengthen the transition to/from that clip later …? …

The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 31 Mar 2018 02:48 #94914

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
I'm making a documentary and have a very organic working process whereby I try things and have to see them on the screen in action before deciding if they work or not. Hence a logical "only import what you will need" way of doing things cannot be applied here.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 01 Apr 2018 15:27 #94931

  • joema
  • joema's Avatar
  • NOW ONLINE
  • Platinum Boarder
  • Posts: 938
  • Thank you received: 203
  • Karma: 15
tj7 wrote:
I'm making a documentary and have a very organic working process whereby I try things and have to see them on the screen in action before deciding if they work or not. Hence a logical "only import what you will need" way of doing things cannot be applied here.

Similarly, documentaries often have high shooting ratios. My doc team can shoot up to 1TB of 4k H264 per day, which equates to about 20 hr of footage. That's two multicam teams, several b-roll shooters, multiple drones, multiple action cams, etc. I can't take time to look through that before import to FCPX -- it's just too slow, plus the multicams aren't synced.

For similar situations it seems better to import everything to FCPX with "leave files in place", then do the initial evaluation, selects and rejects there. At some point you may want to trim that or leave behind the totally unused clips.

I do that myself, but there are several issues:

(1) We never delete any shot material -- it's too costly to obtain and we might use it in the future. Thus it's not really saving disk space because we must keep it around. If we want a trimmed-down more portable version for collaborative editorial, just copying the project to another library (which brings along any used clips and leaves behind all unused clips) works very well.

If you want to trim clips during the copy, not just copy entire clips where a small range is used, there's the 3rd party utility Worx4 X. I personally haven't used this yet because it would take me a long time and many tests to validate it: itunes.apple.com/us/app/worx4-x/id1195903030?ls=1&mt=12

(2) Out of concern over data safety, FCPX never deletes any external media. Even if it identified totally unused clips within a library, and even if you could delete those from the library, they would not be deleted on disk so no space is saved. Only the symlinks are deleted.

(3) Maybe FCPX could be enhanced to (a) query totally unused clips, then (b) tag those external files with a Finder tag (or use you could use FindrCat) so you could manually delete them: intelligentassistance.com/findrcat.html

That's possible but I don't see it as a top development priority when so many more urgent things are needed, and there's already a usable solution of just copying the project to another library. OTOH it seems a very small enhancement, just adding one query type. I vaguely recollect there used to be a trick using a blank smart collection and FCPX would show you totally unused clips, but I can't remember the details. I don't think it works any more.

(4) Internal or managed media can be deleted within FCPX and the file will actually be deleted -- provided no other library or app is referencing that. If a media file is referenced by any other library on that volume, it will show as deleted from the 1st managed library but the disk space will not be cleared up. The file is still hard linked by another reference somewhere. Because of high documentary shooting ratios, I think external media is more common, and in that situation there's no way to delete *any* file (not just unused clips) from within FCPX.

Probably the fastest, easiest way to free up lots of space is using Library File Manager: www.arcticwhiteness.com/finalcutlibrarymanager/
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 02 Apr 2018 20:57 #94951

  • pszilard
  • pszilard's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 138
  • Karma: 0
What a great thread! I was also going to suggest Final Cut Library Manager.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 21 Apr 2018 04:01 #95254

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
Ok, still managing this.
I have not done the whole copy -project-to-new-library thing because it is going to be a major, time consuming head-f%^k doing that for me.
I've been manually deleting totally unused media by using the "show used media ranges" function.

I've just learned the hard way that this function LIES.

If your media is part of a compound clip, IT DOES NOT COME UP AS BEING USED until you ungroup the compound clip, "revealing" the "raw" media and then suddenly FCPX is all like "oh, that's right, sorry, that media IS actually being used, let me put that orange marker on it now."

I guess more than anything I am simply venting frustration that such a basic function is so problematic.

I don't want to f___ around clearing yet more HD space to create a copy of an entire library and play swap arounds just to fool FCPX into doing the clean up job which should be a basic, native function (says me, expecting the universe to revolve around me, haha)

Sigh.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 21 Apr 2018 13:29 #95256

  • Albo
  • Albo's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 44
  • Karma: 0
Great thread indeed.
I'm also working on a documentary.
This conversation is very usefull.
I also import everything as I change edit as I move along the edit.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 21 Apr 2018 15:20 #95258

  • VidGreg
  • VidGreg's Avatar
  • OFFLINE
  • Platinum Boarder
  • Posts: 890
  • Thank you received: 175
  • Karma: 26
Hi tj7
Personally I think the instructions that Joema outlined for creating a new library, copy/move the project to the new library, delete old library is very fast and easy. This will also avoid the problem of not seeing media in compound clips as these will move into new library. You can then move and consolidate the new Library to external drives for archiving purposes if desired. You could most likely create an Automator action for this if you are going to do this a lot.
As Joema stated, if staying on the same drives, creating new Libraries is fast and doesn't really take up very much space at all, only when you copy the media to a different drive does actual copying occur.
The various 3rd party apps that are mentioned provide additional abilities. There is also the use of XMLs and the Timeline Index as additional resources.
Be cautious deleting via Finder.
Finally, if you really won't be adding any additional footage, you can just save a Master File.
Hope this Helps, Greg
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 04:44 #95286

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
Having followed Joema's instructions:

"If your goal is to clean up disk space consumed by library media... this can easily be done by just dragging and dropping the project to a library you create for that purpose....After the copy, click the library and in the Inspector click "Consolidate". "

Done and done.

Outcome:

1.
My library effectively shrunk from around 980GB to 140GB in size after following these steps.
Before consolidating, the library was 40GB in size.
A success in terms of "cleaning up".

2.
Problematically, the process of exporting (or "sharing" as FCPX calls it) is now observably much more CPU intensive, and takes more than ****6 times longer****. The most troubling aspect of this is an evident instability which has emerged. I just completed one more test export of the entire 90 minute feature, and FCPX was observed to hang with the "preparing media for sharing" window, which remained on screen for the entire duration of the export process, "background tasks" was greyed out, and the application would not allow me to quit. It was only by checking the output directory and observing lag in mouse movement that I noted that the export may still be happening despite this hanging window. Before I followed these steps, when my library was completely enormous and in need of a cleanup, the process of exporting the entire feature took no more than 15-20 minutes. Since the cleanup, it now takes between 90 and 180 minutes, and my mac is very sluggish whilst this is happening. Neither were an issue before.

Note that after backing up the original bloated library, and after creating the copy / consolidated library, I deleted the original bloated library and simply replaced it - on the same drive - with the new consolidated copy. So there has been no change in my hardware or process chain to explain the extreme degradation in processing time and efficiency which I have described.

Note that the way I set up my libraries is uniform, ie I always choose to leave original media in its original location. I made no changes to how I set up my original library, which bloated to 980GB after about 18 months of work on this project, as compared with the copy.

So I feel that one step forward has been taken, but two steps back. At this stage, the process of sharing is now so inefficient and worrying that I am tempted to revert back to the original bloated library and continue the stupidly "manual labour intensive" process of identifying and removing unused media, given that FCPX , as evidenced in this thread, can't handle such a simple task natively.

Again: Rhetorically and blusteringly, perhaps - why on gods earth does FCPX not allow for a simple clean up process which allows the user to immediately identify clips which are not used in their entirety, and why, for _____'s sake has it got up to its current version without addressing the diabolically unreliable "show used ranges" feature which fails to identify the used ranges of clips which have been used in compound clips? How enormously frustrating and potentially fatal for those who use a lot of compound clips.

More answerable - I hope - is my final question - what is causing this enormous change - for the worse - in the export process?
Last Edit: 25 Apr 2018 04:45 by tj7.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:14 #95287

  • pszilard
  • pszilard's Avatar
  • OFFLINE
  • Expert Boarder
  • Posts: 138
  • Karma: 0
When you reduced your Library, you effectively also removed all rendered optimised content, which is why exporting took a long time, as it had to be re-created.

I would suggest that you only do a Library reduction after you had done all your exports.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:31 #95288

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Platinum Boarder
  • I am in Durban, safe and warm...
  • Posts: 903
  • Thank you received: 46
  • Karma: -73
It is a great question, something I feel Apple+FCPX dev team has overlooked, back in the day if you used non tape based recording, you used a different ingest method than the tape method in final cut pro 6/7, you could part ingest, in other words range ingest, and only that part was copied to the storage location you set.

Now in X, you don't have this option, some codecs allow for range ingesting, some don't, a bit confusing, after all X relies on a database to store the information, so maybe the database does not allow for deleting a portion of a clip, as I understand it, if you use a range, "flags " are set at the clip in point and out point for that range to highlight in orange.

The clip itself, it is hard baked into the database as 1 clip, maybe I am wrong, but once it is coded with a certain codec or wrapper, it cannot be chopped up into smaller chunks inside of FCPX.

I have always wondered about this from 10.0.1, is this aspect going to change and it has not yet, and I am sure there are many that maybe cannot afford to purchase yodabytes of storage, if you shoot in R3D, and convert, or keep the original, you have storage issues.

Editing, or pre-editing in the browser of FCPX is great, you range select a usable portion of the clip, tag as favorite, keyword, it makes every sense to me, to you, and to Tim's underpants that the next step is to delete from the hard drive clips, ranges within a clip, "send to trash", when I edit, and find a portion of a clip that is garbage, instinctively I press the delete key, large as live, it says what it does right there, DELETE!!! yet in fcpx this marks as rejected...

Sorry but that is just not right, it was not right in 10.0.0 and is not right in 10.4.1, it should mark as send to trash, and when you exit out of fcpx, you are asked "Do you want to empty the trash YES NO" you select yes, and you only have the clips you need the next time you launch FCPX...

After all do we need half a yodabyte of data wasting hard drive space that will never be used??? We all shoot more than we need, back in the tape days, when tapes cost 5 for a dollar, it mattered not, you could walk/drive to the nearest wallymart and pick up a 5 pack of mini dv for $1....run the tape once to ingest, throw the tapes into a shoe box, and shoeboxes went to the basement/attic, under the bed, safe...

Now with storage costing more than the camera, we cannot afford to waste storage, come on Apple get your head out of your rear-end....Sort this garbage out....in other words be professional....
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:35 #95289

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
pszilard wrote:
I would suggest that you only do a Library reduction after you had done all your exports.

Thanks for the advice - however, for me, the problem with this suggestion is that it presumes that everyone has the same way of working, a finite deadline and no need to clean up shop during the project editing process. This project is one which is multi-stage, I do a draft, pitch that, consult etc and then redraft. There is potentially 2 years worth of legal stuff to get through before I even have final approval to use much of the footage I have. So basically, "waiting until it's done" is not practical for me, I need to be able to keep things as clean and tidy as possible, as efficient as possible - now, not later.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:38 #95290

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Platinum Boarder
  • I am in Durban, safe and warm...
  • Posts: 903
  • Thank you received: 46
  • Karma: -73
Then there is only I guess 1 option, flood the feedback page with the suggestion, keep sending in requests, who knows, maybe it will be read, I am sure it won't, other users here swear feedback is read, but if the feedback page itself is obsolete, then how is the feedback read???
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:39 #95291

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
pszilard wrote:
When you reduced your Library, you effectively also removed all rendered optimised content, which is why exporting took a long time, as it had to be re-created.

Which just illustrates yet another "i'll decide what you want" aspect of FCPX which drives me effing insane,
this is just diabolical.

I should be able to simply tell it "please delete the media which was imported but left completely unused up to this point." It's ridiculous that this has been overlooked.
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:40 #95292

  • tj7
  • tj7's Avatar
  • OFFLINE
  • Senior Boarder
  • Posts: 68
  • Karma: 0
MsJustine wrote:
Then there is only I guess 1 option, flood the feedback page with the suggestion, keep sending in requests, who knows, maybe it will be read, I am sure it won't, other users here swear feedback is read, but if the feedback page itself is obsolete, then how is the feedback read???

Yes, diabolical!
The administrator has disabled public write access.

How do you tell FCPX to find and delete (COMPLETELY) unused media from a project? 25 Apr 2018 06:47 #95293

  • MsJustine
  • MsJustine's Avatar
  • OFFLINE
  • Platinum Boarder
  • I am in Durban, safe and warm...
  • Posts: 903
  • Thank you received: 46
  • Karma: -73
What really burns my bum fluffy hairs is that instead of fixing the problems already in FCPX, and there are many, they just keep adding in new shiny new things, as if that will be enough....

yay for 360...not needed by 99% of the user base, yay for captions, not needed by 99% of us...

Delete of unwanted material in the middle of a project is what 100% of users need, even if you have 2000 yodabytes of storage, dealing with garbage all the time is just a waste of time...Hide rejected is a stupid method, it solves nothing, it only in the end makes things worse.

Use the delete key as Steve intended back in the apple 11e days, delete means get rid off... We sort once and range select the garbage, it should vanish like a fart, smelled once and forgotten....But no Apple thinks they know what is best.. we do not need a nanny company looking after us, we are big enough, stupid enough to know what works...You are not qualified to know what I need/want...stop it...
The administrator has disabled public write access.
  • Page:
  • 1
  • 2