fbpx
Welcome, Guest
Username: Password: Remember me
{JFBCLogin}
25 Jan 2021
New boarders will have their posts moderated - Don't worry if you cannot see your post immediately.
Read More...
  • Page:
  • 1

TOPIC:

How can I relink my FCPX clips with newer, compressed files? 28 Oct 2022 23:53 #122632

I have a large project that I've been working on for some time, and in an attempt to save storage space on my drive, I'm trying to replace my existing files with compressed ones. The only problem is I'm greeted with the following message when I attempt to do so:



I've tried using various compressing apps and managed to keep all media specs exactly the same except for the frame rate (which always comes out just slightly different), so I assume the frame rate is the problem. Will the Compressor app be able to accomplish my goal, and if so, how do I go about setting it so that everything remains exactly the same while reducing the file size down to about 60%? All my files are in mp4 format.

EDIT: With one of my clips I just now managed to retain the frame rate with compression. Everything is identical and yet I'm still being given the same message when I try to relink it. I am now at a complete loss.
Attachments:

Please Log in to join the conversation.

Last edit: by Shadowfax.

How can I relink my FCPX clips with newer, compressed files? 29 Oct 2022 01:07 #122633

  • DaveM
  • DaveM's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 351
  • Karma: 1
  • Thank you received: 105
Any reason you haven't tried using the built-in proxy functionality? More recent versions of FCP allow you to create H.264 proxies at 1/8 the frame size (12.5%) of the original (as the smallest file size option).

It's odd that you are having trouble ending up with the (more) compressed file not having the same frame rate, as many transcoding applications have a setting for frame rate. If the original file has a variable frame rate (often the case for screen recordings or online meeting/call recordings) it's going to be problematic in some cases (and it should be avoided by transcoding the original file to a contstant frame rate).

You should also take a look at these sections of the FCP User Guide:

support.apple.com/en-in/guide/final-cut-...c9/10.6.2/mac/11.5.1

support.apple.com/en-in/guide/final-cut-.../11.5.1#verf7b83a22f

The second section discusses filename issues you can run into when using an external application to transcode files (and not just Compressor)...

---

Here's a demonstration of creating an external/manual proxy set of files that you might find helpful:



---

As a test, you could create a new Library and then create two, side-by-side folders on a drive, one "originals" and the other "proxies". Then move a sample "original media" file into the "originals" folder. Use Compressor as shown in the above video to create a corresponding proxy file in the "proxies" folder. You can then test to make sure the workflow works properly. Doing this test will help eliminate other factors that could be involved. Be sure to leave the original media file stored "in-place" (in the "originals" folder) as shown in the video...


Let us know if this helps.

Please Log in to join the conversation.

Last edit: by DaveM.

How can I relink my FCPX clips with newer, compressed files? 29 Oct 2022 23:21 #122642

Hey Dave. Thanks for the response.

I'd say that I've created proxy files for about half of my clips (not all of them, due to the fact that these proxy files are all about 4x the size of the originals, and I simply don't have the storage space for much more). I set the frame size at 50% because I found the video quality to be a little too low for anything under that. I'll try recreating these proxies via the method used in the video you sent me. I still haven't nailed down all the ins and outs of proxies. I originally created the ones I currently have simply because I was told that editing with proxies would help reduce some of the major lag issues I've been experiencing with my massive project (issues which unfortunately still haven't been totally worked out yet...but that's a whole other issue).

The transcoding apps that I've tried out have all indeed had different frame rate options to choose from, but the problem is that none of the options provided are the ones I need. From what you've said, I gather that this is likely because of the fact that all of my files are recorded live streams.

Please Log in to join the conversation.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 01:08 #122643

  • DaveM
  • DaveM's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 351
  • Karma: 1
  • Thank you received: 105

Hey Dave. Thanks for the response.

I'd say that I've created proxy files for about half of my clips (not all of them, due to the fact that these proxy files are all about 4x the size of the originals, and I simply don't have the storage space for much more). I set the frame size at 50% because I found the video quality to be a little too low for anything under that.

If your original files are MP4 (H.264/AVC-encoded), then you would choose H.264 proxies (at 50%) to get smaller files. If you are getting larger files then you may be creating ProRes-encoded proxies. If you are using version 10.4.9 or later of FCP, you should have the choice to create H.264 proxies in addition to ProRes, down to 12.5% of the original frame size. ProRes proxies do tend to take up more space than H.264 proxies, though they don't stress FCP as much during playback/editing.

I'll try recreating these proxies via the method used in the video you sent me. I still haven't nailed down all the ins and outs of proxies. I originally created the ones I currently have simply because I was told that editing with proxies would help reduce some of the major lag issues I've been experiencing with my massive project (issues which unfortunately still haven't been totally worked out yet...but that's a whole other issue).

Yes, if the proxy files are smaller, depending on your system, you should see some improvement in the responsiveness of FCP. The "external proxies" method described in the video should work, though that video was made prior to FCP having the extra proxies features that were added in version 10.4.9.

While editing, you might want to change the quality (viewer menu) to "Better Performance" and see if that helps. You can then change it back for color grading/correction and before you export your finalized project.

The transcoding apps that I've tried out have all indeed had different frame rate options to choose from, but the problem is that none of the options provided are the ones I need. From what you've said, I gather that this is likely because of the fact that all of my files are recorded live streams.

What do you mean exactly by "none of the options are the ones I need"? What is your deliverable frame rate, frame size, and codec? If the recordings have variable frame rates they should still be close to one of the standard frame rates. You need to pick a standard frame rate for the Project/timeline. The variable time rate "issue" may be part of the cause for the "lagginess" when editing. You should also have sufficient extra drive space (at least 20% free space on each drive used) and fast enough drives.

Would you mind using MediaInfo or Invisor on a typical original file you're using and post the results (text file) here or make it available for download? By looking at the file specifications in detail, we might be able to provide better assistance.

Cheers.

Please Log in to join the conversation.

Last edit: by DaveM.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 02:22 #122644

I mean to say that the EXACT frame rate is not presented in any of the options. I was under the impression that every single spec of the compressed file had to be identical to the original one in order to relink with it.
I used Invisor with one of my original files. Here are the results:

File
Name2:19:22.mp4
Size10.2 GB (10188148027 bytes)
KindMPEG-4 movie
UTIpublic.mpeg-4
Location-
CreatedOctober 27, 2022 at 1:37:40 AM
ModifiedOctober 27, 2022 at 1:37:40 AM
Container
FormatMPEG-4
Format profileBase Media
Codec IDisom (isom/iso2/avc1/mp41)
Duration4 h 24 min 10 s 27 ms
Overall bit rate modeVariable
Overall bit rate5 142 kb/s
Writing applicationLavf58.29.100
Video
ID1
FormatAVC
Format/InfoAdvanced Video Codec
Format profileHigh@L3.2
Format settings, CABACYes
Format settings, ReFrames4 frames
Format settings, GOPM=3, N=120
Codec IDavc1
Codec ID/InfoAdvanced Video Coding
Duration4 h 24 min 10 s 1 ms
Bit rate modeConstant
Bit rate5 000 kb/s
Width1 280 pixels
Height720 pixels
Pixel aspect ratio1.000
Display aspect ratio16:9
Frame rate modeVariable
Frame rate59.765 FPS
Minimum frame rate20.000 FPS
Maximum frame rate62.500 FPS
Original frame rate60.000 FPS
Frame count947268
Color spaceYUV
Chroma subsampling4:2:0
Bit depth8 bits
Scan typeProgressive
Bits/(Pixel*Frame)0.091
Stream size9.90 GB (97.2%)
Color rangeLimited
Color primariesBT.709
Transfer characteristicsBT.709
Matrix coefficientsBT.709
Audio
ID2
FormatAAC
Format/InfoAdvanced Audio Codec
Format profileLC
Codec ID40
Duration4 h 24 min 10 s 27 ms
Bit rate modeVariable
Bit rate128 kb/s
Maximum bit rate130 kb/s
Channel(s)2 channels
Channel positionsFront: L R
Sampling rate48.0 kHz
Frame rate46.875 FPS (1024 spf)
Frame count742970
Compression modeLossy
Stream size254 MB (2.5%)
DefaultYes
Alternate group1


Please Log in to join the conversation.

Last edit: by Shadowfax.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 02:52 #122645

  • DaveM
  • DaveM's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 351
  • Karma: 1
  • Thank you received: 105
Okay, thanks Shadowfax for posting the file info.

Sorry, I guess I wasn't very clear when I was talking about frame rates. I would say that, in general, most NLEs don't work well with variable frame rate media (compared with constant frame rates). For files similar to yours, I always convert them to a proper (constant) frame rate before initial import into FCP. I'm not familiar with using variable frame rate files in FCP, so hopefully others will chime in about that.

The file specs you provided show a pretty long file, but it isn't all that large in size. I don't know if it would be better to split that file into say two or four files and whether that would help with performance. Drive space may be the key issue (besides the variable frame rate).

Part of the problem with variable frame rate files is that no transcoding is going to give you the same frame rate as the original, as the original frame rate is an "average" frame rate. Variable frame rate files may indeed be the thing that breaks any sort of relinking (which would be another reason never to use variable frame rate files).

I would guess that the best option (though not for you, possibly) would be to transcode the original files into files with a constant frame rate, and sort of start over by importing those files and replacing clips in the timeline/Project.

Again, I'd suggest creating a test Library and try different workflows with a few files. See if external proxies with a constant frame rate of 59.94 (for example) will relink. Otherwise, try creating proxies with the built-in functionality in FCP. Again, the built-in proxies using H.264 at 50% should be smaller than the original files, since they are using the same codec. And, once you switch Meida Playback to Proxy or Proxy preferred, you should see less lag.

Please Log in to join the conversation.

Last edit: by DaveM.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 03:58 #122646

Unfortunately, starting my project over again from scratch isn't really an option for me, as I'm already well into editing hundreds of hours of video. I will try creating a test library in just a bit here. I admit my novice brain is slightly overloaded with new info at the moment (heh heh).
If I’m unable to do any relinking with modified original files, then, like you’ve said, the main move right now may lie with my current proxy files. As you’ve guessed, all of my current proxies are ProRes encoded. I would love to be able to switch over to H.264 because this would open up an absolutely massive amount of storage space on my drive. The main reason I went with ProRes originally was that I read that it was much easier to process and edit with. That said, editing is currently already very slow, as I only have enough storage space to transcode about half of my original media to ProRes encoded proxies. If I switched over to H.264, I would obviously have more than enough space to transcode everything. So this would be a route you’d recommend? Sorry to ask so many questions. I'm a total amateur.

Please Log in to join the conversation.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 04:24 #122647

  • DaveM
  • DaveM's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 351
  • Karma: 1
  • Thank you received: 105
Okay, thanks for the added info. I would always suggest that anyone should use ProRes proxies, if at all possible. FCP always runs best when using ProRes media. If you have insufficient space for ProRes, I guess you'll have to use H.264-encoded files.

If your drive(s) are too slow (or close to being full) you won't get good performance. It'll be a bottleneck, no matter what file type you use.

If you're using FCP 10.4.9 or later, I'd suggest you select the Library in your browser and use the File -> Delete Generated Library Files... command to get rid of your ProRes proxy files. Then, you could choose your files in the browser and right-click and choose Transcode Media... and create H.264 proxy files inside of FCP. Then, choose Proxy Only from the View menu and Better Performance, too.

I'd say this is your best option, for now...

---

I can confirm that if you create a new set of files using Compressor (leaving frame rate set to Automatic) the resulting files should link with the original files inside of FCP.

I believe you'd be better off sticking with the H.264 proxies inside of FCP.

---

When creating the proxies inside of FCP, you could choose 25% H.264 proxies. It'll take a while for the proxies to generate (good thing to let happen overnight). Choose Proxy Only and Better Performance from the View menu. Use this setup to do your editing.

If you need to see higher quality frames for color correction or VFX, you could switch back to Optimized/Original media playback and Better Quality. Choose these settings when you output your Project. Switch back to Better Performance and Proxy Only to continue editing...

Please Log in to join the conversation.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 07:24 #122649

Still experimenting with stuff, but I noticed something funny. The file I gave you specs for earlier had a duration of 4 hr 24 min 10 sec. I imported the compressed version of this file into FCP and for some reason, this one shows a duration of 4 hr 23 min 54 sec. This difference would obviously explain the inability to relink to the compressed file. Strange thing is, this difference appears within the app, but when I open up the files outside of FCP, they both have identical durations of 4 hr 24 min 10 sec. Thoughts?

Please Log in to join the conversation.

Last edit: by Shadowfax.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 14:25 #122653

  • DaveM
  • DaveM's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 351
  • Karma: 1
  • Thank you received: 105

Still experimenting with stuff, but I noticed something funny. The file I gave you specs for earlier had a duration of 4 hr 24 min 10 sec. I imported the compressed version of this file into FCP and for some reason, this one shows a duration of 4 hr 23 min 54 sec. This difference would obviously explain the inability to relink to the compressed file. Strange thing is, this difference appears within the app, but when I open up the files outside of FCP, they both have identical durations of 4 hr 24 min 10 sec. Thoughts?

Hmm, not enough info to know what's happening.

As I mentioned in my previous post, you should delete your ProRes proxies and try generating H.264 proxies at 50% or 25% (and follow the other directions)...

Please Log in to join the conversation.

How can I relink my FCPX clips with newer, compressed files? 30 Oct 2022 18:22 #122660

  • DaveM
  • DaveM's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 351
  • Karma: 1
  • Thank you received: 105
Some additional comments:

- Shadowfax, I'm guessing that primary reason you're looking into using proxies, either via the built-in functionality or using external files, is to improve the responsiveness of FCP while you're editing. Creating proxies may or may not help.

Your source media seem to be 1280 x 720 ~60 fps (thereabouts) H.264 in MP4 files, with variable frame rate. Those files aren't real large, per se, and the only "difficult" aspect is that they have a variable frame rate...

- If your free drive space isn't greater than 20% (or thereabouts) and you are using spinning hard drives, this may be the limiting factor in the overall performance of FCP. Of course, things also depend on your computer's HW configuration.

- Other factors that can affect the responsiveness of FCP include how complex and long your projects/timelines are, the amount of effects/plugins used, retiming, transformations, etc. If you are using multicam clips, at some point your drive(s) could limit responsiveness in FCP.

- Ways to optimize responsiveness/performance in FCP, aside from creating proxies, include but aren't limited to setting quality to "Better Performance" in the View menu, turning off background rendering, and keeping timelines from becoming hours in length. The timeline length and complexity factor is related to your HW configuration and drive space and speed.

- You should determine (or let us know) what your deliverable project specs are. Unless you are going to deliver a 720p60 master for broadcast, or you really need the higher frame rate, you might be better off with a project frame rate of 29.97 instead of 59.94.

- Again, if you optimize the settings available in FCP, you may not necessarily need to use proxies.

- Before generating a full set of proxies, should you decide to go that route, create a handful of files and compare the sizes to the originals. From that you could estimate how much additional drive space you'll need and be able to avoid running out of drive space.

Optimizing any NLE is a very technical topic and can be frustrating... Hope this helps...

Please Log in to join the conversation.

Last edit: by DaveM.

How can I relink my FCPX clips with newer, compressed files? 31 Oct 2022 04:20 #122668

As I mentioned in my previous post, you should delete your ProRes proxies and try generating H.264 proxies at 50% or 25% (and follow the other directions)...

Yes, we are already in agreement that this may be my best bet. That said, I've yet to begin this process, as it would end up being counterproductive should I find a way of freeing up storage space by accomplishing my goal of relinking my clips with compressed files (though this is seeming less and less attainable). As you've said, it would be preferable if I could edit with ProRes. Not only that, but it took me weeks to generate the terabytes of ProRes files that I currently have, and I'd like that to not go to waste if possible.
But yes, I will definitely be switching over to H.264 if all else fails.

Please Log in to join the conversation.

Last edit: by Shadowfax.

How can I relink my FCPX clips with newer, compressed files? 31 Oct 2022 04:23 #122669

- Shadowfax, I'm guessing that primary reason you're looking into using proxies, either via the built-in functionality or using external files, is to improve the responsiveness of FCP while you're editing. Creating proxies may or may not help.

Yes, my primary reason for looking at proxies is to improve the responsiveness of FCP while editing.

- If your free drive space isn't greater than 20% (or thereabouts) and you are using spinning hard drives, this may be the limiting factor in the overall performance of FCP. Of course, things also depend on your computer's HW configuration.

I always make sure to keep at least 20% free space on both my internal and external drives, and I’m working with SSDs.

- Other factors that can affect the responsiveness of FCP include how complex and long your projects/timelines are, the amount of effects/plugins used, retiming, transformations, etc. If you are using multicam clips, at some point your drive(s) could limit responsiveness in FCP.

The complexity and length of my project/timeline is 100% a factor here, as I’m working with about 200 different files containing hundreds of hours of video. That said, it’s currently been reduced down to about 5 hours on my timeline. This particular project is one that will basically never leave my computer and will forever be ongoing as I continue to add to it and splice things together. This is why it’s grown to be so massive.

- Ways to optimize responsiveness/performance in FCP, aside from creating proxies, include but aren't limited to setting quality to "Better Performance" in the View menu, turning off background rendering, and keeping timelines from becoming hours in length. The timeline length and complexity factor is related to your HW configuration and drive space and speed.

I always have it set to “Better Performance” when editing, and I always have background rendering turned off. Unfortunately, for my purposes, I can’t keep my timeline from being hours in length though.

- You should determine (or let us know) what your deliverable project specs are. Unless you are going to deliver a 720p60 master for broadcast, or you really need the higher frame rate, you might be better off with a project frame rate of 29.97 instead of 59.94.

I'd be perfectly fine with a lower frame rate project. It won't be for professional use or anything, as it's basically just a personal project of mine that I'll forever be tweaking. I'll occasionally export versions of it to view outside of FCP, but it doesn't have to be crazy high quality or anything.

Please Log in to join the conversation.

How can I relink my FCPX clips with newer, compressed files? 31 Oct 2022 04:37 #122670

  • DaveM
  • DaveM's Avatar
  • Offline
  • Platinum Member
  • Platinum Member
  • Posts: 351
  • Karma: 1
  • Thank you received: 105
Thanks for the update.

With respect to your project/timeline, you might consider splitting things into smaller projects/timelines for editing. Then, create compound clips of the smaller timelines/projects and add those to a master project/timeline. Doing so might help with responsiveness. At some point, if your project/timeline grows too large, FCP may crash or responsiveness could slow to being unusable. This could happen sooner or later, depending on system resources...

But, it sounds like you have a good grasp of things to try out. Good luck!


Cheers.

Please Log in to join the conversation.

How can I relink my FCPX clips with newer, compressed files? 31 Oct 2022 15:04 #122682

With respect to your project/timeline, you might consider splitting things into smaller projects/timelines for editing. Then, create compound clips of the smaller timelines/projects and add those to a master project/timeline. Doing so might help with responsiveness.
I've begun doing this, and it is speeding things up dramatically. Thanks a ton. Wish I would've had the common sense to try this ages ago!

Please Log in to join the conversation.

Last edit: by Shadowfax.
  • Page:
  • 1