I am running FCPX 10.1.3 and El Capitan (OS X 10.11.6) on a MacBook pro (Retina, 2.5Ghz Intel Core i7, 16GB Ram, AMD Radeon 2048MB graphics). In order to prepare for the upgrade to FCP 10.4.3 and Mac OS Mojave, I want to make backups of the library files and therefore first would like to consolidate large Library files (those that have media inside). Whilst consolidating (moving original media outside the library file) to make the backups smaller, my FPC crashes with every attempt. I have tried everything. It works with really small libraries. Now some of you might say why don't you ‘upgrade FPC’, and that is the whole point.. I want to do that but I first want to prepare by organizing, cleaning and making backups which is the right thing to do right? FPC does not crash with tasks such as rendering. What is your advise?? I could of course make backups of the library files that have the media in the library but OMG they are huge. Some are over 300GB.
Many thanks for your help!
FCPX.guru wrote: Why pull the media out of the Library? I don't understand the purpose of that. Just manually make a copy of it somewhere safe.
I agree with this. Moving media out of the library doesn't make the backup smaller -- it must be backed up either way.
I understand how before a major OS or app upgrade you might do some housekeeping -- just as a standard procedure.
However in this case a 4+ year-old version of FCPX is crashing for unknown reasons. That will never be supported, investigated or fixed because it's so old.
My advice is just do some basic things, then upgrade to Mojave and FCPX 10.4.4 with media "as is". *After* the upgrade, *then* make the media changes.
Basic things to do. These are just my opinion:
- Run Disk Utility First Aid on all drives (inc'l system volume) and make sure no errors are reported
- All FCPX media volumes should be HFS+ (or nowadays optionally APFS), but if by chance he has some that are exFAT and not HFS+, convert those to HFS+ first. That will require an additional drive, and a copy or clone operation, as I don't know (and wouldn't trust) an "in place" upgrade, even if it were possible.
- Rebuild all Spotlight indexes on all drives, which FCPX uses to rapidly find files:
- Make sure there is plenty of empty space on each hard drive, say 20%.
- Verify there are good file backups of the system volume and all media on all drives.
- Turn off FCPX background rendering
- Do not mount a bunch of libraries, then upgrade. It's possible to upgrade macOS and FCPX without those being mounted. Then after the upgrade the libraries can be mounted and upgraded one by one. This reduces the complexity of what happens during the upgrade.
@Guru/Joema, as a fresh boarder my first question answered within hours. Many thanks. I was recommended to take media out of the library file before backing up as it would make the backup of the library faster (indeed not smaller b/c as you said either way the data it has to be backed up). I can only agree with you on the basics. I have just done the HD repairs, ensured that no disk has less than 20% empty space, but I also just realize that indeed one drive was not MAC OS Journaled formatted (it was NT formatted). But here is the real issue: I tried to copy the entire data to a new Mac OS journal formatted hard drive, however, during the copying of the data I got Error Code -36. It's caused by one library file (large 200GB). I googled it and found that I need to use the command clean_dot in Terminal. I have executed the command in Terminal but I still get the same error. Any thoughts/ideas? The rest of the data is still now copying to this new hard dive (will take hours..) .
DANSGP wrote: ...I also just realize that indeed one drive was not MAC OS Journaled formatted (it was NT formatted). But here is the real issue: I tried to copy the entire data to a new Mac OS journal formatted hard drive, however, during the copying of the data I got Error Code -36. It's caused by one library file (large 200GB). I googled it and found that I need to use the command clean_dot in Terminal. I have executed the command in Terminal but I still get the same error. Any thoughts/ideas?...
This is a macOS, NTFS file system or hardware issue, which could well explain the FCPX crash during upgrade.
Are you using Finder to copy the data? It is not always the best tool, especially on back-level versions of macOS (you are on 10.11.6). You could use the terminal command rsync but I'd still be concerned about NTFS, the state of the file system on that drive, etc.
There are three complications (1) You have a 200GB library on NTFS and (2) Using Finder to copy the data and (3) You got errors when trying to copy that to an HFS+ drive.
MacOS supports read-only access to NTFS, but I don't know how well that is tested, whether Disk Utility First Aid works on NTFS, how that (combined with El Capitan Finder) handles file bundles, permissions or "hard links" which FCPX uses a lot, etc.
The error 36 is associated with the "dot clean" issue but we don't know for certain the cause of the failure during copy. It might be almost nothing or it could be a Finder bug handling NTFS or it could be a file system problem or a hardware problem. I'd be cautious about "poking around" and trying stuff on that drive without first having a block-level backup.
The most conservative approach is treat the NTFS drive as "questionable" and first make a block-level clone, but I don't know what Mac 3rd-party utilities will do that for NTFS. You might take it to a Windows machine but you'd have to get a block or sector-level clone utility there also. On Windows you could try CHKDSK but I'd be afraid to try anything without a block-level backup.
Paragon also makes a block-level backup tool that's part of their Hard Disk Manager product:
You might also need their NTFS product to use this on an NTFS drive; I don't know.
Another option is just defer addressing this since it is unrelated to the macOS or FCPX update. Unplug the NTFS drive, do the macOS update, then the FCPX update, then update your other libraries. Then you could try a Finder copy from NTFS to an HFS+ drive. If it fails you'll be no worse off but at least on current software.
Doing "man dot_clean" from terminal shows the documentation for that command.
The problem is that metadata might be needed for something. The recommendation to just clean those might not envision how FCPX uses HFS+ and how the file metadata is preserved when copying from NTFS to HFS+. But if's already deleted, it's too late. OTOH the dot_clean -m argument implies it might not erase those unless specifically requested.
The "merge" aspect of dot_clean implies it's designed to merge the metadata in the "._*" files to the data fork of files on an HFS+ volume. It's possible that dot_clean does not do a file system check and is silently failing when run on NTFS.
It might be possible to copy the NTFS volume to an HFS+ volume using Carbon Copy, the terminal rsync command, or some other app. Just trying to copy it seems relatively safe since that should be read-only on the NTFS volume. If the copy works, you could *then* try dot_clean (if needed) on the HFS+ volume.
I just tested rsync on a 220GB library and it seemed to work. However this was HFS+ to HFS+, not NTFS to HFS+. The below command would not necessarily copy all the "._*" files which might be later needed to merge into the data fork for the files on HFS+. Maybe the library would work without merging those with dot_clean on HFS+. I suppose you could copy the entire volume with rsync or Carbon Copy.
Here is a discussion page about rsync. Note the syntax stated here requires modification to work on a FCPX library:
I first added the --dry-run argument to verify I had the syntax correct, then removed that and did the rsync copy. The syntax I used was below. Note: you must use the -p argument to preserve permissions and the target location does not require a folder/library name, as it will automatically be created. I don't know how permissions would be handled for NTFS to HFS+.
Re verification when copying a library or folder, I often use Beyond Compare to compare and sync files and folders. It also works on FCPX libraries -- at least on HFS+, I've never tested it between NTFS and HFS+. Beyond Compare is a 32-bit app so you'll get an error on Mojave but it still works OK. They plan on having a 64-bit version by the time the next major macOS version ships:
Beyond Compare is a very powerful, flexible tool and can also do "flat structure orphan verification", where it checks that a list of files on one drive all exist somewhere on another drive having a different folder structure.