Feature: #59312 - Introduce cleanup command for missing FAL records 

See forge#59312

Description 

Files that are deleted outside of TYPO3, for example via SSH or FTP, or that are gone after restoring a database dump without the matching file system, keep their records in sys_file and sys_file_metadata . The storage indexer flags these records as missing ( sys_file.missing = 1 ) and the reports module lists them under "Files flagged as missing", but so far nothing in TYPO3 could remove them. Deleting a missing file through the File Abstraction Layer API is rejected as well, so the only way out was to restore the file and delete it again, or to clean the database by hand.

The new CLI command cleanup:missingfiles of typo3/cms-lowlevel closes this gap. It takes over the name of the command that was removed in TYPO3 v12 together with the uploads/ folder handling, as the File Abstraction Layer is the only file handling left.

The command

  • optionally updates the reference index ( --update-refindex ), as sys_refindex decides whether a file is still referenced,
  • optionally updates the storage index of one or more file storages ( --update-storage=<uid> , repeatable, or --update-storage=all ), so files deleted outside of TYPO3 are flagged as missing first,
  • finds all sys_file records flagged as missing,
  • keeps the records that are still referenced by any record, including soft references such as t3://file?uid= links, and lists them in verbose mode, so that the file can be restored or the references can be removed,
  • removes missing files without references the same way as a file deleted via TYPO3: the sys_file and sys_file_metadata records, their category relations, their processed files (records and files in the processing folder) and left-over deleted file references are removed, and the reference index entries are updated.

Running the command 

Show which records would be removed, after updating the reference index and the index of storage 1:

vendor/bin/typo3 cleanup:missingfiles --dry-run -v --update-refindex --update-storage=1
Copied!

Remove the records:

vendor/bin/typo3 cleanup:missingfiles --update-refindex --update-storage=all
Copied!

Without --update-refindex and --update-storage the command asks in interactive mode whether both indexes should be updated, and assumes them to be up to date in non-interactive mode ( -n ).

Impact 

Administrators can remove the database records of files that are flagged as missing and are no longer referenced with the new CLI command cleanup:missingfiles , instead of manipulating the database by hand.