[Question] Clean old file with Robinhood

[garadar]

Description

I am deploying a Robinhood instance to scan the filesystem and purge unused "old files." We define old files as those that have not been read or written in the last 90 days.

Since BeeGFS is not Lustre (thank you, Captain Obvious 👑), Robinhood needs to scan the entire filesystem to update its information instead of parsing filesystem logs.

Here are my current rules:

rule old_files {
    target_fileclass = user_file;
    condition { last_access > 90d }
    action = common.unlink
}

However, BeeGFS is mounted with the relatime flag, as shown below:

(cluster)-[root@robinhood ~]$ mount | grep /home
beegfs_home on /home type beegfs (rw,relatime,cfgFile=/etc/beegfs/home.d/beegfs-client.conf,_netdev)

Problem

According to the [BeeGFS documentation](https://doc.beegfs.io/latest/advanced_topics/storage_tuning.html#mount-options):

Enabling last file access time is inefficient because the file system needs to update the time stamp by writing data to the disk even in cases when the user only reads file contents or when the file contents have already been cached in memory, and no disk access would have been necessary at all.

Since relatime is enabled, last_access timestamps may not be updated accurately when files are read, leading to potential unintended deletions of recently accessed files.

Question

How can I properly handle this situation to ensure only truly unused files are deleted, without accidentally removing files that have been read but not modified?

[garadar]

According to Redhat 6 (probably valid for higher versions):

relatime maintains atime data, but not for each time that a file is accessed. With this option enabled, atime data is written to the disk only if the file has been modified since the atime data was last updated (mtime), or if the file was last accessed more than a certain amount of time ago (by default, one day).

[iamjoemccormick]

Given your policy is to purge files not accessed in 90 days does relatime work for your purposes?

This does work as expected with BeeGFS. For example given a file last accessed 2025-01-31 if I cat the file the access time is updated:

$ mount | grep -i bee
beegfs_nodev on /mnt/beegfs type beegfs (rw,relatime,cfgFile=/etc/beegfs/beegfs-client.conf,sysMgmtdHost=127.0.0.1,_netdev)

$ stat image_001.jpg 
  File: image_001.jpg
  Size: 5120      	Blocks: 10         IO Block: 524288 regular file
Device: 30h/48d	Inode: 2161091563190622603  Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/     joe)   Gid: ( 1000/     joe)
Access: 2025-01-31 16:56:37.000000000 +0000
Modify: 2024-11-14 21:54:52.000000000 +0000
Change: 2024-11-14 21:54:52.000000000 +0000
 Birth: -

$ cat image_001.jpg >> /dev/null

$ stat image_001.jpg 
  File: image_001.jpg
  Size: 5120      	Blocks: 10         IO Block: 524288 regular file
Device: 30h/48d	Inode: 2161091563190622603  Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/     joe)   Gid: ( 1000/     joe)
Access: 2025-02-27 14:32:33.000000000 +0000
Modify: 2024-11-14 21:54:52.000000000 +0000
Change: 2024-11-14 21:54:52.000000000 +0000
 Birth: -

Note if you wanted to set strictatime instead you would need to set set this on both your BeeGFS mount using beegfs-mounts.conf and the underlying file system(s) used for your storage targets.

[ttys42]

Since BeeGFS is not Lustre (thank you, Captain Obvious 👑), Robinhood needs to scan the entire filesystem to update its information instead of parsing filesystem logs.

Have you had a look at https://www.beegfs.io/c/home/hive/ ?

[garadar]

Yeah, but Robinhood can apply policies during the scan, filter by the file_class you have defined, and offer graphs to provide a better overview.

Or maybe I don't know all the options of BeeGFS Hive.