Universal loss function evaluated with PerAtomMAE

[gabor1]

Discussed in #25

^{Originally posted by doouv February 19, 2025}
Hi, I am currently experimenting with MACE and noticed something unexpected regarding error table selection when using the universal loss function.

According to mace/tools/model_script_utils.py, setting loss="universal" should automatically enable compute_stress=True and change error_table to "PerAtomRMSEstressvirials".

def configure_model(
    args, train_loader, atomic_energies, model_foundation=None, heads=None, z_table=None
):
    # Selecting outputs
    compute_virials = args.loss in ("stress", "virials", "huber", "universal")
    if compute_virials:
        args.compute_stress = True
        args.error_table = "PerAtomRMSEstressvirials"

However, the 2024-01-07-mace-128-L2.sh script uses the universal loss function while computing stress, yet the MACE-MP model seems to have been evaluated using "PerAtomMAE". Additionally, the results in the MACE paper appear to report errors in MAE/atom rather than RMSE.

My Questions:

• Why does MACE-MP retain PerAtomMAE for evaluation, even though universal loss with stress computation should default to PerAtomRMSEstressvirials?
• If energy and force prediction are the primary objectives, is it necessary to use the universal loss function, or would weighted loss function be more appropriate?

I would appreciate any insights you can provide. Thanks for your time!

[ilyes319]

• This is just an old bug, we fixed that in the latest main.
• The universal loss is useful when your target forces and energies span a wide range of values, and are potentially noisy. For most datasets it is best to use the MSE loss.