LPIPS loss can produce negative values

[jklimmek]

When calculating LPIPS you first square distances and then multiply by a 1x1Conv. Since 1x1Conv is frozen and randomly initialized it can produce negative results, which is not desired. So instead:

for i in range(len(self.channels)):
    diffs[i] = (norm_tensor(features_real[i]) - norm_tensor(features_fake[i])) ** 2

return sum([spatial_average(self.lins[i].model(diffs[i])) for i in range(len(self.channels))])

you should move **2 term after diffs are passed through 1x1Conv as so:

for i in range(len(self.channels)):
    diffs[i] = (norm_tensor(features_real[i]) - norm_tensor(features_fake[i]))

return sum([spatial_average(self.lins[i].model(diffs[i]) ** 2) for i in range(len(self.channels))])