Synset.txt generation

[jesse-schallenberger]

Question

I am curious about the proper generation of the synset.txt file for custom model loading. To begin with a toy problem I began with 17 flowers, if that helps in any way.

The question is, what is the proper way to generate the synset.txt file?

[frankfliu]

@jesse-schallenberger The sysnet doesn't have to be a file. It can be List. See: https://github.com/awslabs/djl/blob/master/api/src/main/java/ai/djl/modality/cv/translator/BaseImageTranslator.java#L259-L266
You can build your ImageTranslator with optSynset()

Synset usually comes with your training dataset, see: https://github.com/awslabs/djl/blob/master/basicdataset/src/main/java/ai/djl/basicdataset/AbstractImageFolder.java#L90-L93

If you use ImageFolder as your dataset, you can get it already.

[lanking520]

It looks like this: https://mlrepo.djl.ai/model/cv/image_classification/ai/djl/mxnet/synset.txt

Just use \n to separate your classes and saved to a txt namely synset.txt.

[jesse-schallenberger]

It looks like this: https://mlrepo.djl.ai/model/cv/image_classification/ai/djl/mxnet/synset.txt

Just use \n to separate your classes and saved to a txt namely synset.txt.

Regarding the entries here in the listed file, they seem to be consistently given as similar to n01440764 tench, Tinca tinca for instance on line 1. I'm curious about the first value listed there, what it might mean. Is this common for synset? It is not simply numerical and the class label? Is it problematic to simply utilize the class labels?

[frankfliu]

@jesse-schallenberger n01440764 tench, Tinca tinca is specific to ImageNet, it contains three part, the first column is the ID, 2nd is the short name, and then the last is the full name of the class.

[jesse-schallenberger]

If the NN mapping is integer to class name, would clarification be possible for what the synset should look like in that case? Would both values be needed per line?

[frankfliu]

use ImageNet as example, it contains 1000 classes, each line is the label of the class, the content/format of the label doesn't really matter to the deep learning. The output of NN contains 1000 floating numbers, each of value represent the probability of corresponding class.

[jesse-schallenberger]

To summarize, it seems the synset is required for inference and a 1:1 mapping of the labels is needed for proper functionality. It seems to make the most sense to leverage proper labels, but ultimately the values themselves don't matter? I have found the case of an empty synset, and also no synset to cause errors trying to run inference.

[frankfliu]

For a classification use case, it's required you provide a 1:1 mapping between index to the label. otherwise your output processing will hit ArrayIndexOutOfBoundException.
If you use DJL's default synset loader, you cannot leave the value empty. DJL will remove empty lines in the synset.txt file by default. You can manually load synset.txt file and use BaseImageTranslator.Builder.optSynset(List<String> classes).

https://github.com/awslabs/djl/blob/master/api/src/main/java/ai/djl/modality/cv/translator/BaseImageTranslator.java#L274