Add option --entries to CLI utility

README.md

@@ -28,17 +28,18 @@ Learn how to use optilab by using our demo notebook. See `demo/tutorial.ipynb`.
 ## CLI tool
 Optilab comes with a powerful CLI tool to easily summarize your experiments. It allows for plotting the results and performing statistical testing to check for statistical significance in optimization results.
 ```
-Optilab CLI utility.
-usage: python -m optilab [-h] [--hide_plots] [--test_y] [--test_evals] pickle_path
+usage: Optilab CLI utility. [-h] [--hide_plots] [--test_y] [--test_evals] [--entries ENTRIES [ENTRIES ...]] pickle_path
 
 positional arguments:
-  pickle_path   Path to pickle file or directory with optimization runs.
+  pickle_path           Path to pickle file or directory with optimization runs.
 
 options:
-  -h, --help    show this help message and exit
-  --hide_plots  Hide plots when running the script.
-  --test_y      Perform Mann-Whitney U test on y values.
-  --test_evals  Perform Mann-Whitney U test on eval values.
+  -h, --help            show this help message and exit
+  --hide_plots          Hide plots when running the script.
+  --test_y              Perform Mann-Whitney U test on y values.
+  --test_evals          Perform Mann-Whitney U test on eval values.
+  --entries ENTRIES [ENTRIES ...]
+                        Space separated list of indexes of entries to include in analysis.
 ```
 
 ## Docker

src/optilab/__main__.py

@@ -61,6 +61,12 @@ def mann_whitney_u_test_grid(data_lists: List[List[float]]) -> str:
         action="store_true",
         help="Perform Mann-Whitney U test on eval values.",
     )
+    parser.add_argument(
+        "--entries",
+        nargs="+",
+        type=int,
+        help="Space separated list of indexes of entries to include in analysis.",
+    )
     args = parser.parse_args()
 
     file_path_list = []
@@ -78,6 +84,9 @@ def mann_whitney_u_test_grid(data_lists: List[List[float]]) -> str:
 
         data = load_from_pickle(file_path)
 
+        if args.entries:
+            data = [data[i] for i in args.entries if 0 <= i < len(data)]
+
         assert isinstance(data, list)
         for run in data:
             assert isinstance(run, OptimizationRun)