ultralytics 8.0.114 automatic optimizer selection (#3037)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Burhan <62214284+Burhan-Q@users.noreply.github.com>

.github/workflows/greetings.yml

@@ -32,6 +32,8 @@ jobs:
 
             If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our [Tips for Best Training Results](https://docs.ultralytics.com/yolov5/tutorials/tips_for_best_training_results/).
 
+            Join the vibrant [Ultralytics Discord](https://discord.gg/YVsATxj6wr) 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.
+
             ## Install
 
             Pip install the `ultralytics` package including all [requirements](https://github.com/ultralytics/ultralytics/blob/main/requirements.txt) in a [**Python>=3.7**](https://www.python.org/) environment with [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/).

.pre-commit-config.yaml

@@ -22,7 +22,7 @@ repos:
       - id: detect-private-key
 
   - repo: https://github.com/asottile/pyupgrade
-    rev: v3.3.2
+    rev: v3.4.0
     hooks:
       - id: pyupgrade
         name: Upgrade code

docs/modes/train.md

@@ -72,7 +72,7 @@ task.
 | `name`            | `None`   | experiment name                                                                   |
 | `exist_ok`        | `False`  | whether to overwrite existing experiment                                          |
 | `pretrained`      | `False`  | whether to use a pretrained model                                                 |
-| `optimizer`       | `'SGD'`  | optimizer to use, choices=[SGD, Adam, Adamax, AdamW, NAdam, RAdam, RMSProp, auto] |
+| `optimizer`       | `'auto'` | optimizer to use, choices=[SGD, Adam, Adamax, AdamW, NAdam, RAdam, RMSProp, auto] |
 | `verbose`         | `False`  | whether to print verbose output                                                   |
 | `seed`            | `0`      | random seed for reproducibility                                                   |
 | `deterministic`   | `True`   | whether to enable deterministic mode                                              |
@@ -102,3 +102,60 @@ task.
 | `mask_ratio`      | `4`      | mask downsample ratio (segment train only)                                        |
 | `dropout`         | `0.0`    | use dropout regularization (classify train only)                                  |
 | `val`             | `True`   | validate/test during training                                                     |
+
+## Logging
+
+In training a YOLOv8 model, you might find it valuable to keep track of the model's performance over time. This is where logging comes into play. Ultralytics' YOLO provides support for three types of loggers - Comet, ClearML, and TensorBoard.
+
+To use a logger, select it from the dropdown menu in the code snippet above and run it. The chosen logger will be installed and initialized.
+
+### Comet
+
+[Comet](https://www.comet.ml/site/) is a platform that allows data scientists and developers to track, compare, explain and optimize experiments and models. It provides functionalities such as real-time metrics, code diffs, and hyperparameters tracking.
+
+To use Comet:
+
+```python
+# pip install comet_ml
+import comet_ml
+
+comet_ml.init()
+```
+
+Remember to sign in to your Comet account on their website and get your API key. You will need to add this to your environment variables or your script to log your experiments.
+
+### ClearML
+
+[ClearML](https://www.clear.ml/) is an open-source platform that automates tracking of experiments and helps with efficient sharing of resources. It is designed to help teams manage, execute, and reproduce their ML work more efficiently.
+
+To use ClearML:
+
+```python
+# pip install clearml
+import clearml
+
+clearml.browser_login()
+```
+
+After running this script, you will need to sign in to your ClearML account on the browser and authenticate your session.
+
+### TensorBoard
+
+[TensorBoard](https://www.tensorflow.org/tensorboard) is a visualization toolkit for TensorFlow. It allows you to visualize your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it.
+
+To use TensorBoard in [Google Colab](https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb):
+
+```bash
+load_ext tensorboard
+tensorboard --logdir ultralytics/runs  # replace with 'runs' directory
+```
+
+To use TensorBoard locally run the below command and view results at http://localhost:6006/.
+
+```bash
+tensorboard --logdir ultralytics/runs  # replace with 'runs' directory
+```
+
+This will load TensorBoard and direct it to the directory where your training logs are saved.
+
+After setting up your logger, you can then proceed with your model training. All training metrics will be automatically logged in your chosen platform, and you can access these logs to monitor your model's performance over time, compare different models, and identify areas for improvement.

docs/tasks/classify.md

@@ -6,8 +6,7 @@ description: Check YOLO class label with only one class for the whole image, usi
 Image classification is the simplest of the three tasks and involves classifying an entire image into one of a set of
 predefined classes.
 
-<br>
-<img width="1024" src="https://raw.githubusercontent.com/ultralytics/assets/tasks/im/banner-tasks.png">
+<img width="1024" src="https://user-images.githubusercontent.com/26833433/243418606-adf35c62-2e11-405d-84c6-b84e7d013804.png">
 
 The output of an image classifier is a single class label and a confidence score. Image
 classification is useful when you need to know only what class an image belongs to and don't need to know where objects

docs/tasks/detect.md

@@ -5,8 +5,7 @@ description: Learn how to use YOLOv8, an object detection model pre-trained with
 
 Object detection is a task that involves identifying the location and class of objects in an image or video stream.
 
-<br>
-<img width="1024" src="https://raw.githubusercontent.com/ultralytics/assets/tasks/im/banner-tasks.png">
+<img width="1024" src="https://user-images.githubusercontent.com/26833433/243418624-5785cb93-74c9-4541-9179-d5c6782d491a.png">
 
 The output of an object detector is a set of bounding boxes that enclose the objects in the image, along with class labels and confidence scores for each box. Object detection is a good choice when you need to identify objects of interest in a scene, but don't need to know exactly where the object is or its exact shape.
 

docs/tasks/pose.md

@@ -8,7 +8,7 @@ to as keypoints. The keypoints can represent various parts of the object such as
 features. The locations of the keypoints are usually represented as a set of 2D `[x, y]` or 3D `[x, y, visible]`
 coordinates.
 
-<img width="1024" src="https://user-images.githubusercontent.com/26833433/239691398-d62692dc-713e-4207-9908-2f6710050e5c.jpg">
+<img width="1024" src="https://user-images.githubusercontent.com/26833433/243418616-9811ac0b-a4a7-452a-8aba-484ba32bb4a8.png">
 
 The output of a pose estimation model is a set of points that represent the keypoints on an object in the image, usually
 along with the confidence scores for each point. Pose estimation is a good choice when you need to identify specific

docs/tasks/segment.md

@@ -6,8 +6,7 @@ description: Learn what Instance segmentation is. Get pretrained YOLOv8 segment
 Instance segmentation goes a step further than object detection and involves identifying individual objects in an image
 and segmenting them from the rest of the image.
 
-<br>
-<img width="1024" src="https://raw.githubusercontent.com/ultralytics/assets/tasks/im/banner-tasks.png">
+<img width="1024" src="https://user-images.githubusercontent.com/26833433/243418644-7df320b8-098d-47f1-85c5-26604d761286.png">
 
 The output of an instance segmentation model is a set of masks or
 contours that outline each object in the image, along with class labels and confidence scores for each object. Instance

docs/usage/cfg.md

@@ -94,7 +94,7 @@ The training settings for YOLO models encompass various hyperparameters and conf
 | `name`            | `None`   | experiment name                                                                   |
 | `exist_ok`        | `False`  | whether to overwrite existing experiment                                          |
 | `pretrained`      | `False`  | whether to use a pretrained model                                                 |
-| `optimizer`       | `'SGD'`  | optimizer to use, choices=[SGD, Adam, Adamax, AdamW, NAdam, RAdam, RMSProp, auto] |
+| `optimizer`       | `'auto'` | optimizer to use, choices=[SGD, Adam, Adamax, AdamW, NAdam, RAdam, RMSProp, auto] |
 | `verbose`         | `False`  | whether to print verbose output                                                   |
 | `seed`            | `0`      | random seed for reproducibility                                                   |
 | `deterministic`   | `True`   | whether to enable deterministic mode                                              |

ultralytics/__init__.py

@@ -1,6 +1,6 @@
 # Ultralytics YOLO 🚀, AGPL-3.0 license
 
-__version__ = '8.0.113'
+__version__ = '8.0.114'
 
 from ultralytics.hub import start
 from ultralytics.vit.rtdetr import RTDETR

ultralytics/yolo/cfg/default.yaml

@@ -20,7 +20,7 @@ project:  # project name
 name:  # experiment name, results saved to 'project/name' directory
 exist_ok: False  # whether to overwrite existing experiment
 pretrained: False  # whether to use a pretrained model
-optimizer: SGD  # optimizer to use, choices=[SGD, Adam, Adamax, AdamW, NAdam, RAdam, RMSProp, auto]
+optimizer: auto  # optimizer to use, choices=[SGD, Adam, Adamax, AdamW, NAdam, RAdam, RMSProp, auto]
 verbose: True  # whether to print verbose output
 seed: 0  # random seed for reproducibility
 deterministic: True  # whether to enable deterministic mode

ultralytics/yolo/engine/trainer.py

@@ -627,8 +627,10 @@ class BaseTrainer:
         g = [], [], []  # optimizer parameter groups
         bn = tuple(v for k, v in nn.__dict__.items() if 'Norm' in k)  # normalization layers, i.e. BatchNorm2d()
         if name == 'auto':
-            name, lr, momentum = ('SGD', 0.01, 0.9) if iterations > 6000 else ('NAdam', 0.001, 0.9)
-            self.args.warmup_bias_lr = 0.0  # no higher than 0.01 for NAdam
+            nc = getattr(model, 'nc', 10)  # number of classes
+            lr_fit = round(0.002 * 5 / (4 + nc), 6)  # lr0 fit equation to 6 decimal places
+            name, lr, momentum = ('SGD', 0.01, 0.9) if iterations > 10000 else ('AdamW', lr_fit, 0.9)
+            self.args.warmup_bias_lr = 0.0  # no higher than 0.01 for Adam
 
         for module_name, module in model.named_modules():
             for param_name, param in module.named_parameters(recurse=False):