mirror of
https://github.com/THU-MIG/yolov10.git
synced 2025-05-28 01:24:22 +08:00
Update SAM docs page (#3672)
This commit is contained in:
parent
1ae7f84394
commit
b239246452
@ -30,13 +30,30 @@ For an in-depth look at the Segment Anything Model and the SA-1B dataset, please
|
|||||||
|
|
||||||
The Segment Anything Model can be employed for a multitude of downstream tasks that go beyond its training data. This includes edge detection, object proposal generation, instance segmentation, and preliminary text-to-mask prediction. With prompt engineering, SAM can swiftly adapt to new tasks and data distributions in a zero-shot manner, establishing it as a versatile and potent tool for all your image segmentation needs.
|
The Segment Anything Model can be employed for a multitude of downstream tasks that go beyond its training data. This includes edge detection, object proposal generation, instance segmentation, and preliminary text-to-mask prediction. With prompt engineering, SAM can swiftly adapt to new tasks and data distributions in a zero-shot manner, establishing it as a versatile and potent tool for all your image segmentation needs.
|
||||||
|
|
||||||
```python
|
!!! example "SAM prediction example"
|
||||||
from ultralytics import SAM
|
|
||||||
|
|
||||||
model = SAM('sam_b.pt')
|
Device is determined automatically. If a GPU is available then it will be used, otherwise inference will run on CPU.
|
||||||
model.info() # display model information
|
|
||||||
model.predict('path/to/image.jpg') # predict
|
=== "Python"
|
||||||
```
|
|
||||||
|
```python
|
||||||
|
from ultralytics import SAM
|
||||||
|
|
||||||
|
# Load a model
|
||||||
|
model = SAM('sam_b.pt')
|
||||||
|
|
||||||
|
# Display model information (optional)
|
||||||
|
model.info()
|
||||||
|
|
||||||
|
# Run inference with the model
|
||||||
|
model('path/to/image.jpg')
|
||||||
|
```
|
||||||
|
=== "CLI"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run inference with a SAM model
|
||||||
|
yolo predict model=sam_b.pt source=path/to/image.jpg
|
||||||
|
```
|
||||||
|
|
||||||
## Available Models and Supported Tasks
|
## Available Models and Supported Tasks
|
||||||
|
|
||||||
@ -53,6 +70,33 @@ model.predict('path/to/image.jpg') # predict
|
|||||||
| Validation | :x: |
|
| Validation | :x: |
|
||||||
| Training | :x: |
|
| Training | :x: |
|
||||||
|
|
||||||
|
## SAM comparison vs YOLOv8
|
||||||
|
|
||||||
|
Here we compare Meta's smallest SAM model, SAM-b, with Ultralytics smallest segmentation model, [YOLOv8n-seg](../tasks/segment):
|
||||||
|
|
||||||
|
| Model | Size | Parameters | Speed (CPU) |
|
||||||
|
|---------------------------------------------|----------------------------|------------------------|-------------------------|
|
||||||
|
| Meta's SAM-b | 358 MB | 94.7 M | 51096 ms |
|
||||||
|
| Ultralytics [YOLOv8n-seg](../tasks/segment) | **6.7 MB** (53.4x smaller) | **3.4 M** (27.9x less) | **59 ms** (866x faster) |
|
||||||
|
|
||||||
|
This comparison shows the order-of-magnitude differences in the model sizes and speeds. Whereas SAM presents unique capabilities for automatic segmenting, it is not a direct competitor to YOLOv8 segment models, which are smaller, faster and more efficient since they are dedicated to more targeted use cases.
|
||||||
|
|
||||||
|
To reproduce this test:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from ultralytics import SAM, YOLO
|
||||||
|
|
||||||
|
# Profile SAM-b
|
||||||
|
model = SAM('sam_b.pt')
|
||||||
|
model.info()
|
||||||
|
model('ultralytics/assets')
|
||||||
|
|
||||||
|
# Profile YOLOv8n-seg
|
||||||
|
model = YOLO('yolov8n-seg.pt')
|
||||||
|
model.info()
|
||||||
|
model('ultralytics/assets')
|
||||||
|
```
|
||||||
|
|
||||||
## Auto-Annotation: A Quick Path to Segmentation Datasets
|
## Auto-Annotation: A Quick Path to Segmentation Datasets
|
||||||
|
|
||||||
Auto-annotation is a key feature of SAM, allowing users to generate a [segmentation dataset](https://docs.ultralytics.com/datasets/segment) using a pre-trained detection model. This feature enables rapid and accurate annotation of a large number of images, bypassing the need for time-consuming manual labeling.
|
Auto-annotation is a key feature of SAM, allowing users to generate a [segmentation dataset](https://docs.ultralytics.com/datasets/segment) using a pre-trained detection model. This feature enables rapid and accurate annotation of a large number of images, bypassing the need for time-consuming manual labeling.
|
||||||
|
@ -6,4 +6,4 @@ keywords: RTDETR, Ultralytics, YOLO, object detection, speed, accuracy, implemen
|
|||||||
## RTDETR
|
## RTDETR
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.rtdetr.model.RTDETR
|
### ::: ultralytics.vit.rtdetr.model.RTDETR
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -6,4 +6,4 @@ keywords: RTDETRPredictor, object detection, vision transformer, Ultralytics YOL
|
|||||||
## RTDETRPredictor
|
## RTDETRPredictor
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.rtdetr.predict.RTDETRPredictor
|
### ::: ultralytics.vit.rtdetr.predict.RTDETRPredictor
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -11,4 +11,4 @@ keywords: RTDETRTrainer, Ultralytics YOLO Docs, object detection, VIT-based RTDE
|
|||||||
## train
|
## train
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.rtdetr.train.train
|
### ::: ultralytics.vit.rtdetr.train.train
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -11,4 +11,4 @@ keywords: RTDETRDataset, RTDETRValidator, data validation, documentation
|
|||||||
## RTDETRValidator
|
## RTDETRValidator
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.rtdetr.val.RTDETRValidator
|
### ::: ultralytics.vit.rtdetr.val.RTDETRValidator
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -86,4 +86,4 @@ keywords: Ultralytics, SAM, MaskData, mask_to_rle_pytorch, area_from_rle, genera
|
|||||||
## batched_mask_to_box
|
## batched_mask_to_box
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.amg.batched_mask_to_box
|
### ::: ultralytics.vit.sam.amg.batched_mask_to_box
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -6,4 +6,4 @@ keywords: ResizeLongestSide, Ultralytics YOLO, automatic image resizing, image r
|
|||||||
## ResizeLongestSide
|
## ResizeLongestSide
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.autosize.ResizeLongestSide
|
### ::: ultralytics.vit.sam.autosize.ResizeLongestSide
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -26,4 +26,4 @@ keywords: SAM, VIT, computer vision models, build SAM models, build VIT models,
|
|||||||
## build_sam
|
## build_sam
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.build.build_sam
|
### ::: ultralytics.vit.sam.build.build_sam
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -6,4 +6,4 @@ keywords: Ultralytics, VIT, SAM, object detection, computer vision, deep learnin
|
|||||||
## SAM
|
## SAM
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.model.SAM
|
### ::: ultralytics.vit.sam.model.SAM
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -1,3 +1,8 @@
|
|||||||
|
---
|
||||||
|
description: Learn about Ultralytics YOLO's MaskDecoder, Transformer architecture, MLP, mask prediction, and quality prediction.
|
||||||
|
keywords: Ultralytics YOLO, MaskDecoder, Transformer architecture, mask prediction, image embeddings, prompt embeddings, multi-mask output, MLP, mask quality prediction
|
||||||
|
---
|
||||||
|
|
||||||
## MaskDecoder
|
## MaskDecoder
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.modules.decoders.MaskDecoder
|
### ::: ultralytics.vit.sam.modules.decoders.MaskDecoder
|
||||||
|
@ -51,4 +51,4 @@ keywords: Ultralytics YOLO, ViT Encoder, Position Embeddings, Attention, Window
|
|||||||
## add_decomposed_rel_pos
|
## add_decomposed_rel_pos
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.modules.encoders.add_decomposed_rel_pos
|
### ::: ultralytics.vit.sam.modules.encoders.add_decomposed_rel_pos
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -6,4 +6,4 @@ keywords: SamAutomaticMaskGenerator, Ultralytics YOLO, automatic mask generator,
|
|||||||
## SamAutomaticMaskGenerator
|
## SamAutomaticMaskGenerator
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.modules.mask_generator.SamAutomaticMaskGenerator
|
### ::: ultralytics.vit.sam.modules.mask_generator.SamAutomaticMaskGenerator
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -6,4 +6,4 @@ keywords: PromptPredictor, Ultralytics, YOLO, VIT SAM, image captioning, deep le
|
|||||||
## PromptPredictor
|
## PromptPredictor
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.modules.prompt_predictor.PromptPredictor
|
### ::: ultralytics.vit.sam.modules.prompt_predictor.PromptPredictor
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -6,4 +6,4 @@ keywords: Ultralytics VIT, Sam module, PyTorch vision library, image classificat
|
|||||||
## Sam
|
## Sam
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.modules.sam.Sam
|
### ::: ultralytics.vit.sam.modules.sam.Sam
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -16,4 +16,4 @@ keywords: Ultralytics YOLO, Attention module, TwoWayTransformer module, Object D
|
|||||||
## Attention
|
## Attention
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.modules.transformer.Attention
|
### ::: ultralytics.vit.sam.modules.transformer.Attention
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -6,4 +6,4 @@ keywords: Ultralytics, VIT SAM Predictor, object detection, YOLO
|
|||||||
## Predictor
|
## Predictor
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.sam.predict.Predictor
|
### ::: ultralytics.vit.sam.predict.Predictor
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -11,4 +11,4 @@ keywords: DETRLoss, RTDETRDetectionLoss, Ultralytics, object detection, image cl
|
|||||||
## RTDETRDetectionLoss
|
## RTDETRDetectionLoss
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.utils.loss.RTDETRDetectionLoss
|
### ::: ultralytics.vit.utils.loss.RTDETRDetectionLoss
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -16,4 +16,4 @@ keywords: Ultralytics, YOLO, object detection, HungarianMatcher, inverse_sigmoid
|
|||||||
## inverse_sigmoid
|
## inverse_sigmoid
|
||||||
---
|
---
|
||||||
### ::: ultralytics.vit.utils.ops.inverse_sigmoid
|
### ::: ultralytics.vit.utils.ops.inverse_sigmoid
|
||||||
<br><br>
|
<br><br>
|
||||||
|
@ -23,6 +23,11 @@ keywords: Ultralytics YOLO, downloads, trained models, datasets, weights, deep l
|
|||||||
### ::: ultralytics.yolo.utils.downloads.safe_download
|
### ::: ultralytics.yolo.utils.downloads.safe_download
|
||||||
<br><br>
|
<br><br>
|
||||||
|
|
||||||
|
## get_github_assets
|
||||||
|
---
|
||||||
|
### ::: ultralytics.yolo.utils.downloads.get_github_assets
|
||||||
|
<br><br>
|
||||||
|
|
||||||
## attempt_download_asset
|
## attempt_download_asset
|
||||||
---
|
---
|
||||||
### ::: ultralytics.yolo.utils.downloads.attempt_download_asset
|
### ::: ultralytics.yolo.utils.downloads.attempt_download_asset
|
||||||
|
Loading…
x
Reference in New Issue
Block a user