About
Panoptic Quality (PQ) is an evaluation metric for panoptic segmentation, designed to integrate and assess the quality of both semantic segmentation and instance segmentation. This metric measures how well a single image’s scene understanding task can accurately identify and classify every object present.
Calculation of PQ
PQ comprises two main components:
- Segmentation Quality (SQ):
- This measures how accurately the pixels of a predicted segment align with the boundaries. Specifically, it is assessed by the average Intersection over Union (IoU) between the predicted segments and the actual (ground truth) segments.
- Recognition Quality (RQ):
- This evaluates how accurately objects are assigned to the correct class. It is calculated based on the accuracy of object identification and classification, and how many matching predictions exist for each actual object.
PQ Formula
The formula for PQ is as follows:
PQ = \frac{\sum_{(p,g) \in TP} IoU(p,g)}{|TP| + \frac{1}{2} |FP| + \frac{1}{2} |FN|}Where:
- (TP) are True Positives (correctly identified instances),
- (FP) are False Positives (incorrectly identified instances),
- (FN) are False Negatives (missed instances).
- (IoU(p,g)) is the total intersected area between the predicted instance (p) and the actual instance (g), divided by their total union area.
Caution
In segmentation tasks focused on general object recognition, PQ (Panoptic Quality) is a good metric for evaluating models. However, if the task is more aligned with semantic segmentation, particularly focusing more on “stuff” than “things,” this evaluation may not be appropriate.
