Part-DINO: Complementing DINO Features with Image Structure for Part Discovery

Abstract

Self-supervised Vision Transformers (ViTs) like DINO encode rich semantics, enabling training-free segmentation of objects from the background and discovery of their parts via feature clustering. This paper focuses on part discovery: partitioning a foreground object into semantic parts. Existing approaches produce imprecise part boundaries. We trace this to the rigid ViT patch grid used in training DINO, where patches straddling boundaries contain pixels from multiple parts. While DINO’s invariance-based training could absorb this pixel-level mixing into clean features, experiments with over 26K foreground patches show it does not: boundary-overlapping patches produce features with higher entropy and lower assignment confidence, blurring predicted part boundaries.

Rather than make the massive effort of retraining DINO over geometrically coherent regions, an easier approach is to take boundaries directly from a boundary detector, which supplies the geometric information DINO’s features lack.

Motivated by this, we introduce Part-DINO, a training-free framework that combines boundaries from low-level image segmentation with DINO’s semantic features. We use the boundaries to partition the image into regions, pool DINO features within each region, and group them to discover parts. Without any training, Part-DINO produces structurally coherent part assignments, outperforming training-free baselines across four standard benchmarks (CUB, CelebA, PartImageNet-OOD, PartImageNet-Seg) and remaining competitive with weakly-supervised and unsupervised methods.

Why Do Part Boundaries Blur?

A fixed square patch can cross the boundary between two parts, mixing their pixels before feature extraction. We find that this mixing remains in DINO’s features, making part assignments less reliable near boundaries.

Bird input image — **Boundary patches mix neighboring parts.** A patch overlapping the bird’s head and body combines pixels from both. Edges from DINO similarity are spatially blurred, while image contrast supplies complementary boundary information.

A square patch overlaps the bird head and body — **Boundary patches mix neighboring parts.** A patch overlapping the bird’s head and body combines pixels from both. Edges from DINO similarity are spatially blurred, while image contrast supplies complementary boundary information.

Pixel Mixing Propagates into Feature Space

Across 26,459 foreground patches from PartImageNet, patches closer to a ground-truth part boundary have lower spatial purity, higher feature entropy, and a smaller gap between their two strongest part assignments. 21.9% of the foreground patches contain pixels from more than one part.

Patch purity increases with distance from a part boundary — **Feature ambiguity increases near boundaries.** As distance from the nearest part boundary increases, patch purity rises, feature entropy falls, and the top-two assignment margin grows. Spearman correlations with boundary distance are +0.71, −0.38, and +0.36, respectively.

Feature entropy decreases with distance from a part boundary — **Feature ambiguity increases near boundaries.** As distance from the nearest part boundary increases, patch purity rises, feature entropy falls, and the top-two assignment margin grows. Spearman correlations with boundary distance are +0.71, −0.38, and +0.36, respectively.

The Part-DINO Method

Part-DINO combines boundaries from low-level image segmentation with semantic features from a frozen DINO encoder. Regions replace square patches as the units of part discovery.

Part-DINO pipeline: structural region generation, global prototype generation, and region assignment — **Part-DINO overview.** We generate image regions, pool DINO features within them, and cluster reference-region features into global part prototypes. A new image is segmented into regions and each region is assigned to its nearest prototype.

Generate regions from image structure. Within a foreground mask, the RAMP transform estimates local intensity contrast. Watershed partitions this edge map into regions along its ridges.
Build global part prototypes. We upsample frozen DINO features, pool them within each region, and apply K-means to region features from reference images. The K cluster centers represent parts shared across object instances.
Assign regions to parts. For a new image, each region receives the label of its most similar part prototype under cosine similarity. All pixels in a region share the same label.

Implementation details and optional region merging

Experiments use frozen DINOv2 ViT-B/14 with registers and SAM3 foreground masks. Input resolution is 448 × 448 for CUB and 224 × 224 for the other datasets. RAMP uses four orientations, and watershed seeds have a minimum spacing of 2 pixels.

The default pipeline merges micro-regions smaller than 10 pixels into their most similar adjacent region using DINO cosine similarity. This step only removes existing boundaries; it does not introduce new boundaries. Setting the minimum region size to zero disables merging, leaving the RAMP + watershed partition unchanged.

“Training-free” means no dataset-specific neural-network optimization: the encoder stays frozen, while part prototypes are obtained through clustering. For multi-category datasets, the method maintains a separate part dictionary per category and selects the category using CLS-feature similarity.

Results

We evaluate on CUB, CelebA, PartImageNet-OOD, and PartImageNet-Seg. Part-DINO improves ARI over the foreground-matched patch baseline at every evaluated part granularity on all four benchmarks. It also improves NME on CUB and CelebA, and NMI on three of the four benchmarks.

CUB at K = 16

Against the DINO + SAM patch baseline, NMI increases from 56.92 to 65.45 (+8.53 points), ARI from 35.10 to 46.00 (+10.90 points), and NME decreases from 4.87 to 4.35.

CUB
Parts (K)	Method	NME ↓	NMI ↑	ARI ↑
4	DINO + SAM	9.92	39.77	17.94
4	Part-DINO	8.21	53.93	23.01
8	DINO + SAM	6.26	54.03	27.18
8	Part-DINO	6.00	61.84	37.92
16	DINO + SAM	4.87	56.92	35.10
16	Part-DINO	4.35	65.45	46.00

CelebA
Parts (K)	Method	NME ↓	NMI ↑	ARI ↑
4	DINO + SAM	11.36	1.38	0.01
4	Part-DINO	10.81	45.07	32.20
8	DINO + SAM	10.74	1.12	0.01
8	Part-DINO	10.59	41.53	28.59
16	DINO + SAM	11.42	1.99	0.03
16	Part-DINO	9.15	41.04	25.20

PartImageNet-OOD
Parts (K)	Method	NMI ↑	ARI ↑
8	DINO + SAM	32.86	54.77
8	Part-DINO	30.07	63.27
25	DINO + SAM	37.19	57.87
25	Part-DINO	31.14	64.93
50	DINO + SAM	40.41	59.99
50	Part-DINO	30.13	65.42

PartImageNet-Seg
Parts (K)	Method	NMI ↑	ARI ↑
8	DINO + SAM	25.59	57.69
8	Part-DINO	37.21	66.10
25	DINO + SAM	27.08	62.92
25	Part-DINO	41.54	66.94
50	DINO + SAM	32.12	64.73
50	Part-DINO	41.80	66.97

Selected training-free comparisons transcribed from the paper. “DINO + SAM” denotes the patch-based method of Amir et al., replicated with SAM foreground masks († in the paper). Lower NME is better; higher NMI and ARI are better. Bold marks an improved Part-DINO score within each comparison. See the paper for comparisons with weakly-supervised and unsupervised trained methods.

PartImageNet-OOD trade-off. ARI improves while NMI decreases across all three evaluated part granularities.

Qualitative Comparisons

Representative outputs from the paper compare Part-DINO with MPAE and a foreground-matched DINO patch baseline. The part counts are K = 16 for CUB and CelebA, and K = 50 for both PartImageNet splits.

Input on CUB — MPAE requires dataset-specific training. DINO + SAM and Part-DINO use frozen features without dataset-specific neural-network optimization. Colors visualize each method’s discovered clusters and should not be matched across methods.

Input on CelebA — MPAE requires dataset-specific training. DINO + SAM and Part-DINO use frozen features without dataset-specific neural-network optimization. Colors visualize each method’s discovered clusters and should not be matched across methods.

Citation

@article{rawlekar2026complementing,
  title={Complementing DINO Features with Image Structure for Part Discovery},
  author={Rawlekar, Samyak and Paleti, Nikhil Chowdary and Gupta, Amey and Ahuja, Narendra},
  journal={arXiv preprint arXiv:2610.32293},
  year={2026},
  url={https://arxiv.org/abs/2610.32293}
}

Acknowledgments

This work was supported in part by the Office of Naval Research under Grant N00014-24-1-2169, the IBM-Illinois Discovery Accelerator Institute (IIDAI), and the Amazon–Illinois Center on AI for Interactive Conversational Experiences. This research also utilized resources from the NCSA Delta system, supported by the National Science Foundation under Award OAC-2005572.