MM-CAD: A Multi-Modal CAD Dataset and Benchmark for Cross-Modal Geometric Learning

MM-CAD overview: MM-CAD:A (33,816 models, human annotation) bootstraps MM-CAD:B (192,626 models curated from ABC with MAAS, Gemini captions, photorealistic renders, sketches, and an application ontology) — MM-CAD:A (left) provides human-annotated text and hand-drawn sketches over 33,816 models unified from eleven benchmarks; it bootstraps the curation and annotation of MM-CAD:B (right), 192,626 models distilled from the 1M-model ABC corpus.

Abstract

Computer-Aided Design (CAD) boosts modern manufacturing, yet design reuse remains constrained by the absence of large, openly available CAD repositories with rich multi-modal annotations suitable for search/retrieval. Recent large-scale efforts to annotate public datasets rely on hash-based redundancy removal that leaves no semantic structure, and on captioning by Vision-Language Models (VLMs) using rendered images alone, which struggles to capture geometric and procedural information. We introduce MM-CAD, a multi-modal CAD dataset designed to level-up retrieval and retrieval-augmented generation models for engineering geometry, comprising two complementary parts. MM-CAD:A brings 33,816 unique CAD models from eleven widely used benchmark datasets under a common identifier scheme, with isometric renderings, point clouds, and humanly-curated multi-level text captions, and 4,376 real hand-drawn user sketches among others. MM-CAD:B curates 192,626 models from the 1M-model ABC corpus through a seven-stage pipeline centered on Manifold-Aware Adaptive Sampling (MAAS), which organizes models into semantically coherent neighborhoods rather than merely removing duplicates, directly supplying the hard negatives that contrastive retrieval training requires. Every retained model is annotated through a metadata-grounded pipeline that conditions caption generation on parsed construction sequences rather than rendered views alone, producing three-level text descriptions, multi-level contour sketches, a hierarchical application taxonomy, and photorealistic in-context images that largely preserve source CAD geometry, a modality not previously available at this scale on CAD data. We further introduce a joint retrieval architecture that aligns sketch, text, image, B-Rep, and point cloud encoders in a single latent space through Matryoshka-nested contrastive objectives, establishing the first unified cross-modal retrieval benchmark for large-scale CAD.

Modalities

Every model is paired with geometric and human-friendly query modalities under a single identifier space:

Render

Point cloud

B-Rep

Sketch (light)

Sketch (contour)

Photoreal image

Dataset

MM-CAD:A LIVE ON 🤗

33,816 models unified from eleven benchmarks: MCB, CADNET, ESB, PSB, ShapeNet, ModelNet, DeepCAD, Fusion360 Gallery, CADParser, IFCNet, Thingi10K.

Isometric renders (3 viewpoints) + meshes
10K-point clouds with oriented normals
Multi-level sketches incl. 3,276 real hand-drawn + 1,100 traced sketches
Human-validated multi-level captions (Title / Description / Class)
80:10:10 split — 27,048 / 3,376 / 3,392

MM-CAD:B UPLOAD IN PROGRESS

192,626 models curated from the 1M-model ABC corpus via a seven-stage pipeline centered on Manifold-Aware Adaptive Sampling (MAAS).

B-Rep (STEP) + point clouds + renders
Construction-sequence-grounded captions (parsed FeatureScript)
Multi-level contour sketches
Photorealistic in-context images for ~131K models
Hierarchical application taxonomy (4,862 nodes, recursive DPGMM)
FAISS neighborhood graph for hard-negative mining

Hugging Face upload of the full MM-CAD:B is in progress and will appear in the same dataset repository.

Downloads: Hugging Face dataset · pretrained checkpoints — link coming soon

Application Taxonomy

Joint Cross-Modal Retrieval Benchmark

A joint architecture aligns five encoders — text (EmbeddingGemma-300M), sketch (ViT-Base), photoreal image (SigLIP-Base), B-Rep (BRepFormer), point cloud (DGCNN) — in a shared d=768 Matryoshka embedding space ({128, 256, 512, 768}). Trimodal queries dominate all configurations on the B-Rep gallery:

Text	Sketch	Image	R@1	R@5	R@10	R@25
✓	×	×	15.42	34.86	44.61	57.93
×	✓	×	32.69	57.86	67.31	78.59
×	×	✓	33.20	55.24	64.20	74.81
✓	✓	×	39.97	66.07	74.46	83.69
✓	×	✓	38.10	61.94	70.63	80.30
×	✓	✓	41.81	66.31	74.68	83.79
✓	✓	✓	45.91	70.29	77.86	85.91

Recall on the MM-CAD:B validation set (B-Rep gallery, N=19,263, d=768). Matryoshka truncation to d=128 changes trimodal R@1 only from 45.91 to 45.50, so a d=128 FAISS index serves interactive queries with negligible loss. Released checkpoints expose all four nested dimensions.

Citation

@article{bharathi2026mmcad,
  title     = {MM-CAD: A Multi-Modal CAD Dataset and Benchmark for Cross-Modal Geometric Learning},
  author    = {Bharathi, Anush and Ananthakrishnan, A and Muthuganapathy, Ramanathan},
  journal   = {Computer Graphics Forum},
  year      = {2026},
  publisher = {Wiley},
  volume    = {45},
  number    = {5},
  doi       = {10.1111/cgf.70523},
  note      = {Proc. SGP 2026}
}