ATOMIC: A Domain-Specific Vision-Language Model for Transmission Electron Microscopy

Vision–language models (VLMs) achieve strong performanceon general-domain tasks but can degrade when visual distributions, ter-minology, and reasoning requirements diverge from general-domain pre-training assumptions, including in scientific imaging. We study this chal-lenge in materials-science transmission electron microscopy (TEM), adata-scarce setting that requires nanoscale visual grounding, modality-specific interpretation, and specialized materials-science knowledge. Weintroduce ATOMIC (Assistant for TEM Oriented Multimodal Instruc-tion and Conversation), a TEM-oriented vision–language model adaptedusing peer-reviewed, literature-derived data without manual annotation.Our pipeline extracts figure–caption pairs, isolates TEM subfigures, andgenerates multimodal instruction data via GPT-based instruction gen-eration. The training strategy separates two complementary supervisionsignals: VisionGround, which enforces image-only grounding, and Do-mainContext, which injects caption-conditioned scientific context. Theircombination, Blend, yields consistent improvements across perception-and knowledge-intensive queries. ATOMIC-7B-Blend achieves 75.2% ac-curacy on TEM-MCQ and 37.5% recall-oriented Answer Word Cover-age (AWC) on TEM-VQA, narrowing the gap to GPT-4o on AWC(40.4%) while enabling local deployment. On the TEM subset of MatCha,an independent external benchmark, ATOMIC-7B-Blend+FT outper-forms LLaVA-v1.5-7B+FT by 18.1 percentage points under identicalfine-tuning conditions. We release TEM-MCQ/TEM-VQA annotations,article URLs, figure IDs, normalized crop coordinates for sub-figure re-construction, the data-curation pipeline, and model weights to supportreproducibility within copyright constraints. These results show thatstructured literature-derived supervision offers an effective data-centricstrategy for adapting VLMs to materials-science TEM and may provide areproducible foundation for future studies of VLM adaptation in relateddata-scarce scientific-imaging domains.