Download PDF Fall Bridge on the Materials Genome Initiative September 29, 2025 Volume 55 Issue 3 The Fall 2025 issue explores the Materials Genome Initiative’s progress and future outlook, showcasing advances in autonomous experimentation, sustainable polymers, next-generation batteries, and the broader role of AI in engineering. Self-Driving Microscopy for AI/ML-Enabled Physics Discovery and Materials Optimization Tuesday, September 30, 2025 Author: Sergei V. Kalinin, Steven R. Spurgeon, and Vinayak Dravid From human intuition to machine intelligence: the next frontier in microscopy. Introduction: Our Material Future Materials are the bedrock of economy and foundation for all real-world technologies. The viability of space travel, grid energy storage, solar to fuels conversion, methane removal, and photovoltaic energy solutions hinge on the discovery and optimization of novel materials and rapid scaling toward manufacturing. The last 20 years have seen an exponential growth in the theoretical predictive capability for crystalline materials and small molecules. However, it is only in the last five years that we have seen the rapid expansion of high-throughput synthesis enabled by laboratory robotics and microfluidics, as well as a resurgence of combinatorial synthesis (Abolhasani and Kumacheva 2023; Epps and Abolhasani 2021; Jiang et al. 2022; Rajan 2008; Soldatov et al. 2021; Szymanski et al. 2023). Combinatorial synthesis, microfluidics, and ultimately dip-pen megalibraries have demonstrated the ability to “write” multicomponent nanomaterials at high throughput scale, generating millions of material examples in the 3D, 4D, and 5D composition spaces (Chen et al. 2016, 2019; Jibril et al. 2022). The lesson of the past two decades is that scaling computation or synthesis individually, even by many orders of magnitude, is insufficient for expediting materials discovery. Rather, the key is to accelerate the feedback loop between theory and hypothesis making, experiment planning, synthesis, and characterization with subsequent update of theoretical models. With the recent advances in high throughput computational screening and synthesis (Colón and Snurr 2014; Daglar and Keskin 2020; Greenaway et al. 2018; Manly et al. 2001), the remaining bottleneck for accelerated materials discovery now is closing the characterization loop and, in particular, extending characterization from static to dynamic to learn materials evolution during the processing. This is a highly complex challenge. While theoretical workflows are often homogeneous and experimental efforts typically focus on a single material family or preparation method, characterization spans multiple techniques, often providing information on complementary aspects of materials behavior and requiring integration into single model. A key requirement for accelerated materials discovery is shifting characterization cycles to time scales on the order of seconds and length scales on the order of microns and smaller to match the throughput of existing fast synthesis methods. This consideration propels microscopy techniques to the forefront. Electron microscopy and associated spectroscopies offer a high-veracity method for probing the structure and chemical properties on the single nanoparticle level (Christopher et al. 2020; Colliex et al. 2016; Egerton 2011; Kociak et al. 2014), but generally suffer from slow sample preparation (El Kirat et al. 2005; Ghomrasni et al. 2020; Thompson et al. 2016). Similarly, scanning probe microscopy excels in probing functionalities including piezoelectricity (Martin 1972), photovoltage (Kuk et al. 1991; Weaver and Wickramasinghe 1991), and electrochemical properties on the nanometer scale (Bentley et al. 2019; Gewirth and Niece 1997; Takahashi et al. 2017), and are ideally suited for exploring combinatorial spread libraries[1] and droplet libraries.[2] Microscopy experiments can vary greatly in quality and reproducibility, even among researchers in the same group, let alone across groups or over time. Materials discovery via microscopy gives rise to a unique challenge. These instruments can operate much faster than human decision making and often create bespoke data sets. At the same time, exploration of positionally encoded combinatorial systems via grid search methods is still intractable. The confluence of these two factors makes a case for the machine learning (ML)-enabled autonomous microscopy. Here, we provide an overview of the emergent trends in automated microscopy, highlighting both challenges and opportunities for these methods to aid materials discovery and design. Why Automation in Microscopy Classical microscopy has traditionally relied on rectangular and fixed-time scanning patterns, a practice shaped by both technological limitations and human biases. These approaches are easily implemented and visualized, leading to a standard sequence of operations: tuning the microscope, imaging, and spectroscopic measurements. The human operator performs multiple and repetitive simple operations, such as tuning the microscope, selecting objects for investigation, and conducting spectroscopy (Figure 1 [Top Left]). However, the sequence of these seemingly straightforward operations reflects a complex decision-making process informed by the operator’s expertise and guided by experimental goals, with results continuously monitored and goals refined during the experiment. As a result, microscopy experiments can vary greatly in quality and reproducibility, even among researchers in the same group, let alone across groups or over time. Here, we discuss AI-enabled microscopy as a tool for materials and physics discovery, focusing on the levels of autonomous and human in the loop decision making. Exploring the materials at the limits of microscope data acquisition rates requires translating the operator’s objectives into a structured framework that ML methods can define and execute. This is particularly important because experimental goals like physics discovery or materials optimization vary between scientists and are often not explicitly defined. Rewards, Objectives, and Hyperlanguages Translating the complex decision-making process of human operators into automated workflows requires building the corresponding ML framework. This in turn requires defining several core concepts including reward, objectives, and hyperlanguage. A reward is an immediate, measurable outcome of an experiment—typically quantifiable and directly tied to the optimization goals. Rewards can be simple, instrument-specific metrics such as achieving higher resolution or maximizing signal intensity, or more complex domain specific metrics such as minimization of the peak corresponding to the undesired impurity, limited certain types of deleterious defects, or establishing preferred grain orientation. Objectives, on the other hand, are long-term scientific goals. These can include fundamental goals such as understanding material behavior or discovering a new physical law or applied goal as making better solar energy material or stable battery. Objectives typically require multiple experiments and iterations to approach, and the relationship between experimental reward and objectives is often uncertain. The key part of experimental planning is building rewards that align with objectives. This process is probabilistic in nature, often involving specific hypotheses and heavily depending on human heuristics informed by prior observations, physics, and domain-specific intuition. Finally, hyperlanguage serves as a unifying framework that defines the possible operations that can be executed on the microscope, connecting the ML abstractions to the executable real-world operations. In the language of reinforcement learning, actions are expressed in hyperlanguage. It is important to note that the rewards in ML are necessarily domain related. Quoting Peter Norvig (Russell and Norvig 2020), “Somewhat remarkably, almost all AI research until very recently has assumed that the performance measure can be exactly and correctly specified in the form of utility or reward function.” Traditional rewards in ML or physics are very simple, including loss functions or parsimony or predictive power of models. These contrast with far more complex real-world objectives and reward functions (Figure 1 [Top Right]). Correspondingly, experiment planning by a domain expert always implicitly or explicitly includes the definition of the probabilistic reward(s) expected to align with the project objective. While humans often perform this intuitively, building automated instruments and self-driving labs requires the rewards to be formulated explicitly. Levels of ML in Microscopy Both electron and scanning probe microscopes can operate at rates well above human reaction times and generate data volumes and dimensionality far beyond human analytical capabilities. ML and AI offer the opportunity to enable discovery at these time scales and with such complex objects. The application of ML in microscopy for materials discovery requires classification of the decision-making levels within the experimental workflow (Figure 1 [Bottom]). Level 1 represents a scenario where data are analyzed post-experiment, but the experimental progression remains fixed (as already determined by the human operator) and cannot be adjusted based on after-the-fact analysis. Level 2 introduces real-time data analysis, allowing insights to be gained during the experiment; however, decisions are still made and implemented manually by human operators. Level 3 involves ML agents that can execute microscope commands during the experiment, driven by predefined reward functions available in real time and potentially with a human in the loop. Level 4 extends this autonomy, enabling the microscope to conduct experiments independently based on reward functions, while also integrating interactions with theory-in-the-loop systems to refine its actions. Level 5 goes further, incorporating upstream experimental planning, where the microscope not only executes experiments but also controls sample preparation methods, creating a fully autonomous end-to-end experimental workflow. In this scheme, Level 5 decision making establishes the connection between a single tool and broader up- and downstream decision making, allowing integration into autonomous instrument networks. Level 1: Post-acquisition Analysis Until the late 20th century, electron microscopy was largely constrained by data analysis challenges, with data typically recorded on film or as relatively small image or spectroscopic datasets. The rapid emergence of personal computing and, more recently, cloud computing over the past 25 years has revolutionized data storage and enabled the use of highly information-rich detectors. This technological leap has made it possible to generate datasets with images containing up to 109 pixels, along with hyperspectral data of comparable complexity. This explosion of data necessitated the development of advanced analysis methods, ranging from physics-based reconstructions to ML-driven dimensionality reduction and segmentation techniques. Physics-based reconstructions condense raw, high-volume data into a smaller set of material-specific, microscope-independent descriptors. Dimensionality reduction methods reduce data volume while presenting it in low-dimensional formats more suitable for human interpretation, spanning simple techniques like principal component analysis to more complex nonlinear approaches based on variational autoencoders. Segmentation methods, meanwhile, use prior knowledge embedded in pretrained networks to identify and isolate features of interest within the dataset, enabling targeted and efficient analysis. Many of these methods are reviewed by Kalinin et al. (2022) and Smeaton et al. (2024). However, a key limitation of post-acquisition analysis is that the experimental data have already been collected, meaning the course of the experiment cannot be altered after the fact. Level 2: Real-time Data Analytics In traditional microscopy, all critical decisions—such as selecting regions for imaging, determining scan sizes and pixel densities, choosing areas for spectroscopic measurements, and defining measurement parameters—were made manually by human operators. These decisions relied on prior observations and the operator’s expertise. However, this process is highly subjective, varying significantly with the operator’s experience. Moreover, many phenomena of interest, such as topological defects or domain walls, are difficult to detect, or even interpret, by eye in raw images. For large, noisy, fast, or high-dimensional datasets, human perception is fundamentally limited, making manual decision making increasingly impractical. This challenge has driven interest in real-time ML-based imaging and spectral analysis workflows that operate during data acquisition, transforming streaming high-dimensional data into forms more suitable for human interpretation. In this framework, humans still decide on the next operation, but the data are processed in real time to enhance interpretability. Despite the availability of advanced ML methods for post-acquisition analysis, implementing real-time ML remains a non-trivial problem. For supervised methods, a major challenge lies in addressing distribution shift, where small changes in microscope imaging parameters can cause substantial variations in ML outputs. This issue is common in computer vision, as shown by the immense effort required for autonomous driving systems. Furthermore, scientific exploration inherently involves encountering objects or phenomena not represented in training datasets. This is especially true for mesoscopic and scanning probe microscopy (SPM) imaging, where objects of interest are often less well defined compared to atomically resolved imaging under ideal conditions. Applications of real-time analytics include representing high-dimensional or complex datasets in a form that supports human perception and identification. This includes visualization of defects in high-noise images and mapping physical order parameter fields such as strain, polarization, and octahedral tilts. An example of a few-shot learning–based model (Akers et al. 2021; Ter-Petrosyan et al. 2024), which leverages pre-trained encoders and requires only a limited number of examples to distinguish features of interest, is shown in Figure 2. While such models may offer less descriptive power, they can be adapted to novel scenarios and updated in real time as new contextual information or tasks arise. Alternatively, lightweight forecasting models, such as those based on simple recurrent neural networks or more sophisticated transformers, may be used in conjunction with real-time imaging or spectroscopy data to maintain a continuous prediction of the state of a system or chemical reaction (Lewis et al. 2022). These models are particularly useful for informing real-time decision making, as described below, since they encode knowledge of the response times of both the imaging system and the material system (Hujsak et al. 2016, 2018). Even unsupervised methods, such as dimensionality reduction, present challenges. They often require careful tuning of numerous hyperparameters and can yield multiple or non-unique representations of the data. To overcome these limitations, reward-based analysis workflows offer a promising alternative. By leveraging defined reward functions, these workflows can construct robust and explainable pipelines, enabling automated, reproducible, and interpretable analysis of streaming microscopy data (Barakati et al. 2024). Integrating real-time ML with microscopy thus bridges the gap between complex data acquisition and human decision making, paving the way for more effective and insightful experimentation. Level 3: Real Time Decision Making The effective timescale for human decision making is typically on the order of 0.3–1 second for rapid, instinctive decisions and considerably longer for decisions requiring deeper thought. These timescales are substantially slower than the intrinsic data acquisition rates of electron microscopes and many scanning probe techniques, which can operate on millisecond or faster timescales. Moreover, sustained human decision making is constrained by fatigue and other limitations, whereas instruments can operate autonomously for days. These factors suggest that transitioning from human-based to AI/ML-driven decision making could increase the efficiency of microscope use by a factor of 10 to 1,000, provided the quality of AI decisions matches or exceeds that of human operators. Realizing this potential, however, requires addressing two key challenges. The first is the development of robust Application Programming Interfaces that provide ML agents with direct access to the same control commands available to human operators, expressed through a defined hyperlanguage. This represents a significant engineering task. While the concept of AI-centric microscopes has gained traction globally, progress has been hindered by the cost and complexity of moving away from traditional human-operated systems at the customer level. At the same time, instrument vendors have been slow to adopt AI-native designs due to limited financial incentives outside niche applications. Consequently, early efforts by the materials community relied on fragmented, custom-built solutions to enable AI integration. These included modular access to low-level data streams, real-time visualization, and embedding AI agents into control workflows (Olszta et al. 2022). Current instruments are still designed around human-centric architectures, limiting their response time, precision, and reproducibility (Fiedler et al. 2023). To fully harness the potential of autonomous experimentation, a new generation of hardware–software co-design is required, incorporating modular operating systems, high-speed data streams, and real-time AI integration. A second, and less obvious, challenge involves building decision-making workflows using these defined hyperlanguage commands. This process represents a highly multistep decision-making framework. One approach is to predefine state–action policies, where a fixed policy determines actions at each stage before the experiment begins. Examples include spectroscopic measurements on defined objects of interest, such as domain walls or grain boundaries (Liu et al. 2023), or specific atom types or defects (Roccapriore et al. 2022a). Another approach is to adopt myopic workflows, where the reward function is defined and optimized at each step. An example is the use of deep kernel learning to explore structure–property relationships, where decisions are made based on a consistent reward function (Liu et al. 2022; Roccapriore et al. 2022c). The applications of these approaches include broad exploration of structure–property relationships, either for known objects of interest or for the discovery of microstructural elements that maximize functionality of interest. For the latter, the key requirement is that the functionality must be estimable from the spectral data. This capability opens the door to a fully automated materials discovery process, where the microscope is not just a passive observer but an active participant in designing and creating new materials. In practice, human decision making in microscopy is far more complex, with reward functions evolving dynamically during experiments. For instance, initial operations may focus on instrument optimization, followed by exploration of statistically significant regions or curiosity-driven investigations of anomalies visible in structural images. Later stages often transition to hypothesis-driven exploration, such as focusing on topological defects based on their potential to exhibit unique properties. Building ML workflows capable of adapting to shifting reward functions requires either setting the reward sequence in advance, designing human-in-the-loop systems (Kalinin et al. 2024) where the operator adjusts the reward functions in real time, or developing multistage decision-making workflows capable of autonomous adaptation. Level 4: Knowledge Extraction from Data One of the primary goals of experimentation is the accrual of new knowledge, which can range from gathering statistical information on microstructures and atomic configurations to uncovering phenomenological laws governing material behavior and even advancing our understanding of fundamental physics. The better the physical models we develop, the greater the predictive power of theory in driving real-world material innovations. In this context, a more objective (though still difficult to quantify) measure of an experiment’s value is its ability to generate new knowledge. Achieving this requires integrating theory into the experimental loop, enabling a synergistic interplay between data and predictive models. Astronomers have long used observations of celestial motion to deduce fundamental physical laws, such as Kepler’s and Newton’s laws, transforming our understanding of the universe. Similarly, modern microscopy provides an unprecedented opportunity to extract physical laws directly from observations of dynamic processes, such as particle growth, microstructure formation, and particle motion. With significant investments in in situ and operando microscopy techniques ranging from environmental electron microscopy to liquid cell transmission electron microscopy and scanning probe microscopy, scientists can now access real-time nanoscale processes under realistic conditions (Smeaton et al. 2024). These advancements have already demonstrated the feasibility of extracting physical parameters, such as diffusion and reaction coefficients, directly from observed data (Ievlev et al. 2015) or learning interaction parameters of spin models from atomically resolved imaging (Valleti 2021), proving that physics discovery from experimental observations is achievable. The future lies in moving beyond purely data-driven approaches by integrating experimental data with external sources, such as material parameters, compositional insights, and density functional theory models. This integration—shifting from isolated observations to context-informed discovery—will allow the derivation of new physical laws while bridging the gap between nanoscale dynamics and predictive, theory-driven understanding. Such theory may inform emerging materials “frontier” models, an experimental analogue to the Materials Project,[3] as evidenced by the successful adaptation of current foundation models to microscopy (Abebe et al. 2025). Incorporating theory-in-the-loop not only enhances the ability to design experiments with more targeted objectives but also ensures that the knowledge generated contributes directly to advancing materials science and fundamental physics. Level 5: Upstream Integration with Materials Synthesis and Processing Traditionally, materials discovery has been followed by characterization, where new materials are systematically analyzed to uncover their properties. However, using microscopes for rapid characterization raises an intriguing possibility: can microscopes directly control synthesis tools? This question becomes particularly relevant in the context of combinatorial libraries, which allow exploration across multi-dimensional composition spaces (2D, 3D, and 4D) by synthesizing films or samples with systematic variations in composition. Advanced microscopes could take this further by not only characterizing samples but also directly controlling synthesis tools, provided sample preparation is fast enough. By integrating sample synthesis and preparation into an autonomous feedback loop, microscopes can iteratively guide the selection of compositions or processing conditions to optimize desired properties. Through combined automated control and real-time data analysis, it is possible to gain new knowledge of kinetic responses and phase transitions, such as radiation-induced oxygen vacancy formation in thin films (Lewis et al. 2022). Currently, human cognition limits the ability to register changes and process relevant feedback, but emerging autonomous systems will not face this constraint. This capability opens the door to a fully automated materials discovery process, where the microscope is not just a passive observer but an active participant in designing and creating new materials. Microscopy for Materials Discovery: The Future To illustrate the concept above, we present three emerging applications of ML-enabled Scanning Transmission Electron Microscopy (STEM) and Scanning Probe Microscopy (SPM) for materials discovery. Materials Discovery in Combinatorial Libraries SPM is built around the concept of a cantilever sensor, which combines exceptionally high force sensitivity with the capability to probe currents flowing through the tip–surface junction. This force sensitivity enables the detection of electrostatic, magnetic, or electromechanical forces, making SPM a versatile tool for investigating phenomena in nanometer-scale volumes at reasonably fast timescales. However, a significant limitation of many SPM modes is that the measured signals often represent a convolution of surface topography and material properties, reducing the amount of directly useful information. Despite this, even weakly informative signals can serve as proxies for tasks such as exploring combinatorial spread libraries, where the materials’ topography reflects composition-dependent growth behaviors. Some SPM modalities, such as piezoelectric measurements, photovoltage measurements, and to a lesser extent conductivity measurements, are less sensitive to topographic variations, providing more direct access to material functionalities. Furthermore, the integration of SPM with optical methods, such as NanoIR, unlocks an even broader range of capabilities. This makes SPM a powerful tool for rapidly probing the evolution of material functionalities within the compositional space of combinatorial libraries. These libraries conventionally encode binary and ternary phase diagrams, while more complex compositional spaces can be explored using droplet or nanoparticle libraries. However, given the finite throughput of SPM, exploring these high-dimensional spaces requires ML-enabled search strategies rather than simple grid-based methods. The emergence of autonomous science represents a watershed moment with the potential to transform the materials and chemical sciences. A limitation of traditional spread libraries is that they typically allow variation of compositional spaces but not processing conditions, apart from thermal gradient samples. Looking ahead, a more advanced scenario involves integrating SPM downstream of synthesis robots capable of varying both compositional and processing spaces in very high dimensions. This integration would enable the systematic exploration of material functionalities across a far more complex experimental landscape, making SPM a cornerstone of autonomous and high-throughput materials discovery workflows. An even broader range of opportunities is offered by electron microscopy, which provides nanoscale analogs of conventional macroscopic characterization methods. For instance, structural imaging in STEM yields direct information on atomic structures, while electron beam diffraction offers insights roughly equivalent to X-ray scattering. Energy Dispersive X-ray Spectroscopy (EDS) provides local chemical composition, and Electron Energy Loss Spectroscopy (EELS) delivers both chemical information and data on local low-energy excitations, such as plasmons and vibrational excitations, similar to infrared spectroscopy. Although the signal formation mechanisms of these methods are considerably more complex than their macroscopic photon- and X-ray-based counterparts, rapid advancements in instrumentation and data analytics frameworks promise significant improvements in their interpretation in terms of relevant material properties in the coming years. Even now, multi-modal ML models can effectively fuse and distill actionable materials descriptors from such data, informing synthesis and processing (Ter-Petrosyan et al. 2024). Building upon these advancements, STEM is rapidly becoming a powerful tool for exploring materials, particularly at the nanoscale. A critical requirement for leveraging its full potential is the integration of sample preparation with imaging and analysis workflows. While electron microscopy measurements themselves are relatively fast, sample preparation remains a bottleneck, significantly limiting overall throughput. One solution to this challenge involves the use of “megalibraries” (Kluender et al. 2019; Wahl et al. 2023), where dip-pen lithography creates multidimensional particle libraries directly on an electron-transparent silicon (Si)-rich silicon nitride (SiNx) membrane as a sample holder. These libraries enable the systematic exploration of compositional and structural spaces, with each particle studied individually as well as collectively. Beyond this, other strategies for exploring compositional and processing spaces could be developed, paving the way for electron microscopy to play a central role in high-throughput materials discovery. In recent years, significant advancements in AI-enabled microscopy have emerged, demonstrating practical implementations of the ML-driven search strategies needed for exploring combinatorial libraries. These innovations include binary classification systems that efficiently distinguish high-quality nanoparticles with exceptional precision, rapid image segmentation pipelines that dramatically reduce acquisition time while maintaining high accuracy, and automated preprocessing frameworks that enhance model performance while reducing computational demands. Collectively, these approaches represent crucial steps toward closing the materials discovery loop by enabling the high-throughput characterization of megalibraries, where millions of distinct nanoparticles with varying compositions can be systematically explored (Figure 3). By integrating these AI-driven techniques with microscopy workflows, researchers can now navigate complex compositional spaces far more efficiently than traditional grid-based methods, accelerating the identification of promising materials for further analysis and development. Atomic Fabrication The last note left by Richard Feynman on his blackboard was, “What I cannot create, I do not understand.” Building solid-state quantum computers, creating nanorobots, and designing new classes of biological molecules and catalysts all require the capability to manipulate and assemble matter atom by atom, probe the resulting structures, and connect them to the macroscopic world. The current paradigm of top-down fabrication, based on lithographic methods, is reaching the limit of ~2 nm device structures—an order of magnitude above atomic spacing—is limited to a single materials class, and relies on a globe-spanning supply and manufacturing chain. Until recently, the two low-cost, low-footprint paradigms for bottom-up atomic fabrication were chemical synthesis and scanning tunneling microscopy (STM)-based direct atomic manipulation. Synthesis is massively parallel, but developing synthetic pathways to complex molecular machines—and especially their assembly—is exceptionally time-consuming, with only the simplest architectures realized experimentally. In comparison, STM-based atomic manipulation has developed slowly and has been demonstrated only for special classes of materials. Electron beam damage has been known in electron microscopy since the first experiments by Ernst Ruska before World War II. It was one of the primary factors (along with resolution) driving the development of electron microscopy over decades, first toward high-voltage instruments in the 1980s and then, from the late 1990s, toward aberration-corrected (AC) instruments. The broad introduction of AC microscopes over the last decade has allowed investigators to localize beam damage to a single atomic column or chemical bond (Markevich et al. 2016), opening a new paradigm for direct atomic-level manipulation of matter (Figure 4). Over the same period, this approach has enabled direct atomic motion (Dyck et al. 2017; Susi et al. 2017), the construction of homo- and multiatomic artificial molecules in 2D materials (Dyck et al. 2018), and atomic-plane sculpting of 2D and 3D materials (Jesse et al. 2015). By combining imaging (e.g., HAADF[4]-STEM) and spectroscopic signals (e.g., EELS, EDS) in a multi-modal approach, it is possible to obtain highly localized information about point defects. The complexity of these interactions—for example, electron beam radiation coupling to oxygen vacancy formation (Lewis et al. 2022)—necessitates new ML models. Such models should process multi-modal data and predict appropriate beam parameters to achieve a desired defect configuration. By incorporating these ML predictions into an automated feedback loop, real-time control is achieved, overcoming human reaction time limitations. The microscope thus transitions into an autonomous atomic-scale factory. Integrating Multiple Tools Paralleling other domains, there is an increasing need to develop ML models capable of multi-modal reasoning to improve discriminating ability and aid in hypothesis generation (Figure 5). It is often tacitly assumed that more modalities (e.g., imaging, spectroscopic, diffraction) are better. While correct in principle, this neglects the real cost of acquiring additional data in terms of time, dose, or instrument usage that may damage the material during analysis. Multi-modal data often exhibit a high degree of sparsity or redundancy that is not fully considered when assessing each modality independently. Thus, it is critical to understand each data stream’s characteristics, how sample features are represented, how the data are collected, and related factors. To date, there have been few, if any, systematic studies of these characteristics in relation to specific material properties, such as crystallinity and alloying element distribution (Ter-Petrosyan et al. 2024). As a result, models may inadvertently bias toward one modality and unintentionally disregard key features of others. These models provide an opportunity to learn how to prioritize experiments to extract the most information at the least cost, effectively shortcutting the discovery process. Summary The emergence of autonomous science represents a watershed moment with the potential to transform the materials and chemical sciences. In electron microscopy, initial engineering has given way to practical automation systems, but more work is needed to make autonomous microscopy a reality. The next generation of hardware must be designed with autonomy in mind. We should carefully consider our data budget and design architectures capable of reasoning in sparse and unexplored settings. It is also increasingly important to account for the cost and idiosyncrasies of handling multi-modal data, which can provide a more comprehensive picture of materials. Nonetheless, early successes have demonstrated the potential of these methods to transform how materials are made, measured, and modeled. Acknowledgments SVK was supported by the National Science Foundation Materials Research Science and Engineering Center program through the UT Knoxville Center for Advanced Materials and Manufacturing (DMR-2309083). Research and contributions by SRS were supported as part of APEX: A Center for Power Electronics Materials and Manufacturing Exploration, an Energy Frontier Research Center funded by the US Department of Energy (DOE), Office of Science, Basic Energy Sciences (BES), under award #ERW0345. This work (SRS) was authored by the National Renewable Energy Laboratory, operated by Alliance for Sustainable Energy, LLC, for the US Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. VPD contributions in this area are largely supported by the US Air Force Office of Scientific Research (AFOSR) with the grant number of AFOSR FA9550-22-1-0300. SVK gratefully acknowledges assistance from K. Barakati with figures. The views expressed in the paper do not necessarily represent the views of the DOE or the US Government. The US Government retains and the publisher, by accepting the article for publication, acknowledges that the US Government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this work, or allow others to do so, for US Government purposes. References Abebe W, Strube J, Guo L, Tallent NR, Bel O, Spurgeon S, Doty C, Jannesari A. 2025. SAM-I-Am: Semantic boosting for zero-shot atomic-scale electron micrograph segmentation. Computational Materials Science 246:113400. Abolhasani M, Kumacheva E. 2023. The rise of self-driving labs in chemical and materials sciences. Nature Synthesis 2(6):483–92. Akers S, Kautz E, Trevino-Gavito A, Olszta M, Matthews BE, Wang L, Du Y, Spurgeon SR. 2021. Rapid and flexible segmentation of electron microscopy data using few-shot machine learning. npj Computational Materials 7(1):187. Barakati K, Pratiush U, Houston AC, Duscher G, Kalinin SV. 2024. Unsupervised reward-driven image segmentation in automated scanning transmission electron microscopy experiments. arXiv preprint arXiv:2409.12462. Bentley CL, Edmondson J, Meloni GN, Perry D, Shkirskiy V, Unwin PR. 2019. Nanoscale electrochemical mapping. Analytical Chemistry 91(1):84–108. Chen P-C, Liu X, Hedrick JL, Xie Z, Wang S, Lin Q-Y, Hersam MC, Dravid VP, Mirkin CA. 2016. Polyelemental nanoparticle libraries. Science 352(6293):1565–69. Chen P-C, Liu M, Du JS, Meckes B, Wang S, Lin H, Dravid VP, Wolverton C, Mirkin CA. 2019. Interface and heterostructure design in polyelemental nanoparticles. Science 363(6430):959–64. Christopher J, Taleb M, Maity A, Hentschel M, Giessen H, Talebi N. 2020. Electron-driven photon sources for correlative electron-photon spectroscopy with electron microscopes. Nanophotonics 9(15):4381–406. Colliex C, Kociak M, Stéphan O. 2016. Electron Energy Loss Spectroscopy imaging of surface plasmons at the nanometer scale. Ultramicroscopy 162:A1–A24. Colón YJ, Snurr RQ. 2014. High-throughput computational screening of metal–organic frameworks. Chemical Society Reviews 43(16):5735–49. Daglar H, Keskin S. 2020. Recent advances, opportunities, and challenges in high-throughput computational screening of MOFs for gas separations. Coordination Chemistry Reviews 422:213470. Dyck O, Kim S, Kalinin SV, Jesse S. 2017. Placing single atoms in graphene with a scanning transmission electron microscope. Applied Physics Letters 111(11). Dyck O, Kim S, Jimenez-Izal E, Alexandrova AN, Kalinin SV, Jesse S. 2018. Building structures atom by atom via electron beam manipulation. Small 14(38):1801771. Egerton RF. 2011. Electron Energy-Loss Spectroscopy in the Electron Microscope, 3rd ed. Springer Science & Business Media. El Kirat K, Burton I, Dupres V, Dufrêne Y. 2005. Sample preparation procedures for biological atomic force microscopy. Journal of Microscopy 218(3):199–207. Epps RW, Abolhasani M. 2021. Modern nanoscience: Convergence of AI, robotics, and colloidal synthesis. Applied Physics Reviews 8(4). Fiedler KR, Olszta MJ, Yano KH, Doty C, Hopkins D, Akers S, Spurgeon SR. 2023. Evaluating stage motion for automated electron microscopy. Microscopy and Microanalysis 29(6):1931–39. Gewirth AA, Niece BK. 1997. Electrochemical applications of in situ scanning probe microscopy. Chemical Reviews 97(4):1129–62. Ghomrasni NB, Chivas-Joly C, Devoille L, Hochepied J-F, Feltin N. 2020. Challenges in sample preparation for measuring nanoparticles size by scanning electron microscopy from suspensions, powder form and complex media. Powder Technology 359:226–37. Greenaway R, Santolini V, Bennison M, Alston B, Pugh C, Little M, Miklitz M, Eden-Rump E, Clowes R, Shakil A, and 5 others. 2018. High-throughput discovery of organic cages and catenanes using computational screening fused with robotic synthesis. Nature Communications 9(1):2849. Hujsak K, Myers BD, Roth E, Li Y, Dravid VP. 2016. Suppressing electron exposure artifacts: An electron scanning paradigm with Bayesian machine learning. Microscopy and Microanalysis 22(4):778–88. Hujsak KA, Roth EW, Kellogg W, Li Y, Dravid VP. 2018. High speed/low dose analytical electron microscopy with dynamic sampling. Micron 108:31–40. Ievlev AV, Jesse S, Cochell TJ, Unocic RR, Protopopescu VA, Kalinin SV. 2015. Quantitative description of crystal nucleation and growth from in situ liquid scanning transmission electron microscopy. ACS Nano 9(12):11784–91. Jesse S, He Q, Lupini AR, Leonard DN, Oxley MP, Ovchinnikov O, Unocic RR, Tselev A, Fuentes-Cabrera M, Sumpter BG, Pennycook SJ, Kalinin SV, Borisevich AY. 2015. Atomic-level sculpting of crystalline oxides: Toward bulk nanofabrication with single atomic plane precision. Small 11(44):5895–900. Jiang Y, Salley D, Sharma A, Keenan G, Mullin M, Cronin L. 2022. An artificial intelligence enabled chemical synthesis robot for exploration and optimization of nanomaterials. Science Advances 8(40):eabo2626. Jibril L, Cheng M, Wahl CB, Dravid VP, Mirkin CA. 2022. Polymer-mediated particle coarsening within hollow silica shell nanoreactors. Chemistry of Materials 34(11):5094–102. Kalinin SV, Ophus C, Voyles PM, Erni R, Kepaptsoglou D, Grillo V, Lupini AR, Oxley MP, Schwenker E, Chan MK, and 6 others. 2022. Machine learning in scanning transmission electron microscopy. Nature Reviews Methods Primers 2(1):11. Kalinin SV, Ziatdinov M, Ahmadi M, Ghosh A, Roccapriore K, Liu Y, Vasudevan RK. 2024. Designing workflows for materials characterization. Applied Physics Reviews 11(1). Kluender EJ, Hedrick JL, Brown KA, Rao R, Meckes B, Du JS, Moreau LM, Maruyama B, Mirkin CA. 2019. Catalyst discovery through megalibraries of nanomaterials. Proceedings of the National Academy of Sciences 116(1):40–5. Kociak M, Stéphan O, Gloter A, Zagonel LF, Tizei LH, Tencé M, March K, Blazit JD, Mahfoud Z, Losquin A. 2014. Seeing and measuring in colours: Electron microscopy and spectroscopies applied to nano-optics. Comptes Rendus Physique 15(2–3):158–75. Kuk Y, Becker R, Silverman P, Kochanski G. 1991. Photovoltage on silicon surfaces measured by scanning tunneling microscopy. Journal of Vacuum Science & Technology B 9(2):545–50. Lewis NR, Jin Y, Tang X, Shah V, Doty C, Matthews BE, Akers S, Spurgeon SR. 2022. Forecasting of in situ electron energy loss spectroscopy. npj Computational Materials 8(1):252. Liu Y, Kelley KP, Vasudevan RK, Funakubo H, Ziatdinov MA, Kalinin SV. 2022. Experimental discovery of structure-property relationships in ferroelectric materials via active learning. Nature Machine Intelligence 4(4):341–50. Liu Y, Yang J, Lawrie BJ, Kelley KP, Ziatdinov M, Kalinin SV, Ahmadi M. 2023. Disentangling electronic transport and hysteresis at individual grain boundaries in hybrid perovskites via automated scanning probe microscopy. ACS Nano 17(10):9647–57. Manly CJ, Louise-May S, Hammer JD. 2001. The impact of informatics and computational chemistry on synthesis and screening. Drug Discovery Today 6(21):1101–10. Markevich A, Kurasch S, Lehtinen O, Reimer O, Feng X, Mullen K, Turchanin A, Khlobystov AN, Kaiser U, Besley E. 2016. Electron beam controlled covalent attachment of small organic molecules to graphene. Nanoscale 8(5):2711–19. Martin RM. 1972. Piezoelectricity. Physical Review B 5(4):1607. Olszta M, Hopkins D, Fiedler KR, Oostrom M, Akers S, Spurgeon SR. 2022. An automated scanning transmission electron microscope guided by sparse data analytics. Microscopy and Microanalysis 28(5):1611–21. Rajan K. 2008. Combinatorial materials sciences: Experimental strategies for accelerated knowledge discovery. Annual Review of Material Research 38(1):299–322. Roccapriore KM, Boebinger MG, Dyck O, Ghosh A, Unocic RR, Kalinin SV, Ziatdinov M. 2022a. Probing electron beam induced transformations on a single-defect level via automated scanning transmission electron microscopy. ACS Nano 16(10):17116–127. Roccapriore KM, Cho SH, Lupini AR, Milliron DJ, Kalinin SV. 2022b. Sculpting the plasmonic responses of nanoparticles by directed electron beam irradiation. Small 18(1):2105099. Roccapriore KM, Kalinin SV, Ziatdinov M. 2022c. Physics discovery in nanoplasmonic systems via autonomous experiments in scanning transmission electron microscopy. Advanced Science (Weinheim) 9(36):e2203422. Russell SJ, Norvig P. 2020. Artificial Intelligence: A Modern Approach. 4th edition. Pearson. Smeaton MA, Abellan P, Spurgeon SR, Unocic RR, Jungjohann KL. 2024. Tutorial on in situ and operando (scanning) transmission electron microscopy for analysis of nanoscale structure–property relationships. ACS Nano 18(52):35091–103. Soldatov MA, Butova VV, Pashkov D, Butakova MA, Medvedev PV, Chernov AV, Soldatov AV. 2021. Self-driving laboratories for development of new functional materials and optimizing known reactions. Nanomaterials 11(3):619. Susi T, Meyer JC, Kotakoski J. 2017. Manipulating low-dimensional materials down to the level of single atoms with electron irradiation. Ultramicroscopy 180:163–72. Szymanski NJ, Rendy B, Fei Y, Kumar RE, He T, Milsted D, McDermott MJ, Gallant M, Cubuk ED, Merchant A. 2023. An autonomous laboratory for the accelerated synthesis of novel materials. Nature 624(7990):86–91. Takahashi Y, Kumatani A, Shiku H, Matsue T. 2017. Scanning probe microscopy for nanoscale electrochemical imaging. Analytical Chemistry 89(1):342–57. Ter-Petrosyan A, Holden M, Bilbrey JA, Akers S, Doty C, Yano KH, Wang L, Paudel R, Lang E, Hattar K, and 5 others. 2024. Revealing the evolution of order in materials microstructures using multi-modal computer vision. arXiv preprint arXiv:2411.09896. Thompson RF, Walker M, Siebert CA, Muench SP, Ranson NA. 2016. An introduction to sample preparation and imaging by cryo-electron microscopy for structural biology. Methods 100:3–15. Valleti SMP, Zou Q, Xue R, Vlcek L, Ziatdinov M, Vasudevan R, Fu M, Yan J, Mandrus D, Gai Z, Kalinin SV. 2021. Bayesian learning of adatom interactions from atomically resolved imaging data. ACS Nano 15(6):9649–57. Wahl CB, Mirkin CA, Dravid VP. 2023. Towards autonomous electron microscopy for high-throughput materials discovery. Microscopy and Microanalysis 29(Suppl 1):1913–14. Weaver J, Wickramasinghe H. 1991. Semiconductor characterization by scanning force microscope surface photovoltage microscopy. Journal of Vacuum Science & Technology B 9(3):1562–65. [1] Combinatorial spread library is a sample prepared such that there is a concentration gradient encoded via spatial location. [2] The library of different materials compositions prepared by drop casting. [3] https://next-gen.materialsproject.org. [4] High-angle annular dark-field. About the Author:Sergei V. Kalinin is chief scientist, AI/ML for Physical Sciences, Pacific Northwest National Laboratory, and Weston Fulton Professor, Department of Materials Science and Engineering, University of Tennessee, Knoxville. Steven R. Spurgeon is senior materials data scientist, National Renewable Energy Laboratory; fellow, Renewable and Sustainable Energy Institute, University of Colorado, Boulder; and joint appointee, Metallurgical and Materials Engineering, Colorado School of Mines. Vinayak Dravid is Abraham Harris Professor of Materials Science and Engineering and founding director, NUANCE Center, Northwestern University.