Autonomous, self-driving microscope accelerates cellular imaging

Traditional microscopy has significantly advanced in speed and resolution over the past decade. However, these advancements are typically limited by the speed at which an individual can image one biological sample at a time. To accelerate this process, Wesley Legant, assistant professor in the Joint Department of Biomedical Engineering, and colleagues have developed a new microscope with an autonomous feature that can more efficiently image cellular events that are typically difficult to capture. “Modern microscopes produce gigabytes of high-resolution data per second,” explains Legant. “However, a student using the instrument may spend an hour trying to find the cells in their sample that are undergoing the particular biological process they want to study.”

Since an individual researcher operating the microscope is a more rate-limiting step than the capabilities of the microscope itself, researchers working with Legant sought to find a way to automate the process to increase efficiency. The result of their work is a new microscope that uses both lattice light sheet microscopy, which is a fluorescence microscope that can rapidly acquire 3D movies of live cells, combined with another type of imaging called epifluorescent inverted imaging. The new microscope, smartLLSM, is designed to image at a scale not previously seen in light sheet microscopy and is capable of imaging thousands of cells autonomously.

Featured on the cover of the February issue of Nature Methods, smartLLSM uses an AI-based control that switches between both types of imaging to capture cells without the need for a human to drive the microscope. “In this work, we took inspiration from computer vision algorithms often used by self-driving cars to develop a self-driving microscope,” stated Legant. “With this new control software, the instrument can rapidly scan the sample and identify specific cells of interest hundreds of times faster than a human operator.”

Legant Lab collaborated with the Giovannucci Lab at UNC-Chapel Hill, who are experts in machine learning, to teach the microscope to recognize what a researcher wants the instrument to find. Yu Shi, a postdoctoral researcher in Legant Lab, along with Jimmy Tabet, now an MD-PhD student at UNC-Chapel Hill, developed the control software and training protocol. “In our initial demonstration, we taught the microscope to image what cells look like when they divide and grow and how immune cells kill cancer cells,” said Legant. “These events are fundamental to our understanding and treatment of diseases.”

Now, instead of spending weeks of experimental time to capture a few possibly representative datasets, researchers can load a sample on the instrument and go to lunch, and smartLLSM will have automatically acquired hundreds of ideal datasets by the time they get back. This new process will significantly reduce experiment time and enhance research efficiency. “Not only does this make the students happy, but it also allows us to dramatically increase the rate at which we can learn new things, particularly for rare biological events that are hard to study otherwise,” said Legant.

By designing a microscope that can save time on imaging cells, the team opens up researchers to focus on the analysis and interpretation of the images they collect. “Moving forward, we are focusing both on extending the range of automatic functions that the microscope can perform and on using our current capabilities to contribute to new studies in collaboration with other groups at UNC and elsewhere,” stated Legant.

The smartLLSM microscope was most recently featured in Nature Methods in Volume 21, Issue 2. The publication can be read here.

Citation: Shi, Y., Tabet, J.S., Milkie, D.E. et al. Smart lattice light-sheet microscopy for imaging rare and complex cellular events. Nat Methods 21, 301–310 (2024). https://doi.org/10.1038/s41592-023-02126-0

For more information about Legant Lab and their research, please visit their lab website here.

This post was originally published in the Joint Department of Biomedical Engineering.