A new tool makes high-resolution imaging data on human tissues easier to understand and use

For many of us, Google Maps has become an indispensable daily tool: We pop open a web browser and instantly access a powerful map where we can get directions, zoom in and pan, overlay features like traffic and terrain, and search for specific landmarks. But what if there were a tool with similar functionality to explore an entirely different kind of map? What if we had a Magic School Bus-esque ability to dive deep into the details of tissues that make up the human body?

This is precisely the idea behind Minerva, a software tool developed by Harvard Medical School researchers in the Laboratory of Systems Pharmacology in the Department of Systems Biology in the Blavatnik Institute at HMS.

Minerva, available online to anyone, is named after the Roman goddess of wisdom. It allows users to access in-depth maps of tissue samples gathered during research, ranging from cancerous tumors to heart muscle in distress. On each map, users can zoom and pan, overlay features such as immune cells, and explore noteworthy areas. The maps also incorporate the expertise of scientists and medical doctors who can create narratives to guide users through the samples.

The researchers hope that as Minerva's library of tissue samples grows, the tool will be used by other scientists, scientific journals, clinicians, trainees, medical students, or anyone who is simply curious, thus allowing scientific information—and scientific expertise—to be shared more broadly.

"We collect a lot of research data, but not a lot of it is very accessible yet, so the question is how do you mobilize complex data so people can get to it in a useful way," said Peter Sorger, the HMS Otto Krayer Professor of Systems Pharmacology, who co-leads the project with Sandro Santagata, an HMS associate professor of pathology at Brigham and Women's Hospital. "That's really what Minerva is about."

Drowning in data

As imaging technology has advanced, so too has scientists' ability to gather extremely high-resolution data about tissues in the human body. With multiplexed imaging techniques, researchers can gather such details at the single-cell level—and that data can contain 50 or 60 layers of information about protein markers and other features. Several years ago, Sorger, Santagata, and their team developed their own multiplexed imaging technique, cyclic immunofluorescence, or CyCIF, that is freely available and uses instruments and chemicals found in most labs.

As the scientists used CyCIF in their own research, they were frustrated by the fact that terabytes of complex, high-resolution imaging data were routinely distilled down into a few small, static images when they published papers in scientific journals.

"What you publish is a postage stamp—a little, tiny JPEG figure, 600 pixels wide—while the real image is well over a few billion pixels, but is stuck on someone's computer," Sorger said.

In recent years, the research community has increasingly emphasized the importance of more complete data sharing, or what has been termed "FAIR" data: findable, accessible, interoperable, and reproducible. In January 2023, a new National Institutes of Health policy on data management and sharing will go into effect, requiring all scientists who receive NIH funding to submit a plan outlining how their data will be managed and shared.

Yet despite these efforts, "our work has shown us that there's insufficient investment in what's actually needed to access any of these data," Sorger said. "We continue to collect more and more data, and we continue to aspire to make it more accessible and shareable, but we're not building the software tools that are necessary to do this."

This is in part because scientific data are often big, clunky, and hard even for experts to understand without a guide (like Google Maps without labels), so designing tools that support information sharing in a useful way is easier said than done.

"You could download one of our images, and it would then be on your computer occupying your hard drive," Sorger explained. "Without specialized software, much of which is still being developed, you couldn't look at it. You couldn't interact with it. And even if you could, you wouldn't necessarily know what it was."

That's where Minerva comes in.

Making sense of things

Based in the cloud, Minerva is organized into stories, each of which centers on a single tissue sample or a set of related samples. As users engage with an image, they can pan from spot to spot and zoom in or out to explore. They can also turn on and off different layers of information, including immune cells, other cell types or protein markers, or the physical architecture of the tissue, among other things.

Minerva stories also include a "digital docent," a narration element consisting of a text panel that appears alongside each image. The idea was inspired by museum tours—both audio tours for in-person viewing and online tours developed during the COVID-19 pandemic—that guide visitors through an exhibit by offering a narrative introduction to different pieces of art. The narration panel in Minerva stories guides users through the key features of a sample, providing additional information about each feature along the way. Users can stop to freely explore the sample at any point, and return to the narration later.

Users can also create their own Minerva stories with narration.

Sorger and Santagata described Minerva—and the idea of narrative online guides for tissue samples—in a recent perspective in Nature Biomedical Engineering.

So far, many Minerva stories focus on cancer, including melanoma, lung cancer, and colorectal cancer. Cancer provides an obvious starting point, Sorger said, because it is largely diagnosed based on tissue samples—in 85 to 90 percent of cases, by his estimate. However, Minerva stories can be created for any condition that shows up or leaves its marks in a tissue, including heart disease, tuberculosis, neurodegeneration, and COVID-19.

Sorger and Santagata have been creating Minerva stories from the images associated with papers they publish in scientific journals, as well as images from unpublished research, resulting in about two hundred stories so far. "These are essentially mini publications," that allow scientists to access a lot more information than is available in a paper, Sorger explained.

For example, the researchers created a Minerva story on melanoma for a recent paper in Cancer Discovery in which they created fine-scale spatial maps showing how individual cells interact as cancer progresses. In the future, the researchers envision each small, static image in a scientific paper becoming an entry point into a full-length Minerva story. "We see it complementing a conventional figure by providing more content and context," Sorger said.

Images in medical textbooks could similarly become Minerva stories. For example, instead of a textbook providing one snapshot of tissue with a myocardial infarction, a student could click on the image and use Minerva to freely explore the more visually dynamic and far more detailed representation of that sample. "It gives participatory engagement with images that we hope will enhance medical education," Santagata said.

Minerva may also be useful for pathologists-in-training who are learning to analyze tissue samples. Currently, pathology students sit at a multiheaded microscope and look at a tissue slide as an instructor walks them through the case and points out the critical features for making a diagnosis. Minerva stories have much of this information baked in, with the advantage that they can be accessed on demand and from anywhere. "With Minerva, we can capture the expertise of a wide range of people, and disseminate that broadly," Santagata said. "It really expands our reach, and also provides more types of data that we can use to educate people."

Sorger and Santagata think that Minerva could also support a broader push to digitize pathology. Currently, pathology still largely relies on physical tissue slides, which must be mailed whenever colleagues at different institutions want to consult with each other. In fact, Santagata estimates that around one million slides from around 100,000 cases move through the Brigham and Women's pathology department every year. With Minerva, pathologists could easily share digital versions of tissue samples, along with the essential information needed to interpret them.

In keeping with their goal of making Minerva widely available, Sorger and Santagata have made the tool and all its code open access. They estimate that 10 research groups from different universities have built their own Minerva stories, and around 25 labs are actively using Minerva—including several labs connected to the National Cancer Institute's Human Tumor Atlas Network. The researchers are currently working on integrating Minerva into existing tools, including cBioPortal, an open-access tool for exploring cancer genomics datasets—a step they expect will greatly expand the use of Minerva. They also think that Minerva may appeal to nonscientists interested in learning more about a particular condition, including patient advocates for cancer or the scientifically-curious lay public.

"Our mission is to create public goods. We're about education and research, and those are in the public domain," Sorger said. "If we want to create something to be useful in a class or in research, we don't want to put barriers on its use."

Beyond making scientific data more accessible, the researchers hope that Minerva will inspire greater interest in science and scientific research.

"It's remarkable to be able to take what was basically once a static image and bring it to life. There's a certain wonder and magic that's part of the experience of engaging and interacting with data in this way," Santagata said. "We hope that Minerva will make science more exciting and more understandable to other scientists, to the public at large, and even to ourselves."