September 3, 2019 — “I’ve always been interested in what happens in the distant universe,” says Manuela Campanelli.

“Since I was 12-years-old, I’ve always looked up at the sky and wanted to know everything about what’s happening out there. And for me, the way to do this is to solve very complex math and physics equations, run them on supercomputers, and then compare the results to what we see through telescopes.”

Simulation reveals spiraling supermassive black holes. Image courtesy of NASA.

As an early user of Frontera, the fastest academic supercomputer in the world, Campanelli — professor of Astrophysics at the Rochester Institute of Technology and director for the Center for Computational Relativity and Gravitation — is living out her childhood dream.

“My research uses supercomputers to simulate very compact objects in the universe, such as black holes and neutron stars,” she explained. “These objects emit extremely powerful bursts of gravitational radiation, and in the case of neutron stars, they also emit very powerful bursts of electromagnetic signals. I work to simulate these events on supercomputers to predict what kind of signals they produce, and then pass these simulation results to our colleagues in astronomy so they know what they are looking for.”

Supported by a $60 million award from National Science Foundation and located at the Texas Advanced Computing Center (TACC), Frontera has been allowing Campanelli to explore the cataclysmic collision of neutron stars that produced gravitational waves detected in 2017 by the Laser Interferometer Gravitational-Wave Observatory (LIGO); the Europe-based Virgo detector; and some 70 ground- and space-based observatories.

“We’re doing the most accurate and longest simulation ever of this collision to answer some of the key questions about what LIGO observed and what type of electromagnetic signals were emitted during this process,” she said.

In addition to exploring the specific neutron star collision, the project advances computational methods for understanding the dynamics of ejection, accretion, winds, and jets in neutron star mergers, work that is supported by a $1.5 million grant from NASA.

“These mergers expose the extremes of gravitational, electromagnetic and particle physics,” said Campanelli. “They are some of the greatest opportunities for multi-messenger science and the combined study of bursts of light spanning across the electromagnetic spectrum and powerful gravitational wave emissions.”

To describe how matter behaves in the densest environments, scientists like Campanelli and her team write complex computer codes that must be run on a very large supercomputer in order to get results in a reasonable timeframe.

“We have to simulate these events on very large supercomputers, because they’re very distant from us, so we cannot go there to learn what happens,” she said. “Frontera is an amazing system because it gives us a very large number of computer nodes that we can use to solve these very complex problems. These types of resources are unavailable on campuses, so you need to have systems like Frontera to be able to do the simulations we do.”

During the early-science allocation grant, Frontera provided Campanelli and about three dozen other research groups dedicated time in order to continuously perform their simulations without interruption.

“Frontera is providing the needed resources for us to perform our very complex simulations at a speed two or more times faster than we could achieve on any local supercomputer,” she said.

According to Campanelli, the Frontera team at TACC was instrumental in helping her and her team get started on the system.

“The TACC team is present 24/7 with online collaboration tools to respond to our questions,” she said. “They have been very encouraging and inclusive in terms of getting everyone on the team involved with these simulations and proactively solving any problems we’ve had.”

Frontera allows researchers like Campanelli to dream big and answer the Grand Challenge questions in their fields.

“I’m very inspired by the science I do because it’s trying to answer some of the fundamental questions, like what happens when gravity and other fundamental forces work together in the very densest environment you can find in the universe,” Campanelli said. “The heaviest elements that we have on Earth were formed in super-high density of nuclear matter. So, in a way, this is related to our understanding of how the heaviest elements we have on Earth formed and how life began on our planet.”

This research is supported by the NSF Division Of Physics within the NSF Directorate for Mathematical and Physical Sciences: Award # 1707946: Collaborative Research: Curvilinear and Multipatch Methods for General Relativistic Astrophysics in the Gravitational Wave Era