If you’ve ever played a game of Marco Polo, you can understand how surgeons use sound to help find where to operate.
Though surgeons (thankfully) don’t put their hands over their eyes, they’re in the dark as far as what exactly they’ll find under the skin. Sonification, the use of audio to convey information, is one of the ways they can “see” more clearly.
A surgeon sends a signal—“Marco!”—out into the tissues of their patient, where they’ve put a small responder in the general area that an incision needs to be made. The responder sends back its “Polo!”, increasing in frequency as the surgeon approaches the area. This still-developing field of using sound in surgery emerged in the late 2000s. And while the techniques were still imprecise, they attracted the attention of Laura Schütz. This was the seed that led Laura to combine her interest in medical applications of design with her love of computer music for her capstone project at the d.school: helping breast cancer surgeons more accurately operate on tumors using sonification.
Now, Laura is a PhD candidate at Technische Universität München, where she continues to work on audio feedback for medical augmented reality. She says her journey was deeply influenced by the d.school.
“The d.school, which is so playful and creative, made me feel that anything and everything is possible,” said Schütz. “It changed my way of approaching things . . . and opened up the world of design for me.”
Adventures in design
Stanford wasn’t Laura’s first experience with design. She was already pursuing a master’s in industrial design at Technische Universität München, which focused on product and object design. But after visiting a friend at Stanford who told her about the design programs, she became interested in attending the d.school once she finished her degree.
“This program seemed so special, that there was so much I could learn and explore. I realized this was an adventure I really wanted to go on and grow from,” said Laura. She was interested especially in the biodesign approach at Stanford, which she heard about during her first master’s degree.
“I went into neurosurgery and realized you can apply design in a way that’s so much more meaningful and impactful than designing a new lamp or chair,” said Laura. “Surgical applications and focusing on making that better will have an impact on probably thousands of people.”
Laura was also interested in the d.school philosophy, which she credits with helping her come up with more creative approaches. “Design has all these other aspects and capabilities that come with it. The d.school encourages interdisciplinary work. Being a designer isn’t just about being a good artist or engineer, but about combining many different things, including social skills, fostering creativity, and other things that aren’t so obvious.”
She chose to foster creativity in another realm too—computer music, through the Center for Computer Research in Music and Acoustics (CCRMA). “I like music, I used to play instruments and sing. And I was fascinated by the fact that computationally, you can make music. It was such a good, creative environment where people did cool and sophisticated technical things. And I think that really helped make my capstone, that unlikely mixture of music and biodesign.”
Shock and sounds
For her capstone, Laura worked with a breast surgeon at Stanford named Jacqueline Tsai. She came to speak to Laura’s class and pointed out that the sound feedback in standard practice is still difficult to locate and can often be unreliable or minimally informative.
Laura was shocked that there was so much guesswork involved. “Just having the sound doesn’t help them know exactly where to cut or which part to take. So, they may take a little more tissue out to be sure. That was the most shocking part to me—that this is the normal way to do it.”
Though preventative guesswork happens in other kinds of surgeries, in retrospect, Laura connected her interest to women’s healthcare. “Research in that field is underfunded and underexplored,” she said. “As a woman, I could really relate to it. When I thought about it, would I want this kind of treatment for myself or for my mom, my grandma, my sister, anybody else? No. I’d want there to be a better technique.”
Along with other students, Laura helped to organize meetings across campus to discuss women’s health, connecting people from biomedical, design, business, and many other backgrounds, which she feels likely informed her work as well.
Her capstone used parameter mapping sonification, a 3D mapping technique which connects spatial coordinates with sound. Rather than just volume, it relates spatial information to various sound properties like pitch or frequency, conveying more precise and sophisticated information to the surgeon than a single property alone.
Instruments of anatomy
Now, Laura works with 3D geometries and structures for one of the most complex organs—the brain. Her work is in physical modeling synthesis, which can let you simulate the sound of any 3D object in virtual space.
Though the software was originally developed for modeling instruments, Schütz and her team made “instruments of anatomy” by modeling anatomical structures in the program—a process she says she wouldn’t have come up with if she hadn’t started from her capstone’s idea.
“This program allows someone to go from any medical diagnostic image to make these 3D shapes which will resonate. Then, when you operate on the brain and touch these, it’ll result in rich sounds that tell you about the geometry and physical properties of the tissue,” said Schütz.
Her work is currently limited to the virtual environment, but she hopes to test her techniques out in a real setting with surgical tools in the future. “My goal is to provide this kind of sound feedback to any surgery and any surgeon in real time to make their work more precise. Millions of patients could benefit from that.”
And Laura continues to carry forward another d.school philosophy—one of joy. “Even though we do these technical things, there’s room for joy in what we design. And it can be super impactful and precise and whatnot, but if it can also bring joy to somebody who creates it and uses it, that’s fulfilling as well.”
Credit
Written by Eli Ramos