On the ground floor of one of the new computer buildings at the University of Maryland, Anna Kelleher played her centuries-old violin while a program running on a laptop in front of her told her to do things such as raise her chin or widen her stance.
These were common mistakes that Kelleher knows not to do. After all, she’s a graduate student studying violin performance. But she also teaches violin to others, and the program she was demonstrating might someday help those she teaches to play even better.
Believers in artificial intelligence say the program will radically transform our lives in so many ways.
It’s designed by Snehesh Shrestha, a Ph.D. candidate at the school’s department of computer science, and is the perfect example of how the University of Maryland is building bridges between AI and every other academic program on campus.
A simple webcam found on your laptop, or even your phone, captures enough movement and audio from your performance that the AI program can tell what you’re doing wrong. Whether your stance is too wide or narrow, to whether or not your chin is in the right spot, it can see and also hear everything you’re doing right and wrong.
The program was designed to try “to understand the whole space, not just blindly building a technology, but understanding how can we fill the gaps that are currently there in the entire music learning process,” Shrestha said. “And by identifying gaps where we can empower the teacher and the students, we could really build something a lot more powerful than just building a single technology. And that really was like the starting point of exploring into what the technology can provide towards the future direction of music education.”
On the monitor, the teacher can see the student in 3D — every angle imaginable — to see how they stand and how they move. Technology, including a piece that looks sort of like a smartwatch, can also send cues to the student through vibrations in the wrist.
The future of music education
While the more glaring mistakes done on purpose to demonstrate the program were obvious to Kelleher, seeing herself in the program also taught her a few things about her playing style she didn’t know.
“I’ve been playing for 15-plus years. And still, there are certain things that I do that are not super great,” she admitted. “My shoulder still comes forward a little bit, and I worked on it quite a bit to try and make sure that this is all really open and even. But when it does the breakdown, where it just shows the dots of my posture, you can see that this shoulder is a little bit forward.”
She also moves in ways she never even noticed before. Shrestha said this new technology is analogous to the way the Peloton helped bring personal training and fitness classes right into your home.
“To get this instruction from a coach, you have to go to the gym or be able to hire a personal trainer, right? But not everyone can afford this,” Shrestha said. “What Peloton did was, by adding screen and equipment at home through video systems, really brought this coach to your home. We want to do something similar where we want to be able to use just your laptop at home … that’s able to listen to you and is able to look at you and be able to, then, provide feedback at home and provide these analytics back to the teacher.”
It’s not a program that will put music teachers out of business. Instead, it will help them reach more students, even when they can’t be there in person. And it will help students get more instruction they otherwise might struggle to pay for. It also reduces the amount of time spent rehashing past lessons and instructions.
“With this 3D view, without any expensive equipment and just using a laptop camera, now you’re able to see all the things that a student is doing, and the measurements that are being made,” Shrestha said. “And now, we’re really able to get in, see how the student moves, and are able to see the angles.”
“So this really helps one be able to see from all different angles on what might be happening, and get a different perspective and appreciation for what the student might be doing when someone’s playing,” he added.
Without this, an instructor watching on a program like Zoom or FaceTime could never see the things they can see if they’re watching with the program up.
Bringing an orchestra into your living room
“It will democratize music education,” said Cornelia Fermüller, a research scientist with the Institute for Advanced Computer Studies at U.Md. She helped bring Shrestha and the school’s music department together to create the program.
“I’m saying democratize — it will make it cheaper for students. They can practice in between (classes). If they have a one-on-one teacher, they can use it to accelerate,” Fermüller said. “But in schools, many students don’t have the benefit of private education, there’s often one teacher teaching many. So they can really use this tool.”
The program that’s in development is strictly for violins right now. But eventually, other stringed instruments will be in play. And then it will keep going from there.
For now, Shrestha said he’s looking for parents willing to let their kids participate in trials this summer that will help them create even better models.
“Education has really changed in the last few years with the online courses available. Lots of the students are getting information online,” Fermüller said. “Individualizing them is the next step forward.”
Instead of watching an instructional video online, for example, you can watch an instructional video and get feedback like never before.
“I think this is definitely in the right direction, in terms of being able to literally change the way we learn at home,” Shrestha said.
Get breaking news and daily headlines delivered to your email inbox by signing up here.
© 2024 WTOP. All Rights Reserved. This website is not intended for users located within the European Economic Area.
