During a news briefing at NASA’s Johnson Space Center in Texas, Kelly talked about using the augmented reality headset.
— ISS Research (@ISS_Research) March 4, 2016 “I think that virtual reality has a lot of potential. It worked great. I was really surprised. We messed around with it for about two hours, and immediately I sensed this is a capability we could use right now.” The technology could be useful for astronauts trying to perform a task as instructed by mission controllers.
Microsoft and NASA sent the HoloLens devices up to the ISS to see if it would work as part of Project Sidekick to assist astronauts. The HoloLens glasses can superimpose computer-generated graphics on the wearer’s field of view, and show someone else what the wearer is looking at. Both functions were put to the test in orbit. “It had some cameras on it, and we could also see a display that’s in your field of view. The person on the ground could be drawing things in your field of view, and pointing to things, and I could be doing the same thing,” Kelly explained. For example, during a maintenance procedure, ground controllers could observe Kelly’s HoloLens view. “I could say, ‘Hey, is this the bolt or connector you’re talking about?’ And the person [at Mission Control] could just write an arrow in your field of view,” Kelly said. The glasses also make it possible to display a checklist off to the side of the frame, which could come in handy during a complex task. “Looking through it, you can change the opacity of the procedure so it’s right in front of your field of view,” Kelly said. Kelly came back down to Earth on Tuesday after spending 340 days in space, which marks a U.S. record for continuous spaceflight. But the HoloLens headsets are still onboard and available for testing. “The technology, in general, has got great potential for applications not only in space, but of course, on Earth as well,” he added. However, it wasn’t all work. Kelly got to play RoboRaid, one of the games that Microsoft developed for the HoloLens, and didn’t experience any of the awkwardness or nausea that’s sometimes related to virtual-reality applications. The astronauts’ main worry was fighting off the alien invaders. “”It has alien spaceships coming inside the space station, and there’s these aliens attacking you,” Kelly said of the game. “You’ve got to shoot them with your finger. … It was kinda fun.”