Sarafina Nance Participates In Callisto Technology Demonstration

Astrophysicist Sarafina Nance participates in the Callisto technology demonstration inside Ops Suite 5 in the Mission Control Center at NASA's Johnson Space Center in Houston during the Artemis I mission on Nov. 26, 2022. Callisto is a technology demonstration payload aboard the Orion spacecraft developed by Lockheed Martin, Amazon, and Cisco that will test a digital voice assistant and video conferencing capabilities in a deep space environment.

What is said in the film

so here's what we're going to do um while you're here you're helping us test this this is an integration between Lockheed Martin in partnership with Amazon and Cisco and we talked about why we're doing this really we're trying to test and see if we can provide some new tools for future crewed missions to see if we can provide them something of value that helps them quality of life some improvements helps them be more efficient I am providing some of the Comforts of home as they're on long duration missions to the moon and Mars and hopefully Beyond one day so this is kind of a test just to see hey is there value in this can we do this and that's where you come in you get to be a virtual crew member today so while we're here at this desk we're going to kind of pretend as if we are inside of Orion have you ever wanted to go to space I think about it every day every single day well that's probably more than me so you haven't you have me beat there well as a virtual crew member you should be able to see the views outside of the vehicles so let's go ahead and you can ask Rich to pull up some of the external views and some of the camera shots that were recently taken all right Rich could you bring up some of the camera views I might cry to be honest and that is okay if I start crying that maybe so here we have a couple of images that were taken within the last 24 hours and all of these like you can go to the NASA Flickr page and other things and see some of these I'm familiar okay so uh here we are as far as seeing what the Solar Ray Wings have been able to capture at various points uh in within the last 24 hours so we have the moon on this side and then the Earth on this side well now that you've seen the external views since we are inside the Orion cabin for our virtual crew purposes we should pull up the live feeds and see what's going on inside the vehicle let's do it want to ask Rich can you pull up the live feeds of the vehicle yeah all right so this is live happening right now on board the Orion spacecraft so you can see here we have a close-up of the camera that's up on our payload demonstration just like the the mock-up that you saw in the other room and then we have this camera placed at the back of the crew cabin uh so that we can kind of see out the windows and see what's going on and if you look right there in the center you can see the zero g indicator I'm kind of hanging out there Mr. Snoopy yeah perfect well as astronauts time is very well scheduled and things so we have a schedule for you as a virtual crew member great so uh if you look at status item number one what's the your first item vehicle status report perfect we have a vehicle status report to provide so when you first wake up for your day and you have this to do we need some tools to help you help you do that so I'm going to have you look at the center console in that iPad display in the center this is a battery status and if I were to ask you for the average battery charge can you help me out like 40 perfect 40 I love it where did you get that rating from I looked at the bar on the top that says EPS battery I need to get my eyes checked there you go um you're doing great so far and it looks to be a little less than 50 okay you're going by the the visual bar indicator the white there yep um yeah that actually has nothing to do with it whatsoever so um but we wouldn't expect you to be able to do that um this is a format that is very important for for crew and they've been trained to be able to read and have all of this data Rich information so that they have what they need to fulfill their role on any given Mission so since you haven't had that same amount of training and things uh one of the unique things about uh the payload is we are now running Orion flight software on this tablet and we have access to all the vehicle data so this enables us to create optimized displays so that no matter what your role is on a mission we can create something a little bit more custom for you and your role and because it's on a tablet that's portable you can now use this anywhere within the crew cabin that's kind of the idea behind it so for this since I just asked you the average battery charge and the white bar didn't help us out in this particular case let's switch to the uh the solar array Wing display okay and then I'll ask you the same question and see if it's a little bit easier for the level of training we've had okay in this role all right I do want to ask Brian to pull that that display up for us yeah Brian can you pull up the solar array you got it the solar array Wing display the solar Wing display I would just say the battery display for now okay all right so now if I were to ask you the average battery charge looks like 95 percent perfect you have a nice big blue bar in this case yeah that bar is not lying yeah and so these displays we've kind of created more for the general public again since we're doing this Outreach but this becomes something that is because it's optimized it's a little uh simpler to read for for the audience that we're trying to share this with and bringing everyone along but we could tailor this to anything that the crew would need for their mission so if I were to ask you from this uh the current speed that Orion is traveling almost 2 000 miles per hour almost 2 000 miles per hour and then the last question on this display what is our mission elapsed timeline starting in days hours minutes and seconds 10 days 18 hours 6 minutes and five seconds yes excellent you you are passing this test with flying colors great so now you're being able to provide all this data for your vehicle status report and you're providing these updates let's go to our next screen and see where we are in relationship to the Earth and the moon so we're going to go ahead and pull up a Telemetry display do you want to see Brian can switch that for us again Brian can you switch to the Telemetry display he loves his job I love I love that he can help me out all right so the commands are going through and this is going to update with real vehicle data in just a moment here and then you're going to see the little white dot uh there that's currently showing by the Earth is going to get updated with real vehicle data and it's going to show you where Orion is in relationship to the Earth and the Moon so we have noticed a couple delays with some of that data populating but again this is a test so we're trying to figure out okay what's causing some of that delay in this so while this is up we're going to see if that actually jumps but the readings on the bottom should be right so if I were to ask you what is the distance from Earth for your vehicle status report what would you be able to tell me 250 5000 miles perfect and then this has the same speed and yeah plus plus or minus okay so we're gonna actually go to another display because this one doesn't seem to be updating so we're going to try our Telemetry uh display and see if we can get uh that one to get updated with some of the real-time vehicle data cool all right so as that's going through we're sending oh you see you just saw it jump so it actually worked so Ryan didn't just jump that fast obviously it's been through the mission but because we're not operating this every moment of the mission it's only during these windows there's a lag it there's a little bit of a lag where it finally gets that that data to it and then it updates uh where it is in the timeline uh so here is is another display and again this just provides information at your fingertips and if you're sitting whether you're in the pilot seat that you're in now or the commander seat or the two lower seats for the mission Specialists if you have this portable tablet you now have access to information that normally would be mostly on the displays in front of you on the fix fix consoles but let's say you are working on a science experiment and your hands are busy and you still have a couple of items to ask for your vehicle status report well part of the other integration that we have now is this involvement with Alexa and that same vehicle data Alexa has access to so I think we need to give you an opportunity sorry I've just got to wake up my keyboard here no it is not going to work all right see technical difficulties during a test don't you love it that's why we have time that's why we have tests you think by now I'd know how to operate a keyboard all right so since we are using the deep space Network I think it's important to know which one we're connected to okay I think you should ask Alexa all right Alexa what DSN site is Orion using so now your voice is going up to space so the light ring didn't light up in this case so it seems like we were up maybe just a little bit too close to the mic okay um so just normal voice same thing just like we practice cool and let's give it another try Alexa what DSN site is Orion using Orion is currently sending and receiving data to it from the earth using the camera deep space Communications complex in Australia this site along with similar ones in Madrid Spain and Goldstone California make up the deep space Network or DSN together they make up the largest and most sensitive scientific telecommunications system in the world Orion data is sent from Canberra to the dsn's operations center at the Jeff propulsion laboratory and then on to the flight control team in Mission Control there we go that's the coolest thing in the world you just not only sent your voice to space but now you've asked Alexa a question on board and using some of that real-time vehicle data we were able to find out what satellite we were not satellite what ground station we were connected to for the deep space Network so I think we should do a couple more of these yeah please a couple more for the vehicle status report so um I think this is an important one and let's see how it matches uh what we're showing up on our display Alexa how far away is Earth right now currently Orion is 255 414 miles away from Earth and thousand one hundred thirty five miles away from the Moon wow yeah so it's interesting to think about how this is operating now because your voice recording is going up two space over the deep space Network and then just with that delay it's being played inside the vehicle and then Alexa is now and that's what makes this version of Alexa a little bit different than the one that you use at home normally your voice recording would go and go out to the cloud and then come back with the response and that happens pretty quick in this case because oh it looks like we're having just a little signal trouble so while we're troubleshooting this but with this Alexa it's all programmed inside this inside our demonstration is what Amazon refers to as local voice control so everything is being processed locally which means Alexa can only respond to what we put into it so in this particular case if you were to ask something like um what's a fun fact that you might normally ask uh like how many stars explode okay there you go in the universe because that's important to an astrophysicist how many stars are exploded so in this case if we hadn't programmed that in Alexa can't respond to it got it uh yeah so um that's why we're doing some of these prompts but I want to give you a couple of choices as we finish this vehicle status report with Alexa so here are a couple of other options relating to having access to that Telemetry data that you can ask so go ahead and choose one that you would like to do Alexa what point on Earth are you over right now using Orion's current geodetic latitude and longitude location data calculated by onboard guidance navigation and Control software Orion is currently over South Atlantic Ocean sick I kind of have the same feeling every time we do this and it's fun during the mission because obviously the Earth is still rotating and the moon is going right Orion's trying to catch up and rotate around both as we're finishing the mission so that changes over time and because we have access to that data we can get an understanding of where Orion is at all times and obviously they kind of know that at Mission Control they don't need this to do it sure but this provides a new opportunity for a potential for future crewed missions to have access to that same data but now through a voice assistant in this case all right I think we've done a lot for our vehicle status report I think we're now on moving on to the next part of our of our schedule so what's next on your agenda personal time nice it's always nice after you do an important thing like a vehicle status report to have a little bit of downtime I don't think astronauts are quite as lucky I love I love personal time well good well for your personal time the first thing that we're going to do is we would have a call home yeah but in this case since we're here yeah not actually inside Orion we're going to have you call Orion okay all right cool so this button right here in front of you see that call Orion I do I'm going to have you reach over and push that and we're going to initiate this call and then right here this is the camera on on this unit and this is where the integration that WebEx brought to this is they're now going to initiate this the call is joining here so you can kind of see the camera positioning and things but this is going to appear on the tablet in the space as soon as it finishes initiating the call so I want you to get ready because you're about to be in space I'm not ready and you will notice there's going to be a little bit of that delay um for for the video yeah okay so it looks like um just to make sure that this activates we're gonna have you do one more voice command as well and see if we can get this to come up oh Ryan join the call it's looking like it may have gone through there it goes all right so you are now from Mission Control actually showing up in space inside of Orion oh did you ever think this might happen one day no well see here here it is this is this is your moment um yes you can take a picture of this okay all right so you already have your phone out all right sorry guys this no this is part of it it needs to be talking about if you play with with the delay a little bit you can actually get it up take a picture and then bring your phone down before your phone is in the picture smart okay all right now the other fun thing is because again we're experiencing probably a little bit over six seconds from what it looks like um here's what we can do and I think you should have this selfie moment yourself you don't need you don't need me in this but I want you to look at the camera I'm going to take a picture back here all right here if you actually hand me your phone I can take it on yours perfect so um I'm gonna I'm gonna duck down a little bit out of the way okay um as soon as I'm out of the way I'll give you a count of five you hold that pose of whatever you want for your picture and then I'm gonna have you turn around and then I'll get both you and you in the space between the delay all right okay so stay looking at that camera for five seconds and okay we got you in four five four three two one turn around nice okay we took a couple of those and then now it's catching up so perfect we have a couple so cool thank you no you're so welcome wow now aside from you know this is kind of the of the fun coolest factor of it and you're having a moment in space but there's a real practical reason for doing this and this is when you're on a long duration Mission around the moon and further on to Mars when you're out in space being able to connect with loved ones family back home this is kind of an important step to be able to provide that connection I'd say if anything over the last two years has taught us the importance of staying connected through the pandemic and things this is a way that we can now offer that up to crew on board and provide a new tool for them to to stay connected yeah uh having a successful test demonstration of course yeah but this is you're helping us test this right now so if you are on board this mission who would you call first I'd call my partner first okay Taylor Taylor very nice and it's probably an appropriate one yeah and then I'd probably call my parents um and then I'd call my best friend and then who's your best friend her name is Siri Siri you have to give her a shout out yes she would appreciate it um I feel like comments being left out Taylor and comments are sort of wrapped up it's kind of a command no exactly that's what I figured I feel exactly now aside from being able to call your loved ones being able to interact with the engineers back at Mission Control for their expertise and things you can see how a tool like this could potentially help crew members not only stay connected but have access to information and and the experts back here on Earth in a new and unique way through this it's incredible yeah and being able to debug things that are on the spacecraft in real time yeah you think about the Apollo 13 days in the Square Peg round hole if you were able to hop on a video conference call such as this and be able to work through that with the engineers back here um it it probably would have made that process a little bit faster yeah so um obviously this is this is a great integration to be able to test this and you are helping us do that right now so thank you first and foremost thank you um so obviously we we could spend all day calling other people but let's let's finish uh continue uh testing a few Alexa things because you still have a little bit more of downtime in your personal time and Alexa is is great at providing us with information and access we talked about the vehicle Telemetry Telemetry that Alexa had access to but there's some fun things that we can do as well see we are not controlling anything Mission critical on this there are no commands being sent to Orion as part of our demonstration so everything is is kind of built within right with our the technology demonstration we put in there so that we're not commanding any Burns we're not doing any vehicle rotations and things but doing simple things such as some of the commands that we've been doing and even being able to command the lights would be kind of important and provide a quality of life Improvement so as part of our test demonstration we actually installed three lights with our our payload to be able to command those cool so would you like to change the lights I would love to change the ligths in space all right in space so you have a couple of options uh in front of you here and you can decide what you want to change the the lights okay Alexa set the lights to Blue okay you just commanded the lights to change 250 000 miles away do you want to do another color yeah I do okay I I thought that might be the case Alexa set the lights to magenta yeah that one didn't look like it went through try it one more time Alexa set the lights to magenta Alexa set the lights to magenta there it goes oh those are some those are some magenta lights they are some magenta lights it is always fun we were in this part of our test to see some of the reactions from Mission Control when they see the lights changing I think you have to see if you can get any of them to start moving there is one where we test all the lights oh really yes so if you say Alexa party mode oh my god really okay Alexa party mode all right so watch the lights but then watch mission control as well okay let's try it again okay Alexa party mode okay this time we're gonna try so because there are different ways of asking the same question that's an important part of going into the technology so instead of party mode since it doesn't seem to be registering let's say turn on celebration lighting okay and see if that gets a better response Alexa turn on celebration lighting oh my God celebration lighting I'm seeing if anyone Mission Control reacts this time last time we actually had someone dancing wait I love that it is kind of a fun one so obviously this is just showing the capability of hey we do have the ability to control kind of non-critical functions if that was a desired effect on things if you think about you know turning on docking hatch lights for a future Mission and things providing something where a voice assistant can help in that capacity is is a future implementation that might be helpful but again let's have a test and see how this is so you are doing that for us right now all right so we've done the vehicle status report we've done our personal time and our call home we've done the feature test where we're testing Alexa out just by interacting and even asking that how that question so now what is the almost the the second last thing on our test here collaboration test okay so in order to do that we're going to move away from being inside of Orion here we're actually going to come back over to this board and this is where you now get to write so we're going to initiate this I'm going to give you the stylus once again and we're going to pull up this blank whiteboard and the first thing I want you to do is see the share button yep I want you to go ahead and press that with your finger or the or the stylus so now this is taking this screen and sharing this up to the tablet in space and you'll see this changeover happen just after that delay occurs so now anything you write here is going to show up in space cool inside the vehicle so just like before we're going to have you select the largest pencils excuse me select one of the colors that you would like and then let's say Taylor is up on a mission this is um this is your first time talking to him you want to write him a message what would you send to Taylor make sure you write big right I think that's a pretty appropriate message you got to give comet the shout out to you you did say it was kind of a combined package exactly awesome you know everything that you've written here you are seeing show up in space and it is just that delay in things do you I do you want a picture of this yes okay okay I think you need to sorry I forgot to ask you to bring here no no no no no that's uh this is so cute wow pretty cool now does Taylor like to play games he is currently playing a game he's currently playing a game right now okay so do you ever play Tic-Tac-Toe with him uh sure okay sure so obviously he's not up in the spacecraft right now uncrewed Mission we couldn't get him on for this even with the strings that we pull but let's say we have a tic-tac-toe board here I will play the part of Taylor in this case just from a game perspective okay but you can do something I'll let you go first all right okay I'm gonna go for the Center Square okay so sorry I changed the color to Green you're welcome to change it to anything whatever okay I gotta go for the block though yep all right I don't know I have a feeling I know where this is gonna end I don't know yeah see if you're any of my kids they'd be like no way Dad here it is and then they'd go and fill it in and be like no I beat you now obviously this is kind of a fun interaction you can see the tic-tac-toe game we played if he were on board he would be able to be interacting with you and have what he writes on the tablet show up here and vice versa that's pretty neat now as much as fun as this is there's more of a practical application for this that comes in handy so we're gonna do the same thing here let's say uh Taylor's on board and he just had the tablet took a picture of the lunar surface as they're on a flyby if you go ahead and hit that green share button again we're going to have this screen now take over and we're sharing a new canvas so we're going to switch again to the the largest and kind of blue and green show up the best here cool I'm going to mark two things here I'm going to say here is base camp where we're landing and here is where or here's the landing site and here's base camp where we need to get to and I'm going to have you as an expert here in Mission Control provide input to the crew up there of how you want Taylor to make it from Landing Zone to base camp cool we need you to draw a path okay nice I even like the arrows to make sure he knows which way he's good yeah you gotta okay going up above yeah yeah gotta avoid the large crater yep see my kids would be off-roading completely they'd be like Dad take the fun route nice so you made it now because we always do redundancy we need you to draw another path so it's like another color and have you mark another another option okay similar path but this time we're taking the right right side detour a little yeah okay now again everything you're doing here is you're communicating and drawing these paths is showing up for the crew on board so that they know what the experts here back at Mission Control want them to do and what they're recommending from something that they just took from from the lunar flyby now the last thing is we need to Mark the samples they want you to mark three three locations where they want uh samples taken along the path okay well let's get samples here perfect two more samples here okay and let's get samples here nice well I think that's an important aspect and again this is just provides another tool for the crew to be able to interact with and be able to provide some real-time data and then get real-time expertise back from Mission Control in a different way so you can see how this potentially could could help yeah through on future missions and I know we're kind of playing these roles here we're wearing many hats but that's kind of the idea behind this integration is to see Hey would this be a useful tool so what you just did here drawing on the board may help us so cool I mean it's going to help us from the partnership so the last thing we're going to have you do over here and then we have a little bit of a surprise for you is everyone that comes and helps us test out this demonstration we are having them sign this virtual crew member log so we're going to share this up uh in space and we'd love for you to sign as one of our virtual protesters too and then with today's date okay so yeah choose the large pen size again and then uh go ahead and sign nice and large and you'll see all of this appear up on the tablet and then while you're finishing this I I heard your dad laughing in the back room as we were talking so I think we should probably invite him in oh he would love to experience himself he would love to what's the date today's date is the 27th 20th 26th you were close I had to double check all right and then everything is up here we'll take a picture of you yeah okay awesome do I think let's let's have your dad take a picture of you and stand under it okay so Peter you want to come on over oh hey Dad he is in hey now because you're getting to join I'm we're going to have you uh put this if you if you don't mind oh so sorry if I got you there all right just so that we can capture the audio for this so Serafina if you stand over here we'll get your picture under it and then Peter will let you uh seraphine is a pro now she knows exactly what she's been been doing so we're going to give her a moment to take you through the experience all right she wants another one oh yeah you gotta you gotta get a lot you got to get a couple poses you gotta obviously I'm gonna shoot cool you want to pick so the first thing I think we need to do is let's have you we're going to clear this off and then we're going to share this what message would you like to write in the space he's like wait I wasn't planning on this little bit um yeah I hadn't thought about this so it's kind of spontaneous but it is a little bit it is well let's let's say Seraphina was on a mission in space because I have a feeling she might try and get there one day I I have a feeling you're right what would you what would you write to her I would say write big I'm gonna see if he adds oh come on don't do that so this this actually there is there's this kind of the zoom in and zoom out if you do multiple times yeah um we put them on this box yeah it's like wait I want I'm in space I want to say something inspirational can you be inspirational on a moment's notice he can I believe it I think this is a pretty good message so far you have to look I know because you're in the spacecraft so you have to wait and see see what he's saying to you can't spoil it keep your feet on the ground like down with ground you are in space afterwards yeah I'll be floating dad even more important to uh keep your feet on the ground and keep oh there it is I think we should get both of you under there for for a quick picture of you what you wrote uh uh I'll use mine and then I can send it to you perfect let's get just a couple of those wonderful we still have six minutes you guys want to have a selfie in space we need to get get both we do we want all the Selfies so we're gonna have you both sit here you can you can choose either seat you should use the one on the right so if we had okay we can have you uh you got it ask one or two I'm gonna hit the stop sharing button and this is going to resume the video mode so you'll see that come up and then in just a moment you will both be in space there we go I think you gotta lean in that's space dad so Peter I know you've expected your daughter to maybe make it here one day did you ever think you would be here no way well now you are so thank you thank you for the thank you for it all no you're so welcome I think it's all only fair all right we have don't cry five minutes left so would you like to uh say a couple commands to to Alexa sure and to give it a test I mean that's all what this is about so I've pulled up a couple of others here okay so um I have two two or three lists but you can you can choose uh let's choose one or two of these um and see what kind of response and talk normally yeah okay like just push the button yeah yeah push and hold it okay Alexa how was the launch very well done so now your voice is traveling the launch was incredible it's hard to describe the feeling of 8. 4 million pounds of force generated from the Twin solid rocket boosters and four rs-25 engines but it was a surprisingly smooth ride inside Orion there you go did either of you get to see the launch there you go you did yeah I was awake yes I would I was sleeping should I do one more all right yeah do you do one more on here okay Alexa what new skills have you learned while in space alexa what new skills have you learned while in space here's a song I think you might like three so it sounds like she didn't quite understand that one and now you get the Alexis it's too much Emoji let's go I did appreciate your your uh if if you speak normally okay shot it's too big look you're in space I think you have to get your groove on to the Alexa Space Room you should do party mode party Mode's pretty fun okay I'll try it should I try that again yeah what is it Alexa Alexa yeah yeah we can do that so just any time uh yes you can interrupt it sometimes it works better okay if it's done okay Alexa party mode party mode yeah it didn't quite catch it try it one more time okay the same the same same thing yeah okay Alexa party mode all right we're gonna have to do this celebration we're not having any parties no it's not a party this is a test yeah so is what is it celebration celebration turn on celebration have a celebration okay turn on celebration lighting Alexa turn on celebration lighting turning on celebration lightning so you just commended the lights inside Space here we're going to have you do one last thing to kind of close out our day as now having both of you as virtual crew members so we're going to pull this up because we're getting close to the end we have one final thing since you've had a hard day of work and you've done all this testing and things we now it's time for you to end your day and go to sleep so if you could turn on sleep mode Alexa go to sleep mode what do you think is going to happen starting sleep sequence well turn the lights out you get a little bit of white noise to help you soothing soothing now I want to go to sleep mode that's great well excellent well thank you both for coming in to help us test this and to be able to be part of this demonstration like I as much as I hope you enjoyed yourselves like this is helping us to further refine and learn and be able to to figure out hey is this worthwhile to try and Implement on a future mission is this something that would be beneficial to crew members so as virtual crew members and as our guest thank you so much thank you so much thank you everybody in this room thank you very much so special so we would like to get just a couple pictures with you and your dad great so we

Where this page came from

This page was imported from NASA. NASA material is generally not copyrighted and is in the public domain.

Nobody has written it yet — it is the source material at a new address, which is why search engines are asked to skip it and why no one earns from it. It is up for grabs: take it on, and it is yours to rewrite and to earn from.