LERNLUST #55 // AI in educational videos

Intro:

Lernlust, the podcast for everything related to corporate learning. We are Claudia Schütze and Susanne Dube. And we are learning consultants at tts. And we are the hosts of this podcast. And here we will exchange ideas about topics related to our work, in other words, everything that affects learning in organizations today and in the future. And from time to time we will invite internal or external experts to join us. And we would be happy to have you join us.

Kaja:

Because, of course, when it comes to a learning video, we always want learners to take something away and know more at the end. And the other area is the emotional impact of the video. So what do they feel while watching it? Do they trust the person telling them this information?

Susanne:

Are there enough fingers on it? Is the walk somehow weird?

Kaja:

So there is still so much potential that can be explored in more depth.

Susanne:

And there you have to go into it in more detail, because I just wondered, man, if there are already studies out there, why did you do another one? So, 2024 is slowly coming to an end. It was a very exciting year, also in terms of AI. Many of us only heard about the Gartner Hype Cycle through the topic of AI. And opinions on this topic are still very divided. This is particularly true in the area of corporate learning. All the more reason for us to keep looking at the possibilities of AI in the Lernlust podcast, sometimes more and sometimes less critically. Today I have invited a young colleague of mine who did research on the topic of learning and AI for her master‘s thesis. Kaja McKnight Kaja studied e-learning and media education at the Heidelberg University of Education before joining tts. And here she didn‘t just devote herself to the topic of learning videos, but in her master‘s thesis in particular she researched the topic of implementing learning videos and AI and their effect on learners. What she finds exciting about this topic, what exactly she has investigated and what results she has come to, that‘s what we‘ll be talking about today and I‘m looking forward to it.

Susanne:

Welcome, dear Kaya. Welcome to the Lernlust sofa. How are you today?

Kaja:

I‘m doing quite well. The week is slowly coming to an end, but I‘m really happy to have the opportunity to talk to you about my master‘s thesis in the podcast today.

Susanne:

Exciting. And you‘ve already glimpsed it, right? You‘ve already said, yes, I want to talk about my master‘s thesis. But that‘s perfectly okay. Because in fact, we‘ve been working together a bit and are very curious to see it. From my side, I actually teamed up with you from a very work-related context, because we can maybe already reveal so much now at the beginning. Your master‘s thesis has a bit to do with the creation of educational videos and, let me say, rapid production. So how can I perhaps create quick educational videos directly with subject matter experts and, to make it easier for the subject matter experts, with a bit of help from AI. What I find really exciting about this topic, which we had as a common topic, is the fact that I did a podcast with Johannes Starke at the very, very beginning of this artificial intelligence topic and he said, oh, if we now start making any avatar videos with artificial intelligence, then for me that‘s an absolute dystopia. We have now devoted ourselves to this dystopia a bit, haven‘t we?

Kaja:

Yes, that‘s right. And whether it‘s really a dystopia is also the question, or whether it might be a useful tool that we can use in production.

Susanne:

Exactly, very, very exciting. And now I have said a bit about how I might have come to this question, namely, in fact, this background of saying that in the context of our work, we often deal with experts who find it difficult to go in front of the camera and represent their topic. And it is also, and you know a lot more about this than I do, having this topic, it just supports when there is a face that might tell something and it helps a bit with the social bond. That was also the reason why I found the topic so exciting. But now I‘m talking too much about myself.

Susanne:

How did you come to your research topic?

Kaja:

Yes, well, I studied e-learning and media education and of course I dealt with many different facets of media-supported learning there. And one of these areas is video-based learning, which I was also very interested in. And then, of course, I was studying during the time that ChatGPT was suddenly on everyone‘s screens. And everyone was like, have you heard about that yet? And at university, it was, oh, the hot topic. Have you heard about the latest developments? And let‘s try this out. And all these topics. And then I went to Learntec and saw how AI is now also becoming more and more relevant in the educational context and is becoming a hot topic. And so I thought it would be really interesting to investigate how or whether AIs can be linked to educational videos and what kind of impact it might have on learners if they are no longer guided by a human but by an AI.

Susanne:

Now give me a bit of background. So you said you wanted to look into the videos. You come from the time when ChatGPT was suddenly big. You wanted to learn about the effects. What kind of background research might already exist in this subject area?

Kaja:

Yes, well, there has been some research that has examined whether there is a difference between AI and manually produced learning videos. However, this topic is still quite new, so there is not much research in this area yet. But there is, for example, a study by RASK AI of providers. They are self-providers who have investigated this. And von Leiker et al. also did a study that compared manually produced videos and AI videos to see if there is a difference in learning success between the two groups. That is, the group that had watched an AI video, that had watched a manually produced video. In both groups, it could be said that there was no difference in learning success. And then we also looked at how the learners react, whether there are different motivations or generally how they feel about it. However, in these studies, one success was that both videos were quite different.

Susanne:

You‘ll have to go into that in more detail, because I just wondered, if there are already studies, why did you do another one?

Kaja:

Yes, well, for one thing, technology is always evolving. So the study we have now conducted could also be done again in six months or a year with other technologies that have evolved and might possibly produce different results. So this is generally an area where you can always keep looking and researching because there is so much going on right now.

Susanne:

Exciting, yes.

Kaja:

Yes, and on the other hand, as I have already hinted at, there were very strong differences between the videos. For example, in the case of RASK-AI, one video was given to an agency on behalf of a client, who asked them to make a short video on topic XY. And then this agency filmed a speaker, added explanatory graphics to the video, added sound and did various other things. And then they made another AI video, where they really had AI develop the text from scratch, develop an avatar of AI, the voice, the background and put it all together. And in the end, this resulted in two completely different videos that dealt with the same topic, but it was a different person with a different way of speaking, with a different background, with different graphic backdrops. They are two quite different products.

Susanne:

So apple and pear, so to speak.

Kaja:

Yes, something like that.

Kaja:

And at Leiker, they tried, or according to their own statement, they tried to produce quite similar videos, but you could see that it was a different speaker, a different base color, and the effect can be quite different or different. Which is why I thought, okay, I like this approach, the comparison between AI-produced videos and manually produced videos. However, I would like to focus on one aspect and make the videos as similar as possible to each other so that only this one aspect is examined. Because, of course, there are always so many different factors that affect learners, where you just can‘t say, yes, this factor, AI, is mandatory here, because maybe it‘s just that you find one person more likeable than the other.

Susanne:

Exactly, it‘s the simple things, or someone likes the background music in one video more than in another. Sometimes it‘s actually the little things that make a training course more exciting or simply make one learning product more exciting than another, and quite often you don‘t even get into the people. So the best example is always my colleague Claudia and I, who says that I like to learn by listening and I say that I like to learn by talking, so that I can think aloud and formulate my thoughts out loud. And here, when it comes to videos, there are simply differences that you can‘t even see.

Kaja:

Yes, definitely. And I just wanted to try to counteract that. But my goal was really just to examine the avatar or the learning companion, so to produce two identical videos, where the only difference between the two is the talking head and one was produced with AI and one was actually recorded by a human.

Susanne:

That means that when you have a research task like that, you actually have to narrow it down. I still remember that, I come from a time when people didn‘t write master‘s theses, they wrote diploma theses, but I also still remember from that time that I went to my professor and said I had a topic and I would like to write about it and he then said to me, that‘s much too big, you can‘t write a thesis that big, narrow it down. And if I understand you correctly, you just narrowed it down to this setting, I have a video where just one person is talking, so the so-called talking head, so to speak, and I just want to see if, in contrast to these other videos that have already existed or video studies, I‘m just saying, that there have been in this setting, let‘s see if just this exchange of the talking head makes it possible to exclude as many other effects as possible that exist to the right and left of it. That is, these studies that you just cited are more like studies where you say, well, you could have criticized them well. It just wasn‘t specific enough for me. It was a bit of an apple and a pear. And you wanted to do it really well.

Kaja:

Exactly. I want to know if there is a difference between AI and manually produced videos. However, that‘s just a broad question and statement. And in my opinion, you should examine the individual aspects separately.

Susanne:

Exactly, so one thing is that I can actually examine the concept, how good an AI concept is, how good a human concept is. Personally, I‘m always a fan of saying yes, use AI as support, it doesn‘t have to do everything all at once. Maybe we should still think about that, but in this example we have a case where you say that I take what the end user, the learner, actually has as an effect. So it‘s not about the things in the background, but also about things that are, so to speak, at the front.

Susanne:

In other words, go into more detail, what did you examine?

Kaja:

Yes, so for me there were two areas that quickly emerged as relevant. On the one hand, there was the area of knowledge acquisition or learning success, because, of course, with a learning video, we always have the goal that the learners take something away with them and know more at the end. And the other area is the area of the emotional impact of the video. So how does the video affect the learners? What do they feel while they are watching it? Various aspects, do they trust the person telling them this information? How do they feel? Aspects like that, how does the video affect the learners? These were the two areas that didn‘t exist for me.

Susanne:

And you just described that, on the one hand, I wanted to see if the students had developed their learning skills. That is, whether they really learned something, whether they acquired skills, whether they were able to answer the test correctly afterwards, along those lines. But the other thing is, did they feel like learning? So this emotional component, like everything that is emotion. That was obvious in your work. Okay, and now I would like to know a little more. What exactly did you do? What kind of videos were they? What was your setting? What exactly was the testing? We‘ve already hinted at it a bit, but now describe it in detail. Let‘s cut to the chase.

Kaja:

The setting was at an automobile manufacturer, where I was allowed to interview the apprentices. And so I produced two videos in which a process in a software system was mapped and a learning support was provided. In this case, it was a talking head. And our colleague Lisa agreed to be the face for these videos.

Susanne:

You all know Lisa, by the way. Lisa has also done a podcast before. Thanks, Lisa, for providing your likeness and your voice.

Kaja:

Yes, and the great thing was that we were allowed to use her for both the human video, where we filmed her, and for the AI video. That means she provided us with images and her sound recordings and we were able to create an AI avatar that spoke in Lisa‘s voice. And then we had two videos in which the same content was explained with the same text, but once by AI-Lisa and once by human Lisa.

Susanne:

That would be the setting we would have if a subject matter expert said, I get so stage fright, I can‘t get a word out, and then you would say, just give me your photo and then I‘ll either pick a nice voice for you or you can even give your voice and then there‘s a good video if you wanted to do that.

Kaja:

Yes, exactly that.

Susanne:

Okay, so I‘m curious now, how did you find out, so how did you do it? You had these two videos and then?

Kaja:

Exactly, so first the students had to complete a test in which we basically assessed what prior knowledge they already had in the area. And then they watched the video and after that they took the test again. To see if there was an increase in knowledge in this area. So the students learned something. After they watched the video.

Susanne:

Exactly, that‘s what the first research also tested a bit. Is there a difference in learning? Okay.

Kaja:

Yes, and then of course two separate groups. One group that received the AI video and one group that received the video with the real Lisa and that had no access to the other video. So two separate groups that were randomly assigned. And then, after they watched the video and took the test again, they were given a questionnaire in which they were allowed to give feedback on various emotional aspects. There were questions like, “What feelings did the instructor evoke in you?” “How trustworthy did you feel the instructor was?” “How clear was the video?” Various aspects were addressed, mainly the instructor in and of herself, because that was what I was studying. But also the video in general, how it was received as a learning tool.

Susanne:

Okay, and now you‘re done. Now you‘ve done everything. Now we can really get to the important point, namely, what insights have you gained? Let‘s start with the simplest things. Are there any insights that you noticed in the creation of the videos, so from the material that you had there.

Kaja:

Exactly, so in the creation there is already the question of time saving.

Susanne:

Yes, that‘s what everyone always thinks. Now we have AI and now everything is so super fast. Yes, everything is very fast.

Kaja:

So in the end I actually saved about an hour and a half between the two videos.

Kaja:

So for the human video, Lisa and I sat down together and I filmed her while she clicked through the software and explained what she was doing. Of course, it took a few tries because not everything was perfect the first time. That means we did a second take and a third and in the end I edited it all together. While I was working on the AI video, I uploaded a picture of Lisa and a voice profile of Lisa, and was able to generate videos relatively quickly, although a few things were always being adjusted. You can‘t enter too much text at once, you can only enter a few sentences, otherwise the sound quality wouldn‘t be as good. Would we like a neutral, a serious or a happy AI Lisa? We decided on a happy one, because some studies have actually shown that the emotional presentation of avatars also has an effect on the willingness to learn and the motivation of the learners. And so I decided on a positive AI Lisa, who smiles while she tells you everything. I then integrated that into the video and edited it.

Kaja:

And for me, that was a time saving of an hour and a half. But it could, of course, be different for different people depending on their experience, whether that‘s more or less.

Susanne:

I also think that‘s very, very important to know when you‘re giving time estimates. You were kind of a beginner with us, not really a pro, but not a real beginner either. So somehow, in a semi-professional way, you have already dealt with these topics. And for you, it saved time. You just touched on another exciting point. I don‘t know if we‘ve made it clear enough yet, why these talking head videos are important for us to learn. We could just as easily say we‘ll leave it out. We don‘t need these faces. But there is this social component in learning and past studies have found that there can actually be a connection, also with the assessment of learning or with how someone felt connected to the learning topic, just because a video is there.

Kaja:

Yes, exactly. So the talking head topic is a bit controversial in research, I‘ll say. There are studies that say that talking heads are distracting. But there are also some that say no, talking heads motivate through this social aspect. You listen to someone and this togetherness that is created by a talking head, that it is conducive to learning and that it is also positive to have different points of focus and not just look at the same thing all the time. So, opinions differ on that.

Susanne:

And that‘s perhaps also the reason why it becomes a bit of a dystopia for Johannes when we suddenly replace the talking head and he‘s not really a real person at all. That‘s a bit of a fraud that‘s in there. So who am I letting myself be socially tied to now? So who does it work well for? Okay, but we‘ve decided we want to do this. It‘s something that‘s widely used by enterprises. And this is where we just need to know or want to know if there‘s a difference.

Susanne:

We‘ve noticed that there‘s a slight difference in the time required for production. Are there any other things we need to be aware of?

Kaja:

Well, of course there are always the legal issues that arise with regard to the image and the voice and where the data is then uploaded and stored. Such topics are of course always relevant to AI. Is there now a question for learning videos or other content?

Susanne:

What I find very exciting in this context is that even if you record someone for a learning video, so really for the real video, we also come into an image rights issue. So in any case, we need to have the person‘s consent when creating this video.

Susanne:

The only question is, to what extent does a person agree? Do they agree that the video is simply made using a tool like Camtasia or something similar, then cut and packed into a learning management system? Or does that person agree to their face and mood being uploaded to a system that stores voices, stores voice information and perhaps builds improved voice simulations from it, and where data may then also go somewhere on the server, which is not exactly an on-premise server in the enterprise. Incidentally, with these insights into production, I also find that, as we briefly touched on earlier, there is this question of what the expectation of AI is and then what is actually implemented, which is actually an exciting topic area because it also has a lot to do with the extent to which dealing with AI has become common practice. That is, if I have long since clarified these legal issues and then wrote, where someone puts their signature under it, then it‘s not a big effort. If tools have gone through a software audit in the enterprise and are the tools that the enterprise also releases, then you can do it relatively easily. But if you‘re the first to implement it, then of course it can also lead to expenses to build these things at all.

Kaja:

Yes, definitely. And for us it was D-ID and ElevenLabs that we used. ElevenLabs for the voice and D-ID for the video.

Susanne:

Yes, exactly. I also find it interesting, this time saving that you mentioned. That was the second point that comes into play a bit, that you actually pay attention to it. You explained it so beautifully. I also have to pay attention to an AI, how and to what extent can I upload the voice? I can‘t upload the voice, or rather the text, accordingly. I can‘t upload all the text at once, but I actually have snippets that I have to make and in exactly the same way, I don‘t know who has ever tried to have a German software description text set to music by a text-to-speech unit. What tends to happen is that, I can‘t even imitate it, but the text-to-speech unit then somehow starts to develop an English accent. And that‘s kind of a funny effect, where you actually have to go to some effort to do it in production. That‘s just a side note, so to speak, as a little story from the production with technical units as a replacement for the human voice.

Kaja:

Definitely, and it‘s definitely not the case yet that you can just say, yes, I‘ll upload everything, the AI will do it at the push of a button and I‘m done. At least not with the tools we used. it was still a process to upload everything separately, then cut it all back together again and make sure that you influenced the video to the extent that you could still influence it, to where you wanted it to be. I make sure that I still have a paragraph in the text here, that I now leave out one sentence less or however it looks.

Susanne:

Exactly. So it‘s an exciting topic. Nevertheless, it‘s a great support that we have. We don‘t want to say anything bad about it, but from a producer‘s point of view, it‘s sometimes funny to tell the little stories that can happen when you create videos. Okay, now we‘ve produced the things, all fine. Then you did your tests, we did them with Forms. These were normal multiple-choice tests, right?

Kaja:

Exactly, the knowledge queries were all single multiple-choice questions or yes-no questions.

Susanne:

Exactly. And we provided the whole thing via an internal system that the other company had, and then we were able to provide it there accordingly. Now I‘m curious.

Susanne:

So, now you‘ve made your comparisons. What did you find?

Kaja:

Exactly. So let‘s start with the increase in knowledge. Did the learners with the AI video learn more or less than those with the manually produced video? And I have to say, there was no difference. So both groups achieved the same increase in knowledge, the same increase in points in the test then in the post-test as in the pre-test, so in comparison to the pre-test. And from that point of view, it only supports the other studies that have been done on the subject so far, that there is no difference in knowledge acquisition.

Susanne:

Well, then let‘s just do everything with KI-Erstetten videos or with some nice faces telling something.

Kaja:

But Susanne. There is also the aspect of emotional expression.

Susanne:

Okay, then go ahead.

Kaja:

Yes, it‘s not that easy. If we now ask the learners how they felt and what impressions they then have of the video. So the first question the learners were asked was how engaging they found the video. So the question was always asked in English. “How engaging did you find the video?” And there was no difference between the two groups. Both groups gave a median of four out of five points. That means they both found it really very engaging. Okay. That was well received.

Susanne:

That‘s good. It‘s also fun to learn. Okay.

Kaja:

Exactly. And then the next question was how clear was the content in the video? And we also got five out of five points in both groups. So it was super clear what we produced. They were able to understand the content and follow it.

Susanne:

It‘s great feedback that you got for your production. You created good learning content. Okay. It all sounds pretty unspectacular.

Kaja:

Yes, but that‘s also an insight, that there is no difference. So, that maybe it‘s not this dystopia that everything is much worse when we use AI.

Kaja:

But now we also have the question of how credible you found the content in the tutorial video. And as we‘ve already established, it was very clear, well, it was clearly structured, easy to understand, but how credible was it? And it was interesting that the group with the AI Lisa found the content less credible than the group with the human Lisa. It was the same content presented in the same way. The only difference was that in one case it was an AI and in the other a human learning companion.

Susanne:

And was it the fact that it said AI? Or did they know beforehand that they were watching an AI video? Or what was it like? Were there any other influences that could be considered, perhaps regarding credibility?

Kaja:

Well, the learners were not informed whether they were watching an AI video or not. However, as we will see later, everyone recognized that it was an AI video. It wasn‘t so human-like that there was no doubt about it, but it was recognized as AI in front of all the participants. And this perception that it is AI could be the reason why the video was rated as less credible, or the content of the video was rated as less credible. Still rated as credible on the scale, but less credible than the group with the human Lisa.

Susanne:

Okay, but nobody said you were losing me here. That wasn‘t it then.

Kaja:

No, that wasn‘t it. Later on, there will also be a few questions that go into more detail, with reasons why the learners then expressed their judgments that way. And very often it has to do with the way the AI avatar is presented.

Susanne:

Okay, so it was really about this little head in the corner, not about the beautiful graphics or anything else that was there, but really only about the little head. Okay, we have credibility. Were there any other aspects that produced such interesting results?

Kaja:

Yes, namely trust. How trustworthy did you find the instructor? And here there was the biggest difference between the two groups. So the group with the human instructor found the instructor to be very trustworthy, while the group with the AI instructor rated the video as not particularly trustworthy or the instructor as not particularly trustworthy.

Susanne:

Do you have any examples? You just said that there were sometimes textual descriptions. Do you have anything like that? How did you feel about this instructor if she was not particularly trustworthy? Did you say anything to the participants?

Kaja:

Well, what I can also say about this is that we also asked which emotions the learners felt while watching the video. And statistically, we didn‘t see any relevant difference between the two groups. Both groups indicated a neutral emotional state in the median. However, it is really interesting to see what information was provided at all. And with the video with the human instructor, there was information from the neutral to the positive. And with the AI video, there was information from the very negative, i.e. very uncomfortable, to the very positive, very comfortable. So the whole spectrum of emotional states was represented. However, the median was again neutral, a neutral emotional state. Why? So you can definitely see that the AI instructor, simply this aspect, that the instructor was an AI instructor and that you also recognized that it was AI triggered a feeling of unease in some learners. And that can then also be linked to the low level of trust.

Susanne:

Okay, but you didn‘t say anything specific about that, because most of the time you hear something like, yes, I‘m entering the uncanny valley, it‘s kind of weird when I look at it and stuff like that.

Kaja:

Yes, so we asked the question whether they found the AI instructor distracting.

Susanne:

Ah, okay.

Kaja:

And two-thirds of the way through the AI video, they said they found the instructor distracting. And when they were allowed to say why, to explain, there were answers like “It‘s creepy.” So we probably have this uncanny valley effect. “Or it‘s artificial, spoke very slowly. It‘s AI generated and looks into the camera without moving. It‘s unnatural and obvious that it‘s an AI. And her teeth aren‘t moving in sync with her lips either.”

Susanne:

You can almost think that it‘s a bit framed, also by all the discussions that are currently taking place or have taken place on the market. You have to say that the study was in the summer, right? We did the whole thing. Now you could argue against it, also again the news situation that you might have had, whether that was the time when so much was talked about AI-generated videos because you were afraid of fake news and that kind of thing, whether you might have seen effects there. You said it at the very beginning, maybe you could do this study again some other time, because that was actually in there.

Susanne:

Yes, but what I also found so interesting against this background is that this topic of trustworthiness or the fact that you just say we‘re distracting is that completely or something similar. I watched both videos too. I watched both and then actually thought I liked the AI video a bit better. It seemed more chic. Somehow, I found it more pleasant to watch because it went so smoothly. There were no ups and downs and it was a nice accompaniment to what I saw. Actually, now I wonder if this is because I‘ve been dealing with the topic of AI for a while now and have therefore seen more of it, or would you say, no, this effect doesn‘t exist?

Kaja:

That sounds obvious, in any case, and there is also the mere exposure effect, which says that the more often you are confronted with content, the more normal you feel about it. However, this could not be confirmed in my study or in any other study I found.

Susanne:

Okay.

Kaja:

So I also asked how much previous experience the learners had with AI, whether they had ever dealt with human-like AI avatars. And none of this information had any effect. On the well-being of the learners. So no connection could be identified.

Susanne:

I find that really, really exciting. By the way, and for anyone who still wants to do research, this is a starting point that I have. I talked to a colleague this morning about making this podcast and he told me that he had also done a training course, and there was an AI video in it, so it‘s already out there in the world, so we actually have a current topic. And he said that at the beginning I thought it was a bit strange that there was one, but the training went on for quite a while. So I watched a lot of videos. And at some point I didn‘t care and I just thought it was a good companion. So maybe it‘s actually something that could be checked again, whether it‘s not about having fundamentally dealt with AI, but whether it‘s actually about this one face that I see over and over again. I find that very exciting.

Kaja:

Yes, definitely. And the direction was also along those lines, Pellas‘ research in 2023, where they also analyzed AI-generated educational videos in educational projects and they didn‘t find any connection between demographic factors like age, gender, previous experience with AI either. The only thing they had that had an impact on learners‘ perceptions was some kind of training on AI that some learners had gone through before. And these learners then had a more positive attitude towards AI video than those who had not undergone this training. But all the other factors, where you would think that maybe age plays a role or maybe how many videos you‘ve seen in the past, all these factors had no effect.

Susanne:

That‘s really interesting. Were there any other special results that you might have missed? It all seemed to come out in dribs and drabs.

Kaja:

So what I asked the learners was why they have these feelings towards the AI instructor when they were asked to describe their feelings. And I found it interesting that the result for the AI group was that the facial expressions and movements of the AI instructors seem to have the greatest influence on the emotional reaction of the learners. They say that a great deal of focus should be placed on developing the AI avatars in this area so that their facial expressions and movements appear more natural, because that seems to have the greatest influence of all the various aspects.

Susanne:

That‘s also the aspect that I find really exciting in this context, it‘s this Uncanny Valley, where you actually say, well, either it has to be really good, then the whole thing works well, but if there is this small deviation, which might be these movements, then it‘s somehow creepy, weird, strange, what I see there and then it seems repulsive to me. And I always wonder whether it wouldn‘t be easier to just use some, let‘s say, avatars that aren‘t real people at all, so they‘re really in a comic style or something. But that would have a completely different effect. It then goes back into this social topic again. So how much social connection do I then have to this talking head that is shown. That is another very different, larger topic.

Kaja:

Yes.

Susanne:

Okay. Were there any strange things? Well, in every study you have, sometimes things don‘t work out so well. Are there any little special stories you can tell us from this study, from the results that came out? For example, everyone didn‘t like the color blue or something?

Kaja:

There was one thing that was surprising in my opinion. As expected, 100% of the participants in the AI group recognized that it was AI. Or rather, that was not necessarily to be expected, but everyone in the group with the manually produced video, i.e. with the humanly filmed Lisa, still 35 percent of the respondents believed that they were watching an AI video. So 35 percent of people thought that this real Lisa was AI. And I found that really surprising.

Susanne:

That‘s right. And you still found her trustworthy though? Or did you put her in the other group? You still found her trustworthy? Then that was the good AI video. Wow. I‘m just wondering where it all came from. Well, I‘m experiencing it with myself right now, that whenever I watch videos, there are these great videos on social networks, where I always find it most beautiful when friends of mine show some baby videos, walking on catwalks and stuff like that. And then I see myself watching it, wondering if there are enough fingers on the hands. Is the walk somehow weird? So that could have played a role in it, right? This critical view that you also have this year of videos that are presented to you. Or are there perhaps other influences that you think could have led to this idea?

Kaja:

Well, the fact that there is a higher awareness of AI could definitely play a role. I deliberately tried to be vague in the description of the study when we called for participation in the study, so as not to anticipate that it was about AI videos, to avoid people approaching the content with the attitude, “I‘m now going to watch an AI video.” However, it is also possible that employees discussed it among themselves and that one person said, “Yes, I took part in this study. It was about AI videos.” Then the colleague with whom they talked about it was assigned to a different group with the human instructor and had this prior knowledge, “Ah, my colleague saw an AI video...” and they carried this knowledge with them and then assumed that they would also see an AI video. But that‘s all just speculation, I couldn‘t verify or prove it at all. These are all just theories about what it is like.

Susanne:

Speculating, building conspiracy theories about what might have happened. Exciting, exciting. So there are many points where you can start by saying that you could take a closer look again. So dear master‘s students out there in the world, there are still many topics where you can do research. And please write to me with the results, I‘m really curious about it.

Susanne:

Okay. An exciting study, I think. Even if it‘s always just this small aspect that you look at, which is the problem with master‘s theses, you always look at a precise aspect. But the question is, what do we do with it now? I still have Johannes in the back of my mind, who tells me that this is a dystopia and we‘re not allowed to use it. At the same time, I have this use case in my head the whole time, where I think it would be great, it would help people. But it doesn‘t really seem to work. Are there any conclusions you can draw from this, any recommendations you could give?

Kaja:

Yes, so we also asked the participant if they were willing to watch more videos with this instructor than both groups. In the manual group, the approval rate was very high, at 82. Then they said, yes, I would definitely be willing to watch more videos with this instructor. And in the AI group, the general approval rate was only 40 percent of the participants. 32 percent rejected it outright. They don‘t want to watch any more videos with the AI instructor. And 28 percent said that whether or not they would be willing to watch more videos with this instructor would depend on the context.

Susanne:

And what would be a good context for that? So you also said that?

Kaja:

Yes, you also said that and I also found it really interesting that it was mainly exactly this context in which we used it. And you indicated that you would like to watch similar process videos with her, similar topics, explanations of processes, simple explanations, short videos, instructions for processes. So it seems to have been particularly well received for this context, in which a process was explained. For these goals.

Susanne:

Exactly. Okay. Nice, nice topic. That means, this is something where now, of course, every learning manager in the enterprises out there can consider how it is with my colleagues, how it is with my group, how they would actually take it up. And then actually, so often customers come to us and say, we want it to be more efficient to create the learning material. We all have price pressure or budget pressure, which then means and says, please do it with technical support, with artificial voices, with whoever, because it is then created faster and has less production effort. But here we have now looked at what effect this has on learners and actually also on skill building. Because if you think about it, someone who doesn‘t trust what they‘re getting as content, how much of the topic do you think they‘ll want to take with them? And the test we did on learning was shortly after.

Kaja:

Yes.

Susanne:

It would also be an interesting topic for me to consider again. And how about two months later?

Kaja:

Whether there is a long-term difference. So there is still so much potential for further in-depth research. But I think the results of this study also show that you can‘t make a general recommendation for or against it. Because it always depends on the learners and on the context in which we use it. And this emotional state, this emotional reaction, I think, should not be underestimated, because emotions also play a very important role in learning success. And if there are some learners who really feel very uncomfortable with it, and that was also the case in this study, you should perhaps consider whether the learners you have are whether an AI video is right for them or not. I don‘t want to make a general recommendation here, even though I think there is a lot of potential behind AI and also in the field of education and in the production of learning videos. As the technology currently stands, it should still be used with caution.

Susanne:

Carefully. I think that‘s a nice way of putting it. Actually,

Susanne:

I think I‘m more of a person who is open to new ideas, who tries a lot of things, and I think the world of learning has become so broad. Let‘s test what we can offer that‘s good. But I think your warning is very good, to say that in the end it all comes down to whether the learner likes the whole thing, and that also corresponds a bit to our credo of putting people first and in this case the learners for whom we present the content. So thank you for that. Thank you for talking to me in such detail about your master‘s thesis. Actually, compared to reading a master‘s thesis, it has been quite compact here. Exactly, so thank you for that. I enjoyed it very much. I hope you did too.

Kaja:

Yes, thank you for having me today.

Susanne:

And we will continue to follow the research and remain curious and see what is going on. So feel free to give us input if you have any new stuff. And I‘ll see if I can put more exciting stuff into the show notes on the various topics. And then I wish you lots of fun and say goodbye.

Kaja:

Thank you. Bye.

Susanne:

By the way, have you subscribed to us yet? You can do that wherever you prefer to listen to your podcasts. We look forward to your feedback and, above all, to interacting with you. How can you reach us? You can write to us on Podigy or follow the Lernlust podcast on Mastodon. We also exist as a real person on Mastodon Twitter or LinkedIn. So tell us what you like about our podcasts and where we can improve. Until then, I look forward to seeing you in the next episode of the Lernlust podcast.

LERNLUST #55 // AI in educational videos

Shownotes

Transcript

Related articles

LERNLUST #53 // If AI knew what the expert knows...

DAP tuning with generative AI

Use AI compliantly: Legal Challenges

DAP as an Efficiency Booster for Generative AI

LERNLUST #55 // AI in educational videos

Shownotes

Transcript

Newsletter

Related articles

LERNLUST #53 // If AI knew what the expert knows...

DAP tuning with generative AI

Use AI compliantly: Legal Challenges

DAP as an Efficiency Booster for Generative AI