Film sets from Google Maps

0:00 Okay guys, I'm really excited for this week's episode because it has some of the more hilarious examples of AI films that I've seen up to this point along with a ton of really great updates. Let's get to it. The first thing that I want to talk about this week is an interesting AI filmmaking workflow that we saw over on X. A filmmaker named Chikai put together a really interesting demo where they took images from Google Street View, combined them with runway references, and basically allowed them to create an entire story where they were able to transport a character and have them walk down a street and into a shop and they never even stepped foot in the location.

0:45 The thing that makes this really interesting is the person that put this together is the co-creator of Google Earth. So essentially they took a tool that they developed years ago and integrated it with AI technology and they actually are now a filmmaker which is pretty cool. They put together a really interesting breakdown over on X, but I thought it'd be fun to show you the exact workflow in this video. So let's get started. So first things first, I'm going to go over to the Google Street View website.

1:09 We'll click on go to street view. And let's say that we're working on a film project and we decided for whatever reason it should take place on Rodeo Drive. Well, all you have to do in Google Street View is grab the little guy and we'll just drop him into the street. You have, of course, done this before. And the cool thing is you have the ability to, you know, walk around the street. We'll go over here, maybe find the right angle.

1:33 And let's say that we want this orientation here of the Beverly Hilton Hotel, which was the setting of Pretty Woman. Let's say that I want to relive the Pretty Woman experience and have a dream vacation at this hotel, which uh has to be fiction here because this hotel is like $900 a night. So, all you have to do is take a screenshot from Google Earth. I'll go ahead and grab a screenshot here just like this. And there you go.

1:56 And you can have that for multiple locations. And you can even do a shot reverse shot which is pretty interesting where you have one person who's let's say they're oriented in this part of the frame and then the camera moves to the right and then you have another person here. It's pretty cool because you basically allows you to take the entire world and turn it into a film set. Now I'm going to take an image of me wearing this denim jacket and we will bring it into the video.

2:19 And let's say that we're just working on a project that should take place in different scenes inside of the hotel. Basically, I just took images of the hotel here. So, we have a hotel room and we have the pool. And let's say that we want to composite me into this environment. So, next up, we need to create the imagery for this project. And I'm going to use Runway Frames to make it happen. So, all you have to do is go over to the Runway website.

3:03 And we will go to image. And let's go ahead and bring in our images. So, we have the 360 photo of the environment here. All you have to do is drag and drop it into runway. We also have a picture of me here, and I'll go ahead and drag that into the scene. You also can bring in an additional image if you wanted multiple characters or the character holding an object. It's completely up to you, but I'm just going to keep this uh basic for now.

3:27 And I'll call the hotel Beverly Hilton. and we'll call the character, we'll call it Caleb W. Okay, cool. So, now let's type in our prompt. And I will say at Caleb Walking down the street outside of Beverly Hilton. So, keeping it pretty basic. Obviously, we can get into more detail if we wanted to, you know, adjust the environment at all or if we wanted certain characters to be in the scene. We're just going to keep it really, really basic for now and go ahead and click generate.

3:52 Okay, let's take a look at a few of the images here. And you can see it did a pretty good job. That's definitely me in front of this environment. Now, are there changes to the environment? Yes, there are. Like the uh the sign here says Fivar, which I'm pretty sure that is not the name of that street. I may be wrong. You have a few other images here. This one here, not too bad. Again, not too bad.

4:15 So, as you can see, you can put the character inside of the environment very, very easily. And let's just ignore the fact that this is a red light in the middle of this light and a green light down here. It's really throwing some mixed messaging here. So, let's say that we want to use this image for our video. The cool thing is inside of Runway, all you have to do is click use and you will import the image into the video model, which you can see up here.

4:38 And now you can type in a prompt. And for our prompt, I'll say light camera shake colon. I always like defining what the camera movement or details are first and then a smiling man walking past the camera, flags waving in the background. So basically, he just kind of walks out of the frame and go ahead and click generate. So while we wait on that to render, I do want to note that you can download these images and just animate them in other AI video tools.

5:00 So you could take this inside of Google VO, cling, luma, it doesn't really matter. Some of those other tools are better for physical dynamics, especially if you're using Google VO or cling. The kind of overall realism and naturalism of the model is just much better in those tools. But Runway is great because they have an unlimited plan, so you can just run the generations again and again and it doesn't count against your credits. Okay, let's take a look at our video here.

5:23 Okay, not that bad. And you can see he's just walking down the street here. It's kind of morphing on the hands, but all in all, I think it's doing a pretty good job. Obviously, the face does begin to break down there. So, you may want to use a different tool to either stabilize the face or animate the actual video itself. And I should note that some of our prompting did not come through in the generation. So, you might have to run the prompt a few times to get exactly what you're looking for.

6:15 Now, I went in and created a few other images here. So, we have this image of me in the hotel room and me at the pool. You can see it did a pretty good job. And then here are the video generations, you know, just looking around the room, walking around. Not too bad. And then we have the shot of me at the pool, or should I say, I think standing on the water, like walking on the water.

6:15 So, uh, miracle happened at the Beverly Hilton. And you can see the face really begins to break down here. I have no idea what's happening with his eye there. So, just one thing to note if you're generating video directly inside of Runway. So, I think workflows like this are really important for filmmakers because one of the bigger challenges we have is continuity, not only with the characters and objects, but also with the location. Having the ability to control the background and do all of your cinematic coverage is really important, and I think that this workflow is very impressive.

7:01 And I should note that if you're ever working on a film or advertising project, be sure to take a scan or a 360 image of the actual set because you now have the ability to do insert shots inside of post-prouction. So, if you need to reshoot anything, I highly recommend using this workflow to help fill in the gaps before you actually go on location and do the actual re-shoots. Unless you've been living under a rock this last week, then you probably heard about Google V3.

7:23 Long story short, it is the next evolution in AI video and everybody has been completely amazed by the generations that we get from the tool.