# Redefining Marketing, Design, and Art with AI Auto-transcribed by https://aliceapp.ai on Wednesday, 18 Sep 2024. Synced media and text playback available on this page: https://aliceapp.ai/recordings/dLBJSRWyM9b6reoH8nc_32G4E7fxFTa1 * Words : 5,512 * Duration : 00:29:29 * Recorded on : Unknown date * Uploaded on : 2024-09-18 01:16:30 UTC * At : Unknown location * Using : Uploaded to aliceapp.ai ## Speakers: * Speaker A - 29.41% * Speaker B - 70.59% ---------------------------- Speaker A [00:00:00] For being here. My name is Kat Holmes. I am the chief design officer at Salesforce. Just making sure we can see and hear. And then a quick intro for our guest today. This is Chris. So Chris is a chilean born technologist and software developer, and he's the co founder and CEO of Runway. It's an applied AI research company building AI models and products for the next era of arthem entertainment and human creativity. In Runway's latest model, gen three alpha has taken the world by storm. It's capable of generating hyper realistic environments, the physical world, as well as humans and objects. And it's being used by some of the best creatives and artists around the world, from Hollywood studios to advertising companies and brands and all kinds of creators. So before Runway, Chris was a researcher at, uh, the NYU Tisch School of the Arts. And please join me in welcoming him to our session today. Speaker B [00:00:59] Thank you. Thank you. Speaker A [00:01:03] And to help us get oriented, this is a very visual medium. So we're going to start with a video kind of showing the evolution of the models at Runway to get us set for the stage in our conversation. Really amazing. Really amazing. Um, so we're going to spend a lot of time talking about Runway and the technology that is absolutely groundbreaking. But first, wanted to start with a little bit about you. Um, you've had this journey from being a design and technologist, um, studying economics even, um, and bringing that all together to then co founding an AI startup. So could you share a little bit about how you've changed as a person during that whole journey? Speaker B [00:03:01] Um, I think a lot has happened since we started the company. The company is now six years old. Speaker A [00:03:07] Ah. Speaker B [00:03:07] We've been working on a lot of the research and products behind Runway for the last eight years or so. Um, and we've grown a lot. We've learned a lot. Um, and I think what happens and what has become really authentic to Runway right now is, um, that set of backgrounds and experiences that not only I have, but a lot of the team members have has allowed us to merge both art and science really nicely. Um, and that's the sweet spot, I would say, for us as a company is that we're able to speak the language of design, of art, but at the same time speak the very technical engineering aspects of how to build grid models. Um, and we want to learn more about the in betweens of those two things. Speaker A [00:03:45] And that's, I mean, this frontier that's being pushed by creative thinking and seeing how technology can help serve goals that are far beyond our imagination, I think is that combination of science and art is something that brought me to design. But I think it's also you're opening up a whole new chapter of what's possible in design with the work that you're doing. Speaker B [00:04:01] Yep. Speaker A [00:04:01] Um, Runway started as a thesis project. Is that right? Um, so how could you describe the moment of when you realize that this thesis project could become a full fledged company? Speaker B [00:04:12] Um, it did. It started as a research project we were doing at school. Um, and I think, uh, a big realization at the time was we always thought as a early way of understanding applicable ways of using frontier models in art, but we never understood how useful most of the things we're trying to do where for many folks outside of, ah, our kind of like discipline or focus, we're aiming to like focus on. And so the moment we started putting some of the software out, realized we were getting the attention of like VFX editors and uh, photographers and designers and, um, that a lot of the models and the core research that we had was very applicable to a wide spectrum of creative industries. Um, and I think it was more of the realization of it becoming very useful for many people that, uh, we thought we should at least spend time investigating. Um, and it was just that discovery phase of trying new things, finding all. Speaker A [00:05:11] Those different ways that folks, uh, who maybe before, never before really became m. Speaker B [00:05:16] I remember, um, um, we made, um, I think one of the first, like text to video models or image models in 2016, 2017. And it was a really bad model. I mean, you saw what 2020 used to look like, 20, uh, 16. It was even like low res and even like, uh, different. Um, but I think it captivated everyone's imagination that you can suddenly type in text and generate something. Now, uh, we might be used to it, uh, but a couple of years ago we weren't. And that, I think, was the appetite for us to like, well, we should try to go and understand what's there. Speaker A [00:05:49] Well, looking at that video, I mean, it is so amazing. It looks like 2020 is where it started. And where you are now, like that is just rapid, rapid change. So what are some of the most significant technological advancements that happened along the way to unlock all those moments? Speaker B [00:06:07] Um, well, there are a few. The first one is, ah, scale, um, in the sense of more, uh, compute larger models parameter wise, uh, more data. Um, and then some, of course, um, architectural innovations that happened, um, have given birth to, I would say, the progression of the models you see right now. Uh, but for us, there's two parts of it. The first one is if you want to make more progress in the field. There's the large jumps that every new model will do that, um, would allow you to do new things, new capabilities. And those are the ones that are really hard and take time. Um, but at the same time, once you get to a new frontier, there's so many things you need to uncover and discover. It's a new field, it's a new set of opportunities. If we stop research right now, we will have years to experiment and play with. Um, that also requires another mindset, because you need to start understanding, well, how do I take this model and make it so it's useful and effective for all these specific tasks? Um, I think a lot of what we're trying to focus on is kind of doing a bit of both the frontier models that are driven by that, uh, leaving, like, computing and data and algorithms, but also the use cases that's led by just tinkering and experimenting. Speaker A [00:07:21] Now you're using the term frontier model. I have to admit, I could not really concretely tell you anybody here the difference between a frontier model and other kinds of models. Could you give us the quick. How would you describe what a frontier model is and how that plays a role in the Runway work? Speaker B [00:07:34] Um, it's an arbitrary definition. Uh, but at least for us, is, uh, the jumps that you've seen in resolution, uh, capabilities, uh, and just overall consistency on the videos, um, are transitions between generations of models. And so we are right now in our third generation of models. So Gen three, the videos that you saw at the last part of video, those are from Gen three alpha, which is the last generation of models. Before that, we had two previous generations, uh, gen two and gen one. Gen one was released on May of 2023. So every six months or eight months, you'll see a new, like, generation of models. And to make it so, it's a generational model or a new frontier model. Um, it's a big jump in quality and resolution and a bunch of other metrics. But, um, those basically, you can think of those as, like, big jumps, basically. Speaker A [00:08:22] Okay. The research I was doing before we met, yeah, I watched some videos. I won't name other companies, companies out there that are trying also in this, uh, space. But what Runway does is so clearly leaps and bounds ahead. And part of it is because the video that comes out of it feels very human. It feels like the continuity the characters created. And then you cut away and you cut back, and it's still the same character. Simple things, but really important human kind of needs. As we're looking at visual media, uh, if you can. Like, what are some of the things that really distinguish your model or your approach? Maybe, um, the data you're using, what is it that's creating that real human, um, continuity in the. Speaker B [00:09:06] I think there are a lot of different parts that distinguish how we turn our models. The first one is the composition of the team. So we have a, uh, core research team that makes these models. Um, but we also have a production and a studio that comes mostly from traditional Hollywood like disciplines, uh, and are people that have worked in filmmaking for years. And so the really interesting combination is when you have that research scientist that's pre training a large model sitting right next to a filmmaker who's been making movies for the last ten years. And so being able to, both of them speak the same language and collaborate on a new model, that's like part of the secret sauce in a way. Um, the other one is, uh, there's a component of training models specifically for visual outputs or for the arts that's mostly about taste. Like having a good sense of what is worth doing. On the research side is that's one kind of form of taste, but also choosing the kind of data set, the curation of the data set, ah, the way you train the models and the way you fine tune them as well. Um, there's an artistic component and a, uh, artistic creative direction in all of those, uh, applications. And as well as the things that you build on top, um, how do you allow people to control the models if you want to make a film? Um, you probably will never make a film with text. It's very hard to describe in natural language all the nuances of communicating a really good story. Um, and so that means that text to x, uh, specifically text to video, in this case for me, is a suboptimal point. Like, I'm not, I don't think we're ever going to reach a point of this becoming mainstream and unusable for everyone, where you're only going to be using text to communicate with these models. We need other interfaces, we need other primitives, we need other set of grammars, and those haven't yet been invented because it's only been a couple of months since you saw the last models. And so those as well, for us are very, uh, equally important as the pre training of the models. Speaker A [00:11:02] That's such an important point. I think we've seen chat, GPT, you've got the conversation. Kind of natural language instincts is something everybody can do, but as you start to think about many different types of input, many different types of, um, primitives that are going to go into communicating with models and also bringing the right kind of generated content out of them. Um, at the end of the day, um, I hadn't really thought about the range that you would use for multimedia. Speaker B [00:11:28] Yeah, there's many ranges. And again, you're sampling from a model, and the way you can sample from the model might be very diverse and specific. And it not only has to be via text, Tacs is the simplest and most convenient and cheapest way of doing it. Um, the best way I try to communicate this specifically to folks in Hollywood is think about these models as a camera. They're a new form of, it's a new medium, it's a new technology. And that camera, same as like DSLRs or professional cameras, has different options and different modes. So if you're shooting at night, at date, you can change the lens, you can change the type of camera, uh, that you use. Um, and right now I'm showing you, we're all experiencing a camera that has one option. You can click once for now. So the challenge is, well, if you want to make this a medium, ah, of expression for everyone, then the camera has to have more options, first of all, more control. It has to be flexible enough where you can take it anywhere. So in models that could be, they need to be cheap and fast. If you make a video and it takes ten minutes, it's gonna be hard. Um, that's, for me, the equivalent of, like, you taking the camera and putting it anywhere. And the third and most important aspects of this is a camera. Uh, if you have the best camera in the world, I can give you the camera, uh, James Cameron used to make avatar. Um, that won't necessarily make you a filmmaker. Like, if you have the same equipment, that won't necessarily get you to be a great artist. To be a great artist is not about the tool, it's about how you use the tool. And so then the next frontier is like, well, let's educate and let's help people understand how to use this medium of expression and this camera in the best possible way. Speaker A [00:13:02] I love the point you made about taste and the importance of taste, um, in curation of the data that you're bringing into the model, but also the skill set and the expertise of what great visual content and storytelling is. Um, do you see the roles, like, how are the roles of people who use these technologies to create stories? Uh, are they coming from different backgrounds? Now, does it change who gets to be a filmmaker? Does it change who gets to be a creator? Speaker B [00:13:28] Yeah, they're coming from everywhere. I think the natural first step is to approach those who are already thinking of themselves as filmmakers and artists, because it's a natural transition from them. But then the interesting thing about, uh, radical new technologies is that they allow for a broader set of the population to do things that used to be very constrained to a small population. And so if you think about filmmaking, I always wanted to be a filmmaker. It's really hard. I'm, um, from Chile, and it's really expensive to get good equipment. It's really hard to, uh, get to a good school, to have the opportunity to, like, no film. Um, I think there's too many people like me 20 years ago who might want to tell some stories, who might want to become filmmakers, but they're constrained because of the current set of technologies. And so what we're seeing right now is, uh, more people becoming comfortable with using models and tools like Runway to tap into that creative part of their brain. And maybe that's the case, that they don't want to become, like, award winning filmmakers, but they want to exercise the muscle of, like, that creative mind. And what Runway allows you to do is basically that, um, and I think what we need is more of that, that exercise of the creative mind, sometimes. Speaker A [00:14:40] A skill that maybe we didn't even know we had. Speaker B [00:14:42] Yes, exactly. We have so many people that reach out that they weren't good in art, in school, or they never, like, knew how to paint, and so they never thought of themselves as creatives, but now they're just willing to, able to generate and make stuff. Um, and that's great. They feel great. Maybe they're not thinking about it for, um, I don't know, a competition or, like, winning some awards, but it's an exercise. Um, the way I also think about it is if you go to the gym, 99% of the world, I'm assuming, goes to the gym not because they want to win the Olympics or not because they want to be an athlete. Just you go because you feel good, and it's good for your body. Right. Speaker A [00:15:17] Runway. Speaker B [00:15:17] Um, and what we're trying to do needs to feel, and it's already feeling like going to the gym, but for your creative mind, right. You're exercising your brain just for the sake of it feeling good. Um, and then if you want to share it, you can, but you don't necessarily have to share it. Speaker A [00:15:31] That kind of collaboration, I think that it's such a, it's so clear at the center of Runway, you have a focus on human stories, human, ah, expressions, and then also, it sounds like also evolving how people collaborate and create in the future. Um, along the way. I always love specific examples, if you could share one, just any challenges you faced in developing these technologies. There's so many, um, kind of bumps and bruises that I've certainly experienced as I'm just experimenting with things that we were even using in Salesforce. So, um, do you have an example of just a challenge you've had and how you've overcome that with the technologies? Speaker B [00:16:09] Um, I think one thing that has become obvious is managing expectations. Um, I think the polarization of new technologies, that's always. And I think with AI, we're seeing this, it's presented as the world, most transformative technology we've ever created, or we're doom and it's the end of the world. Right. Pick aside, um, and the truth is, it's neither of both. It's somehow in between, like anything else, like everything else. And so managing those expectations has been. One of my biggest focuses right now is making sure you understand that this is not some sort of, like, binary thing where you type in movie, enter, and you suddenly get a movie and you're done. It's way more complex, and it's way more involved, and you're in control. But for you to understand how it works, you need to use it. And it might be scary, it might be difficult, you've never done it before, but you should try. Just try the first time. And then a lot of, uh, challenges will start to be overcome, and more people will think about them the moment you have first hand of experience with that. Um, I think right now, the biggest challenge is to more people to use properly use the tools. Speaker A [00:17:16] That point about setting expectations is so important because the experimentation and the iteration is the heart of how we learn to use, because there's no set answers. That's kind of what I hear you saying. It's not like everybody's going to produce the same result, giving the tools that Runway has. But if you're up for experimenting and you're up for learning, then you can create something maybe you never imagined before. Speaker B [00:17:38] I think the common misconception might be to think that, um, every artistic or creative endeavor is some sort of like a linear process where, like, oh, everyone knows where they want to go. It's like a very simple step, um, when in reality it's not. It's like a lot of the best art and a lot of the best creative projects have been born out of pure experimentation. You kind of don't really know where you're going, but, like, you're tinkering, you're setting a, uh, stepping stone, and you're, like, expanding the space of being able to wander around. So you're wondering and you're showing more things, and you're doing more things, and m sometimes you arrive at something, you're like, okay, like that. Um, so a, uh, design principle that we try to embed in runways is the idea of wondering. The tools should allow you to wonder. Like, it shouldn't be a tool that's prescriptive. It's not trying to solve a problem for you in the sense that if you want to tell a movie, there's no one single button that you would press. It will make a movie. It can help you visualize those ideas faster. And in the process of doing that, you might see things and discover avenues that you never thought of before. And then you can take those, and then taking the risk of going into those directions should be cheap, should be fast, and should be easy enough, which is, like, not really the case in, uh, filmmaking right now. Speaker A [00:18:51] That makes so much, it's like, um. So I've been working in design a long time, enterprise software a long time. So much of it's about finding a problem and trying to solve the problem. The problem solved the problem. And I think what you're describing is being able to wonder and dream and imagine and experiment is a really important part of how we also do problem solving. You just may not know where you end up. It's not quite such a linear path, um, thinking something like that. I think about the industries that might be transformed by this or industries this could apply to. Right? Like, it's certainly seen filmmaking or brand, but knowing that it can be that kind of collaborative partner and that wonder that you talk about, what would be the industries you hope could really improve this? Speaker B [00:19:35] I mean, the obvious ones are the ones that are where you are already using technology to tell stories. So Hollywood, right? Yeah. The history of Hollywood is the history of technology. And I think, more broadly, the history of art is the history of technology. We've been using technology to tell stories since ever, forever. Um, and so the first set of industries that will be, uh, impacted and I think already are embracing this, are, of course, storytellers. Hollywood, uh, as the first, like, frontier. But then the next frontier is to realize that those who can call themselves part of Hollywood is very small set of the world. And so, more than, like, defining a specific market we want to work on. I think our ambition is like we should creating a market. And that market should be everyone who wants to tell a story, everyone who can know how to use these tools to make something meaningful. Um, and that comes in the form of creatives, independent, like brands, uh, studios, agencies, which is where we're starting to see already. A lot of the brands and agencies who have historically rely very complicated, like workflows, are now moving to easier and faster ways of making stuff of. Speaker A [00:20:46] Mhm. Uh, just that storytelling is so much of how, even as we develop different industries, healthcare and even finance, like, we're all people working inside of companies. And the stories that we can tell to ourselves and to our customers really creates. Speaker B [00:21:00] I think every company, um, is a media company. Every company tells a story in some way. And so being able to tell the story should be not a fact or, uh, constraint based on your budget or youre, uh, technological access should be, how good your story is. And that's kind of the frontier. Yes, I love that. Speaker A [00:21:15] I love that. Um, kind of moving into Runway's impact on culture. Um, and you talked quite a bit about art. You know, every generation of tools kind of unlocks another generation of what's possible in art. Um, so what are the shifts that you've observed or are inspired by around visual storytelling that kind of brought you to this thesis? With Runway? Speaker B [00:21:38] Um, it's hard to properly tell, to be honest, because I think we're still living through that transition. We organize a film festival every year, uh, both in New York and in LA. We've done it twice, and now we're working for the third version of it. It's a great event. Thousands of people submit videos and films. Um, work with traditional institutions like the Tribeca Film Festival, and with new institutions, we're trying to promote new artists. And I think a lot of it has to do with hearing from new voices who are willing to take this technology and find new grammars and new ways of using it creatively. And it's really early because most of the things are very new. I think the impact you'll see long term is we have the saying at Runway that I think perfectly might encapsulate that feeling, which is, um, the best stories and the best movies are yet to be made. We haven't made them. And the fact that we haven't made them is because we haven't reached everyone who needs to, like, be willing or able to tell the stories. Um, and so it's early to fully, like, I think, assess, but, uh, I'm excited to just get this more out. Speaker A [00:22:40] To more people kind of finding, again, how and who shows up when you change the technologies and what people willing to play with, like who shows up at the film festival might look very. Speaker B [00:22:49] Different, I think ten years. Um, I mean, we've seen this before with like, um, I mean, cinema is like, it was a, it was a new medium and a new form that had to evolve from painting. It had to, like, detach itself from the way painting used to work. And the way you detach it was you had people experimenting with it. Like, things that are, for us, obvious right now, like editing or montages or visual effects were things that people discovered. And the first use case of moving, uh, pictures of filmmaking was to record theater, because that was a performative aspects of combining. But then you realize that you don't have to necessarily just record a theater play. You can do more things. But what are those things? Well, let's go outside and try to record. And that in itself, it's a, ah, playful aspect of using, um, technology. And I feel like, again, it's still very early in this journey of AI and art and design. And what we need is people to take that camera and just put it out in the world. Speaker A [00:23:47] Yeah. The way that, uh, what's going to start to show up in all these different ways as people are experimenting is, um. And I think the fidelity that Runway creates is going to be mind blowing. Um, the question that I think we always have a responsibility to ask is, how do you think about the authenticity and the originality of content in this space with generated? And so what are the things that are unique to, you know, what Runway is creating that, you know, we got memes and whatnot. That's one thing. But you're on a whole nother level here. And the originality and kind of the copyright and all those kind of questions along the way. What are the unique challenges in this space? Speaker B [00:24:29] I mean, I always like to think about, like, the human aspects of any technology. And in this case, originality is like, it's, it's, it's the taste and decisions that we make as creators. You probably never watch a movie because of how it was made. You don't care how it was made. You watch a movie because it's a good movie and it's a good story. Right. Um, and so for me, I want to get to a point where we stop referring to either a video that was made with AI or not with AI, or in combination with both. Like, it was just media, it's just content. And you watch it and it's original because it's a good story. And I think detaching, like the, starting with the conversation with the technology first, I think, makes the focus on the wrong thing. The focus should be on the people using the technology, and then everything should steam out of that. Um, and I think the way you can do that is, first of all, what we were saying before, more literacy and more understanding of the challenges, the limitations and the expectations. If expectations are for you to get into Runway and type movie entertainment, you're going to get frustrated. It's not going to work like that. Um, you need to understand how it works. And the second one is, of course, every new technology comes from, with new challenges on how we deploy it safely to many people. And that's something we are as Runway, um, thinking and working thoroughly to make sure we have the right safeguards in place for it to be useful and at, uh, the same time, safe for everyone to use. But I don't think no one right now has an answer to say, like, that's the one thing everyone needs to be using and doing. It's more of, like, recognizing still very. Speaker A [00:26:04] Early, makes a lot of sense. Um, just because I have to ask, in the realm of storytelling, what's one of your all time favorite movies? Or, like, just what are some of the stories that drawn you over the years to do you love, you know, I love action movies. I admit it. Um, you know, are, ah, there certain stories along the way that have inspired you personally as you've thought about what is possible with Runway? Speaker B [00:26:29] Yeah, um, you know, I don't tend to have favorites. I think on the spot. Yeah. It's like a boring answer. It's like, I usually watch. It's more of a. I think I've learned and I sometimes obsessed with specific filmmakers or specific, like, um, works or films that I think are interesting and new. I like the novelty. I like filming filmmakers that are willing to go above and beyond, um, and there's a lot of, uh, new voices and new people doing new stuff. I like avant garde cinema, but also more traditional cinema, um, and just embracing a diversity of things, um, more than anything in particular. And I think that's the same case for Runway. I like to see the breadth of things people can make. Speaker A [00:27:11] That's amazing. That's good. Um, so I think, uh, this kind of transition, we've got a few minutes left, but, you know, you've had this foundational and clearly rapidly changing and accelerating, um, platform and tool for people to really create with. You know, where do you go from here? What are the things that you think are, you know, your communities that you're working most closely with are going to demand as they're trying to do more and more exploration going forward. Speaker B [00:27:39] So where do you go from here? Well, the first thing I would say, um, is, um, I think there's a lot of technical challenges to be solved, uh, to make this even more easier and accessible and more steerable in the sense that you can use these models in really artistic like processes. Um, my philosophy, and I think what I believe is that at some point, you will see we're using the words filmmaking and media to refer to this, because it's the closest set of languages and words that we can have to describe what we're saying. Um, I think at some point we start really labeling this something else. We just don't have the words for it. Um, photography, when it first appeared in the 18 hundreds, uh, people didn't have the word to describe what they were saying. And so the way they describe it was like, this is a mirror with memory, which I think perfectly encapsulates the photographers, right. And so I think what's next is for us to, like, discover those new words, those new languages, the new grammars, the new possibilities of how we're going to make this even more, um, accessible, uh, and usable. And for communities that we're working with, um, I think it's again, pushing the ones that are currently within the story, the storytelling like worlds, Hollywood, but also beyond that. So, for example, we just hosted a, uh, 48 hours competition on the weekend. It's 48 hours to make a film with thousands of applications around the world. And those are the communities that we're trying to, because those are the people who are stepping in and experimenting as much as they can with the technology. Speaker A [00:29:08] Is there a place where we can see all of that amazing work happening? Speaker B [00:29:11] If you just search gen 48, uh, Runway, you'll basically see the amazing set of stuff people are making in just 48 hours. Speaker A [00:29:18] Uh, we could definitely go check it out. Chris, thank you so much for your time today and for sharing all this with us. Thank you, thank you, thank you.