Foundation models

0:09 Hey folks, welcome back to generative AI for beginners here at Microsoft Learn. I am Pablo Lopez, cloud advocate in .NET in artificial intelligence and with here with Carlotta. Say hi Carlotta. Hi, hi everyone. I'm Carlotta Casaluce. I'm a cloud advocate focused on artificial intelligence. That's incredible. So, today let's talk about a little bit on the basics. You're starting you started on the introduction. Now let's go deeper. Let's see how can we explore and compare different LLMs.

0:32 Of course, now that we have on Azure AI our LLMs, but if you want to take a look at that we can take a look later. But we're going to talk a little bit about comparisons on language models. So, going from here you're going to see a little bit of foundation models, language models and later Carlotta you show a little bit on Azure AI. So, what are you going to learn today? You may have heard of foundation models.

0:55 So, let's understand foundation models versus LLMs and language models. How to classify a language model because they have a lot of classifications for them. And then Carlotta you show a little bit of Azure AI. Let's go with foundation models versus LLMs. But to understand this let's start with foundation models themselves. Why? Because you may have seen a lot of a little bit of heard of uh foundation models, LLMs, etc. You may have been confused on how those work.

1:20 Let's think about it a little bit. The foundation models they do the following. So, usually it's the base is a the basis of you're deploying something new. So, what does you mean about it Pablo? Okay. Let's think about like this. Imagine that I have a foundation model that I want to do multiple tasks. So, here I have a teaching one. So, it can both like a multi-modal data source to train it. So, it can have teaching materials, videos, interaction, and the subject mat- matters.

1:46 And then we have like our task goals. So, it can have multiple assets, students, educators, facilitate learning, teaching, and there is a subject matter. So, you can see that we have a lot of input. Here we have a bunch of things we are training with, and they we can have a lot of tasks and goals. But, you see that it does a lot of things, right? But, it may not do them perfectly. We may need to, you know, guide them in how to do X or Y.

2:31 So, as educators, how to assist them, right? Because we don't have like instructions to, you know, understand how it's helping it. Then, it's foundation. Why? Because it's the basis of our constructing new solutions. So, that's why you're calling them foundation models. And let's bring this graph as well. So, let's think about this. And foundation models were brought by Stanford researchers researchers. So, that's why they are using like that. So, the foundation models here are very simple.

2:55 They have some prerequisites. They need to be pre-trained, generalized, adaptable, large, and self-supervised. So, you may see right here that foundation models can cover a lot of things. And you might may know this well if you know a little bit about LLMs that they fulfill all those five. So, yes, LLMs are foundation models because they fulfill all those five. However, not all foundation models are LLMs because they can they don't use language sometimes. As I said, they use a multi-modal way to, you know, learn.

3:20 So, they have a multi-modal way to, you know, to answer, to interpret. They don't use uh tokenizers sometimes. So, that's the difference. So, yes, foundation models are very important because uh our LLMs are foundation models, but not all foundation models are LLMs. That's incredible, but now that we understood those, we're going to start talking a little bit of our language models themselves, right? So, then it brings a classic, right? Open source versus proprietary. So, let's think about open source a little.

4:03 So, when you have open source, nowadays, we had like a big extension, right? A lot of companies are doing open source language models, which are incredible. But, let's think about a little bit. So, what are those means, right? So, open source, usually they are open sourcing some of the parts that they use to, you know, or train the LLMs. So, imagine that you have like the code that they used to train. Some provides the weights. If you don't know the weights, are basically the the fine-tuning of the the language model, so I can like go inside and try to investigate.

4:28 Or even some provide like the full model that you can just download and go and, you know, start having fun with. And you can tweak and do everything. So, these are incredible, but here's the thing, a lot of them are provided by, you know, research groups or NGOs. So, those usually, what they have, they don't can have a lot of no resources to be supported and to be updated with new features or, you know, any updates in security on especially if you think about prompt injection, etc.

5:14 So, you may see here that yes, open source is incredible, but they can have some faults here and there. Or maybe their license aren't permissive enough with what we want to do. So, open source opens a lot of these small issues here and there that you need to think about if you want to solve. Proprietary, usually they are much easier. They are already provided. A lot of those, of course, they're provided by APIs if you go to cloud services.

5:37 But, what is a good about proprietary is that you know you're going to get, you know, a lot of things done correctly. So, you're going to have like updates constantly in the models. Of course, it limits you the the opportunity to fine-tune the model, but usually it means that you're going to have a lot of things that you can already use inside a ready-made solution that you can go and grab and go. Okay. So, you learn a little bit of the open-source models and about the proprietary.

6:00 If you ask open-source, you have a bunch of them, right? You have Falcon, we have a Llama, we have old Llama as well. So, we have a whole bunch here. And our proprietary you can ask about, you know, ChatGPT. So, why I'm saying this? Because you notice that I didn't talk about embeddings, right? So, what are all the embedding one? So, let's pick an OpenAI Ada. So, let's talk about here about their categorizations. You noticing that they have like very strict categorizations here.

6:25 So, let's go through a language model that converts everything to embedding. So, we have a string, we have a prompt, you need some way to communicate. Imagine that, you know, I have a format that, you know, I can communicate with different models, right? So, getting it to convert to embeddings are great because then I can easily imagine that I can use it for systems like Rock that I can store all the embeddings and I can search and then got my information back.

7:12 So, embeddings are very important for systems today on generative AI. And it's not only that, they are used to communicate multiple types of language systems and language models. So, it's just very simple. Compare a string, convert to embedding that everyone can understand. Then we have language models to image generation. That's everyone knows DALL-E. Everyone already played in the heaven as well on web co-pilot. So I have a prompt here. So I have like in quotes and then you're going to put in your generative AI neural network and then boom, you're going to have an image.

7:36 So here I have a painting of a flying dog and here you have a Now, this incredible image. Then the classics. Who did it? Already used text generation, right? Everyone here already had some fun of it. So you go to your classic co-pilot web as well, chat GPT. You can go as a If you even go to hugging face, right? You can ask them on hugging face, a lot of the other models that they have Falcon or Llama.

7:59 We have Mistral and those are incredible. So basically put a prompt and then you have prompt engineer that you're going to see later in this course and then the generative AI will start to write it and then it generates a text. And it can be text, it can be code. We have of course some that are optimized for code, but you can see here that is how it works. So we have Usually those are the three more defined ones.

8:21 Well, of course if you want to break it down, you can do like specialize, you can do a lot of, you know, extra things here and there. Remember, you can always, you know, interact with this those language models and do what you need for, you know, your business, your hobby or anything you want to. Great. Let's talk about service versus model. Remember what I talked before that you can easily access some of those incredible models in the cloud?

8:44 Exactly. Why I said it? Because those are on the service. What it means? Those are already, you know, stored in the cloud, you cannot change that yourself without, you know, going in more inside and try to just fine-tuning, but basically those are stored in there, easy. Usually API calls, you just, you know, send it send your prompt and then you get back an answer. You don't need to deal with scalability. The scalability with the cloud already use of you.

9:31 The security is defined on how is your security on the cloud as well. So, everything is tight and niche and not only that, you can actually integrate with all your other services. So, that's the good part about having this service, right? You don't It's easier to manipulate into sometimes if you want to change a little bit, but of course we're all fine-tuning a bunch of other services, but that's important for you to know. But, after this, you can have like easy way to interact with this.

9:55 The model is a little bit harder because you need to download the model, you need to set up. You imagine that you have a server, right? And you need to take care of the server, take care of the scalability. Imagine that you have multiple clients coming in. So, the model is for you to interact directly. It makes easy to interact with directly. It goes It goes much more easy to try to understand how this is working and the pipeline.

10:18 But, imagine that you need to interact with your cloud service, I need to transfer something. It makes much harder with all the ecosystem. But, you have more full control, but you need to deal with infrastructure cost,