Yesterday, Meta released its biggest and baddest large language model ever, which also happens to be free and arguably open source.

It took months to train on 16,000 Nvidia H100 GPUs, which likely cost hundreds of millions of dollars and used enough electricity to power a small country.

But the end result is a massive 405 billion parameter model with a 128,000 token context length, which according to benchmarks is mostly superior to OpenAI's GPT-4.0 and even beats Claude 3.5's SONNET on some key benchmarks.

But benchmarks lie, and the only way to find out if a new model is any good is to vibe with it.

In today's video, we'll try out LLAMA 3.1 Heavy and find out if it actually doesn't suck like most Meta products.

It is July 24th, 2024, and you're watching The Code Report.

AI hype has died down a lot recently, and it's been almost a week since I've mentioned it in a video, which I'm extremely proud of.

But LLAMA 3.1 is a model that cannot be ignored.

It comes in three sizes, 8B, 7DB, and 405B, where B refers to billions of parameters, or the variables that the model can use to make predictions.

In general, more parameters can capture more complex patterns, but more parameters doesn't always mean that the model is better.

GPT-4 has been rumored to have over 1 trillion parameters, but we don't really know the The cool thing about LLAMA is that it's open source.

Well, kind of.

You can make money off of it, as long as your app doesn't have 700 million monthly active users, in which case you need to request a license from Meta.

What's not open source is the training data, which might include your blog, your github repos, all your facebook posts from 2006, and maybe even your whatsapp messages.

What's interesting is that we can take a look at the actual code used to train this model, which is only 300 lines of Python and PyTorch, along with a library called Fairescale to distribute training across multiple GPUs.

It's a relatively simple decoder-only transformer, as opposed to the mixture-of-experts approach used in a lot of other big models, like its biggest open source rival, Mixtral.

Most importantly though, the model weights are open, and that's a huge win for developers building AI-powered apps.

Now you don't have to pay a bunch of money to use the GPT-4 API, and instead can self-host your own model and pay a cloud provider a bunch of money to rent some GPUs.

The big model would not be cheap to self-host.

I used LLAMA to download it and use it locally, but the weights weigh 230 gigabytes, and even with an RTX 4090, I wasn't able to ride this LLAMA.

The good news though is that you can try it for free on platforms like Meta.ai, or platforms like Grok or Nvidia's Playground.

Now the initial feedback from random weirdos on the internet is that Big LLAMA is somewhat disappointing, while the smaller LLAMAs are quite impressive.

But the real power of LLAMA is that it can be fine-tuned with custom data, and in the near future, we'll have some amazing uncensored fine-tuned models like Dolphin.

My favorite test for new LLMs is to ask it to build a Svelte 5 web application with runes, which is a new yet-to-be-released feature.

The only model I've seen do this correctly in a single shot is CLAWD 3.5 Sonnet, and LLAMA 405B failed pretty miserably, and seems to have no awareness of this feature.

Overall though, it is pretty decent coding, but still clearly behind CLAWD.

I also had it do some creative writing and poetry, and overall it's pretty good, just not the best I've ever seen.

If we take a minute to reflect though, what's crazy is that we have multiple different companies that have trained massive models with massive computers, and they're all plateauing at the same level of capability.

OpenAI was the first to make a huge leap from GPT-3 to GPT-4, but since then, it's only been small incremental gains.

Last year, Sam Altman practically begged the government to regulate AI to protect humanity, but a year later, we still haven't seen the apocalyptic Skynet human extinction event that they promised us.

I mean, AI still hasn't even replaced programmers.

It's like that time airplanes went from propellers to jet engines, but the advancement to lightspeed engines never happened.

When talking about LLMs, artificial superintelligence is still nowhere in sight, except in the imagination of the Silicon Valley mafia.

It feels wrong to say this, but Meta is really the only big tech company keeping it real in the AI space.

I'm sure there's an evil ulterior motive hidden in there somewhere, but LLAMA is one small step for man, one giant leap for Zuckerberg's redemption arc.

This has been The Code Report, thanks for watching, and I will see you in the next one.