The topic LLM From Scratch is a hands-on workshop where you write every piece of an AI from… is currently the subject of lively discussion — readers and analysts are keeping a close eye on developments.
This is taking place in a dynamic environment: companies’ decisions and competitors’ reactions can quickly change the picture.
Have you ever wondered how an LLM works? Perhaps you know some of the basics, such as how they use tokens to get the job done, and how their text generation is somewhat like the autocomplete feature on your phone; however, how would you go about making one from scratch? Where would you even start?
If you want to learn how to make an LLM from scratch, then you’ll need to check out this DIY course, called, uh…”LLM From Scratch.” As one of the best examples of the phrase “does what it says on the tin,” LLM From Scratch starts you off with nothing and walks you through a miniature course to make your own AI.
Most people can find a use for a local LLM on their PC, and here’s how I use mine.
As spotted by Hackaday, LLM From Scratch is a free course where you go from absolutely nothing to creating your own AI. You’ll start off by learning about tokenization and how LLMs parse data, and then finish by training a model on poetry until you’re happy with it. You’re not going to be taking on ChatGPT or Gemini with it, but it will run on a laptop just fine, so it’s great for hobbyists.
Character encoding, vocabulary size, why BPE fails on small data
If you’d like to learn more, be sure to head over to the LLM From Scratch GitHub page to read more about the inspiration and what to expect when undertaking this workshop.