AI with Meta Llama 3 — Coding Tutor’s Impressions

I came out unimpressed with the developer ecosystem for Llama 3. I will stick with Microsoft AI which has far better documentation, better developer support and profitable $$$ prospects for years to come.

I recently picked up a small contract related to AI, and I have been reading about getting into the AI industry. I have been putting some foundations for many weeks now to venture into AI tutoring work. Now, I have reached a point where there is some grasp about the foundations. How, I can make money by expanding my tutoring offering to include AI related teaching.

When I think of AI, there are many divisions of course.

There is generative AI for images, which I have already dabbled in. more about that can be found in some of my posts.

To me playing around with Stable Diffusion was like a fun gateway into the world of AI development. I like fashion and it is always fun to the creative ways AI is able to generate variety of clothing. It also helped me get into the AI zone.

Further, whether you are generating images or generating text, the underlying technology is really the same. Like how, the underlying knowledge of math is common for a civil engineering and computer science engineering, even if they both end up in completely different industries.

So, I got my beak wet with some AI. Exploring and deciding to see where I can learn AI from a developer/tutor perspective. To me, it seems, there are three main names that comes to mind with generative AI.

There is ChatGPT OpenAI, the big daddy.

Microsoft AI (tightly integrated with ChatGPT), the big daddy’s daddy.

Meta Llama 3

I am already part of the .NET ecosystem because of my work since 2012. However, before I embrace something, I like to look at the ecosystem and see where it is going and such. For instance, for the backend and front-end framework, in my tutoring plans, I decided to choose Next JS, a few months ago.

So, I went to the meta site, filled up their form and agreed to their terms and got started.

After downloading about 50 GB worth of models, I realized, we cannot actually use these models directly. The download page makes no mention of this. But I blame myself. I was in a rush of excitement, but it is my fault. I should have spent more time reading the confusing literature.

Then, I realized to run the LLM, I need to use the Ollama library.

It downloaded the models again. and I got some AI chat working alright. It was nice.

However, I need to figure out how to train things, because that is what AI related tutoring and developing comes down to. This is where things got confusing. Once again, the developer documentation keeps making references to using paid API services to start using Llama.

Now, that bothered me. It also would ask me to work across multiple open-source frameworks and libraries. Further, there were frequent references to an A10 GPU. That is, a GPU with 16 GB of RAM. We are talking a modern gaming PC with at least a RTX 4080 or something similar.

That would cost me about 5000 USD in India. I already have multiple gaming PCs, but they top out at RTX 3060 which is the most practical GPU machines I can afford in India, and even they are super expensive. Meta does mention that some models can work on something less than A10 GPU, but the frequent links to paid services kept shaking my confidence.

I understand open source does not equate to free. However, it’s almost as if, unconsciously, open-source projects are frequently pushing the developer and users to paid services by making things impossibly difficult to use and also learn.

Perhaps, I am complaining a lot. But, ultimately, why bother with open source if it’s still going to cost me a lot of money, and on top of that, I am forced to juggle dozens of different tools and libraries for the rest of my AI involved life?

The cost of money is understandable (To make money, I have to spend money. I am staunch follower of that policy), but the cost of time? Getting all these disparate pieces to work.

Ultimately, I must conclude, may be, a one-person developer trying to learn and do things in AI is not really what Llama is about. It’s probably meant for already wealthy developers (who can afford stuff like A10 16 GB RAM GPU) or enterprises.

I am just not their target market.

So, I must choose something that is more convenient and easier to use. All said and done, it seems like, even if will cost me money (and open source will cost me money anyway), Microsoft AI is the better choice going forward.

this is a copy of my original post available here.