AI has the world panicking right now, but I do think it is funny seeing the responses from the inevitble increase of LLM computation. Tokens are far from cheap and a moore's law effect in this realm is debatable to say the least, but F500 companies everywhere are really, really concerned about these financials.

I like to think its quite similar to the standard project management triangle of Fast, Cheap, and High Quality. For LLMs, it is no different. You can really only pick two.

The Pick-Two Problem

If you want High Quality and Fast, you’re looking at frontier models with massive parameter counts, much like whatever the latest Opus Model is. They’re certainly brilliant, but it almost feels like an understatement to say they are expensive.

If you want Cheap and Fast, you’re looking at smaller, distilled models. They’re great for simple tasks, but the consensus that I've seen among developers is that they're genuineley just not good enough for production code environments.

If you want High Quality and Cheap... eh, honestly I don't know how viable this is. May as well be worth it to go back to older development cycles.

The Token Anxiety

Because of this it seems as though many companies are genuinely worried about token consumption. I've heard more and more about developers gettting thorough documentation on how to reduce token usage and reduce AI usage as a whole. Really interesting to think about nonetheless, quite interesting how fast things are changing. Anywho, this is all just procrastination thoughts I'm writing down, back to work.

Signature
../blog