Been experimenting with self-learning (And if you don't like me calling it self-learning, mentally replace it with doing the raspberry) LLMs, and the biggest takeaway I've taken?
Things like governance, observability, and auditability are pretty poor right now. This is just my take on how to improve things.
Also, and surprisingly? The most valuable thing I've seen with my experimentation is failures. Not success. Failures generalize, successes don't. A large amount of tasks can get generalized information for failures, but only specific tasks can get information from successes.
Project is open source, no paid SAAS, not commercially viable. I'm just an excited hobbyist with absolutely no business sense whatsoever.
I've had my 5080 running 24/7 over the past 2 weeks to try to understand the impact of quants on models. Came to some strange conclusions which were interesting to me.
I didn't really see a decline until going under Q4 for the most part, and MoEs were impacted far less then dense models.
Kind of cool results, and unexpected. Although this is just an initial set of tests to validate if the quant itself damaged the model, I've got a longer article coming in the future that's going to test these models in devops, coding, and long sessions. Also, I'm aware it's only a 16GiB card! I've got another article series coming out in the near future for much larger VRAM!
The benchmark code, dataset, and results are open sourced. Take a look, tell me I'm wrong (Wouldn't be the first time!), or run the benchmarks yourself!
Wow. This is super neat. After having gotten my hands dirty on a 30+ year old car, I can definitely see why it would be desirable to have a relatively new frame to start from vs. a WWII Jeep.
Monetization. It's incredibly hard to monetize text content, particularly these days where Google's search AI has all but killed clickthrough rates. Youtube however, that will reliably give you revenue share.
Been experimenting with self-learning (And if you don't like me calling it self-learning, mentally replace it with doing the raspberry) LLMs, and the biggest takeaway I've taken?
Things like governance, observability, and auditability are pretty poor right now. This is just my take on how to improve things.
Also, and surprisingly? The most valuable thing I've seen with my experimentation is failures. Not success. Failures generalize, successes don't. A large amount of tasks can get generalized information for failures, but only specific tasks can get information from successes.
Project is open source, no paid SAAS, not commercially viable. I'm just an excited hobbyist with absolutely no business sense whatsoever.