Yep. It's worth checking out GPT-6 to see what state of the art actually looks like.

https://m.youtube.com/shorts/sJl1KoSL_o8

In my GPT session today the model was convinced Orange and Blue are not the proper fibers for OM4 LC-LC TX/RX, and it could not tell me the correct ones until I told it they actually were and it can check because there's actually a worldwide standard for it.

Gemini on the freaking free tier got it right at the first try.

And this is not the first time I experienced that GPT - on any version - completely got lost, even when the info for the correct answer is widely available on the Web.

If you ditch generative AI you go back to needing to pay an artist/designer for their work, and they likely charge more than ChatGPT because they might have rent/bills and that sort of thing to cover.

The sort of use cases I was think of (village fête etc.) would never have paid for a professional designer. We're talking about an event that's delighted to raise £500 for the church roof repair fund. Even £50 for a poster is 10% of the fundraising lost.

If it wasn't AI, it would be Comic Sans and some clipart.

I don't think the people pushing that narrative genuinely believe what they're saying. They're just influencing public opinion.

> But don’t be the out of touch Kung Fu master, heir to lifetimes of tradition, that gets mauled by an amateur MMA fighter.

I have an issue with this analogy, since a professional MMA fighter is a highly skilled individual who has honed their body through years of training.

A more apt analogy might be "don't be the kung fu master who believes they can survive an atomic bomb, dropped from far above by a pilot who does not understand how it works, but who has nevertheless been gifted the ability to deploy it."

Some context. Many styles of Kung Fu/Gung Fu do not spar in any meaningful sense. Consequently, they have an overinflated opinion of their own ability and generally do very badly against MMA fighters (who do lots of sparring and pressure testing). There are lots of videos on Youtube of Kung Fu 'masters' getting battered by relatively inexperienced MMA fighters. This was sufficiently embarrassing for Chinese martial arts that they created a sport oriented form called Sanda.

I once saw an early black and white film of 2 Chinese Kung Fu masters fighting in a ring. Some sort of grudge match I think. It was embarrassing. You would see better technique outside a Wetherspoons pub at 2am.

I would say analogy is still apt.

When you look at King Fu master as a software dev who refuses to use AI and MMA fighter as a dev who embraces AI.

Vibe coders who know nothing about software development are untrained rubes who got mechanical arm that thinks for itself, they can maul MMA fighter, they can maul Kung Fu master but at the same time arm might just rip off their own face.

But he said amateur.

You could do this before AI, though. Just download a big repo off github and build the binaries yourself. Or is that not enough obfuscation to convince yourself you made it?

> we cannot RSI

Speak for yourself.

You could train an LLM on a dataset that intentionally excludes certain data, and see if it is able to extrapolate outside of its dataset and come up with those excluded items independently.

For example, remove all code from your training data, and then see if the model can code regardless.

Expecting to go from having seen no code to being able to code within a context window is a pretty high bar. More useful would be asking for code that does something you're pretty confident no-one has done before.

(TBH, when people claim this, I do wish there would at least be occasionally an actual pointer to something that was copied. I understand it's probably not going to be doable in all cases, but without examples it sure feels like a weak statement. In my experience the recent models are good at doing things I'm pretty sure there is not a close reference for in the training data, though for most part I wouldn't classify them as particularly difficult tasks either)

Thanks for the elaboration. (Though I'm not sure dmarinus would agree with your criterion?)

> For example, remove all code from your training data, and then see if the model can code regardless.

Would you accept this weaker version: make up a new programming language (that's guaranteed not to be in the training set), and see if the model (which has trained on programming in existing languages) can cope?

Yes, things are really great everywhere else.

You're speaking directly to his marketing trap.

I don't post there anymore but I do still check my DMs once a week, because every now and then someone will find my work through there, and I need people to find my work in order to survive. I unfortunately can't easily trade that in for a platform that no one visits.