Discussion about this post

User's avatar
dan mantena's avatar

great post and I share a lot of your thoughts on this. I think regarding the current deficiencies of LLMs, I am reminded a phrase from Sam Altman at the World Economic Forum earlier this year that haunts me still "these are the dumbest the models will ever be".

"What would the effect of providing people with such capabilities be? I don’t know, but it’s certainly an interesting direction to explore!"

One reason I am not a fan of frontier LLM models is that don't really support human-AI collaboration but promote straight displacement of work. In this case, I think student will use LLMs to just do all the work for them, similar to how high school kids used GPT3 when it was first released. Hopefully I am wrong and this technology forces us to try to evolve our critical thinking skills to be more innovative than what LLMs can produce.

In terms of the data viz process I think GPT is around the 50% percentile mark in terms of performance compared to the average knowledge worker. I am expecting GPT5 to be in the 80-90th percentile based on the scaling laws of AI.

Based on your experience with GPT with respect to data viz, where which level do you think the latest GPT model falls under? https://arxiv.org/pdf/2311.02462

Level 1: Emerging, equal or somewhat better than an unskilled human

Level 2: Component, at least 50th percentile of skilled adults , ex. Watson, LLMs, writing/simple coding

Level 3: Expert, at least 90th percentile of skilled adults, ex. grammarly, Dall-e

Level 4: Virtuoso, atleast 99th percentile of skilled adults, DeepBlue (chess)

Level 5: Superhuman, outperforms 100% of humans, ex. AlphaFold, AlphaZero, AlphaGo

Expand full comment
3 more comments...

No posts