Tobold's Blog
Saturday, June 01, 2024
 
An awareness of mediocrity

Mathematically, in a normal distribution, 95% of all data points are within 2 standard deviations of the mean, and 68% within 1 standard deviation. And a lot of human characteristics, skills, and talents follow a normal distribution. Which is to say that if you take a random human characteristic, e.g. the innate talent of somebody to be good at chess, and a random human, it is extremely likely that this human is mediocre at that. Being mediocre is the norm, not the exception. The awareness of this is slim. 65% of people believe they are above average, which is an obvious mathematical impossibility. And many people with just a sliver of talent believe their talent to be exceptional.

Whatever my talents are in the areas that I am most talented and skilled at, these areas can possibly only cover a rather small part of all human activities and talents. It follows that I, and everybody else, must be mediocre at most things. You might, for example, consider Ernest Hemingway to be among the greatest writers of all time; that doesn't exclude the high probability that he was mediocre at car repair or doing his taxes, but I simply don't know. My personal experience of my work life was that my job, for which I was hired due to having certain certified skills, involved a wide range of different skill requirements, some of which I wasn't as good at. The more different requirements you add to a job description, the higher the likelihood that you can't find a candidate that is exceptionally good at each of them.

The awareness of your own mediocrity can sometimes be very helpful. For example when deciding what career you want to pursue: There are some careers in which only the exceptionally good make money, e.g. rock stars, athletes, painters, and influencers, while people who choose that career with only mediocre skill end up not making even enough for a living. Other professions, like plumber, accountant, or engineer, do not really require people to be exceptional, and a person with a mediocre talent in that field will do perfectly fine.

I do have a few talents regarding gaming. But again, gaming has a lot of different forms, and spans a wide range of different required talents. It is only normal that I am at best mediocre at some of those talents, and thus at the games which require those talents. Which is why I am much in favor of games having variable skill requirements. Games where you need to be "this good" to even play the basics automatically exclude the part of the audience that is average or worse at that particular skill. Any game that requires any skill is usually most enjoyable when the skill requirement is a bit challenging, but not frustrating, and you can't achieve that with a single fixed setting for everybody. The guy on the internet gloating about how good he is at game X is still extremely unlikely to be as good at games requiring a different skill set, and in other aspects of real life.

There is currently an ongoing discussion of how close we are to achieve artificial general intelligence. The discussion is somewhat misleading in the context of people fearing for their jobs. General AI will always be rather expensive compared to narrower AI. A company would replace you with AI if that is a) cheaper than you are, and b) less likely to result in some catastrophic failure. So a) is unlikely to be true for general AI, while the current generation of narrower generative AI is still failing on b), and recommends sticking cheese to pizza with glue, or produces pictures of humans with the wrong number of fingers. However, it is rather likely that large language model generative AI achieves a mediocre level of writing more or less consistently; after all, if you basically take all writing, disassemble it, and put it together again, what else can you get than a mediocre result? Thus AI will never write at the level of Hemingway, but might very well be able to write Video Game Release Dates: The Biggest Games of June 2024 and Beyond (link more or less randomly chosen, no insult meant to the human author) without anybody noticing it wasn't written by a human. While writing legal briefs is probably still better to be left with humans. In the example mentioned above, I was unsure whether Ernest Hemingway was any good at car repair or doing taxes; I am pretty certain that I wouldn't want ChatGPT to do either for me. Nothing suggests that the skills of an AI follow a normal distribution, and it is more likely that mediocre is the best you can get, while a bad result at a task it isn't optimized for is highly likely.

Comments:
I think you are a little too pessimistic on the ability of AI to achieve above average or higher levels of accomplishment. In fact law is the perfect target as such a robust body of well documented examples exist and most cases exist within a pretty narrow spectrum. I think the vast majority of legal briefs will be AI authored in short order. Same for coding, marketing, etc. where exist a clear set of patterns to follow.

True innovation and research is a very different animal. Still no evidence that LLM will achieve sentience that might allow for that. .
 
For good legal briefs you need something better then our current generative ai, they can fake up a brief that looks legal but they dont understand that the precedent controls what you can argue, or visa versa if you argue x you need to come up with a precedent that is pretty close to s

I am with kobold, good research and accurate legal briefs are beyond our current generative AI.

people are also bad at estimating the average when they are not. ie someone who is playing at say a 1% percent game level will put the average at a level say only 5% achieve.

 
Rules of precedent and such are all well documented. Current frontier models will have no problem with that. Having worked on and off with legal on topics over the years i think you are vastly overestimating the average associates competence if you think a current gen model can not replicate them at the 80th percentile at least. The vast majority of work done in law is mostly regurgitation of a limited number of scenarios.

Folks tend to over focus on the errors that AI make and forget the humans they replace make plenty of errors as well. Humans just make their errors at a glacier pace. AI are doing it at the speed of electrons.
 
Strictly speaking, 65% of population can be "above the average" (not in the normal distribution, but still). 50/50 spit is enforced by median.
 
At least on chess, it has been demonstrated that GPT-like AI can be a better player than 90% of the player, and that it has created an internal model of the board.
Without being an expert in AI, my understanding is that AI will be better than 65% of the population on all tasks. But most of us are better than 65% of population on at least one task.
Si if some jobs will disappear I am not afraid for the vast majority
 
Post a Comment

<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool