The Dunning-Kruger Effect and a Few Million Aspiring Authors
A few nights ago I was at the agents party for the British Society of Children’s Book Writers and Illustrators. It’s where mostly unpublished writers get to listen to, then mingle with, literary agents—the first-level gatekeepers to the arcane world of publishing. It is a curious thing to be in a room full of people bursting with ambition, people who believe that with patience and a little luck their talent will be recognised and they will be published and maybe, just maybe, they will take over the world.
No doubt others with a more sanguine and nurturing nature gazed upon that room and saw nothing but potential. Me? I saw a room filled with dreams and hopes that I know are borderline delusional.
The simple fact is, most of those people will never be published. They will keep writing, keep submitting, keep believing in themselves, but they will never make the grade. They will blame the system, bad timing, bad luck, but will never be able to understand that they just weren’t good enough.
This quirk is called the Dunning-Kruger effect, and it is evident everywhere we look for it. The psychologists Dunning and Kruger put their names to the phenomena whereby incompetent people over-estimate their own skill and under-estimate the skill of others. Even once they have been educated about their incompetence, incompetent people tend to remain both incompetent and obstinate in their belief that they are above average.
So, for instance, a majority of people rate themselves as above average drivers, as being of above average intelligence, as above average lovers, humourists, logicians, candle-makers and pretty much anything that requires skill.
Yet it’s simply not possible for a majority of people to be above average—it defies the concept of average. Someone must be in the below average group—in fact, half of all people are below average, but almost no one knows that they’re the one letting the team down. Too incompetent to recognise their own incompetence.
In some situations, though, being below average doesn’t equate to absolute incompetence. Take driving. It is a complex skill. The fact every road is not a trail of burning wrecks demonstrates that the vast majority of drivers are skilled. They are good drivers. But there is still an average point of skill. So even someone below average can be a good driver. The thing is, it’s hard for that individual to recognise they are below average when they notice they are not trapped in a burning wreck.
Driving is the tip of the iceberg. Fifty percent of neurosurgeons are below average. Fifty percent of crane operators are below average. Fifty percent of pilots, counsellors, astronauts, nuclear physicists, submariners, nutritionists (especially nutritionists), literary agents, editors, publishers, and, yes, my tribe: writers. Below average.
One of the frustrating things for all aspiring authors is how difficult it has become to be heard above the maelstrom of voices seeking publication or publicity. There are just so many people writing novels. So many people approaching agents. So many people self-publishing their own work.
Literary agents have what they call the slush pile: the pile of submissions often hundreds deep where aspiring authors pitch their work. Most of it is not worth their time. Most of it should not be published. Most of it is slush. Which is to say, most of it is either incompetent writing or incompetent story-telling, or both, submitted by incompetent aspiring authors who do not know they are incompetent. If they knew how incompetent their work was they wouldn’t go to the effort of submitting, only to suffer the pain and humiliation of rejection, time and time again.
The problem is what we know. Aspiring authors know—indeed, are obsessed with the fact—that the occasional gem is plucked from the slush pile. They also know (or think they know) that their work is a gem, so… one plus one equals every literary agent receiving thousands of pitches each year.
Pity, then, the poor rare bastard who really is wielding a literary gem. How easily they can be looked over amidst all that slush. Their crisp diction is drowned out by the white noise of incompetence. And if they self-publish it is much the same. How to find the self-published gem when there are half a million or more titles published by incompetents each year?
So, what is it about writing that makes so many people think they have rare talent? That they are worthy of being published?
What is it that encourages every man and his dog to casually mention that they were thinking about writing a novel, only to then mention that they haven’t yet got round to it (like it’s some easy thing to do, when even shit novels are hard to write)?
Before I answer that, consider the counterpoint. How many people contemplate going into neurosurgery without training? How many people think they could just drive a submarine if they wanted to? No, there’s something about writing.
It’s that we are engaged with writing every day. We use words to communicate all day long, and most of us use the written word daily, perhaps hourly, perhaps every waking moment depending on the job. Reports, emails, messages, memos, birthday cards, angry notes to housemates, letters to a teacher. Literacy is almost unique in that, in western society, it is a skill nearly everyone has and uses every single day and which we know takes great skill. If it didn’t, there would be no such thing as illiteracy.
So we all know we are skilled when it comes to words and writing (unlike when it comes to neurosurgery and submarining). But being skilled, and being amongst the elite of the elite, are two very different things.
This is where Dunning-Kruger comes into play. We all know that most people are not good enough writers to be published, but we also have first-hand evidence of our own skill at writing. So, we try to honestly estimate our skill.
Which is the exact thing that both a pretty good writer and a comparatively incompetent writer who knows no better would think. Thence, the slush. Piles and piles of slush.
Even knowing about Dunning-Kruger, even having an honest, objective assessment of your writing that comes back in a report that tells you that you are incompetent will not be enough to pry you away from your dear delusions.
Even the person reading this will refuse to countenance the fact that they are the exact type of incompetent aspiring author that I am referring to—despite the fact that is exactly what I am telling you. That’s right, you.
The other guy?
No, you. You are incompetent. You don’t know what you are doing. And you are too incompetent to know otherwise.
Yeah, those poor schleps are deluded.
No—statistically I know for a fact that you are probably amongst the deluded.
Yeah, but I know me, so I happen to know I’m one of the rare gifted ones.
But that is exactly what you would say if you were deluded.
Or if I’m not deluded and really am gifted, destined for greatness.
Is there nothing I can say to get my point through your incompetent head?
There is nothing you can say to convince me—a competent writer—that I am incompetent and ignorant of that incompetence.
Well at least we agree on some of that. If I could convince you of your incompetence—if you were the sort of competent person whose competency allows them to perceive their own incompetence—then you wouldn’t be the person I’m talking about at all.
Of course, this whole rant of mine is also symptomatic of self-serving bias. Take a person, not unlike myself, who wants to be a successful author, but is currently unpublished (though through no lack of trying). Why have they failed so far?
The author tells himself that it’s down to external factors. It’s because of the absurdity of the agenting process (agents with narrow interests, petty rules, the needle in a haystack approach of the slush pile). Because of the whimsical trends in publishing (vampires the year before, wizards this year, bigfoot erotica next). Because of bad luck (they probably lost my submission). Because of another writer stumbling across the same idea and getting it published just as I was about to submit. And because of the proliferation of other aspiring authors who flood the markets with dross that diminishes all of us.
That’s why I’m unpublished. It has nothing to do with my choices (writing something H. Rider Haggard might submit if he was still alive) or my style (modern archaic, anyone?) or the inclusion of chainsaws as central plot-points. No. It’s the other guys who screwed me.
Obviously the subtext throughout this is that I am not a victim of the Dunning-Kruger effect even while suggesting I am. I won’t lie; you’re right; I believe I’m above average. More, even. I think I’m well above average.
Now, don’t judge me for that. You think the exact same thing about yourself. The only difference between us is that I at least recognise the possibility that I may be so incompetent that I don’t know it. You? You’re a true believer. No doubt in your mind.
 As an aside, an audience poll at the SCBWI agents party suggested that only about 10% of those in attendance were illustrators. Illustrations are such a huge part of children’s books that by all rights the percent should be much greater. But it might make sense when you consider that illustrating is not something we are engaged in every day. Though we see it every day, we do not practice it every day (as we do with words). If we did, God only knows how much more white noise there would be for aspiring illustrators. [It would be interesting to find out from agents whether there is the same ratio of poor:good illustrating submitted as with writing. It would also be interesting to hear the perspective of aspiring illustrators: how much white noise is there, and how hard is it to get noticed?]