The Wordcount Project, part 1

[mitrian.livejournal.com]

Many, if not all, of us in this community have been lamenting the deterioration of LKH's writing - especially after OB. But is it the content of her writing, or the actual quality, that has changed? Has her lack of editing caused that much of a change, or did the multiple re-writes of her first few books improve the quality of writing in them? And is there some way we can analyze it objectively?

Well, my dear fellow Lashers, I stumbled across some readability tools online this evening. I then found that LKH has provided the first 2 chapters of each of her books as a free download on her site...

What did I find when submitting the first chapter of each of the AB books through a readability analysis tool? The data is

The first problem was to determine which readability test to use for this project. There are many in widespread use, and they all assign different weights to the factors they consider - number of words in each sentence, number of syllables or letters in each word, etc. In my researching of these tools, I found a site that combines several of the most popular, and gives results for each. The site I used is here. It even provides links to the Wiki page for each of the tests used - so for further information on the tests, including the formula used to calculate each one, click on that link.

The text: Since LKH has kindly made the first 1 or 2 chapters of each book available free on her website, I used those. To be consistent, I only used chapter 1 of each book. I followed the same methodology every time: open .pdf, copy chapter 1 text, paste to a .doc, remove page numbers and headers, then copy and paste into the readability test. I made no changes to the text itself aside from removing page numbers and headers, since those would cause inaccuracy in the results. I did use the wordcount tool in my text editor to find the total wordcount for each chapter, and a calculator to divide wordcount by number of pages.

The books: I was able to use most of the AB books for this. However, the files for Bloody Bones and Skin Trade were uncooperative - the one for Bloody Bones does not have selectable text (it looks to me as if the pages may have been scanned), and the one for Skin Trade appears to be corrupt (it thinks it's a .doc, but when opened contains numerical gibberish). So these two books were excluded from analysis, and from the statistics presented below.

The data:

Guilty Pleasures, chapter 1
Pages: 6
Wordcount: 1906 (average 317.67 words/page)
Analysis:
Flesch-Kincaid Reading Ease 90.80
Flesch-Kincaid Grade Level 2.70
Gunning-Fog Score 5.20
Coleman-Liau Index 7.20
SMOG Index 4.00
Automated Readability Index 1.30
Average Grade Level 4.08 Excludes Flesch-Kincaid Reading Ease score.

The Laughing Corpse, chapter 1
Pages: 10
Wordcount: 3408 (average 340.8 words/page)
Analysis:
Flesch-Kincaid Reading Ease 89.50
Flesch-Kincaid Grade Level 2.80
Gunning-Fog Score 5.30
Coleman-Liau Index 7.70
SMOG Index 4.20
Automated Readability Index 1.60
Average Grade Level 4.32

Circus of the Damned, chapter 1
Pages: 7
Wordcount: 2379 (average 339.86 words/page)
Analysis:
Flesch-Kincaid Reading Ease 90.80
Flesch-Kincaid Grade Level 2.60
Gunning-Fog Score 4.60
Coleman-Liau Index 7.00
SMOG Index 3.80
Automated Readability Index 3.80
Average Grade Level 3.78

The Lunatic Cafe, chapter 1
Pages: 6
Wordcount: 1840 (average 306.67 words/page)
Analysis:
Flesch-Kincaid Reading Ease 91.70
Flesch-Kincaid Grade Level 2.30
Gunning-Fog Score 4.90
Coleman-Liau Index 6.90
SMOG Index 4.10
Automated Readability Index 0.50
Average Grade Level 3.74

The Killing Dance, chapter 1
Pages: 12
Wordcount: 4230 (average 352.5 words/page)
Analysis:
Flesch-Kincaid Reading Ease 90.20
Flesch-Kincaid Grade Level 2.90
Gunning-Fog Score 5.10
Coleman-Liau Index 7.00
SMOG Index 4.00
Automated Readability Index 1.40
Average Grade Level 4.08

Burnt Offerings, chapter 1
Pages: 6
Wordcount: 1998 (average 333 words/page)
Analysis:
Flesch-Kincaid Reading Ease 91.40
Flesch-Kincaid Grade Level 2.80
Gunning-Fog Score 5.30
Coleman-Liau Index 7.00
SMOG Index 4.10
Automated Readability Index 1.40
Average Grade Level 4.12

Blue Moon, chapter 1
Pages: 3
Wordcount: 1356 (average 452 words/page)
Analysis:
Flesch-Kincaid Reading Ease 87.50
Flesch-Kincaid Grade Level 3.30
Gunning-Fog Score 5.50
Coleman-Liau Index 7.20
SMOG Index 4.30
Automated Readability Index 1.40
Average Grade Level 4.34

Obsidian Butterfly, chapter 1
Pages: 7
Wordcount: 2441 (average 348.71 words/page)
Analysis:
Flesch-Kincaid Reading Ease 84.40
Flesch-Kincaid Grade Level 3.80
Gunning-Fog Score 6.10
Coleman-Liau Index 7.40
SMOG Index 5.00
Automated Readability Index 1.90
Average Grade Level 4.84

Narcissus in Chains, chapter 1
Pages: 10
Wordcount: 3461 (average 346.1 words/page)
Analysis:
Flesch-Kincaid Reading Ease 85.00
Flesch-Kincaid Grade Level 4.10
Gunning-Fog Score 6.50
Coleman-Liau Index 7.60
SMOG Index 5.30
Automated Readability Index 2.60
Average Grade Level 5.22

Cerulean Sins, chapter 1
Pages: 13
Wordcount: 3942 (average 303.23 words/page)
Analysis:
Flesch-Kincaid Reading Ease 88.10
Flesch-Kincaid Grade Level 3.40
Gunning-Fog Score 5.80
Coleman-Liau Index 7.00
SMOG Index 4.40
Automated Readability Index 1.70
Average Grade Level 4.46

Incubus Dreams, chapter 1
Pages: 10
Wordcount: 4208 (average 420.8 words/page)
Analysis:
Flesch-Kincaid Reading Ease 80.40
Flesch-Kincaid Grade Level 5.70
Gunning-Fog Score 7.70
Coleman-Liau Index 8.40
SMOG Index 5.50
Automated Readability Index 5.20
Average Grade Level 6.50

Micah, chapter 1
Pages: 15
Wordcount: 2597 (average 173.13 words/page)
Analysis:
Flesch-Kincaid Reading Ease 87.00
Flesch-Kincaid Grade Level 3.60
Gunning-Fog Score 5.70
Coleman-Liau Index 7.10
SMOG Index 4.70
Automated Readability Index 2.00
Average Grade Level 4.62

Danse Macabre, chapter 1
Pages: 15
Wordcount: 5713 (average 380.87 words/page)
Analysis:
Flesch-Kincaid Reading Ease 86.60
Flesch-Kincaid Grade Level 3.90
Gunning-Fog Score 6.10
Coleman-Liau Index 7.20
SMOG Index 4.60
Automated Readability Index 2.50
Average Grade Level 4.86

The Harlequin, chapter 1
Pages: 12
Wordcount: 4185 (average 348.75 words/page)
Analysis:
Flesch-Kincaid Reading Ease 87.90
Flesch-Kincaid Grade Level 3.60
Gunning-Fog Score 6.00
Coleman-Liau Index 6.90
SMOG Index 4.60
Automated Readability Index 2.10
Average Grade Level 4.64

Blood Noir, chapter 1
Pages: 13
Wordcount: 4361 (average 335.46 words/page)
Analysis:
Flesch-Kincaid Reading Ease 91.20
Flesch-Kincaid Grade Level 3.00
Gunning-Fog Score 5.20
Coleman-Liau Index 6.40
SMOG Index 4.00
Automated Readability Index 1.40
Average Grade Level 4.00

Flirt, chapter 1
Pages: 18
Wordcount: 4920 (average 273.33 words/page)
Analysis:
Flesch-Kincaid Reading Ease 85.80
Flesch-Kincaid Grade Level 4.60
Gunning-Fog Score 7.20
Coleman-Liau Index 6.90
SMOG Index 5.20
Automated Readability Index 3.50
Average Grade Level 5.48




Interesting bits of data: The average words/page, and average grade level per book. The numbers, with a few outliers, seem to fall into a small range. To get a better idea of this, I graphed them.

Important note about the graphs: The x-axis numbers are the order in which the books were published. So 1 is Guilty Pleasures, 2 is The Laughing Corpse, etc. Thus reading the graphs from left to right gives an idea of how her writing has changed (or not) over the years she's spent writing the books.

First, the graph showing average words/page. This count is probably a bit low for most of the books - text starts, on average, about halfway down the first page, and usually the last page of the chapter is not completely full. Still, this gives us at least a rough idea of how many words are, in average, on a page of each book.



See the huge dip for Micah? Many of us have been saying that the huge margins and insane line spacing were to pad the pagecount. Even if she kept the extra blank pages at the beginning of each chapter (which were not included in the words/page calculation), this book has fewer than half the words/page of most of the other books. So just formatting it in the same way as the other books would cut the page count by almost half.

The average words/page for these books comes out to 335.81.

Ok, now on to grade level. I have another graph! Most of her books are written on a fourth-grade level. There are a few that are upper third-grade level, two at a fifth-grade level, and one - Incubus Dreams - is all the way up a 6.5. Her aspirations to be recognized as a Great Literary Genius kind of fail in the light of this data, I think.



Now, what happens if we graph the data differently? Is there a correlation between words/page and grade level?



Not really. But then the readability tests don't look at pages, they analyze number of words/sentences, number of syllables/word, etc. It is interesting to look at the data this way, though - it seems that the number of words/page have nothing to do with the grade level of her writing. It also shows clearly that the majority of her books have a roughly similar word/page count. This opens itself to some speculation (such as the relationship, if any, between her "pages" written per day and actual pages of the finished book; the question of whether she can write at a higher grade level, but is following instructions from her publisher; etc.), but that is not the purpose of this post. My intention here is to present the data, and reserve speculation and interpretation of the data for the comments.

Coming soon, in Part 2: Similar analysis of the MG books (later this week). And then the thrilling conclusion in Part 3, where I will compare results for both series to see if there is a significant difference.

Comments

[electricpower.livejournal.com]

*shivers with pleasure*

D... data...

Thank you not only for this fascinating post but for linking that readability tool as well, which will provide me with nigh uncountable hours of entertainment!

[mitrian.livejournal.com]

Thank you! I'll have Part 2 up later in the week (I'm not letting myself work on it until I've finished at least another week's worth of lesson plans).

There are many authors I'd like to compare like this. I also would like to find a tool that includes grammar and complexity of sentence structure, not just sentence length...

I'm also tempted to run some of LKH's blogs through the same analysis, and see how her more casual writing compares to her published work. I suspect the painfully long run-on sentences will skew her results a bit.

[mad-troll.livejournal.com]

Even if I'm skeptic about those measurings - as far as I'm concerned both syllables-count & sentence lenght are unimportant next to syntax - this is awesome, detailed work.

*impressed*

[kyle-blake.livejournal.com]

Hopefully the OP won't mind but I'm bored so I ran the first chapter of The Dresden files books 1 - 12 and Backup through the readability tool and came up with an average grade level of 5.61. The highest scorer was Small Favour with 7.66 and the lowest Gravel Peril with 4.50.

That compares pretty favourably with LKH's average average of 4.56, which frankly was a little surprising to me since I find Butchers writing easier to read.

Awesome work OP, and has you can tell, it has definitely sparked my interest. Looking forward to part 2!

[dwg.livejournal.com]

Okay, poking around with the analysis tool, the only thing I can think of for ID's first chapter having a higher grade level is because there's less dialogue. I've been putting in a couple of my drafts to see what happens, and the grade definitely goes up if I have a chunk that's more narrative. ID starts off with Larry & Tammy's wedding, so we're treated to all kinds of exposition on who they are, the Halloween theme and why Anita gets to wear a suit and won't get married blah blah blah. This probably explains why there's so many more words in the opening chapter. It's interesting that Blue Moon is the only other one that has more words, because that one starts off with a description, too.

I'm looking forward to parts two and three of this, it's most interesting.

[dreamstrifer.livejournal.com]

Wow, I scored pretty well (higher than LKH's highest) on my first chapter of my latest, half finished novel. I tried to paste in all 55K that I"ve written and it seems to have broken it. Oh well.

[pith]

This post pretty much epitomizes what I love about this community: we put more thought into LKH's works than her "true fans" do... and quite often, more than she does herself.

[phyrra.livejournal.com]

This was very cool to read! Thank you for taking the time to put this together. It's fascinating. I'll be checking back for your parts 2 and 3.
I always felt like Obsidian Butterfly (the first book I started with, which caused me to go back and read the rest) was the last good book.

[anaquana.livejournal.com]

Woo! My first chapter got an average grade-level of 6.14!

Thank you for posting this. I will be having lots of fun with this new tool. :)

[jerel.livejournal.com]

I teach middle school English, and this was totally fascinating to me. Thanks for doing this analysis.

[kisstheground.livejournal.com]

that's what i've noticed with my own writing: the narrative scores higher, as does third person versus first person. i'm guessing because first person and dialogue tend to be more casual because it's actually in a voice, and few people speak the same way they'd write?

[countessof-roth.livejournal.com]

My husband who used to read these books NEEDS TO SEE THIS DATA. I KNOW if he sees it he will see WHY I keep ranting about how her writing is CRAP.


I'd love to see how the Twilight books (which I also hate) compare. I'll bet they are higher on the grade-level......

[tehuti.livejournal.com]

This is extremely interesting stuff. Thank you for your effort. :-)

[rodentfanatic.livejournal.com]

WHOA

I just...no words. Amazing work you put into this, and the results are indeed interesting!

For me, it was the content, not the writing level, that pushed me away, personally. Still, amazing to see it all laid out like this

Thank you!

[denouement16.livejournal.com]

Thanks so much for doing this. I love the graphs :)

My perception of LKH's writing in the last few books is that the books aren't as tight as they used to be. There's more filler and less plot. Too bad there's no simple test for plot-to-filler ratio.

[mardelwanda.livejournal.com]

She's been having a lot of painfully long run-on sentences in her published works also. She's been kind of rambling, with no "tightening" up of the book that a lot of other authors work on with their editors.

[maiskorn.livejournal.com]

Really interesting statistics! I didn't even know these tests exist. While they don't really define the quality of the work, it's nice to see a new take on things (we're discussing the books all the time under different more subjective aspects, after all).

(Hm, random one shot fanfiction of mine got an average grade level of 8.7, and my English is bad. *mulls over that* To read this you need to speak English rather well so you can still recognize it when it's butchered?)

[dwg.livejournal.com]

I think that might be it. I write mostly in third person, and I tried putting in something that's got heeeaaaaaps of dialogue and it came out with a low score, then I went back and found another draft that's got more exposition, and it went higher. Then I took out all the dialogue and it went up even more. So I'm thinking that dialogue pulls it down because people don't talk for pages at a time, and if there's a discussion going on, then their sentences will be reasonably short. In narrative, you get to use words that you wouldn't ordinarily put into dialogue, and can linger on the details with love and exposition.

That said, none of these tests take into account what words are being used and the context, because yeaaaah ID is a bad book and it should feel bad.

[jazzymegster.livejournal.com]

Interesting stuff! (I wish I understood what some of the scores meant, but since there's linkage, I can check that out on my own).

Can't wait to see the next two parts :)

[shadwing.livejournal.com]

For giggles I used Jim Butcher's Turn Coat as another test, first chapter pulled from his site and here are those results

Flesch-Kincaid Reading Ease 83.90
Flesch-Kincaid Grade Level 3.90
Gunning-Fog Score 6.30
Coleman-Liau Index 8.00
SMOG Index 4.90
Automated Readability Index 2.30
Average Grade Level * 5.08

[hellozombies.livejournal.com]

I'll bet they are higher on the grade-level......

I think they are. Meyer, at least, uses some good vocabulary words every now and then.

[world-dancer.livejournal.com]

Interesting. I can't wait to see the rest of it.

Though I'd argue much of the LKH problem is repetition in "catch phrases" like "Taking one for the team." Or there's that one about tasting a pulse on the wrist or some such (been awhile since I read them). And the lack of plot movement/character development, so I'm not terribly surprised that a simple word analysis shows she's still using the same "voice" when writing.

Also, when looking for the problem, at the start of NiC, There was a note (or so I recall) in which she stated she was no longer using her writing group, a bunch of authors who would sit around and critique each other's work. That's much of where I put the blame for the drop in quality. LKH clearly needs outsiders critiquing things to summon opposing viewpoints for her characters that aren't merely evil.

[kisstheground.livejournal.com]

i'm definitely not arguing on behalf of ID, and i haven't even read it! i stopped halfway through laughing corpse and don't feel a bit bad about it.

still, this is a fun project. she makes words feel bad in their souls.

[ladyniko.livejournal.com]

She also said in some video taped appearance around NiC that Darla & Jonboi were now her editors. (And, Darla is apparently out of the picture now? I've lost track of LKH's drahma llahma'ings.)

She doesn't use a professional editor! She apparently believes she's too good for 'em. (*snort of major disbelief here*)

[magdalen77.livejournal.com]

Yeah, Darla quit/got fired some time early this year, maybe February or March. Now she has Carri as her personal assistant and she's doing the same thing with her. Carri and her wife Pili are LKH's BFFs now and Carri is her editor. It's probably not a good idea to have people whose lives and financial welfare depend on pleasing you as your editors.

[ladyniko.livejournal.com]

It's probably not a good idea to have people whose lives and financial welfare depend on pleasing you as your editors.

Agreed, 100%! Nothing else could say it better than that. :) (Except for maybe hitting her upside the head w/ a clue-by-four, but that hasn't happened yet. :p)

[flirtswithfan.livejournal.com]

Somewhere in one of her blogs during DM she said she was self-editing as she wrote. Of course with a writer who produces as much "gold" as lkh how much editing is required? (Yes, I am being sarcastic)

[magdalen77.livejournal.com]

Yeah, she claims that she self-edits so she never has to do any more than one draft. Quit frankly she had no time for more than one draft with Divine Misdemeanors. The problem is that you may be somewhat able to edit your own work, but you do ultimately need somebody, preferably several somebodies to edit you. I have two co-workers who I trust go over every letter or legal document that I send out for that very reason (my boss is terrible at checking things and admits it). Obviously the lawyers check the legal wording, but my co-workers check for typos and general goof-ups.

This is because, unlike LKH, I don't enjoy looking like a barely literate moron.

[whispertwistair.livejournal.com]

She sends her manuscripts to her NY editors for two rounds of editing, according to her blogs.

[subtle-shades.livejournal.com]

Goldfish don't count as editors.

*nods seriously* Now if she had cats...

[magdalen77.livejournal.com]

Maybe for some of them. Unless she didn't blog it (which is possible) her most recent books she only sent for one round of edits. It's really all they likely had time for. I've also heard that she has a no edit clause. IDK if that means she's free to refuse any edits or it means that they're restricted to only doing basic edits for spelling, etc.

[gwynethfar.livejournal.com]

A note about the readability scales: in genre fiction, a lower score is desirable. I'm seeing a couple of comments in here that seem to be insinuating that for a book to be good, the number should be higher. A book that is overwritten for the purpose of making the author or reader feel smart is a friend to no one.

And, while this is a really good project, I have to point out that readability is usually determined by using three sample pages taken at random from the first, middle, and last thirds of the book.

[world-dancer.livejournal.com]

There's an old saying in the publishing industry that "He who edits his own writing has a moron for an editor."

The general point is that even the best writer/editor can't effectively edit herself. When you write something, you know what you meant. Editing is a chance to get feedback as to whether you've actually effectively conveyed that.

[runetraverse.livejournal.com]

*fascinated* Definitely bookmarking that site for some later look-through. Very awesome, and that kind of info is pretty interesting, not to mention cool to have. Kudos for even attempting this.

Though I have to admit, I did find it funny that my NaNo entry - first draft, rough as hell (to me) and nowhere NEAR my personal "look for a publisher" phase - got a 6.88 to her fourth-grade average. And the sample chapters of my first novel (written at 10-13yrs, re-written at 16yrs respectively) got 6.84. I wonder what the average score for genre / YA fiction actually is? Could it be posted somewhere?

[runetraverse.livejournal.com]

Total word - it just makes more sense to have someone else look at the work. You know what you meant, so you're twice as likely to read / see what you meant rather than what's actually on the page.

[x-trickster-x.livejournal.com]

Agreed. I don't really think it indicates much about the quality of a book, to be honest. It's interesting but I'm more worried by the idea that a book that's harder to read/has bigger words is automatically better.

[gwynethfar.livejournal.com]

I think it does prove an important point about the length of her books, though. The jump from the other books to Micah confirms what we've all thought: charging eight bucks for that book is a huge rip off.

[x-trickster-x.livejournal.com]

Oh, I totally agree with that. It's astonishing what she manages to get away with.

[gwynethfar.livejournal.com]

The readability grade doesn't imply that a low grade score means the content is juvenile. As someone else pointed out, Jim Butcher-- whose novels are infinitely better than LKH's-- hit right around the fourth or fifth grade reading level average, too.

I think what you'd find is that the genre of fiction would have more to do with the readability. I would assume that high fantasy and historical fiction would have pretty high scores, grade wise, just from the nature of the prose, and that contemporary fantasy and urban fantasy would have lower scores because the writing would be more conversational.

[gwynethfar.livejournal.com]

Why would someone WANT a no-edit clause? The mind boggles.

[magdalen77.livejournal.com]

I used 15 pages from my NaNo work in progress (not much progress lately) and it scored 7.0 on the Flesch-Kincaid Reading Scale. Though I only got 68 on the Reading Ease part.

[alice-the-small.livejournal.com]

Yeah, Obsidian Butterfly was the only book that I actually liked of the series...