Blog

This post originally appeared on my LiveJournal in this entry, which may include a lively discussion in the comments.

Sara’s reaction to the news of her father’s death feels like one of the most emotionally real passages in the story. (And this isn’t meant to disparage the rest of the story!) Her shock, her desperate self-control in public, and then her retreat to privacy and nearly incoherent attempts to reconcile herself to the news feel both utterly in character and entirely realistic. For all Sara’s popularity, she is an intensely private and self-contained person. And, as Becky later notes, sometimes it’s best that people in trouble should be left alone. (It might not be true of all people, but it’s certainly true of Sara. And once again, Becky is spot-on in her emotional reactions.)

When Sara emerges again to face Miss Minchin, she is subdued, but self-controlled and determined. This reaction might almost be designed to set Miss Minchin off. Miss Minchin wants to be the one in control, the one who causes others to react. For the rest of the story, she will view Sara’s self-possession as a personal affront—as a challenge to her authority. The more self-control Sara shows, the more out of control Miss Minchin becomes.

Miss Minchin verbally strips Sara of every scrap and vestige of her previous luxury and privilege, failing only when Sara refuses to give up the doll, Emily, who represents her emotional bond with her father. And when Miss Minchin thinks she’s putting the last nail in the coffin of Sara’s pride, by telling her she will need to work for a living, Sara subverts the situation by seizing on this as a positive. She will earn her living and prove her worth. And she will keep her self-possession and dignity, symbolized by her refusal to thank Miss Minchin for her “kindness”.

But it isn’t until she is shown to her new room up in the attic that she lets down her guard once more and feels the full weight of the transition in her life. And it is only there—when Becky comes in to comfort her—that Sara comes close to crying. When she first met Becky, she observed that they were alike—both two little girls. And now she repeats that observation: that there is no difference between them now. Becky, however is wiser. “Whatever happens to you—whatsoever—you’d be a princess all the same. And nothing couldn’t make you nothing different.” She recognizes the deepest truth to the story, that Sara’s “princess nature” is an inherent characteristic and not something that can be taken away.

But it will be a while before Sara rediscovers how to use that “princess nature” in her new situation, and first she needs to renegotiate her relationship to the world around her.

This post originally appeared on my LiveJournal in this entry, which may include a lively discussion in the comments.

It was pointed out in comments last week that one thing that may be behind the very uneven time-flow of the story is its origins in a much shorter work. So now I'm curious to what extent that shorter work focused primarily on the two time-periods that take up such a disproportionate amount of the page: the day of Sara's 11th birthday party, and the day of Mr. Carmichael's return from Russia. In any event, today's discussion continues with the fateful birthday party with Chapter 7 "The Diamond Mines Again".

We begin with a detailed re-emphasis of Sara's wealth, in reviewing the presents her father has arranged for her. His lack of sense with regard to spending is only emphasized by her reception of those gifts: she seems most delighted with the books, and while she philosophizes over "the last doll", it's clear that dolls themselves--with all their opulent accessories--aren't particularly important to her. The doll Emily is important as a story-telling locus and an emotional focus (as sort of an object-diary to whom she tells her inmost thoughts), but not quite so much as an object for play-manipulation. So the "last doll" with its elaborate clothing and accessories serves as the ideal symbol of excess and waste: important to Captain Crewe to represent wealth, unimportant emotionally to Sara (witness how little regret is involved when she eventually disclaims ownership of the doll, compared to how she clings to Emily), and a thorn in the side of Miss Minchin who has had to front the money for the gifts and will be left holding the bag.

But the beginning of the birthday part also has two key emotional scenes. Sara's request that Becky be allowed to stay to witness the opening of the presents is both a kindness and an imposition. A kindness, in that she publicly acknowledges not only Becky's basic humanity, but her right to "be a little girl" and enjoy girlish pleasures like dolls. Mind you, at the age of 16, a working class girl like Becky is the farthest thing from "a little girl" in this sense. And Sara's inclusion of her in the party is, in some ways, the farthest thing from a "kindness", as it brings her to the disapproving attention of Miss Minchin and--as we will see--ends up trapping her in a location where her accidental eavesdropping could have severe consequences. It's one thing to feed Becky stories and meat pies in the privacy of Sara's rooms, and another to single her out on a public setting. But Sara isn't always wise when her sense of justice is riled up, as we see on other occasions. And I don't see this as a flaw in her, but rather a consistent aspect of her realistic complexity.

The other key emotional scene is the foreshadowing when Lavinia asks how easy it would be for Sara to pretend to be a princess if she were a beggar and lived in a garret. The scene is a bit clumsy only for the fact that it occurs immediately before the announcement of the arrival of Captain Crewe's solicitor who (as it happens) is bringing news of his death and ruin. But it gives us a chance to glimpse how Sara thinks her imaginings would function if she were destitute, before she has to deal with the reality. (And, as we'll see, the reality is that using imagination to fight immediate physical and emotional hardship is not quite as easy as one might think.)

So we will leave this week's discussion in the same way that Sara leaves the schoolroom with the presents: with "the Last Doll sitting upon a chair with the glories of her wardrobe scattered about her; dresses and coats hung upon chair backs, piles of lace-frilled petticoats lying upon their seats." And, like Becky, we will linger just long enough to need to duck into hiding when Miss Minchin returns with the solicitor, so that we can listen in on their conversation...

Today is even more random than usual, as any possibility of applying my brain to a new creative topic is toast. It looks like I'm finally circling down to being able to close the investigation that I've been working on the last two months. And I set up my new laptop last night and have yet to go through and do a complete functionality verification regarding programs and data-transfer, so I don't dare do anything on either the old or new machine that involves changing files yet. But fortunately I can do some more nattering on the question of what sorts of useful meaning can be squeezed from Amazon and Goodreads book ranking data.

I've had a delightful amount of engagement on this topic, and somewhat unusually it's come from a lot of different intersections of my online life. People are popping over from the LesFic mailing lists to comment, and Tuesday's blog got a mention on File 770, which resulted in pointers to some other similar data-crunching that ties in to the topic.

[Note: these two posts were originally posted on LiveJournal then migrated to Dreamwidth. Comments on the May 3, 2016 post can be found in DW here, and those on the DW version of this post can be found here.]

Correlation of Rating Volume and Sales

A commenter on File 770 pointed me to a fascinating analysis by SFF author Mark Lawrence that looks at the correlation between the number (but not magnitude) of Goodreads ratings and book sales. He used his own sales data plus data provided from colleagues writing in similar genres who were willing to provide numbers. (Note that book contracts sometimes explicitly prohibit authors from discussing income or sales numbers in public.) Lawrence's conclusions are stated as:

[T]here is a pretty close relationship between the number of Goodreads ratings a book has and its total sales. PROVIDING that the books are of a similar age and the same genre.

(A rough squint at his graph suggests that the correlation for his specific dataset works out to around $7-8 per Goodreads rating.)

Correlation of Goodreads Rating Volume and Average Rating

(This originally appeared in slightly different form as a comment I made at File 770.)

Having poked around a bit on Goodreads, I’ve pulled a data set from a list entitled “Best Historical Fantasy” (the genre chosen to be thematically comparable to one of the sets I pulled from Amazon) and I took the top 100 titles from that list. Goodreads lists (in contrast to the sales-based Amazon Top 100 lists) are created by reader/reviewers and books are added and up-voted by other reader/reviewers. Internal ranking in the list is dependent on some combination of the average rating and the number of different people recommending the book for the list.

Overall, the results confirmed my analysis from Amazon with a couple of interesting details.

The first interesting detail is that the Goodreads correlation (at least for this dataset) continues roughly linear throughout the ranking scale, rather than losing linearity toward the bottom of the scale. My knee-jerk hypothesis is that this relates to the ability to leave a ranking on Goodreads without leaving a review. This may suppress the number of lower rankings on Amazon because someone who doesn't care for a book may be less motivated to say so if they have to do it in a text review rather than a one-click star ranking.

The second interesting detail is something that most people who pay attention already know anecdotally, which is that Goodreads rankings have a wider spread. This may be simply a consequence of the "suppression of lower rankings" effect on Amazon noted above. (To be clear: I'm not saying that Amazon actively suppresses lower rankings, only that the structure of the interface may have that as an effect.)

The lowest ranking in the Amazon Historical Fantasy Top 100 was 4.0, while Goodreads rankings (though obviously not for the same exact books) went down to 3.1 In addition to the abovementioned suppression effect, the suggested interpretations for Goodreads ratings are also shifted down from Amazon ratings. If you go by the suggested interpretations, an Amazon 4* is a Goodreads 3*. Amazon rating distributions are more likely to descend from a maximum at 5-stars, while Goodreads distributions--even for popular books--are more likely to have a maximum at 4-starts, indicating an overall shift of the curve. More on this below.

The third interesting observation is more particular. As I was entering the rating/# of reviews data, I started noticing a particular name standing out as an outlier. Not an outlier in terms of matching the average-rating/no.-of-ratings trend, but a set of books–all by the same author–who fell high in the list (based on the number of people voting them for the list) while having surprisingly few ratings. So although I wasn’t tracking author or title in general, I flagged every time that author came up in the list: a total of 11 times (out of 100 books). When the books were sorted in descending order of average rating, that author occupied 7 of the top 10 slots (average ratings of 4.7 or more) and completely followed the overall correlation of having relatively few reviews. (In a dataset where a handful of titles had rating entries in the 6 figures, all but one of those 7 had been rated fewer than 100 times.) Having taken a look at the author’s bibliography on wikipedia (and I’m not going to name names because that’s not the point), the best hypothesis seems to be that this is someone with a small but extremely dedicated readership. That dedication extended to ensuring the authors position on the list by voting for the books there, but reflects the general conclusion that the average rating must be understood in the context of the overall number of ratings.

(A commenter on File 770 suggested an alternate interpretation than "small group of enthusiastic fans", noting that some Goodreads authors engage in list-vote trading to move each others' books higher in lists. I have no speculations on this particular author's situation.)

Correlation Between Average Amazon Rating and Average Goodreads Rating for Popular Books

While it's an easy anecdotal observation that average Amazon ratings tend to be higher than average Goodreads ratings, I wanted to confirm this impression with data. So I started with Amazon's Top 100 Historical Fantasy books, looking only at Kindle sales this time (for consistency), and stripping out omnibus editions (to avoid redundancy). I used Amazon data as the starting point because I figured sales figures were harder to game than Goodreads lists.

Because I was going to have to search on each book individually in Goodreads, I only took the top 40 from the Amazon list, which trimmed down to 33 when omnibus editions and not-yet-released titles were excluded. I then calculated the Amazon:Goodreads ratio, as well as the absolute difference. I played around with these results in several ways, plotting them as a curve distribution and running a mean and standard deviation.

At a very rough approximation, the absolute difference in reviews seems to follow a standard curve: average = 0.4 stars, standard deviation = 0.2 stars. In fact, roughly 90% of the data fell within one standard deviation of the mean.

The ratio of reviews isn't quite as pretty a distribution, but has an average ratio of 1.09 with a standard deviation of 0.06. Here only 80% of the data fell within one StDev.

I leave it to a better statistician than I to say something meaningful. I expect that differences in average rating may be much more variable for books with smaller distributions, due to the larger effects of individual choice (both in what rating to give and in which rating site to participate in).

Only one book in the set had a higher average ranking in Goodreads than in Amazon. It's a pre-release listing and has relatively few reviews, which may account for its position as an outlier. Its position on the Amazon Top 100 list seems to be due entirely to the fact that it's a 47 North publication (i.e., an Amazon imprint) and "sales" are probably artificially inflated by internal promotion activities, such as making it available to "book club" arrangements (based on comments in the reviews). Some of the text reviews on Amazon are...um...harsh.

Some Random Overall Conclusions

Mark Lawrence’s sales correlation is very interesting. And it doesn’t necessarily mean that the average rating number by itself isn’t meaningful. I suspect that comparing the combined average rating + number of ratings data to the overall trend line for the relevant genre will tell you more than the absolute average rating alone, in terms of whether the reading community likes or doesn’t like the book. But all of that may be irrelevant to sales.

Just for fun, what does all this say about my books?

Daughter of Mystery has an average Amazon rating 0.5 stars higher than for Goodreads. Plotting the cumulative average Amazon Rating against total ratings over time, I'd say the average rating more or less settled in to a relatively stable number around the 15-20 review mark, but is still highly subject to single inputs. The first 5 reviews were all 5-star.

The Mystic Marriage has an average Amazon rating 0.7 stars higher than for Goodreads. The average Amazon ranking doesn't seem to have stabilized yet and the first 6 reviews were all 5-star. This book is still at the stage where the ratings are artificially high due to low numbers of readers. (Excuse me while I go sob in a corner for a while.)

Both books fall pretty solidly on the normal trendline for Lesbian Romance, but don't have enough data points to compare meaningfully to the Top 100 Historical Fantasy data. As for sales data...let's just say, "Inadequate data for meaningful analysis." (The next related phenomenon I examine may be changes in rating patterns for series books. My knee-jerk hypothesis is that later books in a series will tend to have increasingly higher average ratings because readers will typically continue with the series only if they liked it.)

It's very important to keep in mind that these comparisons and correlations are only meaningful when looking at sets of books with similar potential distribution. A lesbian romance with 200 Amazon ratings isn't comparable to a blockbuster best-seller with 200 Amazon ratings. The latter is only starting to scratch the surface of its most dedicated readership while the former may well have already saturated the market.

In conclusion, much as it pains me to admit it, I do myself no favors in begging my readers and fans to leave reviews if they weren't already inclined to do so. Strongarming existing readers into leaving ratings/reviews does not necessarily generate new readers. It certainly doesn't directly generate additional income. And to the extent that looking at the average rating + # of ratings provides useful data, artificially inflating one's average rating by solicitation to existing fans isn't meaningful.

Instead, the only useful thing to do is beg people to encourage other people to read my books. And the rating stats will fall out of that on their own.

Usually I like to focus this blog on the creative part of the writing process, but I'm in an unusual pause at the moment so I thought I'd talk about the analytic end. I know the common wisdom in mainstream publishing is that an author should pay no mind to reviews and ratings. At most, we should do comic readings of our one-star reviews to show how little we care. (Only cry in private behind locked doors.) So this essay isn't really for anyone whose book came out from a major publisher. But I have this weird bi-cultural existence, suspended between what I consider my "home" writing community (mainstream SFF) and the community in which I was published (small-press/self-published lesbian fiction), so I get a lot of opportunities to compare and contrast. This essay is for people who don't have mainstream publication and for people who may be bewildered by some of those cultural differences.

There is major anxiety within the LesFic community (quite possibly within all marginal publishing communities) around the crowd-sourced rating-and-review sites like Amazon and Goodreads. A big reason for this anxiety is that it's what they (we) have: they don't get a big publicity blitz, they don't get bookstore placement, they don't get advance reviews in all the highly-respected sites. What they do get is an aggregate of individual reader opinions when those readers are motivated enough to post them.

In my experience, this anxiety is expressed in two major ways. There is a strong community pressure that readers--that is, readers "within the community"--should only ever review and rate books that they absolutely love, and therefore that they will rate highly. This philosophical position is expressed explicitly by many LGBT review sites, and in social media forums for LGBT book communities. This anxiety walks hand in hand with the second: the tendency to react to less-than-perfect reviews as a personal attack. Given a supposed "community standard" of only reviewing/rating books you love, there is an interpretation that to rate a book badly (where "badly" is anything less than a four-star review…or sometimes less than a five-star one) must have been done out of personal malice against the author. That either the reviewer is deliberately giving a false opinion (because, of course, the book must be objectively excellent!), or that, even if they genuinely didn't care for it, the act of publicly expressing that opinion could only come from personal malice.

Viewed from within the community (and it is very much an expression of the assumption that the reader/writer/publisher nexus is a community whose purpose is to support each other against the world) this can look a lot more reasonable than it does from outside the community--where it tends to look fairly toxic.

But beyond the damage to the usefulness of ratings/reviews when only glowing opinions are authorized, there is a damage to authors' perceptions of their own work. Express skepticism of the usefulness of all-five-star ratings and some authors will loudly proclaim that their book is so great that of course it earned all those five-star ratings.

No. I believe that almost every book can earn some genuine and sincere five-star ratings. But no book is universally beloved. Let me repeat that with emphasis: NO BOOK IS UNIVERSALLY BELOVED.

Because I wanted to throw some data at this essay, I took a look at Amazon for the top 100 sellers in Historical Fantasy and the top 100 sellers in Lesbian Romance. You know who has spent a very long time in the top 10 books sold in Historical Fantasy? Diana Gabaldon's Outlander. Do you know how many one-star reviews Outlander has on Amazon? 749. Seven hundred and forty-fucking-nine one-star reviews (4% of the total). No book is universally beloved.

Do you know what unfavorable reviews and ratings mean? They mean that your book is engaging readers who are outside your narrow inner-core target audience. Not just that it's reaching them, but it's engaging them sufficiently to express their opinion in public. And up to a certain point (I'll talk about that point later) the more reviews you get, the lower your average rating is. Because the more people you engage, the more likely you are to engage people who may have liked your book but didn't absolutely love it. Sure, that hurts. It would be nice to be universally beloved. NO BOOK IS UNIVERSALLY BELOVED.

Ah, but I'm a data person, so I ask myself, is it possible to quantify to what extent a less-than-perfect average rating reflects getting your book in the hands of people outside your core target audience? Let's see.

I went through the top 100 Amazon sellers in the Historical Fantasy category and recorded the number of reviews and the average star rating. Then I calculated the average number of reviews at each rating. I mention Diana Gabaldon above because I ended up pulling her out as her own little category for the analysis. And then I plotted those pairs of data.

I had to put Diana Gabaldon on a separate y-axis that differed by nearly an order of magnitude from the rest of the data. But here's the take-away: from around an average rating of around 4.3 on up, a lower rating correlates with a larger engagement (expressed as overall number of reviews). This holds true for the overall average of that top 100 and it holds true in the specific case of Diana Gabaldon. The books that had only five-star reviews? (And keep in mind, these are ones that are currently in the top 100 sellers of the category.) None of them had more than four (4) reviews. Your first, most engaged reviewers are quite naturally going to be people from your core target audience. But an extremely high average is a sign that you haven't expanded far beyond that (yet).

Now let's take a look at the same sort of data for the Amazon category of lesbian romance. (In this case, I looked specifically at Kindle sales because for small press books the dynamics of e-book versus paper are peculiar.) I also cut the data off at a rating of 4.0, not only to compare better with the historical fantasy data (for which that was the lowest average rating) but because the values below that represented only one or two books each and so are less reliable for trending purposes.

And what do we find? Pretty much the same thing. In this case, a rating of 4.4 and up correlates very closely with the average number of reviews at that rating. No book that had an average five-star rating had more than five (5) reviews. Interestingly, if you look at the plot for the maximum number of reviews at each average-rating point, you get the same effect: a very strong correlation between number of reviews and a lower average rating.

Now, of course, at some point this effect breaks down. At some point a lower average rating does start to reflect people's opinion of the specific book, even in the aggregate. And that seems to start somewhere in the lower 4's depending on the data set. And these trends are looking at aggregate behavior. It doesn't mean that there's no difference at all between a 4.4 rating and a 5.0 rating. What it means is that the meaningful difference between a book that has a 5.0 rating with X number of reviews, and a book in the same marketing category with a 4.4 rating and 50X number of reviews is not necessarily one of quality. What it means is that the second book is reaching outside its core audience. And is engaging them.

As someone who has been working very hard to reach outside what my publisher believes to be my core target audience, this is what I keep reminding myself when I get a "meh" rating or review. It means that I've succeeded.