The Theory of Related-ivity: Segment IV

The Theory of Related-ivity:

A History and Analysis of the Best Related Work Hugo Category

by Heather Rose Jones

(This is a serialized article exploring the history of the Best Related Work Hugo category in its various names and versions. If you’ve come in at the middle, start here.)

Contents

Part 2: Methodology

2.3 Data and Eligibility

2.3.1 Data Sources and Available Data

2.3.2 Eligibility Notes

---

Part 2: Methodology

2.3 Data and Eligibility

2.3.1 Data Sources and Available Data

Data Sources

This section documents the sources of the data used, the types of data available, and any administrative requirements that affected that availability, concluding with a discussion of how the available data determines the types of comparisons that will be made across the several identifiable “eras” of the award category.

Lists of Best Related Work nominees were taken from the official Hugo Award website (www.thehugoawards.org) and documents linked there. The documents at the Hugo Awards website are generally copies (either electronic or scanned) of reports released after the awards ceremony. Other than reports of Finalists and Winners, reports of this type may not have been created prior to the administrative requirement for reporting the Long List, and the type of data included on these reports is variable. The most complete possible data would be:

• Total number of Hugo nominating ballots
• For each Hugo category, the number of ballots including at least one nomination in that category
• For each Hugo category, the number of different works nominated
• The number of nominating ballots each work was included on
• Any disqualifications, exclusions, or transfer of nominations to another category
• The name/title of the work, the name(s) of the author(s) or creator(s), and in the case of published Books, the Publisher

After the application of the E Pluribus Hugo nomination processing system, the report also shows the calculation data that produces the “score” for determining Finalists.

From this, each work was then researched online to confirm the correct and complete title, author(s), and publication date, as well as to assign tags for the Media type and subject matter Categories of the work.[1] It is also noted if a nominee is part of an ongoing Series of some type, or is a repeat nominee with different content. Whenever possible, a URL link has been identified for reference purposes.[2]

The basic accuracy of the official Hugo Award website is assumed with regard to the nominee lists. Additional details including full titles and full credits have been researched in Wikipedia and archive.org, as well as sites relevant to the individual works.

A reasonable effort has been made to identify the gender of all authors and subjects, as reflected in public information. (See the section on Categorization Process in the chapter on Gender for details of this process.)

Ideally, individuals would also be tagged for nationality, ethnicity, or other identity factors, however as these cannot be consistently determined from publicly available data, the results would not be statistically meaningful.

History of the Administrative Reporting Requirements

While Finalist data is available for all years, the availability of additional nominee data is affected by changes in the reporting requirements for this data. In 1980 (Worldcon 38, Noreascon Two), when the Best Related category first appeared, it is coincidental that a new amendment appears in the business meeting minutes[3] requiring reporting of the final voting data (presumably rather than a simple report of the results). There is no reference to nomination data in this proposal and the existing constitution did not require reporting of extended nomination data. The requirement to report final voting data was ratified in 1981 (Worldcon 39, Denvention Two) and made part of the WSFS constitution.[4] The present study focuses on nomination data rather than the final voting process, therefore the voting data is not relevant here.

The 1994 (Worldcon 52, ConAdian) business meeting minutes[5] include the following proposed amendment affecting the available nomination data.

Release of Hugo Nomination Totals

MOVED, to add the following to the end of Section 2.9.4 of the WSFS Constitution: During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the 15 highest vote-getters and any other candidates receiving a number of votes equal to at least 5% of the nomination ballots cast in that category.

(submitted by Mark L. Olson, Rick Katze, Anthony Lewis, and Sharon Sbarsky)

(The rules now require the publication of the final-ballot Hugo voting counts. (It is not presently required that nomination totals be released, though it has become customary for Worldcons to release them.)[6] This motion would require that the nomination counts also be published, including runners-up down to 15th place or 5%, whichever represents fewer votes.)

After some debate regarding whether this requirement should be a resolution or an amendment, the original amendment passed its initial vote. The amendment was ratified at the 1995 (Worldcon 53, Intersection) business meeting.[7]

This requirement was therefore in place officially starting in 1996. Data consistent with this requirement is available at HugoAwards.org for 1996, not for 1997, then consistently thereafter starting in 1998.[8] Note that in years when data is available for both the total number of nominating ballots for Best Related and the number of nominations received by the 15th place nominee, the 15th place work always received fewer nominations than 5% of the total nominating ballots in the category, therefore it should never have been the case that additional nominees were listed below 15th place because they were on at least 5% of the nominating ballots. Additional nominees are sometimes listed, but not for this reason.

The approved version of the nominee reporting requirement is documented in the archived 1999 version of the WSFS constitution[9] which has the following text. (No archived business meeting documents are available for 1998.)

3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, ... places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category.

The next change to relevant reporting requirements was proposed in the 2007 business meeting.[10] This involved some sort of change to the Long List of nominees, but the specific text is not included in the minutes that year. The amendment passed and was ratified in 2008 Worldcon 66, Denvention 3)[11] as follows (new text is underlined), becoming effective in 2009:

Moved, To amend section 3.11.4 of the Constitution by adding the following words to the end of Section 3.11.4: During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five (5) votes.[12]

For years when the full Long List nomination statistics are available, there appear to be only 2 years when the “not less than 5” rule would need to have been invoked. In 1998 (before the requirement of “at least 5 votes”), there was a tie for 11-14th place with 4 votes each and, as noted previously, the 15th ranked nominee (which presumably was a tie for items with 3 votes) was not listed. In 2007, the year the 5-vote restriction was first proposed, there was a tie for 15-19th place with 4 votes each.

Note that in 2007 a far more extensive Long List than usual was published. For Best Related, the list included every item receiving at least 2 votes. (Other categories reporting extended nominee lists that year had different minimums.) The data reporting for this year was unusual in other ways, in that it did not include nomination data for total ballots, ballots for each category, or distinct works in each category, which data had been fairly standard in the previous decade. This means it’s not possible to calculate how the more extensive Long List relates to the 5%-of-category cut-off.[13] The business meeting would have occurred prior to the nominee data being published, though the data reports were almost certainly prepared earlier. It seems likely that the extended nominee lists were related in some way to the debate over reporting requirements, but in that case, the omission of the category totals is baffling.

In 2009, the first year the revised reporting requirements were effective, there was also an unusually extensive Long List reported. All categories reported every nominee that received 5 or more nominations. For Best Related, this included works down to 25th place, which had 6 nominations.[14] The 5% cut-off that year would be 13 nominations. It isn’t clear whether this was a deliberate choice to publish non-required data using only the “at least 5 nominations” rule or whether it was a misreading of the requirements of the new rule.[15]

Changes to the nomination process under E Pluribus Hugo[16] affected data reporting primarily in that it functionally eliminated ties during the evaluation of nominees.[17] As noted previously, the two changes to the nomination process (6 rather than 5 Finalists and use of EPH) combined with the reporting requirements for nominees appears to have been generally interpreted as “Finalists plus 10 runners up,” i.e., a total of 16 works, however the occasional year reporting 15 items on the Long List may be following the letter of the requirement to report the top 15 items.

Timeline of the Available Data and the Effects of Reporting Requirements

Overall, here is the timeline of reporting requirements and actual available data, as it relates to the changes in the category name/scope. (The requirement for at least 5 votes isn’t included as it had no statutory effect on the data.)

• 1980-1995: Best Non-Fiction Book, only Finalists required to be reported
  • Additional nominees reported in 1980 (10 total), 1989 (12 total), and 1993 (8 total) but otherwise only Finalists, which may be 5 or 6, presumably due to ties. (Actual nomination numbers are not consistently reported.)
• 1996-1997: Best Non-Fiction Book, Long List required to be reported
  • Long List omitted in 1997, otherwise requirements are followed.
• 1998-2009: Best Related Book, Long List required to be reported
  • Reporting as required with the following items noted:
    • 1998 only lists 14 items (see discussion above).
    • In 4 years, there was a tie involving 15th place and therefore more nominees were listed (2000, 2001, 2003, 2004).
    • 1999 16 nominees reported with no tie for 15th place, therefore this was not required.
    • 2002 17 nominees reported with a tie between 16 & 17, which should not have required this addition.
    • 2006 17 nominees reported with a tie between 16 & 17, which should not have required this addition.
    • 2007 All nominees with at least 2 votes reported for a total of 40 items.
    • 2009 All nominees with at least 5 votes reported for a total of 25 items.[18]
• 2010-2016: Best Related Work, Long List required to be reported
  • 2 years involved a tie for 15th place (2011, 2013) and therefore listed more than 15 nominees.
  • 2010 23 nominees listed with the lowest number of votes being 8. As the number of ballots for each category was not reported, the 5% threshold cannot be calculated. Votes for 15th place were 13. Therefore, it does not appear that this more extensive list was based on reporting requirements.
• 2017-Present: Best Related Work, EPH in effect
  • Long List consists of either 15 or 16 items (see discussion above).

Reporting is more erratic for the total number of Hugo nominating ballots, the number of ballots including each specific category, and the number of distinct works nominated in each category. It isn’t clear that any of this data is required to be reported. The incompleteness of this data will be relevant when tracking certain trends in nomination data.[19]

• Total nominating ballots is available for 4 of the Non-Fiction years (22%), 9 of the Related Book years (75%), and 8 of the Related Work years (50%).
• Nominating ballots including Best Related works is available for 2 of the Non-Fiction years (11%), 11 of the Related Book years (92%), and 15 of the Related Work years (94%).
• Number of distinct works for Best Related is available for 1 of the Non-Fiction years (6%), 9 of the Related Book years (75%), and 7 of the Related Work years (44%).
• Overall, only 13 years (28%) report all 3 types of data.

Due to certain coincidences of timing regarding changes to the category and changes to reporting practices, we can conveniently group the data into the following comparison sets.

Best Non-Fiction Book

• Finalist data is complete
• Only minimal Long List data is available, therefore Long List data during this period will be considered anecdotally but not used for between-group comparisons.

Best Related Book

• Finalist data is complete
• Required Long List data (through #15) is functionally complete.
• Additional, non-required data is available for several years but in most cases is equivalent to the number of required nominees in other years due to ties for #15.
• One year is anomalous listing all 40 nominees with at least 2 votes while 20 nominees have 3 or more votes. One year lists all nominees with at least 5 votes. As the number of nominees receiving 3+ and 5+ votes respectively are roughly equivalent to the Long List size for years with large ties for #15, they will be included in Long List comparisons, with the remainder being analyzed anecdotally.

Best Related Work

• Finalist data is complete
• Required Long List data is complete
• One year has additional non-required nominees, however the numbers are similar to those included in Long List sets under Best Related Book and will be included in group comparisons.

Comparison Sets

Therefore, the analysis will include the following:

• Finalist data will be compared across all three eras.
• Finalist + Long List data will be compared between Best Related Book and Best Related Work. This provides a useful comparison for the effects of the increased scope of formats.
• Within the Related Book and Related Work eras, year-by-year differences in Finalist + Long List will be analyzed and compared to the overall group to identify any directional shifts.
• Any year with an anomalously large data set will be compared to its truncated data set(s) to find anecdotal differences in content of the long tail.

2.3.2 Eligibility Notes

Eligibility Questions

Works that make the nomination cut-off for Finalist are evaluated to confirm that they meet eligibility requirements for release date, format, categorization, etc. Some aspects of this evaluation are clear-cut while others can be subjective. Works on the Long List that don’t make the Finalist cut-off are not necessarily evaluated for eligibility, although in some cases there are notes indicating a Long List work would not be eligible. Therefore, the two data sets (Finalists and cumulative Long List) answer slightly different questions. Finalist data tells us what eligible works have been nominated, but Long List data can tell us what the nominators think should be eligible, or perhaps what their impression of the category’s scope is without reference to the eligibility rules.

When a clearly ineligible work appears on the Long List, it could be a sign that nominators aren’t studying the eligibility requirements carefully, or that they are unaware of key information (such as publication date), but it could also indicate that nominators think the work should be recognized in some way regardless of whether it fits the eligibility requirements at the time.[20] This last motivation would be difficult to identify in the absence of documented discussions on the topic. Given that ambiguous works (especially during the Related Work era), when such discussions are well-documented, can show a conscious interest in exploring and stretching the boundaries of the category’s scope, it’s probably best to assume similar motivations in cases where the motivations aren’t well documented. For example, when works of Fiction or Fiction collections are nominated, it should be presumed that nominators considered the work to be significant for some other aspect. It can’t entirely be ruled out that there may have been organized bad-faith campaigns to nominate works that the nominators knew to be out of scope, yet nominated anyway. But this study gives the benefit of the doubt, given the regular appearance of clearly ambiguous works.

Eligibility in Multiple Years

That said, there are contexts in which apparent eligibility concerns can be explained. Works are sometimes nominated in more than one year, or in a different year than the year of creation, due to the allowance for extended eligibility or circumstances which allowed renewed eligibility.

If a new edition of a Book was published, the new edition might be nominated as a substantially new work. This is the case for the following works:

• Anatomy of Wonder, Second Edition (Finalist in 1982), Anatomy of Wonder, Third Edition (Finalist in 1988).
• The Encyclopedia of Science Fiction, unnumbered 1st Edition (Finalist and Winner in 1994), The Encyclopedia of Science Fiction, 3rd Edition (Finalist and Winner in 2012)

Some works seem to appear twice based on a short version of the title, but on further examination this is due to different volumes of a multi-volume work being nominated.

• Robert A. Heinlein: In Dialogue with His Century, Volume 1: (1907–1948): Learning Curve (Finalist in 2011) and Robert A. Heinlein: In Dialogue with His Century, Volume 2: The Man Who Learned Better: 1948-1988 (Long List in 2015).
• In 2024, nominations were received for both Chinese Science Fiction: An Oral History vol. 3 (中国科幻口述史第三卷) (Long List), and Chinese SF: An Oral History vols. 2 & 3 (中国科幻口述史, 第二卷, 第三卷) (Finalist). The usual approach when both a work and its subset are nominated (which happens more often in the Graphic Work or Dramatic Presentation categories) is a process to transfer nominations to best reflect the overall intent of the nominators. In this case, the Hugo voting report notes that as the combined volumes 2 & 3 nomination made the Finalist list on its own, no nomination transfers were considered. (The prohibition on a work appearing more than once on the ballot was moot, as there were insufficient nominations for volume 3 alone to be considered as a Finalist.)

During the Related Work era, certain ongoing projects have been nominated in multiple years based on continually changing content, essentially functioning as a new edition. This is the case for the following works:

• Writing Excuses, Podcast (Long List in 2010, Finalist in 2011, 2012, 2014, Finalist and Winner in 2013)
• Archive of Our Own, Website (Long List in 2014, 2017, and 2018, Finalist and Winner in 2019)
• FIYAHCON, Event (Finalist in 2021, Long List in 2022)

The question of whether the Archive of Our Own site was sufficiently different from year to year for re-nomination was discussed within the fannish community and raised some interesting philosophical issues. The fact that the site only made Finalist once, and then was not re-nominated after it won that year, has contributed to leaving these issues unresolved. The Event and Podcast nominees can more clearly be considered discrete works in different years. There have been other ongoing projects that could raise the same questions, such as The Historical Dictionary of Science Fiction website, where the lack of multiple appearances has made the question moot.

Extended Eligibility

When a specific work is nominated in a year other than its official year of eligibility, it is typically the following year, either based on extended eligibility or possibly a mistaken assumption that eligibility would be extended. Eligibility may be extended if a work had limited availability in its release year, or specifically had limited availability to English-speaking readers in the USA.

While it’s common for Dramatic Presentations to have petitions for extended eligibility[21] it’s far less common for other types of works. Fictional works originally published in non-English languages have a different allowance for the year of first publication in English, due to the Anglocentric nature of the Worldcon nominators. Similarly, due to the USA-centric nature of the Worldcon nominators, there is an allowance to renew eligibility in the first year of USA publication for works originally published outside the USA.

The following shows the timeline that can explain dual appearances in adjacent years due to extension:

• In Year X: work is released.
• Early in Year X+1: nominations are made for year X.
• Summer of Year X+1: petition is submitted to the business meeting and approved for extended eligibility by 2/3 of the business meeting.
• Early Year X+2: nominations are made for year X+1 and works with extended eligibility are included in consideration.[22]

One exception to these allowances is that if a work was a Finalist in a previous year, it cannot receive extended eligibility regardless of other considerations.[23]

Extension of eligibility may be documented in the Business Meeting minutes (if voted on) or may be determined by the Hugo administrators and documented in the Hugo voting report (if procedural), but this latter isn’t always explicitly stated. In some cases, no written documentation for an apparent extension could be identified.

One work was nominated early (i.e., nominated in the same year as creation rather than nominated in the following year) as well as being nominated in its eligible year.

• The 2023 Hugo Awards: A Report on Censorship and Exclusion, (published in 2024 in which year it was declared ineligible for the Long List, Finalist in 2025).

A somewhat unusual case is a work nominated in translation after its original publication date, but as the original publication was English/USA, the translation did not get extended eligibility.

• The Art of Ghost of Tsushima, original English publication in 2020, Chinese translation published in 2022 (nominated in 2023, but deemed ineligible due to the prior publication, per the Hugo voting report).[24]

The following works appear as nominees in the year after the eligible year (i.e., 2 years after publication) and there is specific documentation that eligibility was extended.

• The Cambridge Companion to Science Fiction, published in the UK in 2003 (Long List in 2004, Finalist and Winner in 2005 based on extended eligibility, per the author).
• The True Knowledge of Ken MacLeod, published in the UK in 2003, extended eligibility in 2004 based on the USA publication date, per the Business Meeting minutes. (Long List in 2003 and 2004.)
• Up Through an Empty House of Stars, published in the UK in 2003 (Long List in 2004, extended eligibility in 2005 based on initial non-USA publication, per the Business Meeting minutes, when it also made the Long List).
• Worlds of Ursula K. Le Guin, film, released in 2018 (Long List in 2019, given extended eligibility due to limited release in 2020, per the Business Meeting minutes, when it was a Finalist).

The following works appear as nominees for the year after the eligible year (i.e., 2 years after publication) and may have been given extended eligibility but there is no documentation to that effect.[25]

• Algernon, Charlie and I: A Writer's Journey, has a copyright date of 1999, appears on the Long List in 2001 with no commentary. It appears that the original publication in 1999 was by Challcrest Press Books, which may have been determined to have low enough distribution to allow for extended eligibility.[26] It was republished in 2000 by an imprint of Harcourt Books.
• J.R.R. Tolkien: Author of the Century, published in 2000 (Long List in 2001 the year of official eligibility, Finalist in 2002). Presumably there was extended eligibility but no documentation of this was identified as the Business Meeting minutes for 2001 are not available. The work was published by HarperCollins and won 2001 World Fantasy and Mythopoeic awards, which would seem to suggest that limited distribution was not an issue. The reason for it not being disqualified is a mystery.
• The Arrival, published (in Australia) in 2006 (Long List in 2007, Finalist in 2008). This would automatically be eligible for extension due to the initial publication being outside the USA, but there is no specific documentation of this.
• The Seven Beauties of Science Fiction, published in 2008 (Long List in 2009 and 2010). It was published by Wesleyan University Press, a US company, but may have been considered to have had limited distribution. There is no specific documentation of extended eligibility.

Extended eligibility is not always documented even when it appears to have been granted. However, in some cases a lack of extended eligibility is specifically documented, as for the following.

• The Way the Future Was: A Memoir, published in 1978, (would have been on the Long List in 1980 but noted as ineligible, per the Hugo voting report, due to the publication date).
• Bare-Faced Messiah: The True Story of L. Ron Hubbard, published in 1987 (would have been on the Long List in 1989 but noted as ineligible due to the publication date, per the Hugo voting report).
• Myths for the Modern Age: Philip Jose Farmer's Wold Newton Universe, published in 2005 (listed among the extended list of nominees in 2007 with the publication date called out, but it would not have been on an official Long List). It’s unclear whether the note highlighting the publication date was intended to be understood as an ineligibility comment.
• The Anticipation Novelists of 1950s French Science Fiction: Stepchildren of Voltaire, published in 2010 (would have been a Finalist in 2012 but noted as ineligible in the Hugo voting report due to publication date).

Extended eligibility is excluded if a work has been a Finalist in its official year of eligibility. That’s the situation for the following, though the notes on ineligibility do not mention the prior Finalist status.

• Imagination: The Art & Technique of David A. Cherry, published in 1987 (Finalist in 1988, Long List in 1989 but noted as ineligible in the Hugo voting report).

It is much rarer for a work to be nominated later than the year after official eligibility, however the following item appears in the data set.

• Greetings From Lake Wu, published with very limited distribution in 2003 (appears in the extended list of nominees in 2007 with the publication date noted but it would not have been on an official Long List, therefore there is no reason why it would have been vetted for publication date).[27]

Other Disqualifiations

There are several other reasons why nominated works might be disqualified. The following additional works have disqualification reasons listed.

• A Brief History of Time: From the Big Bang to Black Holes (would have been a Finalist in 1989 but the Hugo voting report notes it as “ineligible - withdrawn” with no reason given).
• Visions in Light and Shadow (would have been on the Long List in 2001). There is no ruling in the Hugo voting results regarding eligibility, presumably because it didn’t meet the threshold for Finalist, but this is a collection of short Fiction and therefore should not be eligible.
• L. Ron Hubbard Presents Writers of the Future Volume XVII (would have been a Finalist in 2002 but ruled ineligible as it is classified as Fiction and does not meet the requirement for being notable for some other reason).
• The Return of the Black Widowers (Long List in 2004 but ruled ineligible, per the Hugo voting report, as it is a collection of Fiction).
• L. Ron Hubbard Presents Writers of the Future: The First 25 Years (would have been on the Long List for 2011). There is no specific note in the Hugo voting results regarding disqualification, presumably because it didn’t meet the threshold for Finalist, but the same ruling as the previous would apply.
• 20 世纪中国科幻⼩说史 (Er Shi Shi Ji Zhong Guo Ke Huan Xiao Shuo Shi) / History of Chinese Science Fiction in the 20th Century (would have been a Finalist in 2023 but ruled ineligible due to conflict of interest as one of the authors was on the Hugo subcommittee that year).

Eligibility and the Data Analysis

Voluntary withdrawals have not been counted as disqualification and are not reported specifically as they do not speak to nomination patterns and the reason for withdrawal may not be known.

All works included in the nomination reports are included in the topical data analysis for Long List and full data sets, regardless of eligibility rulings or withdrawals, as they speak to patterns of nomination and the nominators’ intent.

---

(Segment V will cover Part 2 Methodology, Section 2.4 Categorization Process.)