On this page

Publishing a corpus means more than putting files online. It means placing the data where others can find, cite, and extend it, and connecting it to the scholarship and collections it builds on. This page gathers the frameworks behind the SF Nexus, the legal ground that makes non-consumptive research possible, and the archives, databases, and projects that study the genre at scale.

For a longer running list of speculative fiction data resources, see Grant Wythoff’s SF data list.

Collections as data

The SF Nexus belongs to a movement in libraries, archives, and museums to make holdings available for computational use.

The Santa Barbara Statement on Collections as Data (2017)
Principles for making cultural heritage collections amenable to computational use, drafted at the 2017 Always Already Computational forum in Santa Barbara.
Always Already Computational: Collections as Data, final report (2019)
The IMLS-funded project (2016–2018) that shaped the field. Its authors are Thomas Padilla, Laurie Allen, Hannah Frost, Sarah Potvin, Elizabeth Russey Roke, and Stewart Varner.
Collections as Data: Part to Whole, final report (2023)
A Mellon-funded program that supported institutions developing collections as data, with lessons for sustaining them.
The Vancouver Statement on Collections as Data (2023)
An updated set of principles for collections as data, with greater attention to ethics and communities.

Law and policy for text and data mining

Most twentieth-century science fiction is still in copyright. The following decisions and guides define what libraries and researchers can do with it.

Authors Guild v. HathiTrust, 755 F.3d 87 (2d Cir. 2014)
Held that digitizing books to create a full-text search index is a fair use.
Authors Guild v. Google, 804 F.3d 202 (2d Cir. 2015)
Held that Google Books’ scanning, search, and snippet display are transformative fair uses.
HathiTrust Non-Consumptive Use Research Policy
Defines non-consumptive research, meaning computational analysis that doesn’t let anyone read the works, and governs access to in-copyright volumes.
Building Legal Literacies for Text Data Mining (2021)
An open book and NEH institute led by Rachael Samberg at UC Berkeley, covering copyright, contracts, privacy, and ethics in TDM research.
DMCA §1201 exemption for text and data mining
The Copyright Office exemption, first granted in 2021 and renewed and expanded in 2024, that lets researchers bypass DRM on e-books and films to build TDM corpora.
Data Speculations
The SF Nexus’s IMLS-funded national forum on legal best practices for library stewardship of copyrighted contemporary culture.

Computational studies of science fiction

Ted Underwood, “The Life Cycles of Genres”, Journal of Cultural Analytics (2016)
Uses predictive models to trace science fiction, detective fiction, and the Gothic across 150 years. It asks whether genres are stable traditions or short-lived generations.
Ted Underwood, Distant Horizons: Digital Evidence and Literary Change (University of Chicago Press, 2019)
Includes a study of how genre boundaries, SF among them, are recognized and change over time.
Laure Thompson and David Mimno, Finding Speculative Fiction in HathiTrust
Identifies twentieth-century English-language speculative fiction in HathiTrust. It is the source of the SF comparison workset used in our Word Embeddings study.
Ted Underwood, Patrick Kimutis, and Jessica Witte, “NovelTM Datasets for English-Language Fiction, 1700–2009”, Journal of Cultural Analytics (2020)
Metadata for fiction in HathiTrust, including the 1,501-volume gender-balanced workset used as a comparison corpus on this site.
Andrew Milner and J. R. Burgmann, Science Fiction and Climate Change: A Sociological Approach (Liverpool University Press, 2020)
A sociological study of climate fiction as a strand of science fiction, useful context for our cli-fi corpus.
Natalia Sánchez Querubín and Sabine Niederer, “Climate futures: Machine learning from cli-fi”, Convergence 30, no. 1 (2024)
Recent work bringing machine learning to climate fiction.
HathiTrust Research Center SF Experiments
Grant Wythoff’s experiments with SF in the HTRC.
Sentiments of Science Fiction
Sentiment analysis of SF texts by Tai Xiang.
Data Capsule Appliance
Temple’s local, secure environment for computing on copyrighted corpora.
Digital Mapping of Mary Shelley’s Frankenstein
A digital mapping project on SF’s founding novel.

Digital exhibits of science fiction as data

The Stuff of Science Fiction
An interactive visualization prototype built from a digitized corpus of SF anthologies at the University of Calgary. It is described in Digital Humanities Quarterly.
100 Years of Science Fiction
A network visualization of keywords from SF novels written since 1900, based on Goodreads data.
Digitizing the New Wave
Our Omeka exhibit of book covers and metadata from the digitized novels in Temple’s Paskow Science Fiction Collection.

Reference works and databases

The Encyclopedia of Science Fiction
The standard reference work on the genre, freely available online, with entries on authors, themes, and publishing history.
The Internet Speculative Fiction Database
Bibliographic data and metadata on SF works, compiled by volunteers.
The FictionMags Index
Galactic Central’s index of fiction magazines, including the SF pulps and digests.
The Science Fiction and Fantasy Research Database
A Texas A&M Libraries index to more than 142,000 books, articles, and news notes about science fiction, fantasy, and horror. It provides citations, not full text.
FANAC Fan History Archive
Scanned fanzines, convention publications, and photographs from SF fandom.
The Edward T. LeBlanc Memorial Dime Novel Bibliography
A database of dime novels, story papers, reprint libraries, and related materials.

Datasets and digital archives

Temple University Libraries in HathiTrust
The Temple-digitized volumes, including the SF Nexus corpus, in the HathiTrust Digital Library.
HTRC Extracted Features
Page-level word counts and metadata for millions of HathiTrust volumes. This is the model for our own extracted-features datasets.
Temple University Libraries Datasets
Datasets gathered and curated by Temple Libraries for study and analysis. They include publicly available data, data scraped from media sources, and collections curated as data.
Internet Archive: Pulp Magazine Archive
Digitized pulp magazines from 1896 through the 1950s.
Internet Archive: Science Fiction and Fantasy Fiction
Digitized SF novels from the collections of UCLA, the University of Toronto, and UC Berkeley.
Project Gutenberg Science Fiction bookshelf
Free e-books of public-domain SF.
The Luminist League
A large collection of novels, short stories, and letters, especially by SF writers.

Science fiction special collections

The SF Nexus was designed as a prototype for collaboration among the libraries that collect the genre, including members of the Science Fiction Collecting Libraries Consortium.