Usenet began back in 1980 before there was such a thing as AOL, the World Wide Web, or Yahoo. Discussion of the idea of what eventually became Usenet happened in late 1979 with Tom Truscott and Jim Ellis, two Duke University graduate students, but implementation didn’t happen until 1980 with the help of Steve Bellovin from nearby University of North Carolina. Usenet served (and still serves) as an information exchange mechanism that was shared globally but accessed locally. It allowed for the sharing of messages, called articles at the time, between multiple computer systems, which in turn shared their articles with another set of multiple computer systems. This exchange of messages was a two-way street where computer one passed to computer two messages the first system knew about that the second didn’t, while the second machine passed messages to the first system that the first didn’t know about. This exchange of messages is what eventually led to a global network of, primarily, university computer systems talking to each other by sharing messages across the globe.
Prior to Usenet, people relied on centrally controlled bulletin board systems (BBS’s) for message exchange. A BBS was local to an individual area, and all the messaging content was only available on the single BBS — kind of like a city-specific Craigslist domain, without the cross-sharing to other cities when you search. With Usenet, there was no central administration.
The first version of the software behind Usenet was called A News. This worked sufficiently well until there were too many systems exchanging too many articles. Too many articles back in 1983 was about 50 per day. You can probably guess what came next… B News, which came out in 1981. This lasted for nearly a decade before its successor came about, C News. C News offered performance improvements over B News. C News eventually was superseded like its predecessors. Its replacement was called INN for InterNetNews, which is still in use today for the behind the scenes work of running a Usenet server. It is even still actively developed by the Internet Systems Consortium.
The Big Eight
When you submit an article to Usenet you specify a newsgroup or groups to receive it. Reading articles involves specifying which newsgroup or groups of articles you want to see. Newsgroups are categorized hierarchically, similar to the reverse of a domain name. For instance, www.google.com becomes com.google.www when you reverse it. With domain names, there are several top-level domain registries: like .edu for education and .com for companies, and also regional ones like .jp for Japan and .tv for the island of Tuvalu in the South Pacific (not television). With newsgroups, the primary categories are comp.* for computer-related topics, humanities.* for art and literature type topics (not in original topic hierarchy), misc.* for a catch-all for anything that didn’t fit in the other seven, news.* for Usenet-related announcements, rec.* for sports and entertainment-related topics, sci.* for scientific topics, soc.* for social and societal banter, and talk.* for topics some people like to troll through nowadays, mostly religion and politics.
When you go to post your article, you read through the list of newsgroups available and find the best place for it to be submitted. For instance, if you want to talk about baseball, your message belongs in rec.sport.baseball. The existence of that one group allows football fans who don’t care about baseball to only look for messages in rec.sport.football.
While Usenet groups are not centrally controlled, their names are, at least for the eight groups mentioned above. For a new newsgroup to be added to one of these eight, there has to be some agreement on name and location in the hierarchy. A ninth hierarchy, however, doesn’t require this moderation of group names. That hierarchy is known as alt.*. Anyone can create a group there. While anyone can create the group, if another system in the network doesn’t also carry it, the group is local to that one node in the network. The alt.* hierarchy allows the alt.sports.baseball.bos-redsox and alt.sports.baseball.ny-yankees to coexist without arguing all the time in rec.sport.baseball.
The alt.* Hierarchy
The original intent for Usenet was the sharing of text messages. In addition to holding groups for different baseball teams, the alt.* hierarchy quickly became home to sharing binary content, though the original topics in the alt.* hierarchy were alt.sex, alt.drugs, and, of course, alt.rock-n-roll. All the binary newsgroups ended up under alt.binaries, aka the alt binaries. Early binary categories were heavy on the NSFW image scene as well as what you might find floating around a Torrent site nowadays, albeit in a much smaller variety. There were no high definition televisions back then, and computer monitors just didn’t support that high of a resolution, so, naturally, video files didn’t encode as much information in them, leading to their smaller size.
Sharing alt binaries content across Usenet requires a different set of tools than sharing text messages. First, Usenet messages could only display 7-bit ASCII characters. Also, messages were limited in size. Not many people wanted to share messages with 60,000 characters in them, but binary files were most typically larger. This desire to share content in the alt binaries required tools to combine multiple files together, then break them into individual Usenet articles, and encode them into 7-bit ASCII. And then doing the reverse to put them back together. The format for combining files together was originally called RAR, named after Eugene Roshal, the Russian creator of the Roshal ARchive format. This was somewhat similar to the ZIP format everyone is familiar with, but with the key advantage of being able to easily break the compressed file in to pieces. Splitting a single RAR file into multiple files was done with the help of the Parchive system, short for partial archive. This format included checksums for data integrity checking and even supported data recovery if chunks went missing, as network connections weren’t as reliable back then. The binary-to-text encoding was done with the help of tools like uuencode. Once all the articles with the binary content gets received, the reverse had to be done to reconstruct the original files, starting with uudecode. Does this sound somewhat similar to downloading the latest content with BitTorrent?
NNTP and Newsreaders
NNTP stands for Network News Transfer Protocol. It is how the articles within Usenet are shared between machines. Originally, the protocol described how messages went from the news server on one university machine to the next machine at a different university. If you wanted to read the messages, you connected to your university’s machine and read them there with a newsreader. NNTP also controlled how messages went between the news server and the newsreader. Newsreaders kept track of the newsgroups you, as the reader, were interested in, as well as which messages in those groups you actually read. As newsreaders got smarter, they added support for splitting large files apart and putting them back together. NNTP also evolved to include support for 8-bit characters and binary file attachments, and added NNTPS for a secure version, just like HTTP versus HTTPS with URLs in a web browser. Believe it or not but this evolution of the NNTP protocol also led to how email attachments worked.
While the Internet and the World Wide Web has evolved considerably since those early days of Usenet, Usenet still functions pretty much the same way today. There are several key differences between then and now. Today, users are not limited to research universities. What would happen each fall was a new batch of students would enter college and have to learn proper netiquette (Usenet etiquette). Behavior would improve throughout the school year before the next batch of students would enter school the next fall. Next, companies also came online and participated in Usenet, too, which didn’t have the churn rate of students. However, the biggest disrupter to Usenet at the time though was America Online (AOL). With those AOL floppy discs/CD’s in hand, new users would sign up for AOL year round and have to learn proper Usenet behavior when the signed up, not en masse with the fall freshmen class. Message volume has increased considerably, too, but more significant is the sheer volume of files that spread through the alt binaries groups. With HDTV available, you want to find (or share) files with a higher resolution than before to be able to take full advantage of that new TV set. This resulted in binary file sizes exploding immensely since those early Usenet days.
Today, you don’t have to be the system administrator of a Usenet box to be able to access the various newsgroups. Instead, you just find a web-accessible news server and read/download all the messages yourself.
Looking for a new gadget? How about a Drone? You know you want one!