Why we don't have an NNTP back end for web forums.

[reddragdiva]

(This post is ten years late. Nobody cares any more, including the people who care.)

Old Usenet hands contemplate web forums with aesthetic revulsion. They're a single point of failure, the interface is not flexible, killfiles require Greasemonkey scripts or similar rubbish. Usenet was so much better. (Except even its fans don't bother much any more.)

An NNTP backend on a web forum — or, rather, a web forum interface to NNTP — seems a simple enough idea. So why hasn't it happened?

I think the two key problems are:

1. On web forums, threads are a first-class entity. On NNTP, a thread is an ad-hoc view of a group of messages, assembled on the fly from what References: headers can be found.

2. Web forums are synchronous: you will never get a thread with missing posts. Missing posts are normal on NNTP, as its distributed nature means receiving a message at all, let alone receiving it in the right order, can't be guaranteed.

These are technical differences, but they're fixed in place by user expectations.

To what extent would bolting an expectation of strong threading onto NNTP break expectations? To what extent would an expectation of pretty-much-synchronous strong threading be breakable by NNTP?

(There's other fluff like identity management, which NNTP doesn't do at all, trusting whatever the poster puts in the From: field. But that's not part of the very nature of the problem.)

Even on single-system two-way gateways, the culture clash can be problematic. e.g. the wine-users list, which has a two-way gateway to the forum. No messages are lost, but the different posting styles clash badly.

So. Are there any useful suggestions? How to do forums over NNTP? How to map the user expectations of one to the user expectations of the other?

(Cheers to those providing useful suggestions and comments on Facebook and Twitter. Which will be the next frontiers.)

Comments

[rogerbw.myopenid.com]

To what extent would bolting an expectation of strong threading onto NNTP break expectations?

It would break Outlook Express. Boo hoo.

In the opposite direction: to what extent would bolting the expectation of a message in a thread being a reply to a specific other message, rather than to the thread in general, onto web forums break expectations?

To what extent would an expectation of pretty-much-synchronous strong threading be breakable by NNTP?

If the default webby view is by arrival time rather than posting time - or if the web thing has some way of indicating "I don't have this message yet" and putting a placeholder in the right space in the thread tree - it should be soluble, I think.

Posting styles are the big problem - one article as one big coherent thought, vs one article as a throwaway comment. But this varies between USENET groups anyway.

[vatine]

I have seen a few quite different approaches to "structured, archiveable conversation" and all models have their ups and downs.

There's the LJ/DW model of "the basic entity is the post, with a threaded comment tree underneath".

There's the web-forum model of "the basic entity is the thread, with multiple messages in strict time-order".

There's the Usenet (and to some extent, email) version of "the basic entity is the individual article/email, cross-linked with a variety of headers"

And there's the KOM "the basic entity is the conference, however each text has a list for all its parent texts and all its child texts; and we have ACLs to control who sees what".

All have their appeal, none fit perfectly in the model of the others, since they operate on quite different assumptions.

[hirez]

I think you are trying to re-invent Cix/Bix (CoSy), which, um, oh god.

[heliumbreath]

At the technical level, much as INN builds and maintains an overview file, you could extend to a thread map that gets updated as new articles are posted. Using cookies or such to track newsrc data would let you show each user which threads have new postings since their last visit, and show a thread with new articles expanded and already-seen ones collapsed. (The LJ model not having that kills discussion: a thread is rarely good for a second visit.) And NNTP provides a well-tested mechanism for redundancy and failover, if you grow to having multiple web nodes. Building a web interface on top of an article-storage mechanism that's already well-debugged and robust should be easier than starting from zero.

Part of the trick, on open Usenet or even a semi-private hierarchy, is that other nodes aren't running your software stack and you have to trust them to post well-formed articles and deal with the occasional broken article that doesn't have a References header or such. Software that broken is mostly ancient enough to have been garbage-collected by now, and Usenet does generate social pressure for the last holdouts to upgrade or go away. And sometimes you don't get an article but you do get a followup to it, or more likely the earlier article was cancelled.

Culture clash is going to be the main problem, if part of the community is used to text on 80-column screens and part wants to use HTML markup on windows of whatever width they're dragged to today.

But community is the important thing: I'd rather continue doing crufty old Usenet with well-clued friends than use the niftiest web-forum implementation with only drive-by morons posting.

[jld]

At the technical level, much as INN builds and maintains an overview file, you could extend to a thread map that gets updated as new articles are posted.

And, IIRC, Lars Magne Ingebrigtsen already wrote something like that for Gmane.

[(Anonymous)]

Oh, this stirred some memories! Someone has been over this ground in the past, and it was Jon Udell of BYTE magazine. Back when people will still grappling with the web for the first time, he did a bunch of work creating forums accessible simultaneously by web and NNTP (http://jonudell.net/bytecols/1999-08-06.html).

I don't know how much of that stuff has survived, I can't find very much of it with an initial google search. He did write a book (http://oreilly.com/catalog/9781565925373/), but I've not read it, so I've no idea how relevant it would be today.

[(Anonymous)]

Didn't Baen (the SF publisher) do this with their Baen's Bar forums?

[(Anonymous)]

"Why we don't have an NNTP back end for web forums?" Because nobody's bothered to do the work.

[rone]

"Why we don't have an NNTP back end for web forums?" Because nobody's bothered to do the work.

[reddragdiva]

This is true, but not useful. The interesting thing is that the landscape is littered with past attempts at the easy part, all of which failed.

[rone]

It would help if there were one superior Webforum software package, but it's a poupou platter of abominations, which doesn't exactly inspire anyone to make a decent backend.

What i want, though, is a decent frontend. In a nutshell, something trn-like set up in AJAX, so when you visit a blog, it remembers when you last visited and shows you the next unread entry, and could also keep track of new comments.

[jld]

The thing I couldn't figure out a nice way to deal with, when I was thinking about this in 2005, was quoting styles — if the post is the only thing on your screen, then people should quote, but if it's a scrolling view on a thread then you can probably rely on context. You can probably hack something up, but it'll be ugly.

And that'll probably hit well before the more subtle technologically-encouraged culture clashes like “OMG, how dare you derail my precious thread with your offtopicness” vs. “dude, it's just thread drift”.

[hellsop]

if the post is the only thing on your screen, then people should quote, but if it's a scrolling view on a thread then you can probably rely on context.

One would think so, but that doesn't seem to work in practice, especially given that the forum threads present in strictly chronological order, and (by that very lack of quoting) discourage starting reading anyplace but at the very first post. Most web forums "solve" this problem by locking threads after a certain amount of time/posts have been added to a threat, which prevents people from coming to a conversation history and participating in it. The number of threads out there that exist in a manner of "I'm having problem B with X" "X sux0rz. Use Y." *thread closed* *new thread* "Regarding the B problem (link), I found C, D and E in order resolved it", with little or no means of getting from the first thread to the second, is truly staggering.

[ewx]

I'm surprised to see zero mention of Google Groups, which for all its faults is by a comfortable margin the most popular Usenet client, at least in big8+uk.

[reddragdiva]

That's because it's the last Usenet server standing, not because the interface is any good, because it isn't.

[ewx]

It's certainly not the last one standing yet. The point is that the news/web integration has happened, even if you (understandably) don't like the results.

[reddragdiva]

I've long wanted a web-based NNTP interface that didn't suck (e.g. for the other place); I've yet to see one that doesn't suck, and Google Groups isn't it. I have a nasty suspicion no such interface is possible, though I'm eagerly open to existence disproofs.

[hellsop]

Has anyone any objective data regarding missing posts in non-binary groups on well-connected semi-commercial providers (EG individual.net or eternal-september.org)? I am not sure that this is a widespread or persistent problem anymore and may exist solely as a perception problem maintained by people with long histories with USENET (and therefore, extensive kill-pattern files).

[reddragdiva]

To give some perspective, I used to maintain the slrn-on-Windows webpage and actually last used any form of Usenet in 2006. I may have posted to a group via Google since then, despite the sucky interface, but I'm not certain. That's what I mean by "this post is ten years late." It's mostly an exercise in thinking through a problem, even if the problem is obsolete.

[tcpip]

How about a prettier front-end for Mailman and use the NNTP gateway features of that application?

[liam-on-linux.livejournal.com]

Interesting thoughts about the techie-problems side of it.

I wrote about something along broadly the same lines (http://www.theinquirer.net/inquirer/news/1013886/social-networking-isn-news) a few years back.

I still reckon it could be done and done well. Forum interfaces all suck. The selling point could be just a really good forum interface, and the clever syndication stuff behind it just the silver lining for the sophisticated users.