Scores of Scoring Thoughts

Jul 31 20:24 UTC 2004

Picospan was designed twenty-two years ago.  Conferencing on Grex hasn't
evolved very much since then.  We have a web interface, but that's
really a mostly cosmetic change.  This wouldn't be a problem, except
that I don't think that the currrent model of conferencing is working
very well on Grex these days.  I think it offers inadequate tools to
help users find what they want among what they don't want.

I've been thinking a bit about how conferencing software could be
modified to work better for Grex.  I don't have a plan, but I think it
would be worth doing some brainstorming on this, so I'm going to type in
some half-baked ideas on this.

My idea would not to be replace the current Picospan/Backtalk
combination.  Instead this would be initially be a separate, alternate
conferencing system also offered on Grex.  Or it might be a separate
interface to the same conferences - depends on how radical the design
changes get.

I'm partly inspired by slashdot's moderation system.  Their FAQ
(http://slashdot.org/faq/com-mod.shtml) covers the basic concepts of the
moderation system in a very clear and logical fashion, and I recommend
reading it.  The basic idea is that by a somewhat complex method that is
the result of a lot of expermentation and fine tuning, each response is
assigned a score.  When people read, they set a threshold, and any
responses/items rated below that threshold vanish from sight.  Nothing
gets deleted but low-value comments don't get seen by many people.

The slashdot example, though very interesting, is not quite a viable
model for Grex.  The most important difference is that Grex is a vastly
less focused community than slashdot.  That is, the range of things
people come to Grex to find is very large.  Think of some typical items
from agora:

   - The "Why are you Happy?" item
   - A generic gun control debate item
   - The humor item
   - The license plate item
   - Some one's weight loss item
   - Polytarp's latest post

So let's rate these from best to worst.  Well, the best is obviously...
um, opinions vary.  Well, at least we can agree on the worst ... well,
no we can't.  Polytarp's items evidently make polytarp happy.  All of
these items have constituencies, otherwise they wouldn't keep getting
entered.  In every case there are also people who would rather gouge
their eyes out than read one more of these items.  Slashdot has a much
narrower focus than Grex - it's a technology news site.  This narrowness
of focus is really the norm on the net these days - there are very, very
few conferencing systems that don't have a specific unifying topic. 
This is one of the things that make Grex special - it presents you not
only with material you are interested in, but with material that you
never suspected that you might be interested in.

So a system for scoring Grex responses must differ from the slashdot
system first by recognizing that there are different things different
people are interested in.  The traditional Picospan way of doing this is
the idea of topical conferences.  If you are interested in politics, you
go to the politics conference.  If you are interested in humor, you go
to the humor conference, etc.  This basically works badly.  Over time,
all activity in other conferences dies out, and everything moves to
Agora.  You want to post where the most people will see it.  You want to
catch the interest of people who might not go out of their way to see
your item.  In this day and age, if there is a narrow topic you want to
discuss, you'll probably go to a conferencing system or mailing list
that is devoted narrowly to that topic, not to Grex.  If I have a Linux
question, I'll post it to the Washtenaw Linux Users Group list, not to
the Grex unix conference.  This has accelerated the death of most
non-Agora conferences on Grex.  Agora offers a community of
semi-familiar people, a circle of acquaintances whose value to you is
less predictable than a quick answer to a technical question.

At this point, I think I may have talked myself out of the idea I had in
my head when I started this post, but, hey, that's the point of writing
things down.  I'm going to proceed to describe some of the ideas I had.

My notion of a way to handle differing interests was to loosen up the
concept of a conference.  We start with a big unsorted pool of
miscellaneous items entered by various people, pretty much like Agora
today.  We create a thing called an "agenda".  An agenda is a list of
items that are interesting from a specific point of view.  There might
be a "politics" agenda that hold items of interest to people who want to
talk about politics.  Maybe a "humor" agenda for people who want to read
funny things.  Maybe a "dork" agenda for people who want to see other
people make fools of themselves, a "rude" agenda for people exhibiting
poor manners, and a "meta" agenda for people who want to hear Grexers
talk about Grex.  Each such agenda would have items on it that fit into
that agenda.  Any given item could be on many agendas. 

So far this is just like conferences, except that linking would work
better, so reading something on one agenda would mark it read on all
other agenda's too, and it would be much more normal for items to be
included in many agendas.

Within an agenda, we would also have the capability of rating the
relevanace of each response (or item text) to that agenda.  So on the
"politics" agenda things would be rated for how political they are, on
the "humor" agenda things would be rated for how funny they are, on the
"rude" agenda things could be rated for how rude the poster is.

As a reader, could simply read the agendas you like, like politics, with
a threshold of your choise (so low rated responses get filtered out). 
But you could also read combinations - political things that are funny
but not rude, combining agendas with different weights.  Obviously this
is something you could fiddle with a lot to fine tune to your needs. 
One possible way to make this easier is to give every user a "personal
agenda" - where they can include items they like and rate responses by
how much they liked them.  It might be possible to match an individual
user's preferences against the system agendas and automatically generate
a weighted combination of agendas that roughly matches their ratings. 
(This is a tricky, but much studied computational problem.)

You'd also have some system-generated agendas in the mix, like "newness"
that rates responses by how recently they were entered, "brevity" that
rates them by length and "polygon" and "polytarp" that rates them 5 if
they are by janc or polytarp or 0 otherwise.  You can mix these into
your item selection function, so you can bump up or down things that are
in those categories.

Most of the other agendas would have to be manually maintained by
someone though.  I think there should be different models.  Some might
be created by cfadm with the list of administrators assigned in much the
same way that we now assign fairwitnesses.  Generally this would work
best if you assign people to run it that actually like stuff like that -
say put "klg" and "richard" in charge of the politics agenda.  They
probably wouldn't agree on the truth of many political postings, but
they'd likely mostly agree on whether the subject is political in
nature, and since they are likely looking for such discussions, having
them mark them when they see them is pretty painless.  For something
like "humor" you'd likely want to get more people voting.  For anything
there would need to be a mechanism where other people who happen across
a new political or humorous posting can toss it toward the appropriate
agenda.

I'd really rather see a more open model for agendas though.  One idea is
letting any user create as many agendas as he or she likes, and make
them available for others to use.  This makes a marketplace of agendas.
 Well maintained agendas get subscribed to by more users.  There might
be ways to let interested people feed into existing agendas without
having to formally appoint them.

For newusers and anonymous readers, you could do something like keep
track of how many users used which agendas with which weights in the
last weeek.  Use those to compute a default set of weights for a default
agenda.

An alternative approach would be to ditch the idea of topical agendas
(like "politics") and use only the personal agendas, where each user
rates things by how much they like.  Then you'd pick users whose
judgement of goodness is similar to yours (perhaps by an automated
mechanism that matches their past ratings to yours) and use a
combination of those to select and rate things you haven't seen yet.  As
I'm writing here, I'm tending to drift more toward the idea of personal
agendas only.

Obviously not all these ideas could be used, as things would get wildly
complex if you did, and there are some interesting GUI issues too. 
There are issues in figuring how to keep the labor cost in rating things
low.

Some of these ideas wouldn't fully achieve my goals.  I don't want a
mechanism so perfect that it shows you only content that you expect to
be interested in.  This would be boring, and would tend to fragment the
community into multiple narrow communities.  Maybe there should be a
system generated "random" agenda that assigns random plus scores to
random items and then can be used to mix in a random scattering of other
items into your reading.  The ideas for tools that do pattern matching
to find agendas that resemble your personal agenda and predict interest
based on that would probably throw in enough goofy results to fulfill my
desire for a certain level of chaos all by themselves.

So I'm still playing with ideas here.  I think Grex needs some kind of
mechanism for helping people sort signal from noise, but I'm far from
sure what that mechanism should be.  I'd be intereted in any ideas.

response 4 of 37:

Aug 1 02:09 UTC 2004

I only skimmed through this, so a few general points:

The new ideas for items are nice, though I doubt poly will restrict himself
to one item unless he grows up.

I'd like to see gate extended to provide an emacs- or perhaps joe- or
pico-like interface. I'm not particularly fond of picospan, but it does the
job adequately (apart from the editing function, as I just implied.) But then
I'm not wild about text-based menu interfaces in general.

Of course, these days people have the option of using backtalk.

response 6 of 37:

Aug 1 15:44 UTC 2004

For what it's worth, I like Jamie's ideas.  I suspect that you could write
a newsreader client that mimics the current Picospan look and feel.  Import
all existing conferences into grex.* newsgroups, and go from there :)

I would probably STILL use the text interface that works like picospan and/or
Yapp, because I LIKE this interface.

response 8 of 37:

Aug 1 17:31 UTC 2004

Hey!  Substituting an NNTP server for Picospan was *also* something I
proprosed a long time ago!  Stop stealing my ideas, people!  :-)

response 10 of 37:

Aug 1 17:58 UTC 2004

10. Automatically fixes all my misattributions

response 14 of 37:

Aug 2 03:56 UTC 2004

re 1
>There are a couple of essays on this topic on
 jameshoward.us.

Where, exactly?

response 16 of 37:

Aug 2 22:42 UTC 2004

I have been thinking of something along the lines Jan brought up.  Only
instead of conferences and agendas, there would be lists.  Anyone could
create as many lists as they wanted to, and have any of them be public
or private.

When people create an item they may submit it to as many lists as they
wish.  The owner of the list would then either accept or reject them.
People so inclined could view just the accepted items or all the submitted
items, or I suppose just the rejected ones.

There should also be what iTunes calls smart lists, for example everything
on the science list except for the geology list.

An official hierarchy would also be nice way to get things started.  At the
top would be EVERYTHING and COMPUTERS/HARDWARE/MACINTOSH/IBOOK would be near
the bottom.  When someone tried to submit something to the COMPUTERS list
they would be asked to pick sub-categories.  You couldn't get the hierarchy
right before hand so there would need to be a way to split categories. 

response 18 of 37:

Aug 3 21:42 UTC 2004

Another area for a new system to address is the "fixseen" problem.  Fixseen
sort of works, but it marks things as seen which haven't been.  I would
suggest having old-unseen, new-unseen, and seen items.  By default only new-
unseen stuff would be shown.

When someone joins a list it might be a good idea to mark something other than
the first and last items as new-unseen.  Say all entries in the last week,
or the 50 most recent entries.

Another thought would be to allow responses to responses, not just the
original item.  This would change things from lists to trees, but might help
keep discussions on track.

Having a reader program running on client computers could be great, and if
the system had an SQL interface, might not be that hard to pull off.

response 20 of 37:

Aug 4 18:31 UTC 2004

There's been some talk about using NNTP here and (somewhat bizarrely) in
the retroGrex item.  I'm kind of baffled by that idea.  Yes, you could
use a newsreader to read Grex.  But every newsreader I've seen had an
interface even suckier than picospan.  Yes, you could have a
client/server architecture, but what does that gain us?  It'd only be a
server for content, not for participation information.  Most newsreaders
keep records of what you've seen in local files, so if you try reading
from more than one server, you'd lose your participation information
every time you change servers.  It's a client/server architecture, but
it's not even as good as the fronttalk/backtalk combination.  It really
sounds like a plan that has not really been thoroughly thought through
to me.  It also contributes nothing to solving any of the problems I was
addressing in this item.

I don't know what prp is talking about with respect to "fixseen".  It is
supposed to mark everything in the conference seen.  It is supposed to
mark things seen that haven't been.

Backtalk already allows fairwitnesses to configure which items will be
new for users joining for the first time.  At the moment, I don't think
any fairwitness has bothered to modify this from the default first/last
setting.  Probably because right now it only works for users who first
join the conference via backtalk, not those who first join it via
picospan.

Backtalk has about 20% of a capability to do responses to responses. 
I'm not enthusiastic about this being used in other than very limited
ways.  Deep branching makes it much more confusing to read a
conversation.

response 22 of 37:

Aug 5 06:41 UTC 2004

jp2, why do you never answer any of my questions ?!

response 24 of 37:

Aug 5 20:16 UTC 2004

I like Jan's idea of "agendas" in #0, of getting away from the
current static model into something more flexible and less
hierarchical.  Instead of a conference set maintained by a central
administrator and modifiable only with the administrator's
cooperation, why not a system in which administrators could create
"official" agendas (the equivalent of our current conferences),
but users could also create agendas by grouping existing items
and add new items to the group.  I'm not sure that Grex has the
critical mass for a Slashdot-style rating system to be effective,
but I think that the flexible "agenda" approach has merit even
without that.

The user interface to all this could really look just like
Picospan or Backtalk - "join", "read", "forget", "fixseen",
etc. could work essentially as they do now.

Setups that give you something like this already exist in the blog
world.  They're variously called "news aggregators", "rss
clients", etc.  News sites or bloggers can "syndicate" their
sites, and a user can subscribe to any number of these using
appropriate client software.  Have a look at http://bloglines.com
for a web-based client.

Frankly, I don't find the NNTP-based alternative that's been
suggested to be at all attractive.  Part of the problem with
Picospan-style conference organization is its strict, inflexible
system of disjoint topical conferences, leading to dead-conference
syndrome, as Jan points out in #0.  NNTP makes this even worse by
extending the hierarchy to arbitrary depths (anybody remember
alt.swedish.chef.bork.bork.bork?), so things can get fragmented and
lost even more thoroughly than in Picospan's single-level
conference hierarchy, *and* with NNTP we're still imposing a
centrally-administered structure over which individual users have
no direct control, which is another part of the problem.

For implementing the "agenda" concept, I think we should take a
close look at the RSS ("Real Simple Syndication") format and its
cousin format Atom.  They're what is used to implement
"syndication" or "feeds" of news sites, blogs, and other websites whose
content is updated frequently.  The idea is simple:  A
news site (such as cnn.com) or blogger that wishes to syndicate
its content creates a web-accessible file in an appropriate format
(RSS or Atom) and publicizes its url.  For example,
http://slashdot.org/slashdot.rss is the one for Slashdot.  The
file contains a list of items on the site, and is updated whenever
a new item is added or an existing one modified.  To "subscribe"
to the "feed", users enter this url into their client software.
The client keeps track of what items on the site the user has
already seen and displays any material that is new.  So instead of
going to the Slashdot site to see what's new, you can subscribe to
their rss feed and be informed whenever anything new shows up.

Sound familiar?  Firing up a news aggregator and checking on
what's new is a lot like running Picospan and doing a "browse new"
or "read new".  The .rss file that supplies the feed is
functionally like Picospan's "summary" files, and the records kept
by the client are like Picospan "participation" files.  The
difference is that RSS is an open, non-proprietary, widely-used
format.

I now use bloglines.com to keep track of a number
of news sites and blogs, and the left-hand frame of their web
interface looks a lot like the output of Picospan's "check"
command (10 new items in Slashdot, 25 new items in The Register,
etc.) To access the new items in a particular site, I just click
on its name and item summaries show up in the right hand frame.
When I click on a summary, it takes me to the actual website and
displays the material.

The RSS file that supplies the feed doesn't even have to be on the
same website as the content being fed; it just has to be kept in
sync with changes on the website.  I know of some "bootleg" feeds
not under the control of the site that supplies the content.

So one way to implement agenda-based conferencing on the Next Grex
would be to have the conferencing software maintain an RSS (or
Atom) file for each item and each agenda.  Users could "subscribe"
to agendas (the equivalent of "join") or to individual items.
A user-created agenda could be implemented as a world-readable
RSS file in the user's web directory.  A user's "cflist" would
be a list of RSS files, and could be a mixture of official system
files (e.g. "politics") and user-created ones
(e.g. "remmers-list-of-weird-items-on-various-topics").

If we use RSS as the basis for conference structure, folks could
read Grex material using any news aggregator (e.g. "bloglines"),
which would make Grex material more accessible to a wider
community.  To post, of course, a person would still have to log
in via the backtalk or tty interface.

RSS alone probably wouldn't give us all the functionality we'd
want (e.g. filtering individual users), so Grex's homegrown
conferencing software would have to implement those in addition to
RSS support, should we choose to go the RSS route.

Whether we go with the "agenda" paradigm and RSS, or with
something else, I do think we should move in the direction of
employing open, widely-supported standards and away from
proprietary formats, especially those not under our control.