response 6 of 23:

May 24 20:22 UTC 2017

Re: rewriting fronttalk/backtalk.  Let's talk first about picking
a programming language.  This came up on `party` the other day, and
I thought it was worth recording here.

For the types of programs we run on Grex, the language should be
something type- and memory- safe.  That rules out the entire C
family, unfortunately.  (I have a soft-spot for C.)

I'd like something with garbage collection or clear memory sematics;
that'd rule out anyting like Pascal and is another strike against
the C family.

The desire for strict static typing rules out basically all of the
dynamic interpreted languages, so no python, perl, ruby, Lua, Io,
Lisp, clojure, etc.

I'd like to avoid JVM languages due to the high overhead and startup
cost of the JVM itself, so that rules out Java, Scala, Groovy,
Clojure, etc.  This is kind of sad because I sort of like Scala and
Clojure (and Kotlin).

What's left? SML, Haskell, OCaml, Go, Rust...those are the popular(ish)
options.

SML is too niche; I'm going to rule it out.

For this sort of thing, you need something that can interface with
the underlying system fairly intelligently and has a large(ish) set
of support libraries.  That would tend, I think, to rule out OCaml
and probably Haskell.

Haskell is nice, but it's too unfamiliar to too many people.  There
are probably libraries on Hackage to do all the low-level systems-y
stuff we need, but we don't want to require folks to wrap their
heads around Monads just to understand how the BBS works.

So that really leaves Go and Rust.

Between the two, right now, I'd probably pick Go for purely pragmatic
reasons.  It's relatively familiar, relatively light-weight, has a
good implementation, a wide standard library, but is garbage collected
and type/memory-safe.  Unlike Rust, it's also a relatively stable
and mature *language*.

So...Go feels like a win at the moment.

response 8 of 23:

May 26 01:17 UTC 2017

And /cyberspace/contrib exists now. Some folks can write to it.

response 10 of 23:

May 27 03:31 UTC 2017

Thinking more about backtalk/YAPP/Picospan file formats....

Jan Wolter wrote a very nice write-up of the various file formats
used in a Picospan-like conferencing system here:
unixpapa.com/backtalk/stab/doc/format.html

One will notice, in particular, that these formats are very simple,
line-oriented and mostly text-based.

A valid question is: how many are still relevant?

I suggest that only a handful still make sense: in particular, the
item format seems to make sense because there is so much existing
data in that format (on Grex, about 360MB, on M-Net, about 330; so
nearly a gigabyte total between the two systems). It would be nice
to get rid of the custom format and replace it with something like
say, multipart-MIME-encoded Mailbox format files. I think that Jan
had some idea to do this at some point, but never got around to it.
But the sheer amount of data makes it not unreasonable to retain
the current format.

However, the conference list, conference configuration, etc, formats
no longer make a lot of sense. All of that can be replaced with
JSON and things that format JSON. The question then becomes: to what
extent do we retain *aspects* of the existing formats?

For example, the conference list file contains both a "default"
conference as well as syntax for describing *aliases* for a given
conference. These are great for an interactive program like Picospan;
but to what extent do they continue to make sense for a RESTful
server that's really not meant for direct interactive use? I would
think very little. So would it be reasonable to separate out some
of that metadata from the conflist file itself? For example, does
it make more sense to move the description of the default conference
to another file? Perhaps similarly with the abbreviated names and
aliases?

If nothing else, it's worth a bit of an experiment.

response 12 of 23:

Jun 1 04:06 UTC 2017

I want to take a moment and think about WHY things are, sometimes, the way
that they are.

Now, I'm a firm believer that when it comes to technical and engineering
decisions, there is at least some room for art, opinion, and subjective
decision making. But the bulk of the decision should rest on a firm basis
of data and be based on a grasp of theory. It is perfectly legitimate to
ask WHY a thing is the way that it is and the answer should at least be
supportable on technical merits.

Then there is the idea that a decision made should be open for revisiting
in the face of related change: sometimes, the technical landscape changes
out from under foot, and seemingly-settled decisions can come up again due
to changes in the environment. Decisions come with an implicit context; if
the context changes, perhaps the decision should as well.

But that's not the way that Grex operated historically. Grex was more of a
autocracy from a technical perspective; decisions were made unilaterally by
some staff members with no real regard as to presenting an argument for WHY
they were made. That was mostly fine, except when those things got in the
way of *actually running the system*. Then they became issues and worthy of
debate and justification, but that was rarely done. Instead, the usual
response was to repeat that the solution put into place was the "right" one
and that one shouldn't question it; that it was done by volunteer effort
and thus not permitting of question, etc.

An example was mail delivery. Now, the "modern" solution for delivering
mail is to write it into one's home directory; the older solution is to
write it into a file in a central directory somewhere (/usr/spool/mail,
then /var/spool/mail, and finally in /var/mail/$USER).  That was fine,
but it becomes an issue when you start bumping into limits for how many
files can be in a directory; on a big system with many user accounts,
like Grex, M-Net, or SDF you can actually hit this limit.

On the Sun computers, this was especially bad: SunOS 4 and earlier didn't
gracefully handle large directories with lots of files in them.  The
solution in those days was to create a set of "hierarchical mail spool"
directories. Much like user home directories on Grex now, where the user's
login name determines the path to his/her home directory, so the login
name determined the path to the user's mail spool file on the Suns.

When Grex moved to OpenBSD on x86, there were plans to migrate this
forward.  However, by then the de facto standard of writing into a spool
file in the user's home directory had already been adopted by the major
mailers of the day; postfix, qmail, etc, all worked that way.

Yet, Grex chose not to go that way. Even worse, Grex chose to just go with
the traditional method of a flat set of files in /var/mail; yuck.

Suggestions that we go to delivery into the home directory were met with
a lot of resistance.

Why?

I suspect a lot of it came from the proposals not coming from the right
people. It also came from a resistance to acknowledge that the environment
had changed out from underneath of Grex.

I just changed Grex to home-directory mailbox delivery. It wasn't hard,
and I doubt many (if any) will notice. But it's amazing that it took nearly
15 years to get it to happen.

Why?

I think it's at least partially still an open question.

response 14 of 23:

Jun 8 03:59 UTC 2017

So thinking about types a bit....  To focus the discussion a little
bit, let's talk about some of the concrete data objects present in
a Picospan/YAPP/Backtalk/frontalk-kind of conferencing system.

Picospan-like conferencing systems are actually rather straight-forward:
They are three-level hierarchies.

At the top level, one has a conference; this is analogous to a
discussion forum, newsgroup, mailing list, etc.  On SDF, this is
equivalent to each 'board' in `BBOARD`.  However, conferences are
richer than SDF's boards in that they can have access lists,
bulletins, can display login and logout files, etc.  To make things
concrete: in the existing systems, conferences are represented by
filesystem directories.  Creating a conference requires administrator
intervention.

The next level down is the "item": conferences, in some sense, can
be thought of as containers for items.  Items are unique threads
of discussion about various topics; they can be created on demand
by users.  Items are represented by files in the conference
directories; items can be "linked" into other conferences: this is
actually handled using hard links in the Unix sense.

The final level in the hierarchy is the response: responses are
individual comments or messages in a thread of conversation.  They
are represented as a range of bytes inside an item file; one can
*not* link them into other items or conferences, though there is a
backtalk syntax for *referring* to them via hyperlinks in the web
front-end (this is somewhat analogous to symbolic links vs hardlinks,
incidentally).

Clearly, it seems reasonable that there should be a type to represent
each of these objects.  However, complexity is never too far away,
and we start running into some points for design decisions rather
early on.

Picospan-style conferencing systems have a fairly rich vocabulary
for specifying "ranges" of objects.  That is, sequences of items or
responses (curiously, there doesn't seem to be much in the way of
doing that for conferences) that match some criteria (an obvious
one: "give me a list of items with new responses since the last
time I checked").

So....Should the item type include all of the responses to the item,
or should there other types to represent these ranges?

I'm leaning towards the latter; I think it's cleaner.

response 16 of 23:

Sep 3 22:53 UTC 2017

Question: is this web front-end meant to replace logging into Grex to use
fronttalk, or supplement it?

response 18 of 23:

Feb 6 14:53 UTC 2018

You mentioned sharing conferences between systems. Is this sharing
between  alike systems/backtalk replacement system without a name (yet),
or between  other types of conferencing/forum software systems?

One of the reasons i ask is that, i am interested to see connections
between systems like grex (where alot of people login via a text only
system) and also general web based conferencing systems, like say:
Simple Machines Forums, (which coincidentally is what runs Randy Suess's
CBBS, they have no text based system now after running YAPP - which i
believe is very close to picospan).

response 22 of 23:

Mar 17 22:00 UTC 2018

For me as a non-programmer, and nothing to do with IT person; Grex, and
picospan - cast a spell of the forgotton age, where internet communities were
places you could ask any question and likely get the answer from someone who
really knew what they were talking about. As opposed to todays world, where
each subject, interest and field of eneavour has it's own niche, it's own
seperate web forum where you only go to talk with people in the know when you
want to talk about that paricular topic. It seems kinda one sided belonging
to communites where you only share one interest. I see no reason why people
should not strive to build communities on the net, with the spirit of the
older ideals of sharing in a more general pool of knowledge and experience,
and building new things from many sources and opinions, from many fileds of
interest.
Grex, picospan, the Well, still fire the imagination of the times they were
born, the counterculture, the spirit of freedom, and courage to live life in
a simpler way, without the boundaries of mainstream media and consumerism,
and to marry the fellowship of humanity with the technology that enables us
to span the whole globe, and to keep in mind, the better tomorrow is within
our grasp today- each day. The older generation said: "Go West my
daughter/son!" and with us, as with the founders of Grex, The Well, the
arpanet: West is any direction we point our modem: kinda. :) 
Seriously, there is still magic in places like this, and it's inspiring
finding people still wanting to build, and take part in, inclusive/non
eliteist communities like this.