No Next Item No Next Conference Can't Favor Can't Forget Item List Conference Home Entrance    Help
View Responses


Grex Garage Item 2: Backtalk's Cinnamon Syndication Interface
Entered by janc on Thu Nov 11 03:25:39 UTC 2004:

Backtalk now has an experimental syndication interface called "cinnamon"
(because it rhymes with "syndication").

You can access it at:

   http://grex.org/cgi-bin/backtalk/cinnamon/conf?conf=agora

This will report items that were entered in the agora conference in the
last seven days.  In the future, I'd like to add other versions of this
that report other kinds of things (like responses), and perhaps for more
than one conference at a time, but this is all we've got for now.

The default is to generate an RSS-2.0 feed, making use of some Dublin
Core extensions.  You can change the format by specifying a fmt argument
in the query.

   fmt=RSS-0.91
        An outdated format that may be useful to users with
        outdated software.

   fmt=RSS-1.0
        Latest version of the RDF branch of RSS.  This is not
        obsoleted by RSS-2.0.  Uses Dublin Core elements.

   fmt=RSS-2.0
        Enhanced successor to RSS-0.91.  Uses only the core
        elements.

   fmt=RSS-2.0/DC
        RSS 2.0 extended with Dublin Core enhancements.  The default.

By default, all links back to Grex in the feed will be to the pistachio
interface.  You can override this with another optional argument, like:

    flav=abalone

This should work for all interfaces.

Normally the feed contains all items entered in the last 7 days.  You
can change this by giving an argument something like:

    days=2

or, if you do days=0, it will give all items.

The "descriptions" of the items in the feed are generated by taking the
first 250 characters of the item text.  You can change this with an
argument too:

    len=500
          Changes the number of characters used for the description.
          Values between 1 and 20 are treated as 20.
    len=0
          Causes no description to be sent.
    len=-1
          Causes full text to be sent as description.

At this time, I haven't tested this at all.  All I've done is eyeball
the output to check if it looks like what I think an RSS feed should
look like.  I may well have munged it up.

I'd prefer that these RSS URLs not be widely advertized yet.

18 responses total.



#1 of 18 by janc on Thu Nov 11 03:42:58 2004:

OK, I just tried it in the news feed watcher built into Ximian
Evolution. It barfed on the default version, but took the RSS-1.0
version just fine.

I think I'd like an option to have it include the authenticated URLs
instead of the anonymous URLs.  This means that I'd have to log in when
I click on the link, but I'd actually prefer that.


#2 of 18 by janc on Thu Nov 11 15:11:49 2004:

Spent a little time looking harder at Atom interfaces.  And got
hopelessly confused.

Here's two links to Atom Specs:

 http://www.atomenabled.org/developers/syndication/atom-format-spec.php
 http://www.ietf.org/internet-drafts/draft-ietf-atompub-format-03.txt

They both claim to be version 0.3.  The first one is dated December 2003
while the second is dated October 2004.  But the second says things like:

   The Atom format is a work-in-progress, and this draft is both
   incomplete and likely to change rapidly.  As a result, THE FORMAT
   DESCRIBED BY THIS DRAFT SHOULD NOT BE DEPLOYED, either in production
   systems or in any non-experimental fashion on the Internet.

However, the older spec says nothing of the sort, and there are, in
fact, production deployments of Atom around, like this one from
blogspot.com:

   http://cateybeth.blogspot.com/atom.xml

Blogspot.com seems to think atom has a "consistant, tightly specified,
well-documented XML format".  At least they say so here:

   http://help.blogger.com/bin/answer.py?answer=697

In fact, both versions of the 0.3 spec are seriously incomplete, full of
big gaps.  Notably, both versions contain an annotation saying that
someone needs to fill in the motivation and design principles for the
project.  How is that the last thing you write?  Have these guys got a
compass?

The two 0.3 specs also flatly contradict each other.  For example, the
older one says, of "type" attributes:

  When present, this attribute's value MUST be a registered media
  type [RFC2045]. If not present, its value MUST be considered to be
  "text/plain".

While the newer one says:

  When present, the value MUST be one of "TEXT", "HTML" or "XHTML".
  If the "type" attribute is not provided, software MUST behave as
  though it were present with a value of "TEXT".

  Note that MIME media types [RFC2045] are not acceptable values for
  the "type" attribute.

Seems to me that such dramatic changes in a spec would at least require
an increment in the version number.

The sample feed I quoted from blogger.com seems to conform to the older
spec, not the more recent one, but it also seems to contain a lot of
stuff that is not in the spec.

My conclusion is that Atom does not exist, though various people firmly
believe that it does.  I'm not going to spend any more time doing atom
support at this time.


#3 of 18 by remmers on Fri Nov 12 18:29:57 2004:

Probably a wise decision re Atom.  It's currently under the wing of
an IETF working group whose goal is to produce an official RFC defining
the thing.  I subscribe to their 'atom-syntax' email list, which is
very high-volume and seems more like an arguing group than a working
group.  That's what happens when you design by committee.  One faction
wants just to systematize and clarify field-tested and proven features
of the various RSS versions, and another faction wants to tread boldly
where none have gone before and add all kinds of new features. 
Eventually something good may come of all this, but I'd wait until 
they get things sorted out, if and when that happens.

It's good that you're supporting the various RSS versions that are
out there.  Lots of feed publishers do that.

I looked at the default RSS-2.0/DC output and it looks valid to me.
Offhand I'm not sure if you're supposed to provide a namespace as
an attribute of the document root if you use Dublin Core elements.

Incidentally, there's a feed validator on the web:
http://feedvalidator.org/


#4 of 18 by remmers on Fri Nov 12 18:45:49 2004:

Tried plugging the various fmt options into the aforementioned
validator.  Results:

    RSS-0.91 and RSS-2.0 produced backtalk crashes (consistently)
    RSS-1.0 made the validator happy
    RSS-2.0/DC gave the error:  lastBuildDate must be an RFC-822 date



#5 of 18 by janc on Sat Nov 13 03:43:42 2004:

Hmmm...I generated those dates by calling the same function that I use to
generate dates for cookies.  Dates look like:

   Thu, 11-Nov-2004 03:25:39 GMT

I think that this is fine, except it should have spaces instead of the dashes.
Hmmm...The HTTP Cookie spec says that dates comply with RFC 822, "with the
variations that the only legal time zone is GMT and the separators between
the elements of the date must be dashes".  I'd always assumed that the dashs
were legal under 822 and that the HTTP Cookie Spec was subsetting the spec,
as it was with the time zone restriction, not arbitrarily deviating from it.
However 822 doesn't actually seem to allow dashes.  It also doesn't allow
four digit years, which RSS 2.0 expressly allows and the HTTP Cookie spec
shows in its examples but fails to talk about.

Sure is nice that we have standard date formats so we can write one date
printing/parsing and use it everywhere.  Of course, RSS 1.0 simply uses
ISO 8601 dates instead, or at least a subset of them.

Well, this is easy enough to fix at least.

I'll look into the crashs.


#6 of 18 by janc on Sat Nov 13 04:54:35 2004:

OK, the two formats you named crashed because /usr/local was full on Grex.
Backtalk scripts get compiled the first time they are run and the compiled
version is saved to disk.  Doesn't work when the disk is full.  I cleaned up
some disk space (though more freed up than I think I freed so maybe someone
else did too).  They now work.

I've fixed the date syntax problem, and uploaded a slightly new version of
the Cinnamon interface.  This adds one feature:

   The argument  "auth=Y"  on the URL makes all the links in the feed
   point to the authenticated version of backtalk instead of the anonymous
   version.  I think this will normally be prefered for any regular
   Grex users, but it is not the default.

I fed all four feeds through the validator.  All but the RSS-2.0 feed
passed.  I wasn't actually surprised by this.  The problem with the RSS-2.0
feed is exactly why I created the slightly funky RSS-2.0/DC feed.

It says "author must include an email address".  Well, yes, I knew that.
It'd be easy to format the <author> field like:

   <author>janc@cyberspace.org (Jan Wolter)</author>

which is what they want (it's OK to leave out the full name).  In fact,
Cinnamon has a config switch I can throw to make it do just that.  But I
don't want email addresses in Grex's RSS feed.  That's just spam bait.
So I for the RSS-2.0 feed I wrote:

   <author>(Jan Wolter)</author>

thus leaving out the email part of the email address.  Well, actually it says

   <author>(Jan Wolter [janc])</author>

Sure, anyone who knows anything about Grex can figure out how to send me email
based on that, but that excludes most spammers.

Now, I knew I was bending the spec here, but I thought it was a natural and
logical way to bend it, and very likely many clients would be able to parse
it right.  But I don't like deviating from spec, so that's why I whipped up
the RSS-2.0/DC version.  It uses the Dublin Core <dc:creator> tag instead
of the RSS-2.0 <author> tag, so I can give full names without email addresses.

What I should really do is add an option that would include author names
in the item title, like:

  <title>Backtalk's Cinnamon Syndication Interface - Jan Wolter (janc)</title>

This would be very nice for those clients that show nothing bit the title.
I could then just completely omit the <author> field in the RSS-2.0 feed,
if email addresses are not configured on.  As a Grex reader, the item
author is more important to me that the title.  I really want to see that.


#7 of 18 by janc on Sat Nov 13 04:56:17 2004:

Hmm...I may be renaming the "auth" flag I just added. Having an "authenticate"
and an "author" flag leads to some confusion over what "auth" does.


#8 of 18 by janc on Sun Nov 14 19:24:23 2004:

Things have changed again.  The old URL will no longer work.  To get the
conference feed, you now use a slightly shorter URL:

  http://www.cyberspace.org/cgi-bin/backtalk/cinnamon?conf=agora

The 'auth' option has been renamed to 'login'.  Item titles now include author
names.  Adding 'attrib=N' to the URL turns this off.

There is also a feed to watch responses to an item.  Like:

  http://www.cyberspace.org/cgi-bin/backtalk/cinnamon?conf=agora&item=3

Options are pretty much the same as the other one.

Here's the current options:

       fmt=<format>
          Here <format> is one of the format strings 'RSS-0.91', 'RSS-1.0',
          'RSS-2.0' or 'RSS-2.0/DC'.

       flav=<flavor>
          Here  <flavor> is one of the Backtalk interface flavors that you
          have installed on your site, like pistachio, abalone, or papaya.
          All the links back to your site will be to the named interface.

       login=Y|N
          If login is 'Y', then the links in the feed will be to the
          authenticated version of Backtalk.  The default is 'N', which
          gives anonymous feeds.  Sites that don't allow anonymous reading
          are always Y.  Sites using form authentiction instead of basic
          authentication are always N.

       attrib=Y|N
          If attrib is 'Y', then the names and login ids of item authors are
          included in item titles.  In this case a separate author or
          title field will only be given if there is additional information
          to provide about the author, like an email address.  Note that
          turning this off may cause slightly non-compliant feeds with
          the fmt=RSS-2.0 option.

       len=<number>
          In the conference feed, the descriptions of the items are
          generated by taking the first 150 characters of the item text.
          Changing this parameter changes that limit.  If you set it to
          zero, no descriptions will be sent.  If you set it to -1, the
          full item text will be sent.  Other values below 20 will be
          treated as 20.  With RSS-0.91 this can not be above 500.

       days=<number>
          Normally only items entered in the last 7 days will be included
          in the feed.  You can change that value with this parameter.
          If set to zero, all items will be included.  Note that with
          RSS-0.91, there can never be more than 15 items included.


#9 of 18 by remmers on Mon Nov 15 21:19:34 2004:

This is way cool stuff.

I'm curious about implementation.  When a request comes in, do you build
the entire feed instance from scratch from the raw item files, or do you
use caching or some other mechanism to improve performance?


#10 of 18 by prp on Mon Nov 15 23:06:09 2004:

To make RSS-2.0 happy, you could add an email addess like
donotuse@cyberspace.org, and setup the id donotuse to generate replys that
explain why a real email address is not included. 


#11 of 18 by janc on Fri Nov 19 18:30:48 2004:

There is no caching at all.  The whole feed is built from raw sum files and
item files each time it is requested.


#12 of 18 by janc on Wed Jan 12 17:11:02 2005:

This item has been liked from garage1 to garage2


#13 of 18 by eprom on Sun Apr 3 01:15:57 2005:

Hey! I think it would be cool if we could have a flavor of backtalk
which is more blog like and allows personal avatar (small pictures) to
be seen.




#14 of 18 by gull on Mon Apr 4 15:56:04 2005:

That's a pretty cool idea, actually.  I suppose Grex's prohibition on
serving images would be a problem, though.


#15 of 18 by janc on Mon May 2 13:11:53 2005:

Backtalk already has some capacity to attach arbitrary documents to
responses or users.  My plan was to attach an image to the user that
could be used as an avatar.  Like so much in Backtalk, I need to find
more time to work on that.


#16 of 18 by gull on Mon May 2 14:23:20 2005:

I like that idea.  I've used various web discussion boards that had that
feature, and people seemed to have a lot of fun making custom icons for
themselves.


#17 of 18 by remmers on Thu Dec 22 14:37:03 2005:

Re #8, I request another option:

    secure=Y|N

where Y specifies that the link to the response be an https URL.

If that were available, I would actually use cinammon to do my
Grex bbs'ing.


#18 of 18 by eprom on Fri Mar 3 20:26:59 2006:

I noticed in the Abalone interface, when you compare the number of responses
listed on the 'list all items' page doesn't match the actual number of
responses actually in the item.

Response not possible - You must register and login before posting.

No Next Item No Next Conference Can't Favor Can't Forget Item List Conference Home Entrance    Help

- Backtalk version 1.3.30 - Copyright 1996-2006, Jan Wolter and Steve Weiss