Grex Oldcoop Conference

Item 269: Googable Grex?

Entered by janc on Thu May 26 15:19:43 2005:

Have you noticed that when you google stuff, you never get the Grex
conferences as a result?  That's because we have grex configured to request
search engines like google from indexing it.

It would be trivial to change this.  Grex could be opened up to google in
seconds.  Any selective opening, like opening only certain conferences or
certain items would be much harder, but perhaps not impossible.

Grex already has a feature where you can make all your responses invisible
to anonymous readers (like Google).  ("shy y" adds you to this list, "shy n"
removes you).

Do we want to open Grex conferences to google?

I think it would be a dramatic change.  Make Grex less of a dim little
backwater on the Internet, vastly more visible.  That has it's pluses and
minuses, obviously.  But it's been a long time since we've thought about this.
Grex has changed.  The net has changed.  We've changed.  Where do we stand
on this today?
71 responses total.

#1 of 71 by cross on Thu May 26 15:28:16 2005:

Go for it.


#2 of 71 by mary on Thu May 26 15:51:56 2005:

I agree.


#3 of 71 by albaugh on Thu May 26 16:30:07 2005:

As long as we could change our minds later, it might be worth an experiment.


#4 of 71 by tod on Thu May 26 16:58:49 2005:

You want to remove ROBOT from blocking search engines?  I'm against that.


#5 of 71 by slynne on Thu May 26 17:31:55 2005:

I think this would be great although we might want to restart and 
archive conferences first. There are lots of posts that were made by 
people with the understanding that grex wasnt indexed by search 
engines. Many of those people are no longer regular users of grex so 
they might not see any warnings we post here. 



#6 of 71 by tod on Thu May 26 17:58:48 2005:

 There are lots of posts that were made by
 people with the understanding that grex wasnt indexed
I agree 100%


#7 of 71 by marcvh on Thu May 26 18:15:00 2005:

Restarting lots of existing conferences, many of which haven't ever been
restarted before, seems like a bad idea.

Why not just issue an announcement of the change in policy, and allow
agora to be indexed starting with the summer edition.  We can see if
that does any good and, if other confs want to also be indexed, they can
do restarts as well.


#8 of 71 by davel on Thu May 26 18:52:21 2005:

Wasn't the present policy (no indexing) the result of a member vote?
Certainly the issue was discussed at length in connection with a
very contentious vote.  I don't have time to dig back & try to identify
it right now.


#9 of 71 by tod on Thu May 26 19:02:59 2005:

Maybe popcorn's scribble script should be offered to the userbase for 1 week
prior to removing the member voted and instated robot file.


#10 of 71 by mcnally on Thu May 26 19:30:30 2005:

 I'm not in favor of the idea.


#11 of 71 by glenda on Thu May 26 19:42:50 2005:

I am also not in favor of the idea.


#12 of 71 by cross on Thu May 26 20:02:11 2005:

Which idea; #0 or #9?


#13 of 71 by mcnally on Thu May 26 21:13:42 2005:

 #0.


#14 of 71 by naftee on Thu May 26 22:06:35 2005:

Go 4 it.


#15 of 71 by other on Thu May 26 22:58:02 2005:

I think the idea of allowing search engines to index a Grex conference is a
good experiment to undertake, subject to some restrictions as suggested above.
Only Agora should be indexed, and only after the conference has been restarted,
and not including any old versions of Agora. There should be a reminder header
on every item indicating that this is happening and that it is a NEW feature,
highlighting the change for those who may be or have been concerned about it.
Linked items from other conferences, if at all possible, should not be indexed,
or if that is not possible, then linking items from other conferences into
Agora should be prohibited, at least for a trial period of say, three months
(the duration of a single iteration of Agora). Then, linked items should only
be indexed if they include the same header announcement I referred to above.
Users should be given fair notice of the extent of the obscurity of their
posts. No post on Grex is any less public than any other post anywhere else, in
theory, but the reality is that Grex itself is fairly obscure among all the
noise and fury of the internet, and indexing will raise awareness of Grex and
of the content of its conferences far beyond anything we have ever considered.
This can be a very good thing, and this can be a very bad thing, depending on
your perspective. It is eminently reasonable that those people who wish their
postings to be read only by the small community of Grex conference participants
should be given as clear an indication as possible that their expectations will
no longer have even the flimsy basis they once had. So, to make a short story
long, we should do this; slowly, gradually, gently and loudly.


#16 of 71 by naftee on Thu May 26 23:59:26 2005:

How can you be "gentle" yet "loud" ?  Pleas demonstr8


#17 of 71 by marcvh on Fri May 27 03:05:48 2005:

A reminder in every item?  Bleah!  It's silly enough that we have a reminder
that the conference has been restarted every time we join and we can join
winter for the old version when it's been months since the restart.


#18 of 71 by other on Fri May 27 03:45:21 2005:

This is a major change, and it is far more valuable for Grex to make sure that
people who might be used to things being a certain way are fully and properly
informed of the change than it is to make sure that you never see the same text
twice if you don't want to. I suppose an alternative would be to include a link
or an instruction in the header that would allow users to turn off the warning
by indicating understanding and acceptance of the new practice. Anyway, I'd
suggest the warning only be in place for three months, which is plenty of time
for existing users who may not conference regularly to be considered to have
been reasonably notified.


#19 of 71 by naftee on Fri May 27 13:04:01 2005:

It's not that major.  
I typed this into google:
http://www.google.ca/search?q=naftee

and the third result is this :

http://www.grex.org/grexdoc/archives/minutes/2004-09-10

which is a GreX item, found on google, without any permission asked.
The most interesting was the output of ric(-k)'s finger programme showing up
on google.


#20 of 71 by dpc on Fri May 27 15:47:12 2005:

I think that starting this new accessibility is an excellent idea, for 
Agora only, starting with the summer Agora.


#21 of 71 by janc on Fri May 27 16:20:30 2005:

(1) The robots are currently blocked only from the conferences, not
other Grex pages.

(2) Yes, the current policy was the result of a contentious vote.

(3) I'd suggest that people who don't want their responses indexed should
    use the "shy" command, not the scribble command.

I like the idea of indexing only selected conferences, with the policy
that the conference must be restarted before indexing is enabled.

I like the idea, but I don't know of an easy way to implement it.  The
files used to control indexing aren't that good at fine distinctions,
and the distinction between two different backtalk conferences can be
pretty fine.  I'd have to think about it.


#22 of 71 by janc on Fri May 27 17:46:36 2005:

Hmmm...I think the cleanest way to implement this is to get rid of the
robots.txt file and use

  <META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">

tags on all pages that should not be indexed.  Of course, this requires
changes to an awful lot of backtalk pages.

I could alternately create a copy of backtalk that generates normal
pages except that links to indexable pages are given as links to the
copy and links to non-indexable pages are given as links to the main
backtalk.  The robots.txt file would be set up to allow indexing of the
copy, but not the main backtalk program.  Might be easier, because I
need to change fewer pages, but a fundamentally kludgier approach.


#23 of 71 by jep on Fri May 27 19:13:35 2005:

How do you apply "shy y" if you use Backtalk?  Does it mark your posts 
as being unGoogleable retroactively?

Can we create a test conference which is Googleable, perhaps 
called "Google Agora" or something, and try opening that one first?


#24 of 71 by mcnally on Fri May 27 19:32:14 2005:

 Can I suggest someone whip up a quick man page for the undocumented
 "shy" command?


#25 of 71 by naftee on Fri May 27 20:47:34 2005:

re 21 The web page is an exact copy of a GreX item.  What's the difference?


#26 of 71 by glenda on Fri May 27 23:07:12 2005:

Yes, how does one use the shy command?  I tried it and got "Cannot open
/home/janc/src/backtalk/shylist."  Or does it only work in backtalk?  If that
is the case what do those of us that do not use backtalk and don't want
anonymous readers seeing our responses do?


#27 of 71 by i on Sat May 28 02:35:18 2005:

Do we need to add a level of shyness here, call it "very", that hides your
comments from both anonymous readers and robots, change shy "y" so it only
hides comments from robots, then make the "y" everyone's default (except 
change the current "y" crowd to "very") as the experiment begins?  

I think that many people says things in our conferences that they might 
not want to share (via Google) with someone like an employer... 


#28 of 71 by janc on Sat May 28 13:07:12 2005:

Sounds like shy is broken.  Permission issue probably.  I need to look
at that.

  "shy y" means that your responses won't be visible to anonymous
  readers, including bots.

  "shy n" returns you to normal.

Yes, it's retro-active.  If you are on the shy list, all your past and
future responses are hidden.

I'm not sure Walter's suggestion is practical.  It's hard to tell bots
from other anonymous readers.  I'd have to base it on the User Agent
string sent by the browser/bot and there are an awful lot of
browser/bots out there.


#29 of 71 by naftee on Sat May 28 13:26:14 2005:

I wish shy were pro-active :(


#30 of 71 by davel on Sat May 28 15:42:59 2005:

Re 28: permissions may be a problem, but the immediate problem seems to be
the specified path.  /home/janc doesn't seem to exist.


#31 of 71 by ric on Tue May 31 19:57:53 2005:

So, all the instances of polytwerp calling me a paedophile will be googlable?
great.  


#32 of 71 by tod on Tue May 31 21:31:08 2005:

Yea, "Rick Root" will show up with paedophile on google


#33 of 71 by naftee on Wed Jun 1 00:46:47 2005:

lolz


#34 of 71 by cmcgee on Thu Jun 2 11:51:15 2005:

I'm in favor of the idea. IIRC, minutes have always been searchable.  I'd 
like to see it start with an Agora restart, or with a conference that 
volunteers (by vote within the conference and an agreeable fw).  

I think it needs a member vote however, especially if we are experimenting 
with Agora.  


Ric's comments started some ideas in my mind.  It seems that we might have 
some users who are willing to be members of a conf that has a somewhat 
more active fw than is traditional, leaning toward a "moderator" level of 
involvement.  

Could we have an experimental conference lead the way?  One that was 
clearly intended to be a public Web conversation pit, and inteneded as a 
marketing tool for Grex?




#35 of 71 by ric on Thu Jun 2 12:19:21 2005:

btw, if you're going to publish conferences, you should place google ads on
the pages - at least for people who are not logged in.  Could be a nice source
of revenue if the item threads are indexed well by google.


#36 of 71 by slynne on Thu Jun 2 16:22:22 2005:

I agree that this is a big enough change that putting it to a member
vote is a good idea. 

I also *really* like the idea of putting google ads on the pages
although I dont know if anyone else would agree


#37 of 71 by keesan on Thu Jun 2 18:06:09 2005:

I would consider google ads or any ads in the same category as school vending
machines and I would be willing to pay higher dues to avoid it even though
it does not affect me personally since I don't access grex by web.


#38 of 71 by tod on Thu Jun 2 20:00:58 2005:

Google ads are dangerous and could link people to spyware sites


#39 of 71 by albaugh on Thu Jun 2 21:14:43 2005:

I suppose people who wished to hide their identity even beyond what is already
in place could just create pseudos for themselves, using newuser.  Oh wait...


#40 of 71 by tod on Fri Jun 3 04:01:42 2005:

Grex could always start requiring a National ID.


#41 of 71 by other on Fri Jun 3 19:23:10 2005:

Do Google ads pay only for click-throughs, or based on impressions (times
served)?


#42 of 71 by slynne on Sat Jun 4 21:09:41 2005:

I think they pay for click-throughs but I am not sure. I think polygon
uses them on his site.


#43 of 71 by ric on Sun Jun 5 02:38:12 2005:

Google pays per click.  I have google ads on my blog and some other pages,
I've made about $210 in the last month or so.

Polygon uses them too.


#44 of 71 by dpc on Wed Jun 22 16:05:18 2005:

I think ads are things for which the Grex world is not prepared.


#45 of 71 by ric on Thu Jun 23 12:43:36 2005:

Don't speak rashly.  

If it's possible to only place the ads on pages displayed to users who are
not logged in, then Grex can benefit from google indexing of items, without
affecting any current user of grex.

Google text ads are unobtrusive, and surprisingly effective - much more so
than banner ads, because the google text ads are usually relevant to the page
they're being displayed on.

Without displaying them to users who are logged in, they would only be
effective if grex items showed up in Google or other search engines.  If the
conferences were well indexed, I could see that revenue leading to anywhere
from $3 - $20/month, again, without impacking users who are already logged
in.  Who knows, maybe more.

Certain items could lead to higher ad rates.  Enter an item and have a
discussion about Misothelioma, and you'll probably get $5 or so for every ad
that's clicked on!

Heck, turn on the ads for grex users, and encourage grexers to click on the
ads.  Grex users know that clicking on the ads would support Grex.  I'd do
it.


#46 of 71 by tod on Thu Jun 23 12:48:21 2005:

Google ads would be effective on Grexers?
Are you thinking of selling Dr.Demento boxsets and Dr.Who mugs?


#47 of 71 by keesan on Thu Jun 23 14:31:10 2005:

I would much rather pay more per month instead of going commercial.


#48 of 71 by mary on Thu Jun 23 15:30:28 2005:

How much more?


#49 of 71 by naftee on Thu Jun 23 16:11:42 2005:

I'd pay if there were no ID rule.


#50 of 71 by nharmon on Thu Jun 23 17:37:12 2005:

You can naftee. You can pay without being a member.


#51 of 71 by ric on Thu Jun 23 17:39:56 2005:

re 46 - not necessarily, but when someone google searches something, and views
an item on grex, they may click on one of the content-appropriate text ads
in search of more information on the same topic.

If you're going to make conferences google-able, you might as well take
advantage of the extra hits (if any).  I mean, those people coming in via
search engines to specific items are going to be looking for information, not
Grex.  They may stick around or not if they're intrigued enough, but not
likely.  To most people it probably wouldn't be any different from any other
message board or mailing list archive out there.


#52 of 71 by tod on Thu Jun 23 18:27:04 2005:

If conferences are going to be on google then make sure everyone knows first.


#53 of 71 by nharmon on Thu Jun 23 18:50:26 2005:

Can we like, purge all conferences before putting them on Google?


#54 of 71 by marcvh on Thu Jun 23 18:56:33 2005:

Is this the chase-your-own-tail item?


#55 of 71 by naftee on Thu Jun 23 18:57:55 2005:

re 50 I can, but I won't pay until the ID rule is rescinded.


#56 of 71 by nharmon on Thu Jun 23 19:02:40 2005:

Re: #55. You do know this isn't Grex's rule. It's Michigan's rule...


#57 of 71 by tod on Thu Jun 23 20:37:36 2005:

re #56
Cite that reference.


#58 of 71 by naftee on Thu Jun 23 21:39:46 2005:

what tod said


#59 of 71 by nharmon on Fri Jun 24 13:13:08 2005:

MCL 450.2413: "The officer or agent having charge of the shareholder or
membership records of a corporation shall make and certify a complete list
of the shareholders or members entitled to vote at a shareholders' or members'
meeting or any adjournment thereof. The list shall:
(a) Be arranged alphabetically within each class with the address of each
member or shareholder and the number of shares held by each shareholder."

Grex has to verify your address in order to comply with this rule. This why
a personal check, showing your address, is sufficient.

http://www.legislature.mi.gov/mileg.asp?page=getObject&objName=mcl-450-2413


#60 of 71 by jep on Fri Jun 24 16:07:14 2005:

re resp:47, resp:48: Membership rates have been the same for all of 
Grex's existence.  If Grex needs the money, I'd be willing to see the 
membership rate double.  If Grex already receives all the money it 
needs, then I see no reason to use ads to generate revenue.

How about some discussion on how much money Grex needs, what it would 
buy, and what it is needed for?


#61 of 71 by polygon on Sun Jun 26 14:40:48 2005:

(1) I wouldn't be opposed to having the current Agora be open to Google.
I don't want to see other conferences mass restarted.  (I thought past
Agoras were not available via Backtalk anyway?  But I haven't looked.)

(2) I wouldn't be opposed to experimenting with Google text ads.  They
are not intrusive and pay surprisingly well.  However, Grex's financial
needs are modest (especially now that we're not paying Pumpkin rent),
and we do have 60+ paying members.  Arguably, it's not necessary to go
looking for ad revenue.


#62 of 71 by remmers on Sun Jun 26 18:09:31 2005:

Re #61:  Any conference that's still online at all is available via
Backtalk.  This includes all the Agoras except for a few of the early
ones when we had a lot less disk; they were taken offline to free up space.

I've been taking a sabbatical from Coop for the last month and just
encountered this item.

Some history:  Anonymous conference reading was authorized by member
vote in 1997.  The discussion is in the coop9 conference, item 55, (for
Backtalk readers, the link is item:coop9,55) and was *very* contentious.
 Quite an interesting read, eight years later.  The motion itself does
not mention indexing by search engines, so I suppose one could say that
there's no official policy one way or the other on that.  However, the
Backtalk developers (Jan Wolter, and also Steve Weiss at the time) made
it clear that if the proposal passed, the implementation would exclude
search engines, so I think members had that in mind when they voted.

I hope that the benefit of eight years' experience has helped put the
issue of anonymous reading in better perspective than it was in 1997.  
     I voted for the proposal because I saw it as a way of increasing
Grex's visibility on the web, and thus helping to attract new users, by
allowing people to sample Grex without committing to an account.  I
don't think we have statistics on how much actual benefit anonymous
reading has been in that regard, but I think it's fair to say that none
of the dire consequences envisioned eight years ago have come to pass. 
Allowing anonymous reading but requiring registration to post is common
practice in web forums, so Grex certainly isn't doing anything unusual here.

The web is a big and noisy place - much more so than in 1997.  In
general I'm in favor of things that increase Grex's visibility and help
drive traffic - and hence users - to us.  One of the best ways to do
that is to open up to search engine indexing, so I'm for it.  Anybody
who doesn't want their own responses indexed can opt out via the "shy"
option.

One caveat:  Sites that allow posting for free, and whose content is
indexed by search engines and permits posting of links, tend to attract
spammers.  The fact that posting requires registration protects us from
spam to an extent, but since registration can be anonymous it doesn't
protect us perfectly.  There's a protection that the major search
engines actually support (rel="nofollow" on the anchor tag, for HTML
techophiles) that tends to discourage spamming for reasons I won't get
into here.  Nonetheless, conference spam might be something we'd have to
figure out how to deal with if we open up conferences to search engines.

Re advertising:  Does the fact that we're a 501(c)3 non-profit restrict
what we can do?  (Just asking; I don't know the answer.)  In any case,
I'd be opposed to subjecting registered users to ads, even the
unobtrusive Google ones.  As for anonymous readers - dunno.  I'll have
to ponder Rick's idea.


#63 of 71 by aruba on Sun Jun 26 23:54:08 2005:

I believe we need to get a certain percentage of our income from donations
in order to maintain our 501(c)3 status.  I want to say it's 33%, but I
could be way off.


#64 of 71 by jep on Mon Jun 27 15:02:22 2005:

I do not like the idea of chasing ad revenues just because we can.

I don't see a need for more money for Grex.  I don't see what we would 
use more money for if we had it.  Aside from future upgrades and paying 
the monthly Internet bills, we don't have expenses, do we?

What do we *want*?  Give me that first, then let's discuss raising more 
money.  Is there anything Grex wants money for?  Paid staff?  
Advertising?  A bigger faster computer or better Internet connection?  
What?

If we needed more money, I'd be more willing to increase membership 
amounts than advertise.  But I think we're fine right now.


#65 of 71 by nharmon on Mon Jun 27 15:03:54 2005:

Is Grex in trouble financially?


#66 of 71 by jep on Mon Jun 27 15:06:32 2005:

We have $100 per month in recurring expenses, and $3000 in the bank.  I 
don't think we're in trouble financially.


#67 of 71 by naftee on Mon Jun 27 18:36:23 2005:

your estimate is way off, jeep


#68 of 71 by jep on Mon Jun 27 20:28:51 2005:

I don't think it is.


#69 of 71 by mary on Mon Jun 27 20:38:11 2005:

Our fixed expenses are around $150.00 a month right now.


#70 of 71 by aruba on Mon Jun 27 22:11:12 2005:

Right - John, our two monthly expenses are $100 for colocation and about $45
for phone lines.  Plus a few other miscellaneous things that occur only once
a year.

I think Grex is in much better shape financially than it was before we
moved.  I'm not too worried about it right now.  I'd always like to see
more members, but at our current level I think we'll be operating in the
black. (Note - I haven't figured this out rigorously, and don't have time
to do so right now.)

So I guess I agree with jep.  Let's talk about how people would like to
improve Grex, and if we get a sense that it will take money to do so, then
talk about raising it.


#71 of 71 by jesuit on Wed May 17 02:15:31 2006:

TROGG IS DAVID BLAINE


There are no more items selected.

You have several choices: