Staff Meeting - February 11, 2007
Present:
Jan Wolter
Steve Weiss
Steve Andre
Joe Gelinas
Things to do, in no particular order:
Zoneinfo update.
Daylight savings time starts three weeks earlier. If we leave in old
zoneinfo files, Grex's clock will be an hour off for three weeks.
For new files, upload the latest version of zoneinfo from the latest
OpenBSD-current release and install. Need to run zic to compile
zoneinfo files, which may not be entirely simple.
CVS Server.
Grexdoc is currently maintained on a CVS server hosted by John
Remmers. John wants to shut down this machine soon. Steve Andre
will set up a new grexdoc server in his office.
CVS Checkin.
Jan should check in latest changes to grexdoc.
Spam Filtering.
We'd be interested in trying to set up global spam filtering on
Grex, using spamd. Plan would probably be to discard definate
spam. There is some doubt about whether Grex's machine would be
able to handle the load of running Spam Assasin on every bit of
incoming mail, but it seems worth a try.
Optional Incoming Mail.
Probably a more effective way to reduce the resource drain of
incoming mail would be to make it option. New grex accounts
would not be emailable by default. There would be a command
which users could use to turn incoming email for their account
on or off.
The theory is that many grex users don't actually want to receive
mail on their grex accounts. If they all turned off mail, then
Grex would have much less mail to process.
Outbound Mail Approval Script.
Outbound mail is currently turned off for new account. There are
volunteers who would be willing to handle the process of approving
people who want outbound mail turned on. We need a tool that would
enable the mail-approvers to turn on mail for a user, basically by
adding their names to /usr/local/etc/outbound.
This needs to be implemented so that when an account is reaped and
a new account is created by the same name, then new account does
not inherit email approval from the old one.
Jan may do this, but won't mind if someone else does.
Mail to Staff.
Currently users who are not approve for offsite email can't send
mail to staff because staff is off-site. Need to fix this.
Outbound Mail Limit.
There was a plan approved by board to limit outbound mail to 50
messages per user per day. This could be done by exim script,
but possibly the ouutbound mail approval thing is easier to
implement and might suffice to solve the problem.
Outbound net access script.
Currently new accounts are created in group 1003, which does not
have outbound internet access. We want to be able to have a list
of non-root users move people into group 1002 so they can have
outbound internet access. Need a su-root script to do this.
Probably a task for Jan.
Fix rmuser.
The rmuser script dumps core, so we can't do reaps right now.
Reaping would be good.
Newuser changes
Dan Cross wants to make some fixes to newuser. Jan will
install these when they are available.
RAID and more Disk
There is some interesting in buying a hardware RAID controller
and a number of large fast disks to use with it. This plan
would vastly increase disk space and reliability. Cost might
be around $1600, but STeve will make better estimate.
OpenBSD 4.1 upgrade
OpenBSD 4.1 will be released later this year. We should
upgrade Grex to that.
Might be a good time to also do things like adding RAID disks.
118 responses total.
All the staff people at the meeting recognized that the biggest staff problem right now is a lack of active staff. Too many of the current staff have too much else on their plate, and many of us are not really very active. We really all want to do more, but we aren't promising much. Still, we thought it would be good to have a list of things that we think ought to be done, so that the consensus building part of any staff task is at least partly out of the way, and all we need to do is find someone to do them. The day after the meeting STeve noticed problems with one of the disks. We're going to need to replace that before it entirely fails, probably in a shorter time frame than we could possibly implement RAID in. Luckily we do have a spare disk.
Could grex reap mail accounts which have not been accessed for 3 months even if the user is still active? Not including accounts that are forwarding any mail that comes to them.
Regarding #1; Once again, I can do staff work if needed. Regarding #0; A couple of points. (a) OpenBSD upgrades are only supported through point releases. Right now, we're running OpenBSD 3.8 on grex. 4.0 is current right now. To do a supported upgrade, we really should upgrade to 3.9, then to 4.0, and then to 4.1.... It's a pain, but would be the best way to approach it. (b) With respect to CVS servers for grexdoc, I'd *really* rather see grexdoc hosted on grex and replicated to other machines offsite by CVSup or anoncvs or something similar. (c) What does rmuser do that userdel doesn't? Userdel is an OpenBSD tool; could we shoehorn it into the reap process? (d) Mail to staff could be handled by putting an intelligent agent midway in the process. Something like RT could be leveraged to good effect on grex and solve this problem at the same time. http://www.bestpractical.com/.
I was also at the staff meeting, as a staff member.
What's being done to recruit new staff?
Todd read my mind. Jan got about halfway there, then stopped.
Just curious where the 1600$ estimate came from. Are we planning to keep going with SCSI drives? I know a first order aproximation based on one vendor's webpage was barely over half that price, and drive prices are slowly coming down. Or are we looking at a big-ass 8 drive array so we get mondo space and redundancy?
I've created an item in the garage conference for further discussion/research into a RAID storage solution for Grex.
re #6 I read "...lack of active staff..." "...not promising much..." "...all we need to do is find someone to do them..." So I'm kinda wondering if the Board is going to help or is staff being relied upon? Jan & Dan, you're both on the board. What say you about the recruiting efforts for new staff members?
I think I would classify myself as "inactive staff". I do not see my schedule opening up for large amounts of staff work. I'm still hope to be available for helping out with system upgrades and some light software development and bug fixes, but even that is going to be inconsistant. I recognize that we need regular staff. This includes especially people who are local to Ann Arbor and can do things like reboots and hands-on system work. I haven't the slightest idea how to recruit them. Meanwhile, I've done some checkin of grexdoc, and have implemented a command that would let a selected list of users enable outgoing mail for other users. I'm told some people volunteered for this, but I don't know who they were.
I think rmuser is really called zapuser. As usual, I've forgotten everything
about it, even though I wrote it.
I seem to recall that one of the points of it was that it was optimized for
removing large numbers of users in one fell swoop. Some of the alternative
tools I'd seen would rebuild the hashed password file after each user
deletion. When you are deleting 10,000 users at once, this is a very bad
idea. I don't know of userdel is smart about this or not. The manual page
seems to suggest that it takes only one user name as an argument, which
suggests that it probably would require 10,000 passes through the password
file, and 10,000 hash rebuilts (which take many minutes these days if my
recent experience with vipw can be trusted) to remove 10,000 users.
Looking through the source code, I see it also has a lot of other checks:
- won't delete users with uid's under 1000 (or whatever is configured)
- won't delete members or staff
- won't delete accounts unless home directory is in one of the usual
spots for grex home directories.
- won't delete accounts that are on the immortals list
The user directory deletion is done in a paranoid mode, su-ing to the user
before beginning the deletion. This a final failsafe to avoid traps where
a user subdirectory is swapped with a symlink to /etc between the time
zapuser checks if it is a symlink or a directory and the time that zapuser
cd's into it. Of course, we have other safeguards against this to, like
checking if the inode number of the directory we cd'ed into matches the inode
number of the directory we through we were going to cd into, but it never
hurts to be extra safe, and by running as the user we know we can't be fooled
into deleting anything we shouldn't.
The directory deletion code is also designed to be able to delete arbitrarily
deeply nested directories, something older versions of "rm -R" generally
failed at.
In addition to deleting the home directory, it deletes mailboxes
screen and layer files. I should probably add something to it to
delete people from the exim.outgoing mail file.
My apologies to Glenda for omitting her from the list of attending staff members. Relying on my memory is always a perilous thing.
I've repaired the crash in zapuser. I have not yet taught it to scrub people
from the exim.outbound file.
Grex hasn't done a reap in a long time. I don't know that I know the correct
rules for reaping. I think it is:
Guest accounts which either:
- were created more than 3 weeks ago, and have never been logged
into, or
- have not been logged into for more than 90 days
If that's true, then about 48,000 of the 53,000 accounts on Grex are overdue
for reaping.
Reaping them would be a very good thing. Probably all those accounts are
receiving spam. Getting rid of them would greatly reduce the incoming mail
load, and make running a global spam filter much more likely to be possible.
Many of these accounts probably also have full mailboxes. Getting rid of them
might even allow us to increase the mailbox size limit for users who actually
log on sometimes.
The tools set for doing reaps should now be in place. However I have not
yet run one, because I don't know if the criteria above is actually still the
current criteria.
That seems like a high number of inactive accounts (and a low number of active ones). Are you sure the method you are using to check for logins is reliable, Jan? I gather that whatever finger uses has not been reliable since we moved to OpenBSD.
I think finger's output is reliable; Mark, what gives you the impression that it is not? The procedure for running a reap was to run the reap collection program (which would generate the list of accounts to be reaped), then look at it to make sure there were no `errors' (ie, staffers being deleted by accident), and then run the actual reap.
Thank you, Jan. I will undertake to perform a reap over the weekend. It'll probably take me a little while to uncover the directions; at least I know where to look. :) One other check is for fairwitnesses that have been reaped; I vaguely remember there being a separate tool for that purpose. It's also documented, in the same place as the rest of the reap.
Thank you all.
Followup to #11: I confirmed that if we used the standard OpenBSD userdel to reap users, we'd have to run the program once for each user. It can't do multiple users in one pass. That means the hash files would have to be rebuilt once for each user. Currently, rebuilding the hash files takes almost exactly 10 minutes. Multiple that times 48,000 users, and you have a pretty good idea why we need zapuser. (OK, you can probably divide that by two, since the rebuild times will get faster as the password file gets shorter, so run time will only be about half a year.) Zapuser now deletes from the outbound mail file. Yes, I am concerned that 48,000 of 53,000 users have been selected for deletion. It seems high. But every spot check I've done seems OK. I don't see users with files modified after their supposed last login dates. I've confirmed that http logins are correctly updating the last log file. If anyone has evidence that the last login dates shown in by 'finger' and 'laston' are incorrect, I'd be interested in knowing. I think we just don't have all that many active accounts. Joe is planning to do a reap this weekend.
If someone could do a backup before the reap, I'd have a warmer, fuzzier feeling about deleting 48,000 users and their mail and their files. If we've got drives that may be developing problems it'd probably be an excellent thing to have handy in any event. I wouldn't think it would be a problem to bring over a firewire or USB 2.0 enclosure with a cheap IDE disk in it and do a dump of critical filesystems to it. Even a dump of live filesystems would be better than no dump at all.
Good point, Mike. This will probably delay the reap, but better a late reap than a trashed system.
Regarding #18; Ten minutes to rebuild the password hash on this machine seems like a really, really long time to me. Hey, if userdel won't work, then it won't work; worse things have happened. I'm a bit surprised that it doesn't use the `-u' option to pwd_mkdb, which just updates a single record, instead of the entire hash file (notice that changing passwords and adding users doesn't take that long). But anyway it's a moot point. I suspect that Jan is right: we just don't have that many active users.
Re finger: I used to use finger to decide if members who weren't responding to my hails had disappeared from Grex. I'm afraid I don't remember details, but I quit using finger because it gave me results that seemed wrong. I remember asking about it (in agora?) a while back, and being told (I think by you, Dan) that there was no good way to tell when someone had last logged on. It's possible I'm confusing the last-logged-in time with the last-checked-mail time.
Hmm, I don't remember that conversation. I'd say it might be the last-checked-mail time that we were talking about, if anything; logging in interactively or via backtalk or (maybe) via FTP will certainly will update the lastlog file, which is what finger looks at to tell you when the last time a user logged in was. But, finger's details about a person's email reading habits aren't particularly useful, since people can forward email off of grex, and lots of programs will change the timestamps on the mail spool file without the user actually *reading* the mail.... (For instance, the `from' command will modify the atime of the mail spool file.)
You know, you may be right. I didn't actually try userdel. 10 minutes is how long hte password rebuild takes after a zapuser, which does do a full rebuild (even if only one user has been deleted). So I dunno. Maybe userdel would be fast after all. Some kind of backup would be a good idea. Another possibility would be to run zapuser without the -d flag. In that case, it doesn't delete the user's home directory, but stashes it in /a/deleted. It always saves their passwd file line. However, zapuser always deletes the mail file, so this isn't a great option if you serious expect to want to restore the user. We should backup.
I know in the past I've tried to finger those in conferences that I'm a fw of, to see who and how long its been since those users have checked into the conf. But it was taking forever and a day, so I finally got out of it. Though I'm not sure how relevent that is to this current conversation. :-)
Re #23: The conversation I was referring to was resp:agora56,4,87-106 Unfortunately all of Dan's responses have been scribbled. I can't quite reconstruct what was said, but it's clear I was confused by the answer, and I'm still confused. Bruce Howard said this in response 97: It appears if you log in with a non-interactive shell, for example: "ssh grex.org bash -i" no login record is made.
STeve is planning on doing backups and, if possible, replacing the flacky disk tomorrow.
Regarding #26; Yah, that was sort of for work. Long story (and one I can't really explain anyway). Perhaps the discussion was for non-interactive shells. E.g., when one does, ``ssh cyberspace.org ls'' and things like that (incidentally, that's what is happening when one does ssh cyberspace.org bash -i...).
Well, STeve did a backup, and Joe did a reap (probably the first since we moved to OpenBSD). I guess we'll soon find out if this was a problem. I'd be surprised if there weren't at least a few people among the 48,000 or so that we deleted that maybe shouldn't have been. I did spend some time before the reap adding people to the immortals list whom I thought ought to be there.
That should fix /var/mail for quite a while. Thanks.
Yes, it was the first reap since December, 2004. Since /var/mail is down to 38%, it probably will be fine. For a while.
Did you reap a lot of mail accounts without any mail or spam in them?
Why do some accounts have 20MB of mail in them? For instance munkey, whose mail account is dated Sept 18. I thought we had a 1MB limit. Is something broken? Munkey last logged on Feb 15 but may have abandoned the mail account to the spammers. Is there some way to tell if an account is being used to forward mail, and if not, reap it after 3 months of disuse?
Re your first question: No idea; we don't track that statistic. Re your second question: the quota was raised some time back. As to your third question: What (other) people do, or don't do, with their mailboxes is not really any of my, or your, business.
I agree with Joe.
Re question three, since new users are not being given mail accounts without requesting them, and since many or most of the old users with mail accounts probably are not using them for anything, would it make sense to reduce the number of unused mail accounts of people who are not doing ANYTHING with their mailboxes and did not want them in the first place. Note the '3 months of disuse'. I was not suggesting keeping people from forwarding mail.
I think the reap process works rather well; we just cleared out over 40,000 accounts. Unfortunately, we're not doing opt-in email yet.
What is the new mailbox limit? I have been forwarding anything over 100K to another account. What are 5000 people still using grex for if not mail?
Assuming the reap was the standard "not active in the past 90 days" and that newuser was off for nearly 9 months, I'm amazed and delighted that we still provide service for 5000 people!! Now, for the sobering thought, how do we know how many of those are spammers?
Regarding #38; I have no idea. Maybe you could ask them. I doubt very highly that it's email: most people aren't interested in the 1980's style of email experience that grex offers.
I only keep email on here because I am loathe to lose an internet address I've had for almost a decade (IIRC). I use GREX as a sandbox for shell scripting and for backtalk communities.
I dont really use email here but I do forward it to another account. Every now and then someone I havent talked to in a while sends me email to my grex account and it is nice to be able to still get it. Of course, nowadays I get so much spam at that account I probably would miss any personal emails sent to me.
Actually, there are still a lot of people that are trying to use Grex for email. I've talked with many who've expressed a distrust in the mail systems they have access to and like the idea of the little independant entity. We're getting rid of them however with the amounts of spam they get. People use Grex for all sorts of reasons. There are *still* classes in C programming where instructors point students to us for their homework. There are still a lot of people who hear of Grex as a place to learn about unix, and sometimes about OpenBSD itself. Lots of people seem to like using lynx here because they have some privacy. I helped one person several years ago use it for looking up about a disease a family member had, that they were reticent to look up at work. Lots of people check out party. How they get to Grex and then wind up there quickly I'm not sure, but it happens. This doesn't touch on the conferences.
I think things like, ``a lot'' and ``lots'' are relative. Steve, could you quantify them? If 500 people use grex for email, that may seem like ``a lot'' but compared to the overall population of the Internet, or even of grex users, or even to the places where many of us work, it's really pretty small. Handling email for them, with decent spam filtering, really shouldn't be a problem on grex's over-powerful server. If 100 distinct people check out party over the course of six months, I'll be pleasantly surprised. If a university or high school (outside of the Ann Arbor area) somewhere in the United States or Western Europe points to grex for a C programming class, or *any* programming or Unix class for that matter, I'll be pleased but similarly surprised. To the extent that we're getting these people, I suspect they are coming from abroad, and that as the technology wave continues to roll over even 3rd world countries, their numbers will decline. I suspect a lot of the people who use grex are those who are curious about public access systems, or want a more private place to do web browsing and the like, as Steve said. Some want to move files around, and the like, also. And there's nothing wrong with that; those are the kind of uses we *should* be encouraging. I feel affirmed that someone makes use of grex to do something useful, like find information about diseases that affect his or her life. I think it's great if people use grex for homework or scientific study (I recall a few years ago, some guy sent email to staff asking if he could run some fairly computationally FORTRAN programs on grex; I thought that was really cool. It's certainly the type of use we should be encouraging). We should encourage more of that. Steve, do you have any pointers on who these people are? We should try and figure out how they found grex and how we can get more like them.
If the mail accounts which have not been accessed for three months and do not have forwards on them were removed, how many people would still be using grex mail? Someone should be able to figure this out from last access times at least for non-forwarding accounts. Then check for .forward files. With few enough active users spam filtering would be more practical and then grex email would be more attractive for people to use. I used grex for most of my email until it had too much downtime and I still use it for anything noncritical. I do most of my browsing here with lynx because it is usually faster than directly via my modem connection.
Well, since we did a reap like, three days ago, and only around 6200 accounts, not that says that the number is less than 6200 users. I'm not sure we should be making grex *email* more attractive to use. If anything, we probably should be discouraging people from using grex for email as anything but a last resort. Look: the reality of it is that the email problem has just gotten too hard for what grex can handle with its *staff* resources. And that's not going to change. We can do a reasonable job for a couple of hundred users, but anything beyond that just isn't going to work out without a full-time staff looking after mail. Such is the way of the world; please learn that we're just going to have to deal with that reality. Unless someone just doesn't have another choice, they really should look elsewhere for email (e.g., hotmail, yahoo, gmail, etc). If someone wants to use grex because they don't trust the others, then fine, but they're going to have to deal with all the spam and unreliability. Again, the thing that's lacking here is *staff*, not machine resources. Technically, it can be done, but not without a lot of hand holding. It sucks, but that's what happens when a network takes an infrastructure designed in the 1970's for a closed environment of trusted individuals and thrusts it into the 21st century on an untrustable network.
I only get about 3 spams a day at worst and pine at grex is infinitely faster and more convenient than any webmail. How much handholding would it take to just filter all incoming mail with spamd?
Re #45: If an account is active, it is allowed to have an email account whether it is used or not. Other than not using the account for spamming, it is not our business what email is going into or out of the account. As long as the user account is active, the mailbox can remain full and new mail getting rejected. It doesn't matter if there is a .forward or not. If the user account is active, staff does not make it a habit to go looking in the account to see if there is a .forward or not, it is none of our business. If staff starts doing this or is required to this I will no longer be staff. It is an invasion of privacy. If the user account is not active, then the account and its attached email account will be reaped during the next reap session. I would not want staff poking around in my account just because it thought that I might or might not be making use of the email account that was created along with my user account, nor would I as staff be doing that to other user's accounts. You have been told many times that spam is a problem. It will not go away. There is not a hell of a lot we can do about it. Even if we put system wide spam filters in place, people will have to be able to opt out of them. I for one do not want anyone else filtering my email. My email is my private business as long as I am not trying to spam with it, I will deal with it myself, no one else can make the decision for me on what is spam and what isn't. As for var/mail filling, we are working on ways to deal with it that don't involve invading users' privacy. We just did a huge reap that should take care of the problem for now. We are looking into getting more space for var/mail. It takes time. Please give us the time without doing any more harping about the situation. We are aware of it. We all deal with it in our professional and personal lives. You only have to deal with it for you, we have others besides ourselves that we have to deal with it for. We just can't keep up with the deviousness of the jerks that send the stuff. At least not right now, this very minute. The whole world is having the same problem. A lot of money is being lost in both business and government because of it. Continuous berating staff about it will not make it go away any faster.
Is there some way current users can opt out of having mail accounts? You don't have to look in an account to see if mail is being forwarded, you look to see if there is a .forward file, as far as I know. Could the 1MB limit on mail accounts be restored, with larger accounts on request for those who actually use the accounts?
Regarding #47; How much spam you get each day is irrelevant. The fact that you like pine is irrelevant. It's great for you and all, but 99.999% of the rest of the world would really rather do it the way that, well, 99.999% of the world does it. And I've done timing tests on my home computer showing that, in fact, gmail is faster than you dialing into grex and starting pine. If you want to do your think, go right ahead; no one is stopping you or telling you not to. But please, stop acting like it's `better' some how. You know what I don't understand about you, Sindi? You really, truly, just seem *incapable* of understanding that, while *you* don't like GUI's, webmail, fast comuters, or any of the rest of it, other people do. Your capacity for just not acknowledging that is fascinating and all. The reality of the matter is this: grex email is not going to change very much in the near term. Space isn't the issue, quotas aren't the issue, none of the things you seem to have infinite time to post about are the issue. The issue staff time to babysit email. So you might as well get used to the idea that that's just not going to change any time soon: we don't have the staff for it, and even if we did, honestly? No one wants to work on email. It's a `done' thing and it's tedious and annoying to set up correctly. If you really want to use pine for email, then I highly recommend you start using SDF for it, because it's just not going to change on grex, no matter how much you keep posting about it nor how much you complain about it in the BBS. Now let me say this: I'm sorry that this is the case. I really am. But the resources just aren't there to fix it right now, and quite frankly, you're just going to have to accept that because it's reality. You accept gravity; this is the same. I've offered to do some of the work, but I need administrative access to do that, and I don't know what's up with that. As a board member, I won't strong arm the staff into giving it to *me*, and I really don't know if they want me doing that kind of work or not: it's up to them. But as of right now? No one has the time and inclination necessary to make email on grex work. If you really insist on using grex for email, then I suggest you be happy that it works right now. Long term, the solution is to make email opt-in for new users, enforce quotas, etc. I suspect that few enough users will want to use grex for email that we can then run spamassassin et al for everyone by default.
Here's a suggestion for a very simple opt-in scheme for incoming
mail for new users.
1) modify the account creation program so that it creates a
.forward file that invokes vacation and a a .vacation.msg
indicating that mail to the account is not being delivered
until the recipient opts in to mail delivery. Initialize
the vacation db files before exiting. Frankly the vacation
part can be left out in favor of a .forward that delivers
to /dev/null if desired, but I kind of prefer a solution
that doesn't swallow mail silently, at least at first.
2) create a "mail-opt-in" program that compares the default
.vacation.msg file and .forward file to the ones in the
user's home directory and removes the user's files if both
match the default.
3) create a "mail-opt-out" program (if anyone thinks there
would be demand for it) that backs up any existing
.forward and .vacation.msg files for the user, then
creates new ones with the default opt-out configuration.
I haven't looked at newuser for a while but I think that'd be
very easy to implement and it would do a reasonable job of
doing what's desired.
As for Sindi, if she wants a way for people to opt out of mail
delivery on Grex, she can advise them to execute the following
commands in the shell:
[ -f ~/.forward ] && mv ~/.forward ~/forward.$$; echo "/dev/null" > ~/.forward
I'm sorry, but I am not going to spend my time looking into users' accounts to see if they have a .forward or not. Not my business, or yours, what files a user has or doesn't have as long as they aren't doing something to harm the system. I will not be going into user accounts to check to see the last time they read their email to tell whether they have accessed within a certain amount of time or not. Again, I see that as an invasion of privacy tantamount to looking into a persons physical mailbox to see if they are taking their mail into their house or not, and how much of what appears in that box is junk mail vs bills or personal letters. Not my business, staff or otherwise as long as the owning user account is active.
It occurs to me that if you really want to limit email accounts to non-spamming real people, make it opt in AND make then go through bbs to get instructions for setting up the account. That way sindi could keep her recycling crew on grex by giving them computers AND instructions on how to access bbs. Another layer of "discouragement" could be put in place by requiring a person to actually post something in bbs before staff gives them some random bit of info needed to set up an email account. Hey, for people who want to see more participation in bbs, this will at least cause people to browse it at least once or twice.
Has anybody thought about doing a user study on what features the users find most important? Like, if given the options of e-mail, conferencing, text-based web browsing, programming libraries, etc. Which would they choose as the most important? Least important? Maybe it is none of these and the users have their own suggestions for features that they would like to see us offer. We could create a survey program and place a note in the motd asking for the users' help.
I like the idea of a survey program but have no idea how to set such a thing up.
Plop it into newuser?
Interestingly enough, we're discussing the same thing over on M-Net.. user surveys. I'd actually consider a free account from SurveyMonkey.com to do the actual surveys..
Re 44: we've been having visitors from Quebec in party the last couple weeks. Their computer class teacher sent them here to learn some Unix.
I do use SDF for important mail. I do use spamd at grex. I don't have broadband. I am happy for cross that his setup is fast enough that gmail works for him faster than grex and pine do for me. I have never seen any webmail (even fastmail.fm) that is anywhere near as fast as mail at a shell account using pine, mutt, or mailx. You can tell when someone's mail account was last accessed by typing 'ls -l /var/mail/keesan' for example. Presumably someone could write a script to identify accounts that have not been accessed for three months. I don't know if .forward files are publicly readable. How hard would it be to put back 1MB quotas, at least for non-members?
Regarding #59; No one is going to do it. Just accept that.
re 59 - every time new incoming mail arrives, the mailbox you referred to is "accessed" gmail rocks. So does gmail for domains.
re #52: was that meant to be in response to #51? Because if so, I don't think you understood what I was proposing.
Regarding #62; No, I think #52 was in reponse to the previous posting by Sindi.
And why should we be going into accounts to see when email was last accessed by that account? Again, IT IS NOT OUR BUSINESS to watch whether or not a user accesses his/her email account. Since we have the reap utility back in place now, inactive accounts and their attached email accounts should be getting taken care of on a regular basis. Yes, we have a problem when the reaping wasn't being done, but that has been fixed now. We just reaped a huge number of accounts. Let's wait and see if var/mail filling up becomes a problem again. You may not have an ethical problem with looking into everyone's account to see when they last accessed email, but I do. I wouldn't look into your USPS mailbox either. In fact I think that the government frowns upon that type of behavior. Why should email be treated any differently?
You DO NOT HAVE TO go into an account, just look at the date on it. Spamassassin would work a lot faster on 1000 than 6000 accounts. If you put a 1MB quota on accounts and let people ask for larger ones, would that be a way to bounce spam from those accounts before using spamassassin on mail to other accounts?
Regarding #65; What Glenda is asking is what data that would give you that would be remotely interesting.
You could figure out the best way to filter spam on accuonts which ARE being accessed.
You could do that by looking at logs. Please, Sindi, don't try to solve technical problems that you have absolutely no knowledge of. I know you're trying to help, but relentlessly pushing a solution to a problem that really isn't a solution is just irritating.
So what solution do you propose for reducing the number of unused mail accounts to the point where spamassassin is practical for the rest?
I've already made my proposals. Go look them up.
You seem to be proposing that people stop using grex for mail.
re 69 They aren't looking for an option of that, right now, from what I'm reading. It also seems like this won't be happening anytime in the foreseeable future. You are in an extremly small minority, probably less than 20 people on the system that are trying to use it as a primary email location. This just doesn't justify the staff's time, which is very limited to begin with. If you really like using pine for email, how about running it on your computer, tied to Yahoo via SMTP and POP3? (This does require paying Yahoo a small annual fee.) Yahoo has pretty good spam filtering, and I get very little into any of my accounts with them. I just thought of another possible solution, get a copy of UUPC, and assuming that uucp is still available, bring the mail across onto your computer. Then you can process it however you see fit.
Regarding #71; Wow. That just now sank in? Yes. I am proposing that people stop using grex for email unless they really have no alternative.
We don't propose anything for reducing the number of unused mail accounts. Currently mail accounts come with user accounts. If a user doesn't want to use that account that is ok, it is their business not ours. We just got rid of some 48,000 accounts. And looking at the dates in a person's account to see when they last accessed the mail is STILL an invasion of privacy. Why can't you see that. With 48,000 accounts gone the problem should be mostly gone. We will be doing reaps of inactive accounts more often. I will do them myself if need be. Lets see if this eases the problem. Give it a change for crying out loud. You have been harping on this issue for so long, I think that it has become a habit. Drop for a while and give things a chance before asking staff to do something drastic and invasive.
Re 39: Why do you think newuser was off for 9 months? Or, which nine months do you think it was off for? Last I checked, it was disabled in December and renabled in January.
Regarding #75; It occurs to me that one reason this last reap was so big is because newuser was off for so long. It seems clear that most accounts on grex are unused. Probably, the 90 day window is large enough to account for some percentage of accounts that are created, but then unused after an initial login or two. The long period without a reap probably saw most accounts go dormant, without the influx of new accounts to flesh out the remaining numbers.
Hmm, I think I am recalling the MOTD from last year, which seemed to indicate that newuser was not available. I tried several times (meaning 3 or 4) to start an account under a different login, and was not successful. I don't have data, just my impression.
As I recall, it was closed in January of *last* year and opened again in January of *this* year. 9 months, if my memory serves, is actually three months shorter than the actual closing.
That doesn't really change the numbers, though.
How is ls -l /var/mail an invasion of privacy? I am happy using grex mail with spamassassin already. I am just trying to make it easier for other people to use a spam filter here. Rane objected to having to edit the sample .procmailrc I sent him (to change my login to his). Denise has no idea how to edit anything at grex. Could someone on staff just find the time to write some script to automate the creation of a .procmailrc that uses spamassassin and either dumps anything with five points (or three points, I never had a false positive that way) or sends it to a spam filter, your choice? Or even post my email in motd offering to help people set up procmail to filter spam. Staff was talking about a system-wide use of spamassassin but it would take too much resources if used on everyone's mail, which is why I was looking for some way to close accounts that their owners never used first.
This response has been erased.
keesan, YOU CAN'T TELL WHEN SOMEONE HAS ACCESSED THEIR MAIL BY LOOKING AT THE DATE ON THE FILE! Every time a new message arrives, that file is updated, and gets a new date... I'm pretty sure of that. I already said that once, but you seemed to ignore me. You can't tell the last access time by doing "ls -l /var/mail" -- On a side note.. doesn't grex still follow the 30 day reap policy?
But you can tell that no new messages have arrived in the last 90 days. I guess it would not be relevant to spamassassin to delete accounts that have not received any spam in 90 days and we don't need the disk space again yet. Would it be possible to send everyone an email explaining how to set up spamassassin for their account?
SPAM everyone on grex? Sindi, if people want help setting up spamassin, it is freely available. Flooding the system with a solution in search of a problem makes no sense whatsovever. Do you not understand that staff WILL NOT work on the spam problem? There are higher priorities for staff time. Don't "help" people who a) don't feel they have a problem, and b)haven't asked for help solving their nonexistant problem.
A message in motd linked to an explanation of how to set up spamassassin and offering my help? Everyone I know who is using grex mail thinks spam is a problem but thinks there is no solution available. Or is waiting for staff to fix the problem.
If no new mail has arrived in the last 90 days for a specific user, then why do we REALLY care? -- You could also run pine on your own machine and use Google Mail's pop3 and smtp capabilities with pine... free of charge. The nyou'd also have web access to your mail when you're some place with a modern internet connection.
I suggest you contact everyone you know who thinks spam is a problem and offer your help. Surely you can convince them that there is a solution available if they allow you to set up a filter for them. After that, contact everyone you know who is waiting for staff to fix the problem. Point them to staff responses in this item, and offer to help them set up a filter. Then whenever someone asks for help with spam, you can email them privately and offer to help them set up a filter. You would then have solved all the problems you know about, and would be ready to solve any new ones.
I don't WANT to use either pop mail or webmail. I delete most of my mail after looking at the subject line (freecycle mail). I read mail from three locations. I hate webmail. I don't know most of the people who would benefit from a spam filter at grex. Nobody on staff appears to have time and interest to help them. This is why I suggested something in motd. I am happy with pine and .procmailrc.
Regarding #89; Sometimes life is tough, Keesan. Get used to it.
I recall newuser being closed last January, briefly, while bhoward instituted mail-blocking for new accounts; I *think* it was reopened by February. A check of one of the newuser logs shows errors for every month since November, 2005, EXCEPT for December, 2006: there were no errors between Nov 22, 2006 and Jan 17, 2007. Another log file shows three registrations between Nov 24, 2006, and Jan 14, 2007: one each on Dec 6, Jan 1, and Jan 4. Other than that, there seem to be no significant gaps in registrations. Sorry, folks: newuser was NOT closed for "nine months."
Thanks Joe, I appreciate having actual data.
I don't know most of the people who would benefit from a spam filter at grex. Nobody on staff appears to have time and interest to help them. ------------------------------------- You simply have no way of knowing who would or would not benefit. You are making assumptions about their feelings, desires, and helplessness. Your paternailistic, controlling attitude that you know best for all these strangers is irritating, as is your snide dig at staff. Please, let this drop. When people ask for help here, they get it. When other people start "helping" them whether they want to be helped or not, I get worried. It is none of your business. And staff has repeatedly told you it is none of their business either. The steps I outlined are ways you could personally make an impact. You appear to have the time and interest to help the people who you know would benefit. The rest will just have to live with reality until they are ready to ask for help. And, until they ask, it is no one's business but their own how they deal with spam, email, and their private communication tools.
It is not necessary to be nasty. Board has done a lot of talking about a system-wide spam filter but nothing has been implemented yet. Remmers was working on it before he quit staff. I am offering to help those who still want a spam filter and have no idea how to figure it out on their own. I need some cooperation from someone on staff to stick a help offer in motd. Is everyone on staff reading this item or should I mail a couple of staff members who have not shown a presence here?
Why is Cindy being berated for trying to solve the Grex mail problem? There are plenty of reasons to provide the solution for "everyone". I don't think it is paternalistic at all. And frankly, I think several of you on the board are being rude with your personal attacks.
re #95: there's a difference between "trying to solve the Grex mail problem" and "trying to incessantly badger other people into solving the Grex mail problem by imposing one's own personal preferences on everyone."
There is?
How is it imposing one's personal preferences by suggesting various things that might be done to make it possible to set up a system-wide spam filter, or asking staff to post an offer to help with individual filters?
Requesting something of staff puts you under suspicion of not being an assimilated robot. Please visit the infirmary immediately after taking the blue pill.
This is the first you've actually asked _just_ for an entry in motd, Sindi. All of your previous requests have been "Install this" and "Delete mailboxes." Send the text you want in the motd to staff@cyberspace.org, and one of us (probably me) will post it for you.
My problem with Sindi's requests are that she has absolutely no understanding of the technology. None. It's frankly a bother to respond back to her time and again when she says the same sophmoric thing over and over again.
I would be happy if someone on staff would take over from remmers on the spam filter project. I have asked on my linux list for help creating a script that would automate changing the login in my sample .procmailrc based on what the user types as login, and copying the result to their home directory. Then nobody would have to email me for help, they could just run that script. The author of our linux suggested: echo "Type in your login name" read answer cat file | sed -e /another string/$answer? > new-file (I think another string would be keesan in this case, to replace my login with theirs, and file would be my sample .procmailrc, whereas new-file would be .procmailrc). It would help to first automatically copy my sample file to their home directory, but it was suggested to identify where that is by parsing /etc/passwd cat /etc/passwd|grep ^username|cut -d: -f6 (I am lost here - would the username here be the same as answer above?). They would also need to copy over .forward from my directory. Can anyone here do better than that? I would be happy to put such a script in my home directory and make world-executable. I already have several sample .procmailrcs. Then the motd could just announce the existence of a usable script instead of giving my email address. I do not know how to write shell scripts. Would the above work for all available shells at grex or just bash? Probably people who have changed their shell know how to edit .procmailrc to change the login. How about a contest for writing the simplest script that would copy two files from my directory to a user's directory (or append them to existing files) and then replace keesan with the user's login name? I already had lots of help setting up the filter from McNally and other's, and a kind stranger from Germany explained how to use spamassassin.
I looked at the proposed script and tried to figure out how it worked. Instead of the above, if the user is running the script from their own directory, would `whoami` produce their login? If so: cat ~keesan/procmailrc.sample | sed -e /keesan/`whoami`/ > .procmailrc cat ~keesan/.forward > .forward This would replace these two files if they already existed. How do I append to them instead? Does anyone brave want to test the above from their own directory? (I forget just what I named my most recent sample .procmailrc and there may be several of different names). I am sending anything with three or more spam points to /dev/null, I think, but it could be to a spam folder instead. Do NOT look at my actual .procmailrc, which is a game I am playing with the spammers.
This response has been erased.
A couple of pointers:
1) almost everybody (everybody?) will have the $USER environment
variable set by login, so you probably won't have to ask the
user for their login specifically, just use $USER. If you
don't want to use the USER variable there are other ways,
e.g. touch a test file in /tmp and look at the ownership for it,
then remove it afterwards.. Or use `who am i`
2) when used with the "-i" flag sed will do an in-place edit on
a file. Or, skip the copy phase and use sed to do it, e.g.
# check for existing .forward and back it up before proceeding
[ -f ~/.forward ] && mv ~/.forward ~/.forward.$$
# copy keesan's .forward but change instances of keesan to $USER
sed "s/keesan/${USER}" ~keesan/.forward > ~/.forward
Those are rough suggestions. And they won't work (as is) for anyone
who uses the Bourne shell (/bin/sh) if it really is a Bourne shell
(I haven't tested on OpenBSD to see if it's an "improved" Bourne shell
that understands stuff like "~keesan".)
But the mechanics will work in most cases.
The problem with getting someone to write an officially-sanctioned one
is that there are always the oddball cases, which require a lot more
trouble to anticipate and code for.
Thanks Mike. I will attempt to understand this. Anyone who has switched from the default shell (bash?) should be able to edit .procmailrc without help. Anyone who has their own .forward file should be able to edit this one. I need to read about what sed does. I don't know shell programming. But I think your line starting in sed copies my two files and changes one string in the first. This should be done to .procmailrc not .forward. cp ~keesan/.forward . should work (after backing up). I will test this.
If your .forward file only contains a pipe to procmail, then yeah, you don't need to do it to .forward. The lines I provided only act on the .forward. You would duplicate them if you wanted lines that would act similarly on the .procmailrc sed is a "stream editor" -- that is, it performs alterations on a stream of input and then sends the results somewhere. sed can also be used to do in-place edits on many files using the "-i" flag. The syntax used by sed commands (e.g. "s/keesan/foobar/" comes from a very early Unix editor, ed, but will be quite familiar to users of ed's descendants (e.g. ex, vi, nvi, vim, etc..)
Some users, such as Denise, would also need to type E to exit the Menu system first (from the Menu screen). I need to mention this in motd.
cat ~keesan/procmailrc.sample | sed -e/keesan/$USER/ >> ~./procmailrc This is supposed to append my sample file to a user's .procmailrc rather than back up and replace it. Similarly for .forward. Would that be okay? If I get something working, perhaps staff could put it some place other than my home directory, on the path. Or at least link to it.
Sindi, what you need to mention in the MOTD is that you are available to help, and that there is an easy way to use spamassassin here. The MOTD is not the place for instructions.
If I get more than 10 responses to the offer, perhaps some staff member could set up something more permanent, that would be easier than running a script in my directory.
Would a staff member just put in motd to email keesan@grex.org for help in setting up a custom mail filter based on spamassassin. If I get a lot of requests I will automate the process.
Actually, I've got a better idea: let's get hooked up being one of messagelabs.com's customers.
grex is my officiaol email addrs for lots of stuff. sendmail is JustFine (tm) except for the edit/concatonatinfg provlem.
Grex hasn't used sendmail for years.
well, whatever is the repl;acement then.
Then perhaps you'd volunteer to maintain it?
TS, to the best of my knowledge, *nobody* else but you has ever encountered this problem. I know a way to demonstrate whether or not you're causing it yourself (which is what *I* think is happening..) if you're interested in trying it.
You have several choices: