|
Grex > Coop11 > #7: Grex vs. Malthus, round Sixty-Five Thousand, Five Hundred and Thirty-Six |  |
|
| Author |
Message |
janc
|
|
Grex vs. Malthus, round Sixty-Five Thousand, Five Hundred and Thirty-Six
|
Jul 9 06:18 UTC 1998 |
It appears that Grex is getting about 175 new accounts created every day.
We reap about 100 a day, so we gain about 75 users a day.
This growth rate is steadily increasing.
We currently have about 23,000 users.
STeve estimates that at this rate will reach about 65,000 users sometime in
the summer of 1999.
This is an interesting milepost because SunOS cannot handle more than
65,536 users. This is because all unix systems internally identify their
users not by their login names but by a number. For example, I am 2386,
and kerouac is 9304 (type !id to find out your account's number). Unix
uses these number to identify who owns files, and who owns each process
running in the system. The catch is that SunOS allocates only two bytes
to store the uid number in any of these places. Two bytes gives you
room for only 65,536 different users. We could have more logins than that,
but people would have to share their files and processes. Not a good idea.
So, if current growth continues, Grex will have to face in interesting
technological problem in about a year. There are solutions to this problems,
but none of them are easy (changing to a different version of Unix is
perhaps the most obvious). This conference probably isn't the place to
discuss that.
My intent with this item is to inform people that this issue is coming up,
and to collect people's reactions and ideas about this.
Should Grex attempt to limit growth? If so, how? Turn off newuser when
the user count exceeds some limit and turn it on again when it dips lower?
Expire accounts faster? Cut out some of our services?
Or should Grex's staff keep trying to keep ahead of growth, boldly going
where no gang of impoverished computer geeks has gone before?
|
| 74 responses total. |
davel
|
|
response 1 of 74:
|
Jul 9 10:25 UTC 1998 |
Expire accounts every 4 hours! Those who really want to be on Grex should
stay logged in all the time ... just as they should attack-telnet once we get
rid of the telnet queue!
|
aruba
|
|
response 2 of 74:
|
Jul 9 14:09 UTC 1998 |
Well, I think we could expire accounts sooner than 90 days after last use, if
it comes to that. That would increase the reap rate. But going to a new
version of UNIX seems like a good idea too.
|
jep
|
|
response 3 of 74:
|
Jul 9 14:54 UTC 1998 |
Reaping accounts more quickly seems reasonable to me. Isn't there a
large percentage of Grex accounts created by newuser and then never used
again? I see no reason for keeping all those accounts for more than 30
days.
|
cmcgee
|
|
response 4 of 74:
|
Jul 9 15:04 UTC 1998 |
jep, I assume you mean that a new program should be created that
differentiates between someone who creates a new login with newuser, and then
does not come back for 30 days, and someone else who logs in a 2nd time, and
then disappears for more than 30 days.
If we are simply talking about accounts expriring sooner than 90 days, then
I strongly urge that we go no shorter than 60 days. Or else we write a new
program called "dormant" that lets you protect your login ids when you think
you might be away from Grex for more than a month.
|
jep
|
|
response 5 of 74:
|
Jul 9 15:38 UTC 1998 |
I have benefitted from Grex's long reap-free period; there have been
months when I didn't log in, and I would have been reaped if Grex reaped
accounts more quickly. I appreciate that I got to keep my loginid. I'm
userid 1020; I've been here since Grex first opened to the public, and
I've been 'jep' for 12 years on here and M-Net. I wouldn't want to lose
my loginid due to reaping. I really would hate it if someone else got to
be 'jep'.
However strongly I feel about that, it would be worse for me to disappear
for a year or two, and during that period block some new user from coming
on-line and getting to try Grex, than for me to have to take a chance of
losing my loginid. If I was going to be away for a long period of time,
but expected to come back, I'd send a message to 'staff', and I imagine
they'd do what they could to accommodate my circumstances.
The preservation of old loginids is nice, but if it's a system problem (as
#0 says it soon will be), then I think we need to look at the bigger
picture. If it's a choice between allowing new users and retaining former
old loginids, I hope Grex chooses the new users.
A reap program that gets rid of people first who came here once, looked
around, and never came back would be great. I don't know if it's
possible. If not, well, a quicker reap period will very likely alleviate,
and maybe even *solve*, the problem caused by the 64K user limitation. If
it's needed, so be it.
|
rcurl
|
|
response 6 of 74:
|
Jul 9 15:48 UTC 1998 |
We need to name this problem. Perhaps the "S64K" Problem?
|
krj
|
|
response 7 of 74:
|
Jul 9 16:15 UTC 1998 |
Changing the reap period would buy some time in an emergency,
but I don't think it would buy *much* time. We'd still be adding 75
users per day.
What are all these newusers doing? They aren't active conference
participants; are they lurkers? I had always assumed that they were
mail users, but janc said in party last night when this topic came up
that he didn't think most of them were mail users.
|
krj
|
|
response 8 of 74:
|
Jul 9 16:37 UTC 1998 |
Also, it would be useful to have some input from the staff: guesstimates
on how much work it would take to convert Grex to an operating system,
or an architecture, in which the 65,000 user limit would not apply.
If a solution to allow Grex to grow over 65,000 users would take
a month or two to implement, then there is probably no need to take
steps to limit growth. If such a solution will take a year -- as the
installation of the new Sun 4/670 did -- then we had better get
serious about limiting growth, because a year is about all the time
we have at the current growth rate.
|
toking
|
|
response 9 of 74:
|
Jul 9 17:06 UTC 1998 |
a question:
would it be possible to have a program that checks to see how long the
account has been around and then decide wether or not to reap it?
for example:
sillykid creates an account, logs on a couple times, then never comes
back. The program sees that his account is less that 90 days old and has
been innactive for more than 30 days, so the sillykid account gets
reaped.
another example:
olduser has been around for ages, but for some reason or other he hasn't
been able to log on for a couple of months, the program sees that his
account is more than 90 days old and doesn't bother to see if it's been
innactive for more than 30 days, but checks to see if it's been
innactive for more than 90 days. If the account has been innactive for
more than 90 days then it gets reaped.
I don't know much of anything <O.K. I don't know anything> about how
the current system works, but that's my two cents.
|
albaugh
|
|
response 10 of 74:
|
Jul 9 17:22 UTC 1998 |
In a certain way, it's good that grex is still running ole SunOS, with the
64K UID limit. That limit will keep grex from getting "overwhelmed" by the
resource drag required to create user home directories, user mail files, etc.
If a user can't create a user account one day, due to no available UIDs, then
grex displays a message saying so, asking the user to try again the next day,
when some UIDs will have become available due to reaping. Then if the user
ain't really interested, he won't bother to come back to create an account
that won't be used and get reaped anyway. If the user *is* interested, he
*will* come back the next day. Or, maybe, grex can create a "new account"
queue, yeah, there's the ticket! :-)
OTOH, grex some day is probably going to want to step up to Solaris for other
reasons than just breaking the 64K UID barrier. Yes, in the past staff types
have said "we can function without Sun support just fine, going to Solaris
would require a complete recompile of all grex software, etc." But sooner
or later...
|
albaugh
|
|
response 11 of 74:
|
Jul 9 17:32 UTC 1998 |
Oops, sorry, #10 should have said that grex *ought* to tell the user that he
can't create an account today, please try atain tomorrow.
|
atticus
|
|
response 12 of 74:
|
Jul 9 17:47 UTC 1998 |
IMHO lowering the reap period to 60 days is not going to help much.
Maybe we should find out the number of people who are dormant for 2
months and then log in during the third month. My guess is that it won't
be a significantly large number.
If we hit the limit, then we have no option but to block the creation of
new accounts. It will be nice if there is a provision for reserving an
account so that whenever there is a vacancy, the person can be informed
about it.
Is there any way to keep the BackTalk login database separate from
/etc/passwd? Will we be able to give more new accounts through BackTalk
in that case?
|
other
|
|
response 13 of 74:
|
Jul 9 17:51 UTC 1998 |
I think a combination of the tactics suggested above would be the best
approach. Rewrite the reap program to reap accounts as follows:
* Accounts created by newuser and used on that day, but not used
again for 30 days.
* Accounts less than 90 days old which have not been used for 60
days.
* Any account for which membership dues are not currently paid and
which has not been used for 90 days, except by special arrangement with
staff.
Also, disable newuser at 65,000 users, with a notice saying that
Grex is so popular that it has reached its capacity of 65,000 user
accounts, and would you please try again tomorrow, as old accounts are
deleted every day, making new openings.
|
jep
|
|
response 14 of 74:
|
Jul 9 19:02 UTC 1998 |
re #7: It *would* solve the problem if the reap period was set
quickly enough. The question is, how quickly do you have to reap
existing unused accounts in order to cause growth of the number of
loginids to happen at the desired rate?
Currently, we get 175 new user accounts per day and reap 100 accounts.
The 100 have been unused for 90 days. If you reap accounts after 75
days, you will be reaping some negligible number of more accounts per
day. Set the reap period at 60, and it will be less negligible. Set it
at 30, and I'll bet it's pretty substantial. Set it at some level, and
it will reap 75 more users per day.
The question is, where do we have to set the reap period in order to
either stop growth in the number of users, or at least keep it from
getting to 65,000? When we find out the answer, we can decide if it's a
good idea to change the reap period.
I'll bet someone on the staff can determine the answer to that question.
|
krj
|
|
response 15 of 74:
|
Jul 9 19:43 UTC 1998 |
I think we shouldn't get bogged down in arguments about reap scheduling
and policy.
As I understand the situation: this is oversimplified a bit, but every day
we create 100 Grex accounts which will eventually be reaped, and 75
accounts which will stick around. On a 90 day expiration cycle,
this means that we have a bank of 9000 sure-to-be-reaped accounts.
We could go to a 30-day reap cycle and immediately reclaim 6000 accounts
from that bank: but that's a one-shot deal.
Recovering those 6000 accounts will only buy us 80 days of growth if we
keep adding 75 long-term users per day. That will be useful in an
emergency, but it won't solve our long-term problem.
Our choices are:
1) commit to whatever it takes to remove the 64K limit on the number
of Grex users -- if this is even possible in the time remaining
2) accept that there will be periods, starting next year, in which
"newuser" is going to be shut off. These periods will get longer
and longer, if we keep adding 75 long-term users each day "newuser"
is open, and eventually there will be essentially no new users.
3) Be willing to decide that some uses of Grex are more important than
others; start steering away Grex users who are only here for the
services we deem less important.
Hmm, let me rephrase option #2 vs. option #3:
2) Allow Grex to fill up on a first-come, first-served basis,
and shut the doors to latecomers for all our services --
** including conferencing **
3) Take pro-active steps to shape the Grex community in the belief
that our community-building services are the most important ones.
|
jared
|
|
response 16 of 74:
|
Jul 9 20:04 UTC 1998 |
dynamic uid generation
|
mdw
|
|
response 17 of 74:
|
Jul 10 07:52 UTC 1998 |
I think it's a bit premature to assume grex is just automatically going
to run out of UID's sometime next year. Another important limiting
constraint is the number of users we can support online at one time, and
the kind of performance we can offer those users. I think it's highly
probable that before we run out of UID's, we'll run out of network
capacity, telnet ports, CPU, or so other load bottleneck. It's worth
keeping in mind, too, that with our current text-only operation and no
global media advertising, there are limits to the % of people who might
be interested in us, or who might discover us.
So far as an upgrade path goes, there are a number of interesting
choices. We could continue to go with sun hardware, and run Solaris
2.6, which *does* support long uid's. Or, we could migrate to RS/6K's
and AIX, which has long supported long uid's. Or, we could choose to
run openbsd, and adapt it to run long uid's, perhaps even on the current
hardware. Each has different costs, performance, and security risks.
Another issue to consider in terms of upgrades, of course, is the kind
of computing environment we want to end up in. The kind of "server"
technology that grex is running on is getting relatively scarce. What
is much more the norm is to use "workstation" technology for servers -
and to run services distributed across several such workstations. The
next generation of sun server hardware after the 670 is probably the Sun
Enterprise server 1000, of which I happen to know of 3 in the A^2 area.
The equivalent "workstation" technology would be the SparcStation 10 or
20, of which there are probably at least a thousand in the A^2 area.
It's certainly worth looking at the services we offer. A *substantial*
chunk of the service we currently offer is definitely free e-mail. I
don't think we've done any studies, but it's highly possible that 50-90%
of those "75 new permament users every day" will only ever be using grex
for mail. Perhaps we should resign ourselves to competing with hotmail,
and deliberately set up a number of machines dedicated *solely* to pop
mail, with their own separate arrangements for UIDs and such, and shift
mail to that arrangement. We could then issue "pop-only" mail to the
unwashed masses, and separate full-shell and computer conferencing
access to only those that really want it. We could also probably shove
the mail servers behind some sort of network throttling gateway if they
prove *too* popular, although hopefully hotmail and other free mail
giants, with their big budgets, have a sufficient head start on us that
we won't be serious competition.
|
rcurl
|
|
response 18 of 74:
|
Jul 10 15:08 UTC 1998 |
Do you think it would really be fun offering a free e-mail service? Why
not just drop that and keep just the full-shell and computer conferencing
for "those that really want it"?
|
mta
|
|
response 19 of 74:
|
Jul 10 15:26 UTC 1998 |
Another possibility is to advise people that they can get free e-mail on
hotmail and other free e-mail providers and sing the parises of those other
providers more loudly. Then we hope that if e-mail is all they want, they'll
find it elsewhere...
I don't want to completely stop offering free e-mail -- but if we could find a
way to limit it that would be no bad thing.
You're right Rane, that offering free e-mail is not a particularly fun thing
for Grex to be doing -- but it is valuable to some people and worthwhile.
|
keesan
|
|
response 20 of 74:
|
Jul 10 22:47 UTC 1998 |
Is there some way to try out grex once without creating a user id? I telneted
to some conferencing system in Iowa and had to sign up to use it and then
never went back. Can people just look at the conferences using Backtalk
before they sign up?
|
srw
|
|
response 21 of 74:
|
Jul 11 03:06 UTC 1998 |
Yes you can look at conferences in Backtalk without signing up. In some
conferences, the author may invoke an option which renders his.her posts
unreadable that way, but most things should be readable. Users will have no
participation file, and will not be able to post.
I agree with Ken Josenhans that adjusting the reap policy doesn't buy much.
I strongly urge grex not to reduce its 90 day reap policy.
I agree with Marcus that STeve's projection doesn't take into account other
factors, and thus I too expect growth to slow. With 23000 accounts and a max
of about 80 users on simultaneously, we still have 299 users not logged in
at that time for every one who is. I doubt that this ratio can be forced much
higher. The telnet queue will get too long and people will lose interest.
In answer to atticus in resp:12, backtalk users need to own files too,
because their conference participation file is in their account. therefore, it
is not possible for backtalk users to use backtalk on Grex without consuming a
UID. Note: It *is* possible, but it requires configuring backtalk differently,
and in that mode it could not share conferences with picospan, which is why I
say it really isn't possible.
|
mdw
|
|
response 22 of 74:
|
Jul 11 11:21 UTC 1998 |
People who take out accounts once and then never come back don't really
cost grex much in terms of resources. Our current policy is that we may
delete such accounts as soon as one month after creation.
It was a pretty deliberate decision of mine early on, with m-net, to
give people their own unique full-function account immediately upon
signing up, rather than setting up any sort of 'guest' account. My
reason was that I didn't want to create any barriers towards those
people jumping in and responding, because there are already more than
enough "natural" barriers in people against doing that sort of thing.
If there's a "guest access" provision, then a lot of people who might
otherwise become participants will only observe, and never participate,
because they'll know how to observe, and they'll just never "get around"
to registering. Since this is exactly what backtalk currently offers,
it would be interesting to test my hypothesis against real data from
backtalk usage.
We already offer free e-mail. What I described amounts to making us
less "special" and attempting to streamline the service so that it uses
less resources.
|
valerie
|
|
response 23 of 74:
|
Jul 11 22:53 UTC 1998 |
This response has been erased.
|
arthurp
|
|
response 24 of 74:
|
Jul 12 03:05 UTC 1998 |
I think that wording would do a good job in newuser.
This is a bit techie, but does the file system have room in its
structures for longUIDs? We would have to mkfs anyway. But would we
have to create custom file system drivers?
|