|
Grex > Coop8 > #10: Web-Page Building on Grex |  |
|
| Author |
Message |
| 25 new of 382 responses total. |
scg
|
|
response 75 of 382:
| 
Dec 10 02:44 UTC 1995 |
I'd hope that would go in as a comment, rather than as actal text that others
will see in their web browsers.
|
robh
|
|
response 76 of 382:
|
Dec 10 03:43 UTC 1995 |
You can go ahead, Steve.
|
janc
|
|
response 77 of 382:
|
Dec 10 04:55 UTC 1995 |
I don't understand why it is possible to identify these things from logs, but
not block them.
|
mdw
|
|
response 78 of 382:
|
Dec 10 07:20 UTC 1995 |
Basically, because our httpd defies rational analysis. It was one of
the best available at the time, but there are some newer options that
are likely to be better. I'm currently trying to learn enough about
them to be able to implement the policies I believe we want, but it's
certainly no simple thing. For instance, one of my current worries is
trying to figure out what ".htaccess" files can do, and how to block
them.
|
ajax
|
|
response 79 of 382:
|
Dec 10 08:18 UTC 1995 |
On M-Net, as I understand it, the policy against graphics files in
web pages is mostly enforced by a program that periodically changes
the access rights of all graphics files to owner-accessible only.
I'm not sure if it looks at the file extension or actual file format.
Maybe not a nominee for the Elegance in Administration awards, but
it's quick and easy, and seems to work for them. It also impedes
other public access (ftp, etc.) to graphics files, though that
doesn't sound hard to work around (e.g. rename or encode the files).
|
mdw
|
|
response 80 of 382:
|
Dec 11 06:01 UTC 1995 |
We could certainly teach ftpd not to "get" .gif/.jpeg files. We need to
avoid having programs that crawl through the filesystem doing things
like that - that's one of the things that eats CPU up.
|
srw
|
|
response 81 of 382:
|
Dec 11 06:18 UTC 1995 |
I have enhanced mkhomepage to expound on our policy. This message was
really not being delivered to our would-be web authors, so I am glad it
is in there now.
httpd is somewhere between lame and brain-damaged when it comes to
some of its configuration limitations.
I have been renaming (rather than repermitting) multimedia files.
Either way works well enough.
|
steve
|
|
response 82 of 382:
|
Dec 13 02:24 UTC 1995 |
The problem with teaching FTP to not use certain files is that it
wouldn't take long (probably about 10 hours) for someone to realize that
if 'file.gif' doesn't work, 'file' probably will. ...And it isn't worth
it to try and scan for signatures, as there are too many different things
to look for, and such a search would quickly be defeated with a dummy header
at the front of the file.
|
mdw
|
|
response 83 of 382:
|
Dec 13 04:45 UTC 1995 |
Actually, it's worth it, because the key thing it stops is web browsers.
They're all driven entirely by file extension, and since that's the only
thing we're really trying to stop, we don't have to go any further.
|
srw
|
|
response 84 of 382:
|
Dec 13 07:34 UTC 1995 |
If you rename a .gif file to have no extension, and then use it in a
<src img=foo> tag, the browser will cleverly figure out that it is a gif,
and display the image. At least one user has gotten around the httpd blocksby
renaming the extensions. I tested his homepage with Netscape Navigator, and
it worked like a champ.
Of course his files are gone now, because the log analyzer turned this up.
The bottom line is that file extensions aren't all they're cracked up to be.
I agree with STeve.
|
mdw
|
|
response 85 of 382:
|
Dec 13 11:37 UTC 1995 |
Oh, joy.
*sigh*.
|
steve
|
|
response 86 of 382:
|
Dec 13 18:15 UTC 1995 |
I think we need to build some tools to monitor usage, and use the
social mechanism of asking people not to do this when we discover it.
If we can automate that, then great--I'm not against that at all,
but I think the manual method is going to have to be done, at least
for a while.
|
srw
|
|
response 87 of 382:
|
Dec 16 07:57 UTC 1995 |
Essentially, that it what I am doing. Fortunately, I was able to snag a free
bit of software that analyzes the httpd logs. This has allowed me to take
action. I plan to publish the activity for the week of 12/10 to 12/17 on
Sunday, once the log has been rolled over.
The previous week was weird because the data for two weeks was collected
together into a single log file, since the log rolling did not take place as
scheduled. Probably we were down Sunday at that time. Also the data was
strongly colored by the presence of gif and jpeg files that are now gone.
For the two weeks from 11/27 to 12/10, the total traffic via http was 54.7MB,
or 4.3 MB/day. We averaged 1666 requests/day, and served 1206 distinct files
to 3608 distinct hosts during that 2-week period.
A preview of our data for this week (subject to modification):
It looks like we are now averaging 17% less traffic with the GIF files
disabled for the worst offenders from that period. Our user with the most
popular site is teenzine, with 204 hits against the entry page, and about 4% of
our web bandwidth.
The greatest bandwidth usage comes from user nmarrale, who registered
fewer hits, at 143, but is distributing a FAQ for alt.music.prince, and
consuming about 16% of our http usage, just over 500K/day.
|
sidhe
|
|
response 88 of 382:
|
Dec 16 13:39 UTC 1995 |
Interesting. So, graphic files notwithstanding, the heaviest hit
pages are still not the ones that use the most bandwidth?
|
srw
|
|
response 89 of 382:
|
Dec 16 19:01 UTC 1995 |
Yes, that is not too surprising. Even among non-graphics files, sizes vary
widely. Our policy of denying graphics is a very broad one. It eliminates
the use of graphical files that are small as well as large. Our permitting
of hypertext files permits both large and small ones.
|
ajax
|
|
response 90 of 382:
|
Dec 17 00:40 UTC 1995 |
Interesting...around four megs a day of http traffic would
represent what, around 1-2% of total network traffic capacity?
Sounds encouraging if it's on that scale.
|
sidhe
|
|
response 91 of 382:
|
Dec 25 02:46 UTC 1995 |
Well, then, it would sound that the graphics policy is inherently
unfair. I would say it would be better to limit by size, not type, if
the link is your actual concern.
|
popcorn
|
|
response 92 of 382:
|
Dec 25 13:38 UTC 1995 |
Really? The no-graphics policy seems to me a lot cleaner than any other
policy we could dream up. For one thing, the amount of information per byte
is much lower in a graphics file than in hypertext.
|
adbarr
|
|
response 93 of 382:
|
Dec 26 18:40 UTC 1995 |
I am puzzled by this discussion. What interests are you trying to promote?
Is text-only the "good"? Or, is there something more? Sure, we can all suggest
a monster that hides under the bed, but is not that a reason to shut down,
and be safe?
|
scg
|
|
response 94 of 382:
|
Dec 26 19:06 UTC 1995 |
The interests we are trying to promote is that having users actually being
able to use the system to communicate with eachother, in ways like
conferencing, mail, and party, is what's important, and that if web stuff is
eating all of our bandwidth that can't happen very well. With text, we get
one character for each byte. With graphics, we get a few pixels per byte.
The difference in the amount of room taken up is huge.
|
adbarr
|
|
response 95 of 382:
|
Dec 26 20:13 UTC 1995 |
Time and technology are frozen? Or should be?
|
steve
|
|
response 96 of 382:
|
Dec 27 02:22 UTC 1995 |
Our technology isn't frozen, but rather retarded timewise--we're running
on obsolete hardware, and now it seems that our link speed is the same
way.
|
srw
|
|
response 97 of 382:
|
Dec 27 06:56 UTC 1995 |
The most recent week of web statistics are available. Check out
/usr/local/lib/lynx/stats.html
Our total web bandwidth is holding steady at 4MB/day. This is not too high.
However, a new page has jumped to the top of the bandwidth charts.
It is /u/arthurp/www/sex/brandi
It appears to be 156k worth of hard core sex story. I read the first few
lines. It is responsible for nearly 1/4 of that, or about 1MB/day.
I will be honest. The content of this file worries me. I would feel a lot
better if it were not a permitted use of our web server. If you have read my
comments on the Exon amendment, then you know that I am against that
legislation. Still I feel that this kind of story is an abuse of the privilege
of using our web server. I bring it up now, not because of the Exon amendment,
but despite it. I bring it up now because it has just come into existence here.
Because of the presence of this file, I plan to propose that we
institute a restriction against hard core sex stories (text) on our web server.
If such a proposal passes, I do not propose to be the "censor" who decides
what is allowed. I just want to have a policy in place so that we can remove
blatant examples, should they be discovered.
|
scg
|
|
response 98 of 382:
|
Dec 27 07:49 UTC 1995 |
I'm not so bothered by the content, which is behind lots of disclaimers saying
what it is, as I am by the amount of bandwidth. From what I've heard, things
of that nature generally tend to suck up a lot of bandwidth once word gets
out, and we don't have a lot of bandwidth for it to suck. If it's sucking
a quarter of our web bandwidth, we need to do something about it. It sounds
like it's using more than the graphics we do have a policy about generally
do.
|
robh
|
|
response 99 of 382:
|
Dec 27 11:59 UTC 1995 |
Yeah, the content doesn't bother me in the least - though I
haven't read the whole story, I hope there aren't any curling
irons later in the story? >8) - but one-quarter of our
Web traffic certainly bothers me.
I know that said user has a page somewhere else, I'm going
to check and see if they can move the story there.
Failing that, maybe we can allow the story if it's stored in a
compressed format? Most systems will have some compression
software available.
|