I propose eliminating grex's strange proprietary password hashing algorithm,
and replacing it with the system standard hash.  This involves modifying the
password program in the next system upgrade to *not* use grex's hash when
storing the hashed password.

81 responses total.

moroccan black ?

That seems sensible.  Why did Grex implement its own hash to begin 
with?  Was it to get around the old 8-character limit on DES hashes?

Diffie is just not a mayonaisse brand.

Regarding #2; That was one reason.  Another was to facilitate moving to
Kerberos on the Sun.  But, that never happened (and there are other ways to
get user keys into the Kerberos database - grex's hash required modifications
to the KDC, not just grex's local authentication).  Another was speed: grex's
hash is based on SHA1, which is faster than DES.

Actually, grex's hash is based on HMAC using SHA1 as the underlying hash. 
But I'm not sure why; HMAC adds no additional security over SHA1 for this
application.

Would a changeover impose a burden on existing users?

It shouldn't.  Usually the way something like that works is the old 
hash is simply replaced with the new one the next time you change your 
password.

Regarding #5; David's got it exactly right in #6.  Optionally, to speed up
the process, the login process itself could be instrumented in such a way
that it would detect whether a user is using the grexhash algorithm, and if
so, change to the standard hashing algorithm.  After the next reap cycle,
pretty much everyone would be using the standard hash (except for a couple
of long dormant accounts that people are loath to delete.  In these cases,
the password could be changed until reactivation was requested by the
owners, or people that know them could gently request that they login to
change their passwords, etc).

I'd be willing to modify login_grexpass (the program that authenticates
users who login via telnet, SSH, ftp, etc) to do this.  For the record:
that's also the best way to get users into a Kerberos KDC if that's ever
desired in the future.  Marcus's method was insecure, as on the Sun, large
chunks of /etc/shadow ended up in core files that were readable.  Marcus's
idea was to import grexhash passwords into the KDC directly, as keys; if
someone had ever run strings on those core files, they'd have the user's
Kerberos key.  Clearly not a good thing for that user.  But, honestly, it
doesn't really look like grex will ever move to Kerberos, and those who were
most interested in that aren't active anyway.

That's a long way of saying that changing to the standard hash should be
transparent to the users.

To dig under the covers a bit, the reason this works so smoothly is 
each password entry has a flag at the beginning that tells crypt() 
which hash it's using.  That way it can apply the correct hash when 
checking the password, even if different users have passwords using 
different hashes.  For example, a hash that starts with "$1" is using 
MD5, and one that starts with "$2" is using Blowfish.

Yes.  And the login program has access to the unencrypted password, and can
thus rehash it using a different algorithm.

Got to love simplicity and standards-compliancy.

Well, so what is the general concensus?

I can't think of a good reason not to do it, and anything that reduces the amount of nonstandard software that has to be maintained seems like a winner.

unlucky

Great!  Then...who wants to implement it?  I can write the code, but no longer
have access to install it (or even debug it).

Note: I have written a modified version of login_grexpass (that all users use
when they authenticate to grex by a password) that replaces grexhash'ed hashes
with the system standard hash for the user's login class.  I have not debugged
it, but it was not hard; for debugging, I compiled in a routine that should
make it possible to run the modified version on grex without doing anything
to normal users.

I have also modified newuser, wnu, and the passwd program to make this
proposed change.

I have also installed a throw-away test virtual machine running OpenBSD 3.9,
and have tested all of my changes to the above code (with the exception of
the wnu code, which simply links against the newuser code, which is working.
The only thing I changed for wnu was the Makefile).

All that remains is for some staff member to install this stuff.  I'll do it,
but I'll need root access.

Dan: I have added you to group   wheel    

Let me know if you need anything further.

Thanks, Mic.  Does everyone concur with me doing this?

Well, it appears as though I'm out of wheel again.  Hmm.

   This will not be done without talking about it!

Very well.  That's what this item is for.  Let's talk about it.  Steve?  Your
position?

(By the way: Steve removed me from the wheel account and the staff conference
ulist [which I had added myself to in order to announce when I'd finished
copying all of the programs into the base system.  For the record, yes, I was
planning on waiting until hearing more in this conference before proceeding].)

Regarding #21; By the way, this has been under discussion for more than a
week.  Steve, I wonder, do you regularly read this conference?

   I haven't in the last five days or so, at all. Sorry.

   But we have an large problem here, that of a staff person giving
you root access, without prior conversations about it.  That you
were once on staff is essentially irrevelant.  This issue is not
about trust, eiher.  I personally trust that you would not do
anything to hurt Grex.  But the issue of changing the hash is not
something I care to do without more people talking about it!
That several staff members aren't reading this item (or conference)
is not a reason to go ahead without consulting others.  THAT is
what I am pissed about.  The issue of whether or not to make a 
change is another matter, which I need more time to think on, and
I think others will say that as well.

There are a couple of issues here.  Firstly, there is the whole root
access issue.  Honestly, you folks ought to know better.

Secondly, there is the whole "how things get decided" bit.  There are
reasons why it would be good/bad to switch to using Poul-Henning Kamp's
md5 pw hash, but nobody's managed to raise them.  Raising the same issue
over & over again works great in politics.  It's not a particularly
effective way to make a technical point.  

Regarding your "core dump" argument; if we have things that do that
today, then we definitely better not switch over to using md5.  There are
lots of programs out that will crack passwords given md5 crypt strings.
Few of those programs are going to support grex crypt strings, precisely
because it is non-standard.  Say again why we want to have a "standard"
crypt string that everybody's code understands?

Regarding sha-1/hmac, this is probably the least relevant argument you
could make for grex.  99% of our vandals don't care, and probably at least
50% of our users have security practices that make this entire argument
a moot point.  However, since you mention it: a more modern and standard
password hash than any you've proposed is PKCS#5 pbkdf2.  This is what's
in fact used in kerberos 5 today as well as many other places.  It's based
on hmac, & in kerberos 5, uses sha-1.  From the cryptographic end, md5
is regarded today as suspect, and even sha-1 has recently come under a cloud.
I don't understand your fascination with this argument; not only does it
not really matter for grex, but if you follow the evidence to its end,
it points away from md5 and towards hmac.

Regarding #24; Okay, Steve, enough.  If Mic does something, how am I
supposed to know that he *hasn't* discussed it with the rest of you?
Honestly, I'm not privy to whatever communications he may have had.  If he
announces, in a public forum, that he has given me root acess, don't you
think it's reasonable to assume he's gone through the "appropriate channels"
(whatever they may be) before doing so?  This is the gist of what I told you
in private communication, though you apparantly did not believe me and made
some insulting and condescending remarks about my military experience.  IF
you have a problem with the way Mic did things, I suggest you take it up
with him.  So far, I've found your lecturing to ME on the matter insulting
and rude in the extreme; you have made no been effort otherwise.  And if
it's *really* such a big deal, why am I still listed on the staff web page?

But more to the point, what does your problems with the root access thing
have to do with the topic under discussion?  If you want to continue talking
about the root access thing, create an item in coop or somewhere else and do
it there, please.  If you have something constructive to add about the
password hashing algorithm, then by all means please continue to post here.

Regarding #25; The standard OpenBSD hash has been based on blowfish, not
MD5, since OpenBSD 2.1, and has no known vulnerabilities that I am aware of.
The core dump issue comes from the Sun, not OpenBSD.  SHA-1/HMAC gives no
additional security over standard SHA-1 for this application, as I have
demonstrated on grex in the past.  I've posted all these things before, with
greatly expanded justification for the latter two (the former is true by
simple inspection).

But this isn't an issue of security.  OpenBSD's standard hash is in the same
ballpark as yours, if not better  Sure, you allow longer password strings
(128 or so bytes versus 50-something) but that doesn't matter.  The size of
the hash is the limiting factor in your scheme, and for any long password,
there are much shorter strings that will collide with it.

But this is a matter of grex doing something its own weird proprietary way
for little benefit and people having to support it when it provides no
discernable benefit.  You originally built your hashing algorithm for the
Sun where I agree it was a win: it was faster than traditional DES-based
crypt, arguably stronger, and allowed long passwords.

It is not a win here.  It provides no addition security over OpenBSD's
bcrypt, the speed is largely irrelevant (as it doesn't put a big load on
this machine either way), and bcrypt allows long passwords as well.  Someone
has to maintain your algorithm.  You've been inactive for two years, so
who's going to do that?  Every time there's a system upgrade, someone has to
go and modify the login procedures, modify the passwd program, etc, etc.
Anyone who wants to write a program that authenticates users has to link
against code that must be ported and maintained by grex as opposed to by the
operating system vendor (though fortunately, there's a library for this).
Any operating system patches dealing with the password subsystem have to be
carefully examined to see if they'll break compatibility with us.  I think
that all of that matters a lot more than whatever modicum of security is
obtained through the obscurity of your hash.  That is what I would like to
change, and I have code already written and tested that will get us out of
the custom maintenance cycle within a few months.  Why should we stay
non-standard?  If hashed passwords are in core files, at least they're
not world-readable on modern BSD, like on the Sun.

Btw- A paper on the OpenBSD bcrypt hash is available here:

        http://www.openbsd.org/papers/bcrypt-paper.pdf

That gives a good overview of the bcrypt algorithm that is the default in
OpenBSD.  Note that it is explicitly contrasted with the old MD5 based
password hash that Marcus mentions.  I don't know why he mentioned that at
all, come to think of it.

Marcus did mention PBKDF2.  That's available as an RFC.  A link to it is
here:

        http://www.ietf.org/rfc/rfc2898.txt

While of course they are different, notice how this algorithm resembles
somewhat the internals of the bcrypt implementation: both are based on
repeated permuting the input password based by some function with the result
of the previous iteration as an additional input.  Same basic concept, same
basic function.  Is PBKDF2 stronger?  Perhaps.  Is it worth the maintenance
cost?  As Marcus himself says, it's not likely to be relevant on grex; our
users aren't security concious enough for that.  That's the real attack
vector: the strength of the password itself, not the security of the hash
(and, as Marcus says, some doubt has been cast on the security of SHA-1.
Hmm, another strike against our home-grown algorithm?).

But suppose for a moment that it turns out that the bcrypt algorithm is
weak, or that an HMAC/SHA-256 version of PBKDF2 just becomes clearly the way
to go for password authentication.  Then, if the OpenBSD people are as
security concious as Steve and Marcus claim, surely they will implement the
alternative.

But, then how to switch to it?  You don't de-hash the hashed passwords; the
crypto doesn't work that way.  Instead, you have to get the user to somehow
give you the password so it can be hashed using the new algorithm.  This
highlights another benefit of moving to the standard model: it is easy to
switch algorithms over time.  You simply change the default crypt cipher for
the user's login class in /etc/login.conf to be another algorithm (or a
stronger version of the current algorithm, if that's better).  Then, the
next time the user changes the password, it will automatically be rehashed
with the new algorithm.

Why?  It goes back, again, to the fact that the system stores an identifying
string at the front of the hashed password in /etc/master.passwd that tells
it what algorithm was used to hash the password.  When the user goes to
authenticate, the system uses the algorithm current associated with the
user's hashed password.  However, when the user selects a *new* password,
however, the system looks up what algorithm is associated with the user's
login class (or the system default), and hashes the user's new password
using that algorithm.  If there's a problem, you impose a password
expiration time of, say, a month or two.  Let the system do the dirty work
for you.  All you do is change a configuration file.

Again, though, this is not a security argument, it's a maintenance argument.
One of grex's most precious and scarce resources is staff time.  Marcus's
hash is a drain on that time; perhaps normally not much, but what if the
OpenBSD people change the authentication interface around?  What if the
structure of the passwd program changes suddenly?  As things stand now,
someone has to go in, sort out how things work, and shoehorn our stuff into
the existing model.  Why not just use the existing model, when it offers the
same or comparable security?  In short, it *does* matter.

For the record, my philosophy on things of this nature is that, the standard
for deviations from the base system shouldn't be, "why shold it be the
same?" but rather, "why should it be different?"  Given limited staff
resources, it really should be up to those who want to keep this algorithm
to justify why they want to.

Am I out to lunch here?  What do others think?

Makes total sense, and would be the professional thing to do.

I sense there is a tad too much protectionism of one's code around this 
place that often cloudens clear logic.

Hey, Mic.
Can I have root, too?  ;)

Regarding #28; Perhaps.  It does *feel* that way.

Regarding #29; Go back to lurking or I'll put you on firewatch for a week.
 :-)

so;

if we can bring marcus watts back from the deep unknown, do you think we can
resurrect dave thaler ?

Dunno.
It'd certainly be nice if he open-sourced yapp....

My main problem with this whole process is that nobody on staff
has the endless amounts of time you seem to require to debate this
issue over and over and over and over and over.  Moreover you appear
to believe you are a better expert in this matter than any of the
current grex staff.

I know about bcrypt, in fact, I knew the author
Niels Provos when he was in A^2 - and by an odd coincidence he ended
up in my old office at the argus building for a while.  I was also
involved in part of the design process for putting PBKDF2 into
kerberos 5.  Today, I'm working to improve encryption technology
in AFS.  There are certainly people who know more in this field
than I do.  I haven't seen any evidence you are one of those people.

I find it perverse you waste so much of everybody's time in monotonic
pursuit of this issue.  You claimed just now that "bcrypt" has no
known weakness, yet there's a version of John the Ripper that has
bcrypt support in it.  Solar Designer has respect for bcrypt - yet
our cookbook vandals have the code.  It's a classic security tradeoff;
can we crank our password quality checks and iteration counts to defeat
the future cookbook vandal who slips in & has superior computing resources
to guess passwords selected by people looking for the lowest entropy possible?
You claim "bcrypt 'has no known vulnerabilities'", in fact, any password
based system does, and using standard algorithms on a public system has
interesting implications.

You claim standards compliance justifies saving
staff time, yet other staff members volunteered their scarce time to do just
that. There are times when individual initiative is to be commended, yet there
are other times when consensus building is in fact more important. Pissing off
other staff members is not a valid consensus building exercise.

You may be believing I have some sort of unnatural attachment to the grex
pwhash algorithm.  I don't.  It has its own weaknesses, as well as
strengths.  I built one experimental version of kerberos 5 that
could use any combination of mit string to key, unix crypt, grex's hash,
and pbk's md5.  I don't see grex moving to kerberos in the near future,
and I even agree there's value to running "standard" software.  The
right answer for a grex upgrade might be "bcrypt".  This is not a black
and white decision, nor even shades of grey.  It's also not my decision
to make, nor is it yours.

That is the core problem here.  Grex does not need a divided and contentious
staff.  Grex needs a cooperative staff who can work well together to
achieve mututal goals.  I am quite capable of being as bad a cowboy as
anybody here.  I reign that in hard.  Most of what you see here on grex
is the result of hard work on others, and grex is so much more because of
all that work.  Grex cannot afford to depend on any one person for stuff,
even you, and especially me.  That is a danger in any sized organization,
and it's a special danger to a small organization like grex.
This is *far* more important than md5 vs. sha1.
I don't care if grex goes with bcrypt with a count of 1 or one gadzillion,
or even with md5.
I care a lot if grex becomes unusable, if grex gets compromised, or
if the decision making or implementation process becomes too unwieldly
or controversial that it drives good people away.

Marcus, you're the one who brought up MD5, not me.  I'm just pointing out
that it's the default in OpenBSD and giving pointers to more information
about it so that others can go look for themselves.  I don't know what you
knowing the designer of bcrypt has to do with any of it.

It would be trivial to plug your algorithm into John the Ripper - the source
code is there.  (For those that don't know, John the Ripper is a password
cracking program: it takes a dictionary of words, permutes them in simple
ways [adding numbers and the like], hashes them, and compares them to hashed
passwords.  If they match, you know the original password.  This is
essentially how the login scheme works already, it's just throwing a whole
slew of passwords at it shotgun-style to see if you can find one or two that
work.)  That your hash is not in the basic distribution just means script
kiddies won't crack grexhash'ed passwords.  But then, script kiddies aren't
likely to be able to get to the grexhash'ed passwords anyway; someone who
can will likely be able to do the tiny amount of work to get a version of
John (or crack, or whatever) running that *will* handle grexhash'ed
passwords.

That a version of John the Ripper exists that compares against bcrypt'ed
passwords doesn't imply that there's a weakness in bcrypt itself; no known
weaknesses have been found in the crypto.  I'm sure you knew that that was
what I meant, so I'm not sure where you were going with that.  If someone is
attacking it via a brute force, offline dictionary attack, and that's the
best they can do, then that means things are working the way they should be.

Certainly, grex should not move to an experimental version of Kerberos or
anything else.  We need solid, mainstream, production software here.  If the
decision is made to move to Kerberos at some point, it would be far easier
to instrument the login_passwd program and passwd to add principles to a
Kerberos database than take the hashed passwords out of /etc/master.passwd
and use them as Kerberos 5 keys.  Surely, using a string value from *any*
file on grex as a key would be a bad idea.  If you're assuming bcrypt is
bad, on the basis that attackers will be able run John against it, then how
can you possibly justify using any variant of the same hashed string as a
cryptographic key?

Now, is the main point of your argument that we're better off not changing
because your hash is obscure?  How is it secure?  If someone can get to the
contents of /etc/master.passwd (or the db file) then they've already
compromised grex.  At that point, it would be easier for them to add a
password sniffer than run crack against the password file.

Of course there are times when individual staff effort should be commended.
I believe I said your hash was a win on the Sun.  But there are also times
when decisions should be re-evaluated in the face of changing environments.
I don't think that's perverse, I don't think it should piss people off, nor
is that what I trying to.  I don't see how that's "endlessly debating"
things.  In fact, so far, you're the *only* person who has voiced objection
to changing the hash algorithm.  What, then, is contentious, and why?  If
you, by your own statement, are not attached to this hashing algorithm,
then why are you so upset about the question?

And I've never called into question your expertise.  I don't think name
calling and implied questions about mine are called for.  But, for the
record, I learned crypto from Michael Rabin.  Are there those that know
more about it than me?  Certainly.  But what does that have to do with
the current discussion?

(I also don't see how I appear to think I'm more of an expert than anyone else
in this matter.  Would someone please (a) tell me if I'm doing this, and (b)
tell me how?  Surely, asking for a justification of something that requires
staff effort to support is reasonable.)

Provos and AFS chest thumping.  Holy cow.  What next: Pushup contests?

re #34
 But then, script kiddies aren't
 likely to be able to get to the grexhash'ed passwords anyway
Then why do you care?  Grex doesn't have to be obscure unless its only about
job security for folks like yourself.  Your rant was transparent, Marcus.

STeve, why does Marcus seem to think this was talked about "over, over, and
over" yet you seem to think nothing was discussed?  Which is it, gents?
I'm also amused that Marcus is talking about 1) pissing off staff and, 2)
concensus building; yet, nobody seems to agree with neither STeve or mdw on
how things are being conducted.
Black box, anybody?

Regarding #36; Are you referring to Marcus or me?  I wrote #34, which you
quoted, but you mention Marcus by name.  The reason I care is that I don't
think grex should be maintaining customizations when reasonable standard
alternatives exist (Provos didn't design the bcrypt algorithm, by the way,
someone at MIT did.  He just implemented it.  Perhaps they collaborated; I
don't really know.)

Marcus is probably referring to conversations from several years ago.

It's interesting that Marcus seems to be against my proposal, yet his argument
almost supports it.  In particular, he speaks about how organizations as small
as grex shouldn't rely on any single individual to keep them running.  That
would imply, in cases like this, sticking as close to the standard software
distribution as possible.  It would certainly rule out things like
experimental Kerberos servers that would almost certainly require the
originator to maintain (unless someone were to invest quite a bit of effort
into it).  I'm not at all sure I followed the rest of his argument; the last
time this came up for discussion was nearly two years ago, when grex moved
to this machine.  It's hardly been under continuous debate since then.  At
the time, the decision to leave the custom hash in place was made in order
to facilitate allowing Marcus to plug in his modified KDC.  But Marcus only
logged into grex two or three times in the ensuing two years (his recent
logins seem to coincide with this discussion), and given how the staff climate
has changed over that time, it seems reasonable to re-open the issue.

- Backtalk version 1.3.30 - Copyright 1996-2006, Jan Wolter and Steve Weiss