[PATCH] compiler warning cleanup in libcrypt/des.c

[Al Stone]

On Tue, 2007-03-13 at 23:17 -0400, Mike Frysinger wrote:
> On Tuesday 06 March 2007, Al Stone wrote:
> > Clean up another warning about a possibly uninitialized variable
> > in libcrypt/des.c.  Found while trying to compile default ia64
> > config on Debian (GCC 4.1.2, btw).
> 
> i think this is a gcc bug ... ive looked at that bit of code before and have 
> yet to determine the code path where f would be used uninitialized ...
> 
> we make sure count is non-zero at the top of do_des, which means the while 
> (count--) is executed once ... then the while (round--) loop is obviously 
> executed which means f gets set to "(r48l ^ r48r) & salt" before it is ever 
> used

This appears to be a problem in GCC 4.1.2, and it is not limited to
just ia64 (x86 does the same thing).  I reduced the command line to:

   gcc -c short_des.c -Wall -Os -fno-tree-dominator-opts

where short_des.c is the pared down version of libcrypt/des.c, and looks
like this:

unsigned int saltbits;

static int
do_des(unsigned int *l_out)
{
        unsigned int    l;
        unsigned int    f, r48l, r48r;
        int             round;

        l = r48l = r48r = 0;
        round = 16;
        while (round--) {
                while (round--) {
                        f = (r48l ^ r48r) & saltbits;
                }
                l = f;
        }
        *l_out = l;
        return(0);
}

void __des_crypt(void)
{
        unsigned int    l, r0;
        unsigned char   *p;

        p = 0;
        if (do_des(&r0))
                return;
        l = r0;
        *p++ = l;
}


If I remove almost anything else, the warning goes away.  It looks
like GCC is confused about the loop variable, even though it is very
clearly being set to a constant.  For no obvious reason, it believes
the loop variable could somehow be set so that one never enters the
inner loop.  The warning is bogus, especially since you get this out:

short_des.c: In function '__des_crypt':
short_des.c:7: warning: 'f' may be used uninitialized in this function

Apparently, it's trying to inline do_des() and gets confused about
who 'f' even belongs to.

The good news is that this has been fixed in GCC 4.3.0.  I've filed
bugzilla #31181 against GCC 4.1.2 for documentation purposes, if
nothing else.

So: wait until GCC 4.3.0, or apply the patch as a workaround?  Either
way works for me...

-- 
Ciao,
al
----------------------------------------------------------------------
Al Stone                                      Alter Ego:
Open Source and Linux R&D                     Debian Developer
Hewlett-Packard Company                       http://www.debian.org
E-mail: ahs3 at fc.hp.com                        ahs3 at debian.org
----------------------------------------------------------------------