Bass is a semi-commercial audio library that does decoding, playback, mixing, all that basic stuff. They support just about every format known to man. Pretty simple and sweet.
But so far as I can tell there's no good audio processing library out there !? By processing I mean stuff like windowed STFT, MDCT/IDCT, equalizing, lowpass/highpass/bandpass, quadrature filters, phase vocoder stuff, DSP stuff like effects, etc. Basically a library of fundamentals that you could build on to build more complex audio algorithms like instrument separation or compression algorithms or pitch correction or beat detection. It must be out there, I just haven't found it yet.