Err, I guessed from the comments that you mean, our personal bookshelves? (Unless we are librarians who run actual libraries? Which I'm not, sooooo.) I'd never call mine a library, heh. For the most part I organize things vaguely by subject, then by height. I don't like going, for example, shortTALLshortTALLshortTALL or whatnot. Put short books together and tall books together! Or have some kind of height gradient. But if subject relationships become too dispersed, that's also no good. Then you have the problem of, some books are too tall for a shelf at all, and must go on another shelf! (or must go sideways). ALSO! I like shelves to be horizontally full but not too packed, so books of different width may need to be distributed in order to ensure some kind of equal width-tightness of books on shelves.

And now I'm all staring at my shelf and getting really bothered by these books that are too tall and are currently situated sideways. *twitch*