Much of the way books get classified has to do with marketing decisions. I think it's more useful to think of literary books and sci-fi/fantasy books as existing on a continuum. To oppose them, to suggest that one category excludes the other, always feels bogus to me. The great Leonard Michaels line is "I wanted proximity to darkness, strangeness"? That's what I'd say I want from a book, regardless of where it falls on the fantastical spectrum - that suspense connected to a particular human character, rather than just some mechanized plot.