If you're writing a book where you want to make a positive truth claim, then you should absolutely call it nonfiction or memoir. If you don't want to make that claim - if that's not what's important to you; if you're more interested in storytelling and interiority and interpersonal relationships than in objective, checkable facts about the world - then why wouldn't you call it a novel, and take advantage of what that gets you, of the extra freedom, of belonging to the tradition of the novel?