I second pandoc. I managed to get a book published with sources in markdown. And I can confirm that you can run from latex, but you cannot escape it..
Before you are done, your markdown will typically end up moving through latex2pdf. I think what nailed me down was proper page numbers.
They are one of a thousand small things you expect to be "right" in a book,
but nevertheless only latex seems properly prepared to handle..
And as others have said, the marketing/community for a book is a full extra job, separate from being a writer.
And as others have said, the marketing/community for a book is a full extra job, separate from being a writer.