You think no-one's tried recreating various distros' binaries from their published source, to check they're the same? E.g. Jos van den Oever did that for Debian, Fedora, and OpenSUSE here[1].
Which isn't to say that backdoors inserted into the binary that aren't in the published source are impossible, only that they need something more subtle than the crude/easily-detectable 'merge backdoor, compile, ship'. Something like a Ken Thompson 'Trusting Trust'[2]-style attack. (Though there are ways of at least having a good chance of detecting even those - see [3]).
(More likely, IMHO, are just deliberately-introduced, plausibly-deniable bugs in the source - think [4]. Yeah, they might be found & reported by an outsider reviewing the source, in which case you thank them, fix it, and introduce another couple somewhere else next week).
Which isn't to say that backdoors inserted into the binary that aren't in the published source are impossible, only that they need something more subtle than the crude/easily-detectable 'merge backdoor, compile, ship'. Something like a Ken Thompson 'Trusting Trust'[2]-style attack. (Though there are ways of at least having a good chance of detecting even those - see [3]).
(More likely, IMHO, are just deliberately-introduced, plausibly-deniable bugs in the source - think [4]. Yeah, they might be found & reported by an outsider reviewing the source, in which case you thank them, fix it, and introduce another couple somewhere else next week).
[1] http://blogs.kde.org/2013/06/19/really-source-code-software
[2] http://cm.bell-labs.com/who/ken/trust.html
[3] http://www.dwheeler.com/trusting-trust/
[4] http://underhanded.xcott.com/