The voting population difference was massive. The entire reason for the 3/5ths compromise was because the slave states would have almost no house representation.
Adding up the 1780 numbers on that page, the numbers appear to be almost identical. But there's a catch: those numbers include enslaved people. who numbered at least 500,000 (see [1]).
