My counter argument is that we should not have an electoral college

The electoral college gives electors more power than the average citizen. They can be paid or blackmailed to vote a certain way. Sometimes they don't represent their district properly.