We only need to look at human history, and animals in general. The power of an individual is very limited, so those with shared interests group together for protection. The bigger and more powerful groups dominate the smaller and weaker ones and power naturally concentrates into the modern nation state, political party, corporation, etc. And whoever controls those groups holds the most power.

Your perspective requires that all humans be willing and able to come to mutually acceptable compromises for any disagreement. But that’s obviously not possible, there are many disagreements where humans can’t just split the difference or are simply unwilling to make any compromise for whatever reason. And so power becomes the deciding factor.

A few people will always end up with all of the power, it’s how its always been because that’s how our species works. Having mechanisms for choosing these people peacefully is better than a free for all power grab.