Answering to a Reddit post on whether and how AI could take over and improve our political process.

[Debate] "One day AI will be able to govern a country and manage its economy more effectively than human politicians and leaders."
https://www.reddit.com/r/artificial/comments/71ybpb/debate_one_day_ai_will_be_able_to_govern_a/

My comment:
"The idealized version of what you describe will take a lot of time to materialize, and most probably by the time they reach such level of intelligence we will have already handed over the keys to this planet, willingly or not.
The main reason why this will take so long, is because there is a very significant lag between the decision and its consequences, which makes it very very difficult to train an intelligence on this via trial and error. If you think of it, it’s not serendipitous that the top management positions in corporations are about strategy, and are the highest paid and also the hardest to control of whether one’s doing a good job or not. In such strategic dilemmas we really are reaching the limits of human intelligence and I’m not confident there’s a magic way to circumvent the learning process.
It’s also no accident that history repeats itself. This has to do with what we mentioned above, accentuated by the… limited human lifespan. We don’t live long enough to do enough trials and errors and be able to build an experiential modus operandi for the really difficult decisions and directions in life.
Finally, even if an AI manages to overcome these two hurdles, there is the final one that is always prevalent but seldom openly discussed of "who’s the interests and pleasure you wish to maximize with your decisions". In other words, there are many right decisions; they just satisfy a different set of people. So it might even not be a matter of strategic difficulty to pinpoint the optimal course of action. It could just be a case of you having different interest and wants than the decision makers. And believe me, it is absolutely not certain that if and when an AI takes over the authority to take and effect political decisions, it will have the same interpretation as you on what Good is. Especially if the means of production and the sources of wealth are AI-automated not in the hands of your social group.

TLDR: Political decisions are strategic and thus very difficult for both humans and machines to get them right. If and when the machines overcome such limitation, politics will probably be the least of our worries."