I dont think we need the singularity for machines to take over. The systems that run society are bigger than any of us and nobody can hold every single one of them in their head. If some emergent property or bug cropped up, most people would keep doing what the computer says, because theyre a small fish and we do what the system tells us. Whos the master now?
The Machine Fired Me[0] is the best written account on how people do not question the systems. Even though nominally the machine has (collective)human masters, they are hard to find when needed; the individuals will do as instructed by "the system".
We already see the beginning of it. Some support person tells you "the computer won't let me do that" to rectify some harm done by the programming. I've run into it all the time. I've read about it happening (recently Google Pixel support completely f'ing up an order and it taking a post on HN to fix).
The weird thing is that big organizations just seem to give up when trying to fix these types of things.
Imagine what it will be like when ML is feeding the software values that are broken, wrong, or dangerous. How will they fix those problems? Especially when corporations often lose the people that wrote the applications, and the people behind them have little knowledge or skill.
