I dislike them greatly though because they are extremely general hammers that typically don’t have provably performance or running time guarantees, and don’t reveal any interesting insights into the structure of a problem.
He then goes on to state an instance of the “No Free Lunch” theorem that says that all generic learning algorithms are the same.
To me, the main argument against some generic Machine Learning algorithms is that they “don’t reveal any interesting insights into the structure of a problem”. In some way, some of Machine Learning research has become a form of anti-science. Science is about finding patterns, going down under the surface and make some underlying, often hidden structures, emerge. Science is not about solving problems; that’s what engineering is for.
In French, “researcher” is “chercheur” which could be translated as “searcher”. My father used to joke that he would much rather hire a “finder” than a “searcher”. Back as an undergraduate, I tended to agree with him: researchers should be people solving hard problems. Now, I would disagree. My function, when I do research, is not to solve problems. Solving the problems is only a secondary side-effect to my research work. One way to do research happens to involve the process of solving a problem, usually using the simplest and most elegant approach possible.