Suresh has this to say about some Machine Learning algorithms (neural nets and genetic algorithms):
I dislike them greatly though because they are extremely general hammers that typically don’t have provably performance or running time guarantees, and don’t reveal any interesting insights into the structure of a problem.
He then goes on to state an instance of the “No Free Lunch” theorem that says that all generic learning algorithms are the same.
To me, the main argument against some generic Machine Learning algorithms is that they “don’t reveal any interesting insights into the structure of a problem”. In some way, some of Machine Learning research has become a form of anti-science. Science is about finding patterns, going down under the surface and make some underlying, often hidden structures, emerge. Science is not about solving problems; that’s what engineering is for.
In French, “researcher” is “chercheur” which could be translated as “searcher”. My father used to joke that he would much rather hire a “finder” than a “searcher”. Back as an undergraduate, I tended to agree with him: researchers should be people solving hard problems. Now, I would disagree. My function, when I do research, is not to solve problems. Solving the problems is only a secondary side-effect to my research work. One way to do research happens to involve the process of solving a problem, usually using the simplest and most elegant approach possible.
I edited the post. Sorry.
I think you misquoted me slightly. My beef is with neural nets and genetic algorithms, rather than with “machine learning algorithms” that a quick read of your post might imply
Well, I have the same feeling of ML methodolgy, not only limit to NN and GA. ML has developed a set of testing data and testing context, so that people can compare their algorithms objectively. So what? You can’t compare your algorithm with all the existing improvements on the existing algorithms. So to declare your algorithm is better has no meaning. You have to tell me “interesting insights into the structure of a problem”. I like Daniel’s “solving problem is engineering work”. Exactly what subtle in my mind.
Daniel, are you familiar with Douglas Hosftadter and friends’ work on microdomains ? Tabletop, letter spirit, copycat, metacat, etc. ?
I’ve read all his books, exchanged a few times with him over email, but he’s almost never quoted (nor does he quote much of the usual scientific papers, which might explain why…)
Anyhow, I love his work.
Robin: No, I wasn’t familiar with Douglas Hosftadter but through his entry in wikipedia, I learned a lot about him just now.