Why I think a lot of Machine Learning research is misguided

Suresh has this to say about some Machine Learning algorithms (neural nets and genetic algorithms):

I dislike them greatly though because they are extremely general hammers that typically don’t have provably performance or running time guarantees, and don’t reveal any interesting insights into the structure of a problem.

He then goes on to state an instance of the “No Free Lunch” theorem that says that all generic learning algorithms are the same.

To me, the main argument against some generic Machine Learning algorithms is that they “don’t reveal any interesting insights into the structure of a problem”. In some way, some of Machine Learning research has become a form of anti-science. Science is about finding patterns, going down under the surface and make some underlying, often hidden structures, emerge. Science is not about solving problems; that’s what engineering is for.

In French, “researcher” is “chercheur” which could be translated as “searcher”. My father used to joke that he would much rather hire a “finder” than a “searcher”. Back as an undergraduate, I tended to agree with him: researchers should be people solving hard problems. Now, I would disagree. My function, when I do research, is not to solve problems. Solving the problems is only a secondary side-effect to my research work. One way to do research happens to involve the process of solving a problem, usually using the simplest and most elegant approach possible.

5 thoughts on “Why I think a lot of Machine Learning research is misguided”

  1. I think you misquoted me slightly. My beef is with neural nets and genetic algorithms, rather than with “machine learning algorithms” that a quick read of your post might imply

  2. Well, I have the same feeling of ML methodolgy, not only limit to NN and GA. ML has developed a set of testing data and testing context, so that people can compare their algorithms objectively. So what? You can’t compare your algorithm with all the existing improvements on the existing algorithms. So to declare your algorithm is better has no meaning. You have to tell me “interesting insights into the structure of a problem”. I like Daniel’s “solving problem is engineering work”. Exactly what subtle in my mind.

  3. Daniel, are you familiar with Douglas Hosftadter and friends’ work on microdomains ? Tabletop, letter spirit, copycat, metacat, etc. ?

    I’ve read all his books, exchanged a few times with him over email, but he’s almost never quoted (nor does he quote much of the usual scientific papers, which might explain why…)

    Anyhow, I love his work.

Leave a Reply

Your email address will not be published. Required fields are marked *

To create code blocks or other preformatted text, indent by four spaces:

    This will be displayed in a monospaced font. The first four 
    spaces will be stripped off, but all other whitespace
    will be preserved.
    
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks:

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax