Education in the XX^th century has been primarily industrial: organize the workersstudents in groups under the supervision of a managerteacher.

We all have been in such systems for so long that we take it for granted. How else is anyone to learn? Maybe some can learn differently, but most can’t because they are unmotivated and lazy, they lack the critical skills to differentiate right from wrong on their own and they can’t assess their own level of expertise. At least, that is what I’m told, but I think it is unfair.

To me, this is like saying that we have to keep long-time prisoners in jail because they do not know how to organize themselves when given their freedom.

Indeed, if students who went through years of schooling cannot learn on their own, if they cannot assess their own progress, and if they generally cannot organize themselves without supervision, we have to wonder whether schools bear part of the blame. And I think they do: we enroll students in supervised and regimented systems where they are constantly told what to do, constantly tested by others and where they have to follow rigid rules as to what they should learn. It is no surprise that many students cannot work on their own when they leave school.

There are a few broken individuals who never really became adults. They have to be kept in check all the time because they could not survive on their own. But if these constituted the essential part of the human race, we would have gone extinct a long time ago. Our ancestors, not long ago, had to survive in small bands hunting small animals and grabbing whatever they could eat. They had to be incredibly resilient because human beings spread throughout the globe like no other animal species.

To put it bluntly, most people lack autonomy, they can’t be entrepreneurs, precisely because we have carefully beaten it out of them. I have two young kids and they are crazy. One of them is building a castle out of paper in his room. The project is huge and complicated and has worked on it for days, on his own, without anyone telling him what to do. He made mistakes (which he explained to me) and he had to fix them. How often do schools let students embark on self-directed projects? Almost never.

My sons are not exceptional. Like other kids their age, they behave in unconventional ways, trying crazy things on their own, having crazy thoughts on their own. Eventually, with enough schooling, they will settle down and do as they are told in a more reliable manner. They will become very good at following directions.

How good will they be at emulating someone like Steve Jobs, who repeatedly broke all rules? I fear for them that their sense of initiative and wonder will be killed by the time they finish their schooling. (Thankfully, I am a crazy dad with crazy ideas, so maybe I will mitigate the damage.)

Hence, as a teacher, I reject the industrial model as much as I can. I believe that, in an ideal world, we would not need any teaching at all. There is hardly anything you can’t learn through an apprenticeship. For example, if you just helped out Linus Torvalds for a couple of years, you could become an expert programmer. In fact, I suspect you would fare much better than if you just took programming classes.

The problem with apprenticeship is that it scales poorly. How much patience will Linus Torvalds will have for kids who hardly know anything about computers? How many could he coach? Would he want to have kids over at his house while he is coding?

We still use the apprenticeship model in graduate school. But to accommodate most students, I still haven’t thought of a better model than setting up classes. But should the classes be organized like factories with the teacher acting as a middle-manager while students act as factory employees, executing tasks one after the other while we assess and time them? I think not. My teaching philosophy is simple: challenge the student, set him in motion, and provide a model. I try to be as far from the industrial model as I can, while remaining within the accepted boundaries of my job. I have two rules when it comes to teaching:

• Focus on open-ended assignments and exams. Many professors are frustrated that students come in only for the grades. Probably because they focus on nice lectures and then prepare hastily some assignments. Turn this problem on its head! Focus on the assignments. If your students are not very autonomous — and they rarely are — give several long and challenging assignments (at least 4 or 5 a term). Do make sure however that they know where to get the information they need. Provide solved problems to help the weaker students.

  However, keep the assignments open ended. We all like to grade multiple choice questions, but they are a pedagogical atrocity. In life, there is rarely one best answer: assignments should reflect that. In some of my classes I use “programming challenges”: I make up some difficult problem and ask the students to find the best possible solution. Often times, there is no single idea solution, but multiple possibilities, all with different trade-offs. Quite often the students ask me to be more precise: I refuse. I tell my students to justify their answer. Over the years, I have been repeatedly impressed by the ingenuity of my students. Many of them are obviously smarter than I am.

  What about lecture and lecture notes? They are secondary. In most fields, the content, the information, is already out there. It has been organized several times over by very smart people. Books have been written on most topics. There is a growing set of great talks available on YouTube, Google Video and elsewhere. Your students do not need you to rehash the same content they can find elsewhere, sometimes in better form. Stop lecturing already! Just link to what is out there and encourage your students to find more using a search engine. Only produce content when you really cannot find the equivalent elsewhere. Please link to material beyond the grasp of most of your students: they need to know the limit of their knowledge.

  The famous software engineering guru Fred Brooks agrees with me:

  The primary job of the teacher is to make learning happen; that is a design task. Most of us learned most of what we know by what we did, not by what we heard or read. A corollary is that the careful designing of exercises, assignments, projects, even quizzes, makes more difference than the construction of lectures.

  For my years as a student, I hardly remember the lectures. They were overwhelmingly boring. And I soon learned that even if a teacher was remarkably able and he could give me the impression that I understood everything… this impression was quickly falsified when I tried to work the material on my own.

• Be an authentic role model. Knowing that someone ordinary, like your professor, has become a master of the course material means that you, the very-smart-student, can do the same. That’s the power of emulation.

  When Sebastian Thrun gave his open AI class at Stanford, tens of thousands of students enrolled. Sure enough, the Stanford badge played a role in the popularity of the course, but ultimately, it is Thrun himself, as a role model, that matters. He has now left Stanford to create his own independent organization (Udacity). Thrun must be confident about his success since he left his tenured position at Stanford, reportedly because he cannot stand the regular (industrial-style) teaching required at Stanford. One upcoming course is “programming a robotic car”. I have no idea how good the course will be, but it will be motivating for students to attend the class of the world’s top expert in the field of robotic car.

  The status of the teacher as an expert has always been important. However, the ability of people like Thrun to reach thousands of people every year through his teaching means that there is less of a market for teachers who aren’t impressive AI researchers.

Unfortunately, as long as I teach within a university, there are a few things I am stuck with:

• Deadlines: Some students are able to go through the material of a class in 4 weeks. Others would need 16 months. Alas, universities have settled on a fixed number of weeks that everyone must follow. If you complete the course faster, you’ll still have to wait till the end of the term to get credit. If you need more time, you will have to make special arrangements. Of course, schools follow the factory model: we can’t have workers come in and finish whenever they want. But outside an industrial setting, I think that deadlines are counterproductive. If I take a class in computing theory and end up proving that P is equal to NP, but I end up my paper a few weeks after the end of the course, I will still fail. Meanwhile, the good student who followed the rules but showed a total lack of initiative and original thinking will go home with a great grade. What do we reward and what do we punish?
• Grades: Grades are a very serious matter in schools. Denis Rancourt, a top-notch tenured physicist at the University of Ottawa, was fired after refusing to grade his students. (He would give A+s to everyone.) Grades are effectively the quality control mechanism of schools, where students are the product. Somehow, we have totally integrated the idea that we could sum up an individual by a handful of letters. It sure makes managing people convenient! It all fits nicely in a spreadsheet. Of course, students have adapted by cheating. Schools have reacted by making cheating harder. But I cheated all the way through my undergraduate studies getting almost perfect score in all classes. How? I discovered a little trick: at the University of Toronto, all past year exams were available at the library. If you took time to study them, you soon found out that, at least in the hard sciences, a given professor would always use the same set of 10 to 20 questions, year after year. So all you had to do was to go to the library, study the questions, prepare them, and voilà! An easy A. But it is all rather pointless. In theory, grades are used by employers to select the best students, but serious employers don’t do this. We use grades to select the best candidates for graduate school, but I doubt there is a good correlation between grades as an undergraduate and research ability. I know two top-notch researchers who have admitted getting poor grades as undergraduates. For years, I have served on a government committee that awards post-doctoral fellowships: I am amazed at how poor the undergraduate grades are at predicting how well someone might do during his Ph.D. Conversely, I have seen many graduate students who had nearly perfect scores throughout their undergraduate studies who are totally unable to show even just a bit of initiative. They do well as long as you always give them precise directions.

Credit: Thanks to Michiel van de Panne for the reference to Brooks’ quote.

Further reading: Making universities obsolete by Matt Welsh, an interesting fellow who left his tenured position at Harvard to go work in industry.

Disclaimer: Many people are better and more sophisticated teachers than I am. And the industrial model does work remarkably well in some settings. Yet I think that they the skills it fails to favor are increasingly important. We have to stop training people for factory jobs that are never coming back.

Many papers in Computer Science tell the following story:

• There is a pre-existing problem P.
• There are few relatively simple but effective solution to problem P. Among them is solution X.
• We came up with a new solution X+ which is a clever variation on X. It looks good on paper.
• We ran some experiments and tweaked our results until X+ looked good. We found a clever way to avoid comparing X+ and X directly and fairly, as it might then become obvious that the gains are small, or even negative! We would gladly report negative results, but then our paper could not be published.

It is a very convenient story for reviewers: the story is simple and easy to assess superficially. The problem is that sometimes, especially if the authors are famous and the idea is compelling, the results will spread. People will adopt X+ and cite it in their work. And the more they cite it, the more enticing it is to use X+ as every citation becomes further validation for X+. And why bother with algorithm X given that it is older and X+ is the state-of-the-art?

Occasionally, someone might try both X and X+, and they may report results showing that the gains due to X+ are small, or negative. But they have no incentive to make a big deal of it because they are trying to propose yet another better algorithm (X++).

This process is called citogenesis. It is what happens when the truth is determined solely by the literature, not by independent experiments. Everyone assumes, implicitly, that X+ is better than X. They beauty of it is that you do not even need for anyone to have claimed so. You simply need to say that X+ is currently considered the best technique.

Some claim that science is self-correcting. People will stop using X+ or someone will try to make a name for himself by proving that X+ is no better and maybe worse than X. But in a business of science driven by publications, it is not clear why it should happen. Publishing that X+ is no better than X is an unimpressive negative result and those are rarely presented in prestigious venues.

John Regehr made a similar point about our inability to address mistakes in the literature:

in many cases an honest retrospective would need to be a bit brutal, for example to indicate which papers really just were not good ideas (of course some of these will have won “best paper” awards). In the old days, these retrospectives would have required a venue willing to publish them, (…), but today they could be uploaded to arXiv. I would totally read and cite these papers if they existed (…)

But there is hope! If problem P is a real problem, for example, a problem that engineers are trying to solve, then you can get actual and reliable validation. Good software engineers do not trust research papers: they run experiments. Is this algorithm faster, really? They verify.

We can actually see this effect. Talk to any Computer Scientist and he will tell you of clever algorithms that have never been adopted by the industry. Most often, there is an implication that industry is backward and that it should pay more attention to academic results. However, I suspect that in a lot of cases, the engineers have voted against X+ and in favor of X after assessing them, fairly and directly. That is what you do when you are working on real problems and really need good results.

Credit: This blog post was inspired by a comment made by Phil Jones on Google+.

Whether you submit your work scientific journal or just post it on a blog, you can expect to receive harsh criticism from time to time. Sometimes you are facing arrogant or ignorant readers. Other times, your work is genuinely flawed. My own work is frequently flawed, as you know if you read this blog.

Over time, I have learned that even if the reviewer is wrong, spending time to careful respond can be tremendously useful. If you are 100% correct, then you get to build up your confidence and can later answer similar criticism hastily. Very often, however, you did not do everything perfectly. Maybe your arguments and data are correct, but you might have presented them better.

There are specific strategies to deal with harsh reviews:

• Expose yourself regularly to criticism from total strangers. In my experience, if you rarely publish, you are more likely to have difficulty dealing with criticism. I have been called an idiot, I have had to deal with overly aggressive people and I have been ridiculed on occasion. Of course, I occasionally get depressed after receiving harsh criticism, especially if I thought I had produced great work and feel unappreciated, but I am typically able to recover mentally in minutes or, at least, hours. Part of it is just habit: my brain has learned that harsh criticism does not necessarily signify upcoming pain.
• It is critically important to distinguish yourself from your work. If someone repeatedly produces inferior work, his reputation will suffer. However, everyone (even Nobel prize winners) gets it wrong from time to time. It is important to keep in mind that most reviewers do not care that much about you. In fact, they often quickly forget about you while you ruminate over their review.
• The best way to address criticism is to take it one comment at a time. If someone finds ten different flaws in your work, don’t look at it as one message: break it into ten components and address each one separately. This approach scales up linearly: it just take ten times longer to address 10 flaws than one. Brian Martin describes it well:
  

  I’ve found a way to make the revision process easier. I don’t reread my text, because that just cements my previous approach. Instead, I go through the recommendations of the referees and the editor one by one, making changes. After I finish all those changes, large and small, I print out the whole article and read through it, fixing up expression and making it flow.

  Tackling recommendations one by one is important psychologically. Looking at a list of criticisms, sometimes pages of them, can be demoralizing; the task seems too big. Focusing on a single point is easier. Once it’s done, you can check it off and proceed to the next point, either immediately or tomorrow.

  Sometimes responding to a point requires additional work, such as obtaining and reading some new theory or doing some new calculation. It’s helpful to write down every step that’s required – for example, (1) order Smith’s book, (2) read the theory section, (3) write a one-paragraph summary – and tackle them one by one.

Open access journals make articles freely available. Some of them even allow the authors to keep the copyright of their work. It would seem that they offer a compelling alternative to traditional journals, especially if you hope to reach to people outside academia.
However, open access may allow you to get a free copy of an article, but your rights might still be limited. For example, videos on YouTube are freely available, but you are not allowed to copy or reuse them freely.

The directory of open access journals gives a list of over 300 open access journals in Computer Science. Thus, finding an adequate open access journal where you can submit your work is relatively easy.

However, there are a few sore points.

1. Indexing of open access Computer Science journal is generally weak

A journal needs to be indexed so that your fellow researchers can find out about your work. Most open access journals will be indexed by Google Scholar, but other indexes are important in Computer Science such as DBLP and the ACM Digital Library. Scopus is also often used by hiring and promotion committees. (Scopus is run by Elsevier.)

As I review the open access journals in Computer Science, I find that indexing is often a sore point. The next table shows that the ACM Digital Library does a poor job at indexing open access journals. In fact, I could find only two open access journals indexed by ACM. It cannot be explained by the prestige of the respective journals: some of these open access journals that ACM fails to index are just as good or better than others it indexes. And, of course, no ACM publication is open access. Quite clearly, ACM is doing little to help open access.

Elsevier and Springer allow authors of papers in some regular journals to make them available under an open access format in exchange for a one-time fee. Their journals are typically well indexed so they may offer good alternatives.

2. Many open access Computer Science journals require complete copyright transfer

To publish an article, a journal does not require complete copyright ownership. The only valid justification for requiring that the author gives away his copyright is to restrict access. When reviewing open access journals in Computer Science, I see that several of them inexplicably require complete copyright transfer:

Conclusion There is still much room for progress.

There is a growing list of famous scientists who have pledged to boycott Elsevier as a publisher. If I were in charge of Elsevier, I would be very nervous: academic publishers need famous authors more than the famous authors need the publishers. After all, famous scientists could simply post their work online, and people would still read it.

Elsevier has committed too many sins to give an exhaustive list: they have created fake academic journals so that pharmaceutical corporations could claim that certain facts appeared in a journal, they have sponsored evil regulations, and they have restrictive views on what constitutes fair use. Unbelievably, they were also involved in arms trade. They probably have the devil on their board of directors.

The boycott is currently lead by a famous mathematician, Timothy Gowers. Gowers accuses Elsevier of charging exorbitant prices for its journals.

Focusing solely on database-related journals, I decided to look at how much journals charge per article.

Observations:

• The price distribution appears almost random. I can see no relation between prestige or paper length and prices.
• Elsevier is hardly alone at charging high prices for papers. Wiley and Springer are just as expensive. Of course, it is possible that Elsevier ends up charging more through deals and bundling.
• ACM is very inexpensive on a per-article basis. However, ACM often asks the authors to pay page charges whereas Elsevier rarely does in my experience.
• Though SIGMOD Record is limited to short contributions, its price is unbeatable. And it has no page charge. Moreover, it is generally a well regarded publication venue among database researchers.

My take: The evidence is strong that high-quality inexpensive journals are possible. Current journals are up to an order of magnitude too expensive. However, Elsevier is selling what we want to buy: prestigious journals that people outside the best schools cannot afford. Just like middle-income Americans get into debt to keep up with the top 1%, colleges increase their library budgets to keep up with Stanford and Harvard.

The solution to overpriced journals is to reduce library purchasing power. Most colleges do not have the infinite budgets Harvard and Stanford have, and they should not act like they do. In fact, if we could reduce the purchasing power of most libraries to zero, then researchers and students would be forced to pay $20 or more per article. You can be quite certain that they would mostly read the cheaper (and more competitive) journals. And Stanford researchers want to be cited by the researchers from the lesser institutions so they would also migrate away from overpriced journals. Reduced budgets would still allow publishers like Elsevier to make generous profits, but they would only profit by offering great services at an affordable price.

Disclaimer: I am currently reviewing a paper for Pattern Recognition (an Elsevier journal), and I recently published in Discrete Applied Mathematics (another Elsevier journal).

Update: Though you can get articles from SIGMOD Record for free if you to the SIGMOD Record home page, ACM sells them through its Digital Library for over $10 a piece.