Category: Academia/Research

Through Didier and Nielsen, I found a list of Golden Rules for Successful Scientific Research attributed to Dijkstra.

• “Raise your quality standards as high as you can live with, avoid wasting your time on routine problems, and always try to work as closely as possible at the boundary of your abilities. Do this, because it is the only way of discovering how that boundary should be moved forward.”
• “We all like our work to be socially relevant and scientifically sound. If we can find a topic satisfying both desires, we are lucky; if the two targets are in conflict with each other, let the requirement of scientific soundness prevail.”
• “Never tackle a problem of which you can be pretty sure that (now or in the near future) it will be tackled by others who are, in relation to that problem, at least as competent and well-equipped as you.”

Of the three rules, only the last one seems important. The second one appears self-evident: you want to be socially relevant, but not to the point of producing low quality work. This being said, most researchers go to the other extreme and ignore social relevance and their work loses out its motivation. If you tackle a problem that only you care about, don’t expect much recognition. I actually disagree with the first rule: small problems, technical issues actually often hide interesting problems. Always focusing on the management and top level issue is a bad idea I think. Michelangelo was painting a church! In research, do not be so quick to think that there are noble and not-so-noble problems. All problems can be interesting and knowledge of technical issues can bring much insight.

Didier reminded me to check Nielsen’s last post on Principles of Effective Research. I take a quote out of it…

The foundation is a plan for the development of research strengths. What are you interested in? Given your interests, what are you going to try to learn? The plan needs to be driven by your research goals, but should balance short-term and long-term considerations. Some time should be spent on things that appear very likely to lead to short-term research payoff. Equally well, some time needs to be allocated to the development of strengths that may not have much immediate pay-off, but over the longer-term will have a considerable payoff.

This is a refreshing view.

When I started out as a researcher, as a young Ph.D. student, I thought research was about “having ideas”. Then, it occured to me that it was about “having ideas and ‘selling’ them” because “having ideas” is easy and too many people have too many ideas already. But marketing experts sell ideas all the time… surely, they don’t do “research”. Then, I changed my mind and decided research was about “taking ideas, validating them, putting them in practice, and building tools out of it” where “tools” is to be interpreted in a very wide sense. Turns out it is not a bad definition of what research is. But the part about “taking ideas and validating them” is a networking problem. Where do your ideas come from, how do you know how good they are? Ultimately, “validating” an idea means putting it in front of a community and getting the community to say “this is a good idea”. “Validating” is not the same as selling, though it might be hard to tell what a person is really trying to do.

But to be blunt, I don’t have yet a satisfying definition of what “research” is and I’m not looking very hard… though, networking is a necessary condition for sure. Scientists on desert islands without telecommunication can’t do research. That’s the part that I did not understand until a few years after my Ph.D. Well, maybe I’m hard on myself, maybe I understood it on the surface, but I didn’t internalized until much later.

Michael Nielsen pointed me to an interesting Web page very useful for Ph.D. students and novice researchers: Networking on the Network.

In Networking on the Network, Philip E. Agre accurately describes the world of research as a network. A network isn’t good or bad… so, some nodes will suck energy out of the network, and others will contribute much to it. The network is somewhat self-regulating, but it is possible, nevertheless, for bad leaders to emerge… He has this to say about the relationship between students and supervisor which I find rings very true:

It is good to be powerful, but only in the correct sense of the term. People with the right kind of power, in my view, do not need to manipulate or control others. To the contrary, they are (sic) know that they are well-served when others grow and find their own directions, so they happily support everyone in their growth. They don’t take responsibility for others’ growth, which is a different question. They speak to the healthy part of a person, and they are concerned to draw out and articulate the brilliant ideas and worthy vision that lie beneath the surface of whatever anyone is saying. For example, they don’t try to enroll students as acolytes in their empire-building strategies, but honestly ask what’s best for each student’s own development, confident that their knowledge, vision, and connections will have an important influence on the student’s development in any case.

As you can see, he talks a lot about “Empire building”. Indeed, because research is all about networking, to a large extend, one can build an empire out of thin air, with no substance.

It seems you can either build an empire for the purpose of building an empire, because that’s you definition of success, or else, you can aim to remain “free”. That’s a very powerful idea:

You build networks around the issues you care about, you grow and change through the relationships that result, you articulate the themes that are emerging in the community’s work, and through community-building and leadership you get the resources to do the things that you most care about doing. It’s true that this method will never give you arbitrary power. But the desire for arbitrary power is not freedom — it is a particularly abject form of slavery. If you can let go of preconceived plans then you are free: you can choose whom to associate with, and as you build your network you multiply the further directions that you can choose to go. You also multiply the unexpected opportunities that open up, the places you can turn for assistance with your projects, the flows of useful information that keep you in contact with reality, the surveillance of the horizon that keeps you from getting cornered by unanticipated developments, and the public persona that ensures that people keep coming to you with offers that you can take or leave. That is what freedom is, and it is yours if you will do the work.

I give Agre a lot of credit from bringing in the concept of “freedom” in research. University professors will often talk about “academic freedom”. I think that freedom in research is a stronger form of freedom. You can have “academic freedom” but be a slave to the “publish-or-perish” paradigm for the power it brings you. Or else, you can “do the work”, that is, do your research as a network node, and leverage the strength of the network to make the research you want to do anyhow, much better, much stronger.

I’ve been reading Michael Nielsen’s Principles of Effective Research, he is up to Part IV now.

He makes a very important point about research. When I started out doing research, I thought that research was about sitting in your office thinking up new ideas. God! Was I wrong!

Now, don’t get me wrong, research is not about having meetings with other researchers or spending time chatting, or drawing UML diagrams of what is to be done, or spending weeks on funding proposals. We might do these things, but they don’t make us good researchers. But neither will sitting in your office thinking new ideas. That’s not effective research.

On quasi-desert islands with no telecommunications, you’ll find very few great researchers. The social network doesn’t need to be immediate: I think you can be a great researcher even in a tiny school. And I don’t think your network should be made of students mostly, especially not your own students.

I believe the secret to being a good researcher is to belong to a tightly knitted group of solid researchers. Research is about networking. By tightly knitted, I don’t necessarily mean “military-like”: I mean that you feel peer pressure all the time to do good research. This can be achieved through emails, blogging, phone… whatever the mean…

A must read paper in the Chronicle Is There a Science Crisis? Maybe Not. The paper is about the oversupply of graduate students in science which is brought upon by universities who have a vested interest in producing more and more science Ph.D.s but don’t necessarily need to adjust to the job market.

It brings back memories. At the end of the eighties, they were predicting a severe shortage of science Ph.D. As it turns out, it was totally false and the paper documents very well the fact that life after a science Ph.D. has gotten tremendously worse and that there are clearly an ever increasing number of science Ph.D.s with fewer and fewer jobs.

The truth is that universities are being irresponsible (and so are professors). Training highly specialized students who know how to solve one type of technical problems has no value for society. Whatever you do, train students to have a wide range of skills. This means that we need to reduce drastically the number of science Ph.D.s and focus on well-rounded students.

I’m convinced governments will soon wake-up and stop listening to universities. They’ll be forced soon to look at the numbers and figure out that generously paying universities to produce more science Ph.D.s is a waste of tax payer money.

Some beautiful quotes:

An editorial in Science this year argued: “We’ve arranged to produce more knowledge workers than we can employ, creating a labor-excess economy that keeps labor costs down and productivity high. Maybe we keep doing this because in our heart of hearts, we really prefer it this way.”

Mr. Freeman, like other economists, looks to dollars to make sense of the trends among graduate students. “They’re not studying science,” he says, “because they look and say, ‘Do I want to be a postdoc paid $35,000 or $40,000 at age 35, with extreme uncertainty working in somebody else’s lab, and maybe getting credit for my work and maybe not getting full credit? Or would I rather be an M.B.A. and making $150,000 and hiring Ph.D.’s?'”

With wages stagnant and too few jobs for engineers, adding to the work force will only make those careers less attractive, says one of the authors, George F. McClure, a retired aerospace engineer who studies employment issues for the Institute of Electrical and Electronics Engineers. “The problem is that everybody has focused on the supply side, and very few have focused on the demand side,” he says. “People in colleges and universities are concerned with maintaining the pipeline and throughput.”

In a case study, Ms. Stephan, the Georgia State economist, has analyzed the growth of the bioinformatics field, generally regarded as one of the hottest areas in science. The number of degree programs blossomed from 21 in 1999 to 74 in 2003. “There’s been a tremendous increase in the number of students in these programs,” she says. But, she adds, “we also track job announcements in bioinformatics, and they’ve been declining.” She sees parallels to other leading fields. “Everybody is talking right now that there’ll be lots and lots of jobs in nanotechnology,” she says. “I’ve not seen a convincing case that that is happening, or that it will happen.”

The following page points to some research on the impact of latency on gamers and, in particular, on recent research involving Unreal Tournament:The Effects of Loss and Latency on User Performance in Unreal Tournament 2003.

I think this is an absolutely great way to attract students: do research on gaming technology. In fact, I once proposed to NRC that I could do some research on Web porn technology, but my boss (Bruce Spencer) seemed reluctant for some reason to invest government dollars in the porn industry.

I still haven’t given up on using porn technology as a research topic though. However, I think that just like gaming, you’d need to work extra-hard just to justify your research topic.

Which is not to say that my research is not on cool topics. I think that inDiscover is quite cool. I also have other things coming that may appear as sexy to some people (but no porn research as of yet).

Just like art, I think that research should be thought provoking.

Here’s a cool project: OpenTextBook. When I was at Acadia, we often wondered why students had to pay CAN$80 for a Calculus textbook when it was obvious that all such textbooks are the same, they have to be, and all of the content has been known for quite some time.

What are we paying publisher for, exactly? We could never quite figure out, but because, neither the university nor the professor ends up paying the bill, we keep asking students to buy expensive textbooks for no good reason.

To be fair, professors are not evil. Not all of them. Many wondered whether we could write a free calculus textbooks ourselves. I think there are some free textbooks but who would dare trusting them? I would trust a textbook that the community can peer review though. If people find mistakes, they can come in and correct the mistakes. It would not converge into a free, perfect textbooks, but it might end up producing an acceptable textbook, for sure.

I co-wrote this really cool document on Series. It is free! You can have the LaTeX source if you ask or are clever enough to navigate my hidden Web.

One day soon, I’ll co-wrote an open textbook. I should make a note of it.

Interesting commentary by 17th century on the One room syndrome. Many Ph.D.s have lived through this and know the feeling. You have to sit at your chair, in the same damn room day after day after day. You can imagine that what you’ll produce will be like a movie that people will pay to see… Hmmm…. not so in my experience. The reward is mostly just that: you become a “doctor”. Some of the time, it might good for your ego, and it helps getting some kind of jobs and contracts. But it ain’t a movie and it doesn’t bring you any kind of fame (99% of the time).

Once you become a “true” researcher (read: actually paid real money), things change a bit, at least for some of us. Sitting in the same room is just one piece of it, then you have meetings, networking, endless emails, administrative duties, funding proposal, students, and so on. Many people find they no longer have any time to sit in a room, especially the poor folks who get a 4-4 teaching load. Then, you have industry, where you end up doing pretty much what everybody else do. In my case, this meant a lot of programming (and I got good at it too!) and getting stressed out about having real people depend on your work.

My Ph.D. was a rather interesting experience. I basically lived totally isolated. I chose to cut ties from the university and from everyone else so I could focus on my work. Did it work? Yes. Was it a good strategy? Probably not. I would run every day, I would spend too much time on the Web, but I ended up finishing my thesis and it was ok. I was even lucky enough to have a job waiting for me (if you call a post-doc a job, but that’s for another day). I recall that I made all these sacrifices because I thought that the good life was waiting for me after the Ph.D.

Critical Mass started an interesting debate: are teachers underpaid? The argument why they should be paid more seems a bit on the weak side: they have great job security, relatively good benefits like a summer off, and comparatively decent pay. My mother works 30 something hours a week, she has her summer off, and she makes more than I do as a university professor. Oh. And she lives in a less expensive area, has far less education, but admittedly more work experience.

The debate was taken on by O’DonnellWeb who takes the whole issue apart by bringing it back to supply and demand. Getting a university degree in education is relatively easy (he claims), there are plenty of people qualified to teach, and frankly it isn’t such a difficult job: intellectual rigor is not a requirement (he says).

Here’s what I think. The market is a powerful force to set salaries, but only when people can be trusted to do what’s best for them. People will often aim for job security at all cost. That’s when the system fall apart. Once you have tenure, the school could freeze your salary and many people would never leave… sad… and I’ve recently heard someone say “if I don’t get tenure, where else could I get a job?” These comments are interesting to me. Where else could you get a job? What about the average joe who works at company X and company X fires him… is his life over? Not even close!

People in academia are very insecure. My theory is that most of them have been sheltered for so long, that they have no idea how the world works.

Think of yourself as wolf. You’ve spotted a nice forest where there is plenty to eat… then, one day, you have hard time finding new prey, or maybe there are hunters nearby shooting at you, or maybe they are cutting down the forest. Don’t stand there! Move, go! Any wolf would know this. “But where will I find a new forest? What if there aren’t any?” Well, you’ll die then. But you know what? Failure is ok. Getting stuck in a bad spot is ok. As long as you still feel free to go whenever it is best for you. But my skills are not easily transferable? Ah. See, you should have thought of that earlier on: pick new skills carefully, make sure they can serve in many settings otherwise, be willing to pay a price later on.

I think nobody should go straight from school to a tenure-track position. It is a bad choice: many people will then tend to overvalue their tenure which will be detremental to them if they have tenure at a bad place.

Update: I think we could make the world a better place by requiring teachers and professors to have real work experience (outside schools). Right now, we discourage students to get work experience outside universities: I think we should require it. When I was a professor at Acadia, they wanted to make industry experience a requirement. It was a brilliant move and I hope they did it. Everyone would be better off.

When I was a student, I got fairly lucky in funding applications: as an undergraduate student, I got the C.D Howe Memorial scholarship which explains why I have $0 in student debt. Then as a graduate student, I had my own funding all the way till I got my Ph.D. It was nice. Then, things got tougher. The forms got longer. The requirements got increasingly more difficult to meet. In the last few years, I got into “funding application hell” where the forms are extremely long to fill, the competition is fierce. You are supposed to talk about the projects and research you are doing, but of course, you have no time for research because you are stuck filling out forms so that, hopefully, you’ll be able to do your work. My first encounter with really boring funding applications was when I first started as a Research Officer at the National Research Council of Canada. At the time, I applied for NSERC funding which I still have to this day, but also, many other funding applications. I was the Team Leader for the e-Health Research Group and so, I automatically got involved with many funding proposals. I could not believe what it meant: stopping all your research for several months while you have meetings, and write relatively boring documents one after the other. Right now, I’m fairly deep in funding applications and will be for at least another month, and after that, I’ll continuously have to work on funding up until this autumn.

I don’t think that many students are aware of the fact that research for a professor or researcher often doesn’t not mean thinking about new ideas, there is very little time for that, sometimes it doesn’t even mean writting about new ideas… sometimes research means filling out long forms.

It is a well guarded secret, but more than brand new exciting ideas and great papers and great teaching… the modern value of a scholar is often measured by how much money he can attract. Whether he needs a lot of money or not is irrelevant. I’m not complaining about the system, but this is a part of it that people don’t often talk about. You don’t see on a researcher’s home page “just spent the last 2 months filling out funding applications, I hope it will work”.

There is a special issue of JIME on Semantic Web for Education (as in “Learning Objects”). I picked it up from Downes‘ in one recent post.

Not only is the issue interesting, from what I could tell, it is a journal that gets it. First of all, reviews are on-line, for all to see. Don’t get me wrong: I don’t mind peer review. I cherish it. But I’ve gotten too many poorly written, poorly prepared reviews. I really wish reviews would go on-line when the paper is published. This way, it might give an incentive to the reviewer.

Plus, if a poor paper gets accepted, you can trace back the reasons why it was accepted…

It still doesn’t help if your paper gets rejected for bad reasons, but in such instances, you can go elsewhere with your paper. At the very least, you know that if the paper makes it, you’ll have an exciting review to go with it. Useful for you and the reader.

Update: other blogs have picked up this issue of JIME.

Two links that are very invaluable to researchers who want to know how to succeed…

• How to Have a Bad Career in Research/Academia
• HOW TO ENSURE RAPID SUCCESS
  IN A CAREER IN SCIENCE

The second one was found by Seb.

Offline someone commented that more than half of the Ph.D. students are foreigners and that the Ph.D. is serving as a funding source. True. But that’s somewhat of a cynical view if you ask me.

In any case, you are a young student, and despite reading my blog, you still want to do a Ph.D. Maybe because you come from a Third World country and getting some cash to study is a compelling idea on its own. But whatever your reasons, here’s what you should be looking for, I humbly suggest…

• Look at the past projects and students. Did all the students this prof. supervised ended up on welfare? Or can you google them as Harvard professors now? Can you find traces of the past projects this prof. was involved with or did they all fail? Look beyong the fanfare: look for evidence the prof. can’t control easily. Google past students.
• Is the prof. aware of what the world is like, right now? Does he know the employment rate and career possibilities for young Ph.D.s or does he just pretend he knows? Where does he gets his facts from if he has any?
• Does he give you the full story, with the pros and cons of doing a Ph.D. with him? Pros and cons of the research life?
• Does he need to consume graduate students to get his research going or is the training of students only tangential to his research? In other words, can the guy still do research without students or are students cheap labour?

Nice post on Critical Mass today about a researcher who sold his services as co-author on eBay and actually got 50 bids and many phone calls. It would appear that many people, from industry to students, are willing to pay so they can produce high quality scientific content with their names on it.

I’m not sure it is an interesting line of work though. Most people don’t realize how expensive it is to write a good scientific journal article. Probably upward of $50K. It’d be difficult to sell 10 pages for $50K. Of course, there is other types of “research”, like journalistic research, when you get 10 pages for much less… but real science is awfully expensive.

What’s more interesting to me is the reasons why people where interested: “There’s this whole constellation of things they could get from it. They could get credentials. They would get the ability to have their questions actually answered.”

Why do I pick on this bit of news? Because I was actually offered jobs like this, and I always turned them down. I was offered money to write journal articles at least twice by totally different people. It was meant to promote a product or a service, in the end, or rather, give the product or service some credibility. I think this is misguided since there is an actual proper form for such publications: patents, technical reports or white papers.

In any case, it would actually be doable: sell your services as a scientist who publish papers to give credibility to products and services. It would be similar to a patent consultant, I guess, except that law is not so involved anymore. I found a lot of people everywhere think they have very unique ideas. They’d love them to be validated and have their ideas pushed in a very prestigious publication, just like having patents.

Writting papers is like taking pictures for Playboy. You look at beauty most of the time, and you have to capture the beauty… you have to make sure enough is being shown, but not too much. It is seen as a very romantic job where you are living a dream, but are, in fact, just doing your job. The only difference is that few people write papers attracting as many eye balls as Playboy pictures and most earn less money too.

This time, I found stats for Philosophy Ph.D.s. It would appear that the ratio of candidates to job advertize is shrinking quite a bit. I got this from Leiter Reports. In the same blog, we find evidence that 9 out 10 Philosophy Ph.D.s get a cool middle-class job, assuming they went to Princeton.

Well, there seems to be a lot more evidence than I thought that things are improving for new Ph.D.s

As it turns out, though, my claims are also supported by anedotes: Fang and Nielsen.

Does academic research matter?

I’m not a very good historian, but I seem to recall that rresearch as we know it arose out of the German model. It proved invaluable at least in the Second World War. Or did it?

Of course, Tim Berners-Lee owe to academic research some of the ideas that lead to the Web. Some. But not that much, really. Tim Bray is not exactly from academia, is he? Yet, XML changed the world in a deep way.

It seems like academic research is more and more irrelevant… or is it progressively more underfunded, or mismanaged… or just simply totally irrelevant?

Here’s a theory: we’ve come to define success by the number of publications… yet, amazing folks like Tim Bray don’t necessarily go out of their way to submit papers. They listen, they talk, they write within communities and then they publish proposals. They probably hack some software too. So, maybe academic research is becoming irrelevant because we have success wrongly?

Why would the public respect people whose main achievement is a (smallish) number of 10 pages documents they get in books hardly anyboyd ever read.

According to Owen, or maybe, according to what I understand from his email, overproduction of Ph.D.s is a myth. Schools can’t get decent Ph.D. holders.

True. Maybe.

Owen has evidence: CRA Stats.

More precisely, according to this table, 60% of CS Ph.D. holders go to a Ph.D. granting university (on a tenure-track or not?), 4% to a non-Ph.D. granting school, and 29% go in industry. A meagre 2% join the government, and 1% and self-employed.

I stumbled on a channel setup by Seb called Topic Exchange: Channel ‘invisible_adjunct’. I was an avid reader of the invisible adjunct. For those that don’t know, the invisible adjunct was one of the many Ph.D. holders who have a decent publication record and are in every way competent scholars, but they still fall through the system and end up beggars at some university. For those of you not familiar with the context, let’s just say that there is tremendous pressure on professors (like myself) to train more and more and more Ph.D. students.

This comes in part from the government which likes to measure universities by numbers: how many Ph.D. does this university produce… and so on…

Now, of course, if all these graduates are unemployed… well who cares? And who’s going to believe you when you say that Ph.D. holders can’t find jobs? Who’s going to pity them? Surely, they can’t find a job because they want to earn $500k a year? Right.

No. Most Ph.D. graduates are lucky to find a post-doc. A post-doc, in Canada, pays around $30k. Sometimes more, sometimes (amazingly) less. I’m not saying that some of them don’t find great jobs. It happens. But it is statistically insignificant.

So, the invisible adjunct is someone who just gave up. She’s not alone.

Some people are smarter and they leave early, like wolfangel… but many don’t.

I particularly like a recent post by Erin.

Here’s a comment which rings very true:

My experience with a Ph.D program was that the myopic focus on only academic skills was very damaging to the students in my program. Academic who have devoted their whole life to study of a particular subject in an educational setting have little understanding of the learning that takes place on the job or as part of living life.

But here’s some advice from a couple of science Ph.D.s :

There is only one reason to get a Ph.D. — because the career path you want to pursue requires it. Do not do it because you think it will make you feel important, because it will do the opposite. Do not do it because “there are a lot of things you could do with it”, because there are plenty of things you can do without it. Do not do it because you think it will be an intellectual adventure, because you’d do much better with a library card.

I think that all students should read and seriously consider such statements before undertaking a Ph.D. I’m not saying a Ph.D. is a bad thing… but, well, read the quotes above!

There is more:

your odds of getting the PhD are smaller than you think, your odds of getting a job are slighter still, and your odds of getting tenure at a place yet smaller, and then all of this happening at a place you would otherwise choose to live? Infinitesimal.

 

Universities survive because they are political beasts. Most (canadian) universities would have to close down if the government killed its support. By itself, this only means that I expect universities not to gain any funding in the coming years… However, one can worry about a few things…

• Progressively, the value of a degree has gone down. I think it has gone down in academic quality and also in value on the job market.
• New graduates face a tougher and tougher job market.

What will happen next? Will universities thrive because students will need badly more and more degrees? Or will the race come to a stop… and new forms of training will arise?

Here’s a theory… universities did well before Gutenberg because books were so incredibly valuable and so, any community which had books was well off. Then Gutenberg came about. Well, books were still very expensive, so universities still did well because they allowed students to attend lectures and take notes. Then, books became cheaper and cheaper. Knowledge became easier to get to without teachers. I’d say this happened some time ago. Maybe at the beginning of the century…

Why did universities survive past this point? I think because they offered communities. Young, smart people could come together and ideas would just be transmitted like diseases. It was far more efficient than, say, using mail. Universities provide the people and the proximity.

I’d say, most universities still provide this… and they suggest books… However so does the Internet. Well, not quite.. email, for example, is not as efficient as meeting face to face… So, I think we need broadband. Real broadband, without technical glitches. Then, at this point, I predict that universities will be obsolete.

When is that? 5 or 10 or 15 or 25 years. What do you think? Will there be strong, thriving universities in 20 years?

Update: Universities as gateway to knowledge can’t compete against the Internet, but the Internet doesn’t provide authentification for the knowledge we acquire. The Internet doesn’t, by itself, grant degrees. Are universities the best form of institutions to authenticate knowledge?