Writing tools to improve your research productivity

Researchers—at least in Computer Science—spend most of their days at a desk typing. Picking the right software for writing is important.

Most of my writing time is spent on LaTeX documents. I have tried typical Word processors in the past, but they get in my way. Indeed, by mixing document content and document presentation, Microsoft Word makes it difficult to maintain consistency. Word is meant for short-lived (or throw-away) business documents. It is easy to get started and get 80% of the job done with Word. However, as the document gains complexity, as the number of revisions grow, as the number of collaborators expands, Microsoft Word becomes inadequate.

Oh! I still use OpenOffice or Google Docs to produce quick-and-dirty documents. But for anything that is meant to have lasting value, that is research, I refuse to fall into the Word processor trap. It causes some friction with colleagues, but it is a price I am willing to pay.

I believe every single graduate student should learn to write without a word processor. And serious science students should learn LateX. Even if you do not care for LaTeX, at least explore alternatives to Word such as Scrivener.

In any case, you are unlikely to need more than a text editor to write your prose:  Charles Stross, one of the best scifi writer alive, wrote many of his novels using a primitive text editor (Vim). If you have never written without Microsoft Word, how do you know that Word is not holding you back?

Right now, I write using a regular text editor (Smultron for MacOS) and the TeX Live 2009 distribution. I save all my documents to a subversion tree. Using a version control tool such as Subversion makes collaboration easy, and it allows me to go back in time years ago. It is a good setup.

Programming is also a form of writing. For my experimental work, I program in C++, Java or Python, often using Eclipse. I find it is slightly better for programming than my standard writing setup (using only a text editor). Eclipse has great qualities:

  • It stays out of the way. In particular, you can collaborate with people who are not using Eclipse without any problem.  For example, it will happily let you use handcrafted makefiles to compile your C++ programs.
  • It offers incremental compilation of Java programs. Basically, it compiles as you type.
  • It suggests corrections for many common compilation errors.

Essentially, while Java is still an awful language, Java with Eclipse is almost fun. Eclipse proves that sophisticated software can be helpful to programmers and writers.

Writing is hard and it will always be hard, no matter the tool. But at least, ease your pain!

See also Physical tools to improve research productivity.

Published by

Daniel Lemire

A computer science professor at the Université du Québec (TELUQ).

30 thoughts on “Writing tools to improve your research productivity”

  1. Comments in LaTeX are trivial. For one thing, LaTeX has already comments in its language (begins a line with “%”).

    If you prefer something fancy, you can setup commands like this one:

    newcommand{neil}[1]{textbf{{small{color{magenta}DL}: #1{color{magenta}$circ$}}}}

    (This uses the package color.)

    Because your documents are under subversion or a similar version management system, then you can “diff” the documents to see what your collaborators changed. You can setup emails to get notifications when something changed something.

  2. (1) What you describe is not collaborative writing. Merely appending comments on someone’s document is review, not writing.

    (2) Why on Earth would you want to have lines much longer than 80 characters? Your troubles with diff are the least of your worries. Many text editors, such as vim, work poorly with very long lines.

  3. @Neil

    I think we are getting sidetracked. My statements were “every single graduate student should learn to write without a word processor” and “serious science students should learn LateX”.

    I did not write “everyone should learn LaTeX”, and I certainly did not write “everyone should use LaTeX”.

    All I am saying is that you should consider other writing tools than Microsoft Word. If, after trying out alternatives, Microsoft Word is best for you, then fine.

    I have written several research papers and reports without LaTeX.

  4. I prefer lyx but people I know use texmaker and kile too. To each his own 🙂

    People I admire know one editor completely in and out – vim or emacs. IMO, learning a editor such as vim/emacs is the first thing any serious developer/gradstudent/scientist should do.

    I prefer vim with git.

  5. I use Textpad for almost all my editing. I’ve integrated latex with it, and can compile tex files from it directly.

    It’s not as fancy as TeXMaker, but certainly way faster.

  6. How do you do collaborative writing, e.g. comments on papers in progress? I’ve always found Word to excel in this area. For LaTex it seems to boil down to PDF markup.

  7. That’s true, but I maintain the MS Word collaboration features are far superior. Especially with people – *cough* advisors *cough* – who don’t edit Latex.

    Finally, diff with Latex has never been useful for me, since diff is line-based, and I have soft-wrapping – therefore each paragraph is one line.

    This leads to much manual searching to find the change. Have you found something better than traditional diff?

  8. I don’t get it. All your text documents contain carriage returns or line feeds after X characters? Seems like the wrong solution to the problem.

    The right solution is a better diff. Diff was designed for code, where lines >80 chars made it impossible to read on the terminals of the day (and still a bad idea for readability). But we don’t have these restrictions nowadays.

    As for the extent to which advisors and grad students ‘collaborate’, I’ll leave that to Daniel. I still maintain that Word’s least appreciated feature (that I haven’t seen anywhere else) is “Track Changes” and comments. Sure, it can be duplicated with merge/diff, but that answer doesn’t work for a lot of people.

  9. Well, I’ve followed your blog for quite a while and this is really the first item I take issue with …

    I think you’ve jumped into the “gave the GUI a go, didn’t do what I wanted, I’ll just code it” trap that pervades in the world of mathematical / computational science.

    While it’s true that a quick-and-dirty attempt to generate complex documents in Word is most often frustrating and infuriating, the fault lies in the lack of initial document design rather than with the application. I recently completed my PhD in computational biology and, in line with your current topic, I spent considerable time exploring the writing options available to me – yes, I learned LaTeX !! What I found though is that if the document is correctly planned and implemented from the start, then the simplicity with which downstream complexity can be added in a Word document is impressive. This is especially true of the most recent Word incarnation. Collaboration is also remarkably simple if correctly implemented with the track changes and revision functionality. Just for the record, I’m far from a Microsoft proponent, in fact, linux is my OS of choice and I also use openoffice every day for “quick and dirty” documents.

    Ultimately, I’d stick with your ‘check out all the options’ advice, but would tag on ‘learn the fine details of MS Word ‘ as well. For my money, complex documents implemented correctly in Word are traceable, managable, simple, and time efficient and beat LaTeX hands down …

  10. git is much like svn with one crucial difference: it is a distributed VCS. That is, commits are made locally and changes can be pushed to one or more servers. It may seem like a small difference but it has made a big difference to the way I manage my own code and collaborate with others.

    Definitely worth trying. If you don’t like git’s style, mercurial (hg) is another distributed VCS I have heard good things about.

  11. I support wholeheartedly the push for Latex in research. However, as the previous commenter points out, collaborative writing with Latex is not obvious. I would recommend mercurial over subversion for this kind of use, even for personal version control (no need for a server).

    Otherwise, I am an Emacs guy.

  12. @Neil, I used to believe as you – that a better diff was called for – then found that end result is not necessarily better. The simple act of placing an end-of-line at the end of every sentence is oddly semantic. When you review changes, you are interested in added, deleted, or modified sentences. Adjacent modified fragments in two sentences are more likely distinct changes, not one change. The added line break makes that distinction impossible to miss.

    With content laid out one-sentence-per-line it seems a bit easier to scan through changes, as the line breaks add just a bit of semantics to the layout. Also, overlong sentences REALLY stand out – kind of a side benefit. 🙂

  13. I write a lot of my personal notes (not shown to others) in a personal wiki. When a good idea for a paper forms, I do a first draft of the abstract and/or introduction with pen and paper. If it looks good, I type it up in Latex, rewrite a little, and share with collaborators. At that point, we start writing the paper.

    Although I don’t write my papers in my wiki, it is one of my most important writing tools for developing ideas.

  14. I’ll add my recommendation to texmaker, although I do still enjoy an occasional Emacs loaded up with every TeX mode and extension in existence. I tried Lyx for a while and hated it, but each to his own. Also, I’d really love an editor that was like Scrivener, but with a lot of extra LaTeX functionality built in.

    I second the recommendation to use a DVCS rather than svn. It is so easy to set up a local repository for your own stuff even for quite trivial things without the need for a central server. Then you can throw stuff onto the server when it gets serious or when it is time for collaboration. Also, being able to commit when you don’t have a network connection is very handy. The only advantage that svn has for LaTeXing is that someone wrote a package that prints the svn revision number and other details on the .dvi or .pdf file, which can be very useful for keeping track of drafts. I don’t know of anything similar for any of the DVCS systems.

    Personally I prefer Mercurial (hg) as a DVCS for LaTeXing because I think it is much easier to parse than git for people who don’t do a lot of coding. As a physicist, I have to collaborate with a lot of very unhackerish types and it is hard enough to persuade them to use any VCS at all, so ease of use is definitely at the top of my list of requirements. Also, I love how easy it is to throw a repository up on the web via https or http with the included hgweb scripts and because it is Python I find it very easy to write extensions/customizations for hg. I think git is ultimately a lot more flexible and probably faster, at least in its current implementation and undoubtedly contains features that are essential to complex projects, e.g. the Linux kernel. It is definitely the tool of choice for serious hackers but if you are just going to use it for a few LaTeX documents and a few small coding projects then life is a lot easier with hg.

  15. @Itman do you use any computer algebra system?

    I have used maxima in the preparation of a couple of papers, including one upcoming journal article. It is not as fancy as Maple or Mathematica, but it gets the job done very nicely. And there is a great community around it.

    But honestly, it is very rare that my research will lead me to the kind of algebra that requires a computer algebra system. Usually, for me, it is a sign that something went badly.

  16. Never, never, never use Word for a scientific publication. I had an experience with a survey. You simply cannot get the work done.
    1) References are awful.
    2) Tables and charts are awful.
    3) Most important for surveys and large articles: bibliography is almost impossible!!!

  17. LaTeX, Google Docs, OpenOffice, Vim, and Eclipse… definitely a few of my favourite things. I would put a good word in for Mercurial too.

    I have recently heard of writing wikis to assemble notes and I love the idea. I saw some comments here about it and would be interested to hear about the wiki workflow.

  18. I use TiddlyWiki to store my personal thoughts and notes. Not a true wiki in the sense of collaboration and versioning, though, more of a notes application.

Leave a Reply

Your email address will not be published. Required fields are marked *

To create code blocks or other preformatted text, indent by four spaces:

    This will be displayed in a monospaced font. The first four 
    spaces will be stripped off, but all other whitespace
    will be preserved.
    
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks:

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax