Cool free H323 videoconference software for Macs: xMeeting

Contrary to what the name suggests, XMeeting is Mac software (and not X Windows software) for videoconferencing. I just tried it tonight to call my friend Martin Brooks at NRC and it worked very well. It even gets around my firewall (NAT) without any problem. Oh? Did I mention it was open source? Looks like some kind of GPL-incompatible FreeBSD-like license.

As with all things, there is a catch under Mac OS with the use of the camera. If you are using the camera in one application, for example, XMeeting, then it becomes unavailabe under other applications such as skype. However, you may get the misleading error message ‘No video camera connected’. Simply ensure that you have only one camera-using application opened at any one time.

On video-on-the-web a.k.a. youtube, is going to change teaching!

My teachers in high school would use lots of short movies. But this was bad as it took up precious time in class, you couldn’t watch it over on your own, the sound was sometimes terrible if you were at the far end of the class, and so on.

These days, the technology has improved by leaps and bounds. Any time this year, you can watch a high quality video by UCR’s Eamonn Keogh on time series data mining. It is free, it is good quality, and it is here, now, for us to use.

Wouldn’t you want to have James Gosling explain the finer points of Java as part of your course on Java?

Having guest lecturers in college requires organization, time, and energy. A guest lecturer won’t repeat its lecture, and every single year, you have to get him to come back. Also, can you get the Turing medalist Jim Gray to come over and explain what a data cube is to your class? These days, these same students can listen to Jim by following a link. They can do it whether you want them to or not.

Having talks available through video-on-the-web is not any different than the use of textbooks, you use someone else’s content in your course, except that it is maybe even cheaper.

I can see this becoming a way for authors to boost the sales of their books: watch me talk for free and, if you like it, buy my book!

The world is changing.

Stock-Picking Computers

In my we-do-not-have-strong-AI-yet category, I found this juicy quote on slashdot:

Investment firms fall over themselves advertising their latest, most esoteric systems,’ said Mr. Lo of M.I.T., who was asked by a $20 billion pension fund to design a neural network. He declined after discovering the investors had no real idea how such networks work.’ Why did he decline? ‘I’ll build a stock market neural network for a couple thousand (you’ll just need five thousand years of market data to train it before it actually works).

I was approached myself to work with a company that did stock picking by computer. When I suggested they look at having the computer help the human being make better decisions, they dismissed the whole notion as being silly. Why help the human being when the machine can take over? Why have so little ambition?

Well, the truth is, now wait for it, computers are less intelligent than human beings! I can’t believe I need to write down something like this, but it is fortunate that there are still Machine Learning researchers (at MIT) who do not fall in love with their topic to the point of living under the illusion that it is easy to mimick human intelligence.

Now, you’ll excuse me, I have my daily batch of spam-that-got-through-12-layers-of-machine-learning-spam-filters to delete.

Source: Slashdot.

Why Fink is broken for anything but non-X applications

Update. Since I wrote what follows, I’ve managed to find ways to install X11 and KDE and all my favorite software with minimal pain. So Fink is not broken, but it could be friendlier. Same holds for Apple’s X11. In fact, Fink is an incredibly powerful tool when combined with something like KDE. It extends Mac OS X in incredible ways.

Fink is a tool to install most common Unix applications (gnuplot, Xfig, you name it!) on MacOS. It has a nice and simple command line interface which I love.

My friend Scott Flinn seems to maintain that Fink is a good solution, even if you want to run X applications (such as XFig, but he went so far as to include KDE appllications!!! Daring!!!).

Ok. One wasted afternoon and one wasted evening later, I beg to differ with Scott.

(Disclaimer: I think that MacOS X is about 10 times better than Windows XP. I’m very happy I bought Apple gear. But when things are poorly layed out, poorly organized, someone has to say it. Even when volunteers do the work.)

Let’s review some of the claims people make:

  • Fink supports X11. This is true, but you end up with a primitive X server, without any sensible integration with the OS. I can have the same deal with cygwin/X for Windows, and who uses that? I read somewhere that MacOS was a Unix platform. If it can’t do better than Windows at emulating a real Unix box, well, I will not be impressed.
  • Apple supports X11. This is true, but not by default, and you need to be clever.

    First of all, do not confuse “MacOS X Server” (which is expensive payware to run MacOS version 10 as a server OS) with “support for X11”. I don’t even want to think about what will happen down the line when Apple comes out with “MacOS XII” (Mac OS 12) and offers “Mac OS XII Server”. If your font is slightly bad, you might be lost at this point. Short story: Apple sells a Server OS which has nothing to do with an X server.

    Second of all, Apple has a nice and convenient download page for its X11 server. Except that it says: Note 10.4 customers can install X11 by using the Tiger DVD installer disk. What this means, in Apple talk, is that if you have 10.4, you need your Tiger DVD. It won’t tell you this even as you try to install the freshly downloaded package on your machine: it will tell you that you already have “more recent software installed” (which wrongly suggested to me that X11 was already installed!). Yes, I know, they don’t say “must install”, they say “can install from DVD”, but apparently, for Apple, if you can do at thing, you must do a thing. Did you lose your DVDs? Too bad. Ok, turns out that in my case, I still have them. I put the “Tiger” DVD in (which, interestingly, is not called “Tiger” at all, but “Mac OS 10.4”). Then a window pops up with a few icons, mostly a README, and two reinstallation option. I try to reinstall the bundled application, reboot. No luck. Ok, now I go for the full reinstall of the whole OS. Still no luck. At this point, I reinstalled my OS and rebooted twice. Now, wait… Oh! When you put the DVD in, in the window that opens, if you scroll all the way to the bottom, you see “optional installs”. Ah! I run this! Ah! X11. Ah! Cool. Looks like I installed it, but there is no instruction, nothing. The installer just quits without telling me anything. Now, where is that X11 thing? It has to be on my machine now! In Finder, under Applications, maybe? No. Nowhere to be found. Hmmm… Did it install the thing? Why won’t it leave a trace? Searching for X11.app in Finder gives me nothing. I search for “X11”. Ok, now, beside a bunch of files derived from Fink, I have something called X11 as an application. Where and how I’m supposed to find it usually is beyond me. (Later on, the X11 application appeared under “Applications / Utilities”, but how was I supposed to guess this?) I launch X11 and it looks good. I get some kind of shell. I can’t test it yet, but it looks hopeful. At this point, my afternoon is completed and I must go home.

  • Fink supports Apple’s X11. This would be the best of both worlds. Fink provides the applications, and Apple provides an X server with a nice integration to the OS. Unfortunately, this is painful and can be a lenghty process.

    Here is what fink tells me when I want to install an X application:

    You have an existing X11 installation in /usr/X11R6 and/or /etc/X11.
    This package refuses to overwrite these. Remove them, then tell Fink to
    install xfree86 again. (The package won’t be recompiled.) If you want
    to keep your X11 installation, please see the FAQ entry at
    http://fink.sourceforge.net/ faq/ usage-packages.php#apple-x11-wants-xfree86
    for more information on how to configure your system.

    The link basically says this:

    If you have a current version of fink (>=0.18.3-1), typically what you need to do is reinstall the X11User package, since the installer application occasionally misses installing a file. You may need to do this multiple times.

    Right there, I have to stop. What is X11User? I’ll save you the trouble and about 3 hours wasted. X11User is the very thing I installed from the “Tiger” DVD. Turns out that you can do the install from a shell, so let’s do it again, and again, and again. (They say to do it multiple times. They say that the application sometimes does not install all files? What a piece of software engineering!!! Way to go Apple!)
    cd "/Volumes/Mac OS X Install Disc 1/System/Installation/Packages"
    sudo installer -pkg X11User.pkg -target /

    Ok. I did it about ten times to be sure. Every time the installer tells me “The upgrade was successful”. Now, I’ll save you the trouble, but later on, we learn that we better have something called X11SDK. It is not clear that I need this, but let’s be safe. (Earlier, I reported, on my blog, that “X code”, which is “Apple talk” for “dev-tools” was installed.)

    cd "/Volumes/Mac OS X Install Disc 1/Xcode Tools/Packages"
    sudo installer -pkg X11SDK.pkg -target /

    Ok, so far so good. I repeat this about ten times, just to be sure given the, apparently well known, randomized behavior in Apple’s installers. Now, I’m told that “fink list -i system-xfree86” should output some pseudo-packages. It outputs nothing except “Information about 4816 packages read in 1 seconds.” Given that I have two processors in this machine, I’m not impressed that it takes 1 second to parse 4816 packages, but I presume that Fink is not written in assembly. Fine.

    Not all is lost! Fink tells me that if nothing works, I can try to “flush out my X11 installation and remove any old placeholders and partially/fully installed X11-related packages”. Oh! Ok. The command line is long and it remove my precious X11 application (this is scary! what else does it remove?), plus it tries to update Fink using what appears to be a very slow pipe (I have selected the rsync update method after the CVS method just hung there for over two hours doing nothing). What gets to me here is that clearly, Fink is in error, why do I need to remove Apple’s X11 to fix Fink? Can we say “bad engineering”?

    Ok, now I try to rebuild from the ground up:

    cd "/Volumes/Mac OS X Install Disc 1/System/Installation/Packages"
    sudo installer -pkg X11User.pkg -target /
    sudo installer -pkg X11User.pkg -target /
    sudo installer -pkg X11User.pkg -target /
    sudo installer -pkg X11User.pkg -target /
    sudo installer -pkg X11User.pkg -target /
    sudo installer -pkg X11User.pkg -target /
    (repeat many times)
    cd "/Volumes/Mac OS X Install Disc 1/Xcode Tools/Packages"
    sudo installer -pkg X11SDK.pkg -target /
    (repeat many times)

    Now, I run the damned command “fink list -i system-xfree86” and get…


    i system-xfree86 2:4.4-2 [placeholder for user installed x11]
    i system-xfree86-dev 2:4.4-2 [placeholder for user installed x11 development tools]
    i system-xfree86-s... 2:4.4-2 [placeholder for user installed x11 shared libraries]

    I’m hoping this is good. Short story: Fink is trying too hard to be smart and ends up making you work like a madman.

  • Now, let’s try to install a few things:

    fink install gnuplot
    fink install xfig
    fink install kile

    The first two work, but the last one won’t install. for the following reason:


    dpkg: dependency problems prevent configuration of autoconf2.5:
    autoconf2.5 depends on autoconf (= 2.60-4); however:
    Package autoconf is not installed.
    /sw/bin/dpkg: error processing autoconf2.5 (--install):
    dependency problems - leaving unconfigured
    Errors were encountered while processing:
    /sw/fink/dists/unstable/main/binary-darwin-i386/devel/autoconf_2.60-4_darwin-i386.deb

    Ok, but so, maybe I have gnuplot working, right? Let’s see…


    > gnuplot
    dyld: Library not loaded: /usr/X11R6/lib/libfontconfig.1.dylib
    Referenced from: /sw/bin/gnuplotx
    Reason: Incompatible library version: gnuplotx requires version 1.0.4 or later, but libfontconfig.1.dylib provides version 1.0.0
    /sw/bin/gnuplot: line 6: 24936 Trace/BPT trap /sw/bin/gnuplotx "$@"

    Hmmm… that can’t be good. For completeness, here’s the content of my proud /sw/etc/fink.conf file.


    Basepath: /sw
    RootMethod: sudo
    Trees: local/main stable/main stable/crypto unstable/main unstable/crypto
    Distribution: 10.4
    ConfFileCompatVersion: 1
    Mirror-apache: http://www.apache.org/dist
    Mirror-apt: http://bindist.finkmirrors.net/bindist
    Mirror-cpan: ftp://ftp.cpan.org/pub/CPAN
    Mirror-ctan: ftp://tug.ctan.org/tex-archive
    Mirror-debian: ftp://ftp.debian.org/debian
    Mirror-gimp: ftp://ftp.gimp.org/pub/gimp
    Mirror-gnome: ftp://ftp.gnome.org/pub/GNOME
    Mirror-gnu: ftp://ftp.gnu.org/gnu
    Mirror-kde: ftp://ftp.kde.org/pub/kde
    Mirror-master: http://distfiles.master.finkmirrors.net/
    Mirror-rsync: rsync://master.us.finkmirrors.net/finkinfo/
    Mirror-sourceforge: http://west.dl.sourceforge.net/sourceforge/
    MirrorContinent: nam
    MirrorCountry: nam-us
    MirrorOrder: MasterFirst
    ProxyPassiveFTP: true
    UseBinaryDist: true
    Verbose: 1
    SelfUpdateMethod: rsync

    Maybe my problem is that I now use a binary distribution? I don’t know where the “UseBinaryDist: true” came from as I remember setting up fink to use a source distribution and seeing packages compile (must be the reinstallation that was required of me earlier that wiped my setting). Ok. I’ve changed the setting to false, but uninstalling and reinstalling packages does not seem to recompile them.

    So, maybe now I should wipe out all of fink and start again from scratch? Will this even work? Maybe I should try “fink cleanup”? Will it help?

    Fink should learn from portage which also has binary packages. By default, portage always rebuilds packages. This should be the sane behavior. Maybe “fink rebuild gnuplot” will do what I want? Here’s what I get…


    New package: dists/unstable/main/binary-darwin-i386/text/ghostscript_8.54-3_darwin-i386.deb
    Failed: phase compiling: readline-4.3-1028 failed
    (...)
    Note that many fink package maintainers do not (yet) have access to OS X on
    Intel hardware, so you may have better luck on the mailing lists.

    What??? You mean the guys who maintain this have no idea if it even builds and no incentive to find out? Gosh! Maybe this explains why none of this works.

    Hours go by…

    Ok, I tried rebuilding various packages in various orders and things eventually build. I’ve got no idea what will end up working tomorrow though. And it is very scary the way the dependencies are not automated. (Gnuplot now works though, but it did work before this whole mess.) Next, I’m trying this before going to be late: “fink rebuild kile gimp gnumeric kopete swig”.

Unrelated but important nonetheless: My oldest son is 3 years old from now on. No, he can’t help me yet with my computer problems, but he can find the TV remote from time to time which is pretty useful.

Do not think you have to go to college

There was a day when books were expensive. Then they became cheaper during the last century. Getting in touch with peers was still expensive for a time, but now, you can call, message and email people very efficiently for next to no money. But lectures were still expensive for a time too. Except that now, you can watch, online, for free, lectures given out by Turing Award recipients. It is only a matter of time before someone figures out a better way to get an education than attending 3 hours of lecture in a dirty classroom every week. This model came out because books, chats, and lectures were expensive. It is no longer the case. We need to move on.

In fact, the infamous Will Richardson says you don’t have to go college:

For most of your young lives, you’ve heard your mom and I occasionally talk about your futures by saying that someday you’ll travel off to college and get this thing called a degree that will show everyone that you are an expert in something and that will lead you to getting a good job that will make you happy and make you able to raise a family of your own someday. At least, that’s what your mom and I have in our heads when we talk about it. But, and I haven’t told your mom this yet, I’ve changed my mind. I want you to know that you don’t have to go to college if you don’t want to, and that there are other avenues to achieving that future that may be more instructive, more meaningful, and more relevant than getting a degree.

Stephen Downes has nailed it with a very sensible question:

(…) are the positive aspects only possible via traditional schools? Could kids obtain (vs. create) these positive aspects without the need for traditional schools?

The answer, for me, is clearly yes. We absolutely need to sit down and rethink our educational models. They are obselete.

A few days with MacOS X

As a newcomer to the Apple world, and an avid Linux user, I thought I would share my impressions after a few days. As a disclaimer, I have spent too much time in meetings this week to actually get to know MacOS X in details, but still…

  • Apple has done a good job at capturing the best of Unix and free software. After installing xcode (provided by Apple for free), you have the GNU compiler and most things you need to do free software work.
  • Fink is useful to get a few Unix tools you miss. Don’t bother with unstable packages. Don’t bother with any X application.
  • Pretty much everything under MacOS X can be done remotely using a shell. Even when Apple only says “double-click on this or that to install”, you pretty much can get away with the shell. However, unlike traditionnal Unix systems, there are many scripts missing in MacOS. For example, you can’t quickly delete or add users by default. You need to create or borrow your own scripts if you want to do this. This is slightly annoying, but would be easily fixable if a bunch of people got together to write scripts and provide an nice installer.

To sum it up, I could see a world where both Linux and MacOS dominate and that would be a beautiful world.

My first Mac

Today I finally received an Apple MacPro I ordered several months (!!!) ago. I thought I would quickly review my first impressions.

  • The machine is sexy. There is no other word for it. My Linux box looks like a russian car (not insult intended toward the russian folks) in comparison.
  • The machine is fast, but it takes forever to boot up. It takes longer to boot than a slow Linux box that has 12,000 boot-level services.
  • Took me some time to find out where the console is, but once you have it, you can create a link on your desktop. The console is good, comparable to what you have under Linux. It looks like it uses bash by default. However, I do not seem to have color support in the shell. No hint anywhere how to turn it on. (Update. Will says to check an article on macosxhints for the color support, but says he prefers iTerm.)
  • The on-disk help is pretty good. Clearly, Apple cares about helping you find a solution to your problems. Unlike Microsoft who is happy to confuse you into depression. (And Linux, well… Linux has Google as documentation…)
  • The machine comes with lots of preinstalled software. I don’t know what any of it does so far, but it looks like Apple is not cheap software-wise.
  • The keyboard French Canadian layout sucks. It differs from the established standard found in both Windows and Linux.
  • I could not find the equivalent of the “Home” and “End” keys. I still don’t know where they are. This means that it takes five minutes to select the content of a text box.
  • Having the menu all the way to the top of the screen is really a drag when you have two screens. When my application is on the second screen and I need to go in its menu, I have to go back to the first screen, move up and click on the menu.
  • It took two of us about 15 minutes to even find out if I had a DVD reader. Turns out I do, but it does not appear anywhere. I can open it by pressing one of the keys on the keyboard (the “eject key”).
  • Setting up a ssh server was not too hard. It looks like I can manage my Mac from home just like a Linux box. So far so good. Though I don’t have gcc up and running yet. My main problem is that the connection speed with my lab. is not great, but the sysadmin, Mihai, says it will get better.
  • You can configure the mouse so that you have an actual right button. Very nice. You can even configure it as a 3-button mouse. Excellent! Those of you who don’t know why you need 3 buttons clearly are missing on some great classical software such as xfig.
  • Installing Firefox (first thing I did) created some kind of “mounted disk” that now resides on my desktop. When trying to put this useless icon in the garbage can, the machine complained that it could not unmount the disk. Which disk? I suppose that what I downloaded was some kind of disk image that MacOS mounts as a virtual disk. Fine. But how do I get rid of it now that Firefox is installed? There must be a trick to it.
  • Security seems weak. It appears that I can install everything using my initial account. No root account? (Or maybe I have both a user and root account? I’m confused.)
  • The second thing I installed was Fink. Fink is the MacOS equivalent of “apt-get” (debian) or “portage” (gentoo) or “rpm” (redhat/mandriva). Took me some doing to get it running, and it seems very useful, though, by default, very few packages are available. I tried moving to CVS access which opens up many more packages, but it said, quite rudely, that I needed something called dev-tools. Alas, doing sudo apt-get install dev-tools fails with a comment to the effect that there is no such package. The command sudo fink install dev-tools is more informative as it tells you to go and register as a developer with Apple. You are supposed to guess that dev-tools is “Apple talk” for a package called xcode. I did find it, sold my soul to Apple, and now I’m downloading a huge image of what I hope is the dev-tools thing. This file is really gigantic (1GiB!). By the way, I do this remotely so I had to do sudo apt-get links to use the links browser (links is really a good browser). So far so good. I just hope I’ll be able to mount the disk image I’m downloading through my ssh access. The command hdiutil attach allows one to mount dmg files. It looks like cd /Volumes/Xcode Tools; sudo installer -pkg XcodeTools.mpkg -target / will install Xcode without any need for a GUI. Oh! And fink install python cvs svn gnuplot gnuplot-py xfig kile tetex transfig anacron numeric wine pdftk imagemagick swig koffice kopete looks like a decent way to start. I still don’t know whether it will work, but there is a detailed page on building KDE on Mac.

References:

Are Smart People Dumb?

Scott asks whether Smart People Dumb? He tells the tale of when he joined Mensa and found out that, while these people have high IQ, they may not be the winners you expect them to be:

It turns out that the people who join Mensa and attend meetings are, on average, not successful titans of industry. They are instead – and I say this with great affection – huge losers. I was making $735 per month and I was like frickin’ Goldfinger in this crowd. We had a guy who was some sort of poet who hoped to one day start “writing some of them down.” We had people who were literally too smart to hold a job. The rest of the group dressed too much like street people to ever get past security for a job interview. And everyone was always available for meetings on weekend nights.

I don’t know whether I’m a high IQ person, but I can tell you that I’m the typical nerd who has trouble to manage his life. It is a miracle if I have a beautiful wife and great healthy kids (actually, it is really a miracle, but I’ll tell this tale another day). Over time, I have managed to find tricks to manage my life so that it does not suck. But they are just that, tricks. The real me is very bad at the basics of life.

Money for example. I’m really bad with money. Not so much because I spend it all. In fact, spending money is work, so I tend to be very frugal. Still, I can’t manage a budget. I never know how much I have left in my bank account. My wife recently decided to handle our finances and I bet this is making me much richer.

Getting organized for example. I’m terrible at structuring my work in any way, shape or form. I improvise all the time. This, initially, made me a terrible teacher, but I have since learned ways to compensate. This has also made me a terrible researcher, initially, but again, I have found ways to compensate for this weakness.

Now, I sure hope that I have a high IQ, because otherwise, it would be depressing to have all these weaknesses without the brain power that goes with it. It would be like being Superman without the superpowers, but with the kryptonite weakness. But I sure don’t feel supersmart, so I wouldn’t rule it out.

In vitro kids are smarter

This has been know for some time, but in vitro kids have an higher IQ than the average (whether IQ is an accurate measure of intelligence is a debate better left for another day). Even advanced in vitro techniques lead to smarter kids.

So, people, if you want to have smart kids, in vitro is the way to go!

Consider, also, as a factor, that in vitro mothers are more likely to keep their kids at home with them. Some have argued that in vitro kids suffer from delayed social development. Yet, interestingly, they don’t suffer for delayed intellectual development, quite the opposite apparently.

I think we still need to learn a lot about what makes a smart kid.

RSS hygiene and Google Reader

It used to be that Google’s RSS reader was a catastrophe, but it is now pretty good and pretty smart. Google has turned things around. I’ve switched to it two days ago. The UI is still trying a bit too hard for my taste, but I guess you have to account for the fact that it is still slightly experimental.

Oh! And can we, please, get over the round corners? I actually like rectangular shapes. The Web 2.0 look where everything is round and smooth is really a fad and we will look back on these Web designs, in five years, with a fair amount of disgust.

The really nice thing about Google Reader is that when you scroll down past an item, it marks it automatically as read. At least, this is the default behavior. I found that one annoying thing with most readers is how they either required me to manually marked items as read, or they used a “delay then mark read” approach. I much prefer the Google Reader approach.

See http://www.google.com/reader/.

In general, about RSS/Atom feeds, I have learn that you should not maintain more than 25 or so feeds. It is quite tempting, especially when you are bored or stressed out, to add more and more feeds. I find it is better to migrate and slowly change your list of feeds. Let us call this “RSS hygiene“. You don’t keep 50 books open on your desk, do you? You do not follow 50 TV shows, do you? You do not read simultaneously 50 novels, do you? Why is 25 a good number? Because, on an average day, you will have no more than 5 new posts to read, maybe 10 at the outmost.

One side effect of limiting yourself to few feeds is that you tend to go for quality. What about diversity? What about the long tail, you ask? I think you simply outgrow some of the feeds over time and will naturally replace them. If you are like me, your interests change over time and so will your feeds.

And this whole argument justifies the fact that “feed recommender systems” have never really picked up any steam. People don’t want lots of feed proposals, not most of the time. They want to carefully choose any new feed, and that is not something that can be done automatically.

Querying the library of congress using Search/Retrieve via URL

SRU (Search/Retrieve via URL) is an interesting REST Web Service protocol.

Enough technobabble. Let’s run an example.

Suppose you want to retrieve the data that the library of congress has on a book called “First Impressions of the New World” by “Trotter Isabella Strange”, you issue the following query (follow the hyperlink for the XML result):

(dc.title=”First Impressions of the New World”) and (dc.creator all “Trotter Isabella Strange”)

You want to use this in software? Download my corresponding Perl and Python code examples: srucodeexamples.zip.

Further reading: See the wikipedia entry or even better, check the refbase entry.

(Special thanks to Owen Kaser for making me discover this exciting new technology.)

Looking for an all-recording plug-in for Firefox

Dear reader,

I’m looking to record everything I ever browse on the Web using Firefox. That is right. I want a copy of every single document, Web page, query, and so on, I ever encounter. I also want to record the content of every single form I submit. The result should be adequately protected so that it is not possible to have access to the data without access to my machine. It would be akin to the wayback machine except that it would cover only the stuff I have seen. I have not yet decided whether it should record all the youtube videos I watch, but I guess it should.

Ideally, it should be able write the data on a portable disk.

Why do I want to do this? Why not. Actually, I would then want to use this data to build a fancy database that would support things like drill-down or roll-up queries (à la OLAP).

If you know how to build this, want to help, or know of such a plug-in, please drop me a line.

I’m also running a competition to decide how to call such a thing. I initially thought about calling it a webex but it turns out that’s a trademark. Then I decided that Hammerspace might be a better name, or maybe Magic Satchel? What do you think?

90% accuracy for translation software?

The USA Today research projects whose goal is to deliver, by 2010, software that can almost instantly translate Arabic and Mandarin Chinese with 90 to 95% accuracy. I’m no natural language researcher, but I’d be interested in knowing how they measure accuracy. I have seen some recent results on automated translation (French-English) and the results were alright some of the time, but even if you only screw up one word out of ten, is this good enough?

To anyone who has spent weeks crafting a ten pages paper: do you think you could get away by screwing up one word out of ten? Communication is a difficult task. It requires high accuracy most of the time.

Yes, I’m being hard. Automated translation is good and interesting work, but let us remain critical of the possibilities. While we lack strong AI, we cannot hope to replace human translators.

My Erdős number is 4

Fascinating. My Erdős number is 4. According to wikipedia, the median Erdős number amongst all working mathematicians at the turn of the millennium is 5 and the average is 4.64. In this case, being below average is good. Here is how it happened: Paul Erdös coauthored with Janos Galambos who coauthored with Eugene Seneta who coauthored with Serge Dubuc (my thesis cosupervisor).

Reinventing HTML or, yes we admit it, XHTML failed

Very interesting post by Tim Berners-Lee, the inventor of the Web.

Some things are clearer with hindsight of several years. It is necessary to evolve HTML incrementally. The attempt to get the world to switch to XML, including quotes around attribute values and slashes in empty tags and namespaces all at once didn’t work. The large HTML-generating public did not move, largely because the browsers didn’t complain. Some large communities did shift and are enjoying the fruits of well-formed systems, but not all. It is important to maintain HTML incrementally, as well as continuing a transition to well-formed world, and developing more power in that world.

I say this as one of the few people in the world who spent a year building an XHTML/XML web site: it ain’t worth it. As long as XHTML is the end product, and as long as browsers don’t care too much what you feed them, then XHTML does not and will not matter.

I always assumed that XHTML would find applications as a data source, but alas, it is not and will never be a good data source. I will keep on producing valid XHTML code whenever convenient, because working with XHTML is just better than working with tag soup, but there other formats, such as HTML 4.0, which are better than tag soup.

What is infinite storage?

A colleague of mine, a Ph.D. in Physics, objected to my use of the term “infinite storage” in some lecture notes I posted on the Web.

I think that infinite storage is something that might be possible in my lifetime. What does “infinite storage” means? Let us consider how much is required to achieve the mythical (digital) memex.

  • To record everything you read in a year requires 25MB.
  • To record everything you hear in a year requires 100GB.
  • To record everything you see in a year requires 10TB.

Hence, I argue that whenever I will be able to buy a cheap 10TB disk at my local electronics store, I will have infinite storage. Currently a portable 1TB drive can be had for $569.

What about recording everything I hear? Right now, I can buy a slick 100GB portable drive for $160.

Interesting question: how would I ever search through all these sounds?

And what about all these people who will get upset that I am recording them?

(Reference: Jim Gray, What Next? A Dozen Information-Technology Research Goals, Journal of the ACM, Vol. 50, No. 1, January 2003.)