The amazing thing about this YouTube talk on scalability is that they use MySQL, Apache, and Python. One key comment he makes is that replication does not necessarily scale so well, not unless you also partition your data.

There are Web 2.0 applications to do almost anything. But I just found out that there does not seem to be any UML Web 2.0 application out there. I must admit I am not a big fan of these types of software engineering techniques, but from time to time, you do have to sketch out the design of your application and UML-like diagrams are a sane way to communicate your idea. But UML, almost by design, is meant to be a shared view of your design. This makes it a prime candidate for Web 2.0.

Sounds like a good start-up project.

Update. Eran accurately points out that Gliffy may do the job. Indeed. Nevertheless, there is still, I think, lots of open room for Web 2.0 applications having to do with programming. I’d love to do an entire non-trivial programming project using only Web applications (excluding any desktop application other than my browser).

Engineer and scientists are among the few professionals who work to make themselves increasingly obselete. Scientists try to capture as quickly as possible the most useful results. Engineers try to make product design and manufacturing so efficient that scarcity is irrelevant.

For all purposes, we live in a world of abundance. This means that nobody, at least in North America, is really running out of bread or milk. There are exceptions, of course, but they are artificial and usually the results of broken politics: even if we produced more milk and bread, these people would still go hungry.

Regarding scientists, I think it is fair to say that we never had access to so much high quality research. It suffices to hop on Google Scholar to find hundreds of articles on even the most pecular topic. In turn, these articles might be locked away from people who do not have a subscription, but that’s an artificial scarcity considering the ridiculously low cost of distributing electronic documents.

The more abundance we observe, the more scientists and engineers have to move to new niches, to fill lesser needs. At first, you build the telephone, then techniques to transmit digital data through the wires, then compression techniques to transmit even more data, and finally, you start worrying about which protocol you are going to use to transmit a particular piece of data.

I think there might be another way. Another route for scientists and engineers. I think we should focus on overabundance. In other words, the minute we solve scarcity problems, we almost automatically create overabundance problems. Obesity comes to mind. Pollution and waste management are other examples. The more we produce, the larger are the problems we create, and the more work we have. You first find a way to transmit 24,000 bauds of data, then 10 times more, then 1,000 times more. Finally, you have a virtually infinite bandwidth. But voilà! You get 1 terabyte of spam per day!

Since I do research in Computer Science, I have to wonder what are the overabundance problems we have. Here is a first list:

• We never had access to so many documents. When I was a kid, I was constantly frustrated that I only had a few books in my house, and that these books routinely didn’t have answers to my questions. As a college student, I was frustrated that the library didn’t have all of the references I needed. These days are pretty much over. My kids will have access to many, many more documents than they could ever read. They will almost never be frustrated that the reference is not available to them; they will be frustrated that they can’t find the reference they are looking for. This is the Google realm.
• As a kid, there were basically two TV stations I could watch. Maybe three. And going to a movie was an event. Buying a book meant going to a (physical) bookstore, checking out what was on the shelves, and possibly buying it. Now, I can download dozens of TV shows on my laptop and watch them whenever I like. Same could be said of music: we are quickly exiting the era of the top ten list. I almost always buy books through Amazon. I buy the books I care about, not what a bookstore put on its shelves. Personalization is here to stay. Personalization is hard. Automatically composing content for a given user is really difficult. I have done a lot of work on collaborative filtering and I must review a paper per month on this topic. Yet, I feel that collaborative filtering falls short to solving the personalization problem. User models are also not so exciting. Social networks are maybe more exciting. In any case, I predict that we are going to see a significant breakthrough in this area within 10 years. And everyone will be very excited.
• As a college student, I was frustrated by the lack of choice in my education. Though I attended what must be the largest university in Canada, most courses listed were never offered. I think that, soon enough, we will give our students access to a much, much larger range of courses and training tools. I think that Education will go through profound changes in the near future.
• We never had access to so much software. The number of Web 2.0 applications is exploding and several of these tools are too specialized to be integrated in the Google or Yahoo portfolio. However, life is not getting automatically easier. These applications often do not work well together. Simply logging on twelve different applications on a given day is a major pain. I predict that we will soon see Web applications mashups: tools that help you integrate different tools. The era of meta-Web 2.0 tool is about to happen.
• I have never connected with so many people so efficiently. I can video-chat live with almost anyone at any time. However, I have no good tool yet to manage these social relationships. Even Computer Science is only barely addressing the social networking problems, and I feel they are too quick to transpose it as a graph problem. I do not care about the social graph I live in, I only care about my immediate subgraph and how I can derive more benefit from it.
• It used to be that data was expensive to acquire, store and process. Now? We have so much data that most of us tend to ignore it and play our jobs by ear. Or maybe the data is difficult to find in the giant mess. Storing and backing up the data used to be expensive: no longer. Processing the data used to require powerful machines: my current laptop can probably outgun most high performance computers from the nineties. Why are we still ignoring much of that data? Because spreadsheets and visualization software are inadequate. They simply do not scale well to the amount of data complexity we currently face.
• Bandwidth (with some latency) is becoming infinite. Soon enough, I will be able to broadcast myself in high resolution throughout the world. There might be glitches, delays and so on, but the data will get through. This is sure to create nasty new problems.

Some recruiters are looking for OLAP experts in Montreal. Knowledge of MDX and other Microsoft OLAP technologies required. Knowledge of French probably a must. Get in touch with me.

(No, I do not get a commission or any benefit whatsoever. I am just glad when there are jobs in an area where I do research.)

(Yes, my email address is somewhere on this site. No, it is not hard to find it.)

I’m far behind in sorting out my pictures, but here are two good ones. First, my wife and my son Lohan:

This is my youngest son, Louka:

I’m the luckiest man in the world!