SOAP leads to strongly coupled, poorly scalable, and bandwidth hungry solutions?

Here’s some comments by Joe Walnes on his experience with SOAP. The scary thing is that he comes to exactly the same conclusions as I did on my own… Any SOAP supporter out there wants to answer these:

On the last system I worked on, we were struggling with SOAP and switched to a simpler REST approach. It had a number of benefits.

Firstly, it simplified things greatly. With REST there was no need for complicated SOAP libraries on either the client or server, just use a plain HTTP call. This reduced coupling and brittleness. We had previously lost hours (possibly days) tracing problems through libraries that were outside of our control.

Secondly, it improved scalability. Though this was not the reason we moved, it was a nice side-effect. The web-server, client HTTP library and any HTTP proxy in-between understood things like the difference between GET and POST and when a resource has not been modified so they can offer effective caching – greatly reducing the amount of traffic. This is why REST is a more scalable solution than XML-RPC or SOAP over HTTP.

Thirdly, it reduced the payload over the wire. No need for SOAP envelope wrappers and it gave us the flexibility to use formats other than XML for the actual resource data. For instance a resource containing the body of an unformatted news headline is simpler to express as plain text and a table of numbers is more concise (and readable) as CSV.

Victor Shoup’s A Computational Introduction to Number Theory and Algebra

Through Didier, I got to Victor Shoup’s Home Page. He has an on-line textbook called A Computational Introduction to Number Theory and Algebra. It is unclear whether he intends the textbook to remain free, but it is pretty cool to post the book on his home page. Shoup’s is an expert in cryptography.

The Edu-Blogger: ITI: Stephen Downes keynote

From the wish-I-was-there Department, here’s a review of Stephen Downes’ keynote at ITI.

There are few people that can be called “visionary”. I’ve met very few. Very few can pass my tests over and over because often, you discover they had one idea and the rest is just fluff or posturing. Stephen is the real thing. That doesn’t mean he makes a lot of friends in the way. However, maybe the people who should listen to him just don’t understand him and that’s why he doesn’t get shot.

Marketing will never be the same

Cringely points out that Apple is slowly making the normal retail and marketing process obselete.

This is the end of the RIAA and the big recording industry. Apple in the last year has signed deals with more than 300 independent record labels, most of them not big enough to do much promotion. But now they don’t have to because that promotion will be handled by mtv.com and every music web logger, now that they have a material incentive to make recommendations and print lists. If I recommend a song — IF I JUST TYPE A FEW WORDS — and a thousand people decide to download based on my recommendation, heck, I just made $50 bucks. This is like sending tens of thousands of record sales people out on the road except that they can sell anything THEY like — any of the one million iTunes songs — making them salespeople with real conviction and maybe even with good taste. Maybe.

To me, this is extremely interesting. When the dot-com era started, people began talking about the new economy. It was a catchy phrase, but it turned out to be wrong. There wasn’t a new economy, yet, but mostly an extension of the old one using new tools. However, the new economy is slowly emerging out of the burning ashes of the old one. Here is what is being transformed forever and dramatically: marketing and distribution channels. I think we are moving to a more distributed world. And I have the nagging feeling that Internet publishing will be the core element. We will buy and sell according to what we read and experience on the Web. Who controls that? Right now, the rising force are blogs. Blogs are essentially distributed publishing units. This is where the future lies, maybe.

PlanetMath a free/better alternative to Mathworld?

Mathworld is a mathematical encyclopedia on the Web. Up until now, I thought it was the only one. I was a bit annoyed at having to use Mathworld because it is owned by the Mathematica people and so, you never know when they won’t pull a Microsoft on you.

Didier (who I wrongly assumed to be from France initially) pointed out PlanetMath. The cool thing about PlanetMath is that the content is great and released under GPL. This means that they won’t pull a Microsoft on you! You can copy the content and redistribute it if you so wish. They can close their servers, but the data itself is free, free to go with someone else, free to be reproduced, free.

Living with the fear of failure

Before you start wondering: no I did not fail at anything today. In fact, my life is rather smooth going and while you routinely get bad and good and not so good and not so bad reviews from time to time, all my projects are proceeding forward better than I had a right to expect.

But like so many people, I’m haunted by the constant fear that I may fail. I was reminded of how hard it is by the pressure some Canadian athletes have reported feeling at the Olympics these days. Constant fear of failure is hard because even if your life is beautiful and you succeed in everything, you are still focused on possible failures. Ok. I’ll admit. I’m a pessimist. Or rather, a realist living in a bleak world.

Why do I fear failure so much? Failure is a neutral or even positive force. In fact, many times when I failed, I’ve actually been glad of the failure and found positive things in it… I don’t know… You might not get in the school you want, but you end up getting in an even better school. You do not get to see the movie you wanted to see, but you get to see an even better movie.

I suspect that there is a little cave man in me who fears he’ll get eaten by a dinosaur (yes, I know, I’ve watched too many Flintstones). Failure might be really bad… like having your feet in a dinosaur’s mouth and expecting the dinosaur to start eating you up.

What I know for certain is that fear of failure is a negative force in most of my life. It distracts me. Pulls me away from my family. Makes me dumber. Takes my eyes away from the road and on the ravin where my car will end up.

If you attend all classes, you pass…

Two profs allegedly got fired because they refused to grade students based on “effort” instead of results. Not that I think that recognizing effort in the grading is such an evil thing… and maybe the policy was even acceptable… Saying that students attending all lectures will pass the course might have its advantages… but the fact that the fellows were fired tells us something about the state of education in North America right now… I think there is clearly a downward spiral as far as the academic level goes. Not that I think it is necessarily bad.

It is a bit troubling in the following way however. If Internet is making information more widely available as before, and the university is no long the holder (and certainly not creator) of knowledge… I was thinking that universities could still authenticate knowledge: provide proof to someone that you do, in fact, know about archeology. But I forgot that academic levels have been going down in the last 20 years or so. So what will remain?

Someone commented in one of my earlier posts that universities are good at organizing knowledge. Knowledge might be readily available through Google, but it isn’t validated or organized very well. I guess, this is true: university professors are pretty good at determining what is sensible knowledge, with the unavoidable mistakes and bias. We are also pretty good at organizing it in a sensible fashion. However, time and time again, studies show that students overwhelming enrol in courses and degrees, not to learn, but for the recognition they get. They don’t care so much about the work professors do to organize and validate knowledge. If we lower the academic levels further, could it be that students will just leave universities? I think that if we ever reach the tipping point where corporations lose confidence in the training students receive, and this day is around the corner, we’ll be in trouble.

A Theory of Strongly Semantic Information

Thanks to my colleague Jean Robillard, I found out that philosophers do Knowledge Management too! Following a request I made, Jean suggested I read an Outline of a Theory of Strongly Semantic Information by L. Floridi.

He starts out by asking how much information is there in a statement? Well, in a finite discrete world (the realm where Floridi appears to live), you can reasonably define “information content” in terms of how many possibilities the statement rules out. For example, if my world is made of two balls, each of which can be either red or blue, so my world has 4 possible states, and I say that “ball 1 is blue”, there are only 2 possibilities left (ball 2 is either red or blue) so I could say that I’ve ruled out 2 possibilities and so my information content is 2. If I say “both balls are blue”, my information content is 4. You can see right away that a self-contradictory statement (“ball 1 is blue, both balls are red”) rules out all possibilities as well, so it has maximal information content. A tautology (“ball 1 is either blue or red”) has 0 information content. Floridi is annoyed by the fact that a self-contradictory statement has maximal information content.

In section 5, he points out that statements are not only either true or false, but they have a degree of discrepancy. So, for example, I can say that I have some balls. This is a true statement, but with high discrepancy. However, I can say that I have 3 balls when in fact I have 2 balls and while false, this is a statement with lower discrepancy, and maybe a more useful statement. Apparently, he borrows this idea from Popper, but no doubt this is not a new idea.

He comes up with conditions on a possible measure of discrepancy between -1 and 1. -1 means that the statement is totally false and matches no possible situation (“I have 2 and 3 balls”), 0 means that you have a very precise and true statement (“I have 2 balls”), and 1 means that I have a true, but maximally vague statement (“I have some number of balls”). What he is getting at is that both extremes (-1 and 1) are equally unuseful, but that things near zero are equally useful (either false or true). Let’s call this value upsilon.

Then, he defines the degree of informativeness as 1-upsilon^2.

This solves the problem we had before. The statement “ball 1 is blue, both balls are red” will now have an upsilon value somewhere between -1 and 0, so it will have some degree of informativeness, but nothing close to the maximal. The statement “ball 2 is either red or blue” will upsilon = 1 and so will have a degree of informativeness of 0. Finally, “ball 1 is blue” will have upsilon positive but less than 1, and possibly close to 0, so that it will have a good degree of informativeness.

Cool RDF tools

RDF is everywhere it seems: from Dublin Core to RSS, all to way to FOAF… However, it can be quite painful to parse. Cool tools are starting to emerge however, but google is not yet very good at finding them.

Suppose you have a RDF/XML representation and you want the triples… go to W3C RDF Validation Service and it will do it nicely for you.

On the other hand, the form on this page allows you to go from N3 (the user friendly RDF syntax) to RDF/XML.

How to be creative

Through Downes’, I found this great post about how to be creative. HOWTOs are always interesting and sell magazines, but they are somewhat more interesting in blogosphere because someone you can get to know put his heart into it.

  • Ignore everybody
  • Creativity is its own reward
  • Put the hours in
  • If your biz plan depends on you suddenly being “discovered” by some big shot, your plan will probably fail
  • You are responsible for your own experience
  • everyone is born creative; everyone is given a box of crayons in kindergarten
  • Keep your day job
  • Companies that squelch creativity can no longer compete with companies that champion creativity
  • Everybody has their own private Mount Everest they were put on this earth to climb
  • The more talented somebody is, the less they need the props
  • Don’t try to stand out from the crowd; avoid crowds altogether
  • If you accept the pain, it cannot hurt you
  • Never compare your inside with somebody else’s outside

One of the most interesting one is number 5: Nobody can tell you if what you’re doing is good, meaningful or worthwhile. The more compelling the path, the more lonely it is.

Of course, I don’t buy all of it. Being extremely lonely is no way to be creative I think. Nobody gets awfully creative at the bottom of a cave. I do think you have to look for others. The strength of your network is key because it multiplies your own brain power. I guess we go back to Emerson’s independence of solitude. Be in a network, be in a crowd, but do not be a mere node in the crowd, be your own node. It does require courage though, and you have to expect to fail, fail badly even.

Great Hackers

Paul Graham wrote an essay called ‘Great Hackers‘. I’m pretentious enough to call myself a hacker (though I do not claim to be great), so I had to jump on it!

Here are some juicy quotes…

Good hackers find it unbearable to use bad tools. They’ll simply refuse to work on projects with the wrong infrastructure.

Great hackers also generally insist on using open source software. Not just because it’s better, but because it gives them more control.

They [great hackers] work in cosy, neighborhoody places with people around and somewhere to walk when they need to mull something over, instead of in glass boxes set in acres of parking lots.

There’s no way around it: you can’t manage a process intended to produce beautiful things without knowing what beautiful is.

And this is the reason that high-tech areas only happen around universities. The active ingredient here is not so much the professors as the students. Startups grow up around universities because universities bring together promising young people and make them work on the same projects. The smart ones learn who the other smart ones are, and together they cook up new projects of their own.

If you’re worried that your current job is rotting your brain, it probably is.

Collaborative Filtering Java Learning Objects

Through Downes’, I found an interesting paper on the application of collaborative filtering to e-Learning in ITDL (by Jinan A. W. Fiaidhi).

It makes the point quite well that we must differentiate heterogeneous settings from sane laboratory conditions:

Searching for LOs within heterogeneous repositories as well as within collaborative repositories is far more complicated problem. In searching for such LOs we must first decide on appropriate metadata schema, but which one!

Nielsen’s Extreme Thinking

Blogging is a fascinating past-time. Who would have thought? I just read bits and pieces of an essay on Extreme Thinking.

Here’s a fascinating quote:

The key to keeping this independence of solitude is to develop a long-term vision so compelling and well-internalized, that it can override behaviours for which the short-term rewards are significant, but which may be damaging in the long run.

Update: Independence of solitude: I didn’t know this expression. Found 600 or so hits on Google. Seems that maybe the expression comes from Ralph Waldo Emerson.

What I must do is all that concerns me, not what the people think. This rule, equally arduous in actual and intellectual life, may serve for the whole distinction between greatness and meanness. It is the harder, because you will always find those who think they know what is your duty better than you know it. It is easy in the world to live after the world’s opinion; it is easy in solitude to live after our own; but the great person is one who in the midst of the crowd keeps with perfect sweetness the independence of solitude.

Open Learning Initiative at Carnegie Mellon

The people at Carnegie Mellon seem to have the right idea: their Open Learning Initiative offers you to browse right there and now the content of their courses. It seems to be a driving point: they want to make courses free for individuals, and low cost for institutions. By building communities around their courses, I guess they hope to make these same courses, the de facto standards. It could be very powerful and could make some other online schools obselete.

A primary objective of the project is to build a community of use for the courses that will play an important role in ongoing course development and improvement. The courses are developed in a modular fashion to allow faculty at a variety of institutions to either deliver the courses as designed or to modify the content and sequence to fit the needs of their students and/or their curricular and course goals. These courses will be broadly disseminated at no cost to individual students and at low cost to institutions.

Students as Colleagues for Professors

Stephen gave an interview to Clientology about eLearning.

Some of his statements should be scaring the h*ll out of some people in universities and elsewhere. First, he points out that gatekeepers are slowly losing their power:

It is important to recall how much of our culture – including political culture, economic culture, educational culture — has been shaped by ‘gatekeepers’, elites who, because of their knowledge and position, are the sole arbiters of what we will read, buy or learn. This gatekeeping function has already been disintermediated; new people — what Robin Good calls the ‘newsmaster’ are taking their place, and the result is a much more balanced exchange.

This is so obvious in my daily life and was brought on by the Internet and excellent tools like Google. It actually links with a previous post I wrote this week: non-tech natives are the gatekeeper generation. The new tech natives won’t see a purpose for these gatekeepers who kept their knowledge close and their power even closer. Information and knowledge is always changing and flowing so that controlling knowledge doesn’t make sense anymore. In a very deep way, we’ve become a dynamic society. This is not just class mobility, that is, the ability for a large segment of the population to acquire some key knowledge and then, have a shot at becoming a gatekeeper. No. I think that the notion of gatekeeper itself is failing.

This has very concrete consequences in universities:

In education, the result is the gradual erosion of the power relationship that existed between student and professor. In some senses, we see this already by the designation by many of the student as ‘customer’ rather than, say, apprentice. But it’s deeper than that, and we will see eventually the designation of student as ‘colleague’ — and in an important sense, it will not be possible to distinguish between student and professor online.

I couldn’t agree more. Students do not depend on their professors the same way they use to. Professors won’t be able to hold their status as gatekeepers for very long now. Not when anyone in the planet can quickly become an expert in almost any field just by reading up on the Internet.

I think university professors will remain marketing tools. First, they will serve to sell training, which they have always done anyhow, and second, they will be used to authentify knowledge which will become an increasingly important task. Students won’t look so much for knowledge but for recognition and professors who can bring student some recognition which will be in high demand.