## 26 April 2005

### Personal projects and work

I'm wondering how do people find the energy to do open source projects while working? I have often felt that I do not have enough physical time to do what I want to do. Or maybe it is stamina? Often, after a very productive week comes one that is very much unproductive: I feel tired and I can't make any progress. I feel trapped in an iterative waterfall. In the evening I get home and I am tired. I can't stop thinking that I haven't done anything particularily interesting that day. So, even if I know it is wrong, I take a book with me in bed and start reading. It's wrong because I usually take a serious book that I ought to read with a pencil in my hand and certainly not while staying in bed. Lately it is either The Art of Computer Programming, or collected papers by Knuth or something else (The Art of Problem Solving, an abstract algebra textbook etc.) Anyway, I digress. It is something serious because I feel I must do something serious that day, at least in the evening. But I am so tired that I don't really read actively. Sure, I do absorb some information, but overall I'm wasting my time. The effect is that I fell asleep rather late, the next day I can barely wake up and I go to work feeling tired. At work I am much more than I'd like to be immersed in small, trivial, many problems. I don't know if I mentioned but I still feel tired. I do some work. Maybe it's writting in my notebook my thoughts about designing the next part of the application or maybe I write some code. After some time (two hours) I can't stop thinking: Hey, at least today do something interesting. And, instead of finding something interesting in my work, I shift (or should I say loose) my focus and search for interesting things in my inbox, in forums like the one at TopCoder or Devnet, or in blogs I'm subscribed to. Then I get back to work and continue with trivial stuff. Yuck. I realise that this multi-tasking made my productivity behave like 1/x going to infinity and I feel guilty. I must stay a bit more to do at least something today. I start to think (finally) of what I can do that is both interesting and valuable to my company. I find something. I make a plan. I look at the watch. Oh, my god: it is so late. I have no chance to do this well in one hour. It's probably better to start tommorow. And here we go again.

That's my worst possible day. I probably never did all that in one day until now. How does a good day look like then?

Now you've seen how my bad days look and how my good days look. The problem is that I seem to be incapable of breaking the good-bad-good-bad cycle! Distractions (like a late conference) can break a string of good days. Sometimes even a particularily good day can ruin my rythm because I say to myself "ok, now I deserve some more entertainment than the usual, let's stay up all night". The trasition from bad to good is almost invariably done thru a conscient effort to break the string of bad days.

Today is a bad day. I hope it is the last one in its string (although I do have a late conference today: why on earth is earth spherical? couldn't everybody have the same hour?).

I started to write this entry because I realised how insignificantly I have progressed in improving cfind. I have implemented a html parser and saw that performance plummeted. The format of the invereted index file required it to be constructed in-memory and then dumped in one step. This worked really fast while the swap memory wasn't used. Well, on my computer (192MB RAM) the addition of html tripped the balance. The indexing time went up from 1 minute to 20 minutes :(. So I have redesigned the format of the inverted index to allow efficient incremental updates. It's no rocket science: just good old B-trees. Not even their advanced cousins (B+ or B*). I have started to write the code and... that's where I am now. That's pathetic for almost two weeks! (That is about 10h since I have at most 1h per day to work on cfind). Ah, I have also designed a configuration file that will allow much control over the indexing process (including the ability to index archives, etc.). And... it is quite cool to have the OCaml manual, KDE help, and some html books indexed!

I hope I'll get over it and start a long string of good days really soon.

You know what else helps me have good days? Discussions with smart people. Yeah.. I know a couple so I'll try to do that. (The irony is that while I'm in a bad day I feel shy about talking to someone smart simply because I feel lousy and I don't feel up to the task.) It is amazing how bad habits support themselves. This seem to be a general rule. Many bad things in life are addictive: smoking, drinking, drugs, sex. Ok, strike the last one :)