## 26 April 2005

### Personal projects and work

I'm wondering how do people find the energy to do open source projects while working? I have often felt that I do not have enough physical time to do what I want to do. Or maybe it is stamina? Often, after a very productive week comes one that is very much unproductive: I feel tired and I can't make any progress. I feel trapped in an iterative waterfall. In the evening I get home and I am tired. I can't stop thinking that I haven't done anything particularily interesting that day. So, even if I know it is wrong, I take a book with me in bed and start reading. It's wrong because I usually take a serious book that I ought to read with a pencil in my hand and certainly not while staying in bed. Lately it is either The Art of Computer Programming, or collected papers by Knuth or something else (The Art of Problem Solving, an abstract algebra textbook etc.) Anyway, I digress. It is something serious because I feel I must do something serious that day, at least in the evening. But I am so tired that I don't really read actively. Sure, I do absorb some information, but overall I'm wasting my time. The effect is that I fell asleep rather late, the next day I can barely wake up and I go to work feeling tired. At work I am much more than I'd like to be immersed in small, trivial, many problems. I don't know if I mentioned but I still feel tired. I do some work. Maybe it's writting in my notebook my thoughts about designing the next part of the application or maybe I write some code. After some time (two hours) I can't stop thinking: Hey, at least today do something interesting. And, instead of finding something interesting in my work, I shift (or should I say loose) my focus and search for interesting things in my inbox, in forums like the one at TopCoder or Devnet, or in blogs I'm subscribed to. Then I get back to work and continue with trivial stuff. Yuck. I realise that this multi-tasking made my productivity behave like 1/x going to infinity and I feel guilty. I must stay a bit more to do at least something today. I start to think (finally) of what I can do that is both interesting and valuable to my company. I find something. I make a plan. I look at the watch. Oh, my god: it is so late. I have no chance to do this well in one hour. It's probably better to start tommorow. And here we go again.

That's my worst possible day. I probably never did all that in one day until now. How does a good day look like then?

I wake up early: at 05:30. I do some workout, go outside with my dog and then eat something. Then I take some vitamins. It's 06:30. For one hour I read something interesting: from an article, from a textbook. I solve (or try to) solve a problem or two. I go to work. I try to solve a problem from TopCoder, preferably a medium problem from division 1 because that's where my "limit" stands. The easy problem is to easy and the hard one might take anywhere from one hour to a week. I do read a hard problem and keep it in the background for a few days. By now I feel full of energy. It is ten o'clock. I look at my todo list and make a plan for today. I try to make it realistic. It contains one or two things that without extra constraints (like old code) would take me three hours. Then I start working. I begin by thinking. I lay down what I think in my notebook. I read what I wrote and try to find a better solution. When I finally settle for something I start coding. I think hard about how to organize the code to make it fit well with what is already written. This means that I have to read the code that is related to my problem. Sometimes I find places where it can be improved. I do it: don't wait. Finally I code my piece. Then I test it. Does it work? If no then debug. Oh, and BTW, I try to write code for people to read not for machines to execute. As a result sometimes I write inefficient code if I can't see how to clearly code the more efficient version. I try to follow coding idioms so that the code has a certain uniformity but I'm not shy about breaking the rules if breaking them seems the right thing to do in a particular situation. The most important thing is to know that you are breaking the rules and only do it as a result of a conscient decision. Now it's already six o'clock. It took more than three hours but I also did more: I improved some old code, I found problems I didn't knew existed, I wrote the code more than once because I found an opportunity to improve it and now it looks rather nice. I'm proud of what I did and I feel good. I go home. Now I'm still full of energy. This morning I have read; now it's time to do. I take on a personal project or maybe a harder problem found in a book or on the internet (like Lars' contest problem). I work on it. I follow a cycle resembling very much the one at work: think it thru, do it, test it. It's nine o'clock and time for me to take a shower and go to sleep. Yeah, I feel great!

Now you've seen how my bad days look and how my good days look. The problem is that I seem to be incapable of breaking the good-bad-good-bad cycle! Distractions (like a late conference) can break a string of good days. Sometimes even a particularily good day can ruin my rythm because I say to myself "ok, now I deserve some more entertainment than the usual, let's stay up all night". The trasition from bad to good is almost invariably done thru a conscient effort to break the string of bad days.

Today is a bad day. I hope it is the last one in its string (although I do have a late conference today: why on earth is earth spherical? couldn't everybody have the same hour?).

I started to write this entry because I realised how insignificantly I have progressed in improving cfind. I have implemented a html parser and saw that performance plummeted. The format of the invereted index file required it to be constructed in-memory and then dumped in one step. This worked really fast while the swap memory wasn't used. Well, on my computer (192MB RAM) the addition of html tripped the balance. The indexing time went up from 1 minute to 20 minutes :(. So I have redesigned the format of the inverted index to allow efficient incremental updates. It's no rocket science: just good old B-trees. Not even their advanced cousins (B+ or B*). I have started to write the code and... that's where I am now. That's pathetic for almost two weeks! (That is about 10h since I have at most 1h per day to work on cfind). Ah, I have also designed a configuration file that will allow much control over the indexing process (including the ability to index archives, etc.). And... it is quite cool to have the OCaml manual, KDE help, and some html books indexed!

I hope I'll get over it and start a long string of good days really soon.

You know what else helps me have good days? Discussions with smart people. Yeah.. I know a couple so I'll try to do that. (The irony is that while I'm in a bad day I feel shy about talking to someone smart simply because I feel lousy and I don't feel up to the task.) It is amazing how bad habits support themselves. This seem to be a general rule. Many bad things in life are addictive: smoking, drinking, drugs, sex. Ok, strike the last one :)