Now I got my Raspberry Pi. I am an experienced application software developer, but I've never done hardware stuff or low-level programming before. I want to build a minimal Linux, including drivers for all hardware on my Pi. For learning purpose, I don't want to install any pre-built Linux distribution on my Pi. Where should I start?
You can also check out Buildroot, http://buildroot.org. We have a default configuration for the Rasberry-Pi, and several contributors are working on the support for this platform.