Why don't they teach these things in school?
Over the summer, I was fortunate enough to get into Google Summer of Code. I learned a lot (probably more than I've learned in the sum of all my university coursework). I'm really wondering why they don't teach a few of the things I learned sooner in school though. To name a few:
- unit testing
- version control
- agile development
It seems to me that they spend a significant amount of time teaching other things like data structures and algorithms up front. While I still think those are very important to learn early on, why don't they teach more of these three before them? Or is it just my school that doesn't teach much of this stuff?
Don't get me wrong, I don't think it's desirable for universities to always teach the trendiest programming fads, but shouldn't my professors be teaching me something other than "draw a diagram before you start coding?"
Answer
The simplest answer to your question is that the fields of computer science and software development are both very new, and not very well understood. Although all scientific and engineering disciplines are advancing more rapidly in modern times, other fields have a lot more experience to draw on and there is a much broader shared understanding of how they work.
For example, despite recent advancements in materials science, civil engineers have known for about 2000 years how to build an arch that won't fall over, and this is something that can be taught and learned in university with relatively little controversy. Although I completely agree with you about the techniques that software developers should learn, this agreement is based on personal experience and informal reasoning. In order to be a socially accepted "best practice", we need quantitative data which can be very expensive to gather: how much does version control help? How does it help? Unit testing? We can reason about the effectiveness of various techniques, but actually proving that effectiveness conclusively would be very expensive. We'd need to run a complete, realistic software project from beginning to end, numerous times, with groups of programmers that have equivalent expertise, using different techniques. At the very least we'd need lots of data about existing projects which those projects would be unwilling to release.
Civil engineers have thousands of years of bridges to look at, with lots of information. Software developers, on the other hand, have only a few decades of information, most of which is kept secret, since there's little motivation for organizations to collate and publish information about their developers' effectiveness, even if they are collecting it (which most aren't).
There's also some confusion of fields. Software development, or software "engineering", is really a different thing from computer science. Software developers need a working knowledge of computer science, but working at the boundaries of algorithmic complexity or reasoning about parallelism isn't something that a working programmer will do every day; similarly, a real "computer scientist" will write tons of throw-away code that just doesn't work or doesn't do anything interesting, and won't benefit as much from the sort of rigor that an actual software product would.
The emergence of the internet and the open source community may provide enough data to start answering these questions conclusively, but even if the answers were available tomorrow, it will probably take 100 years for them to permeate international society to the point where everyone agrees on what should be taught in schools.
Finally there are some economic considerations. It has been a relatively short time since almost everyone involved in software development had cheap, easy access to dedicated machines to run whatever development tools they want. A few decades ago, completely dedicating a machine to just running your tests, or even housing an infinite history of source code, would have seemed frivolously expensive to a lot of people.