It’s just a basic engineering problem. When you have more than one person working on the team they will end up making changes in the same file, use the same function, or in another object using another object in a critical way. And things are going break unless they are disciplined in their approach. I think this is a problem that basically generally people haven’t got a clue how to share code. That’s what we had there. A usual thing was that someone would stand up and go, “Okay, who’s changed such and such?” with, you know, “You Muppet” going unsaid.
People would not check in for as long as they could possibly get away with it because they knew it was going to be hell. That was a vicious cycle, because the longer they waited the worse it was going to be. On top of all this, because we’d expanded quickly, we had lots of engineers working on this who were perhaps good at maths or good at lots of detailed stuff, but who didn’t know anything really about design or anything and so were just putting code wherever they saw fit. It just got really, really, really congealed.
There was one time (this was actually later after we’d realised there was a serious problem), the team went to make some changes to do with rounding and other floating point numbers and somebody had already done all the work, but they merged it in and it didn’t work, so they had to repair that and that didn’t work, so they tried to repair it and that didn’t work. That went on for three months of five people full-time essentially doing nothing, adding nothing that could be sold. Three months lost.
As a result of all this we tried to introduce continuous integration, but it’s really hard to get people to understand that it is really, really important. University teaching and most companies enforces the idea that the most important thing you can do as a computer programmer is write code, which is utter nonsense. The most important thing that you do is whatever needs to be done - that doesn’t necessarily mean programming, often it means the delete key. The problem wasn’t so much explaining what continuous integration was, although there was a bit of a challenge, it was actually getting people to accept that it was important and therefore follow it. I think that’s why you need big flashy lights, markers and things like that, because it makes the problem visible.
delicious
digg