The Power of Moore's Law
One of my personal projects has been writing a buildbot that builds Mono on Windows. Although we have an official one on monobuild, it takes about 4.5 hours per build, so it can't build many revisions. Mine doesn't do everything that the official one does yet, like running tests, but it looks promising as far as cutting down on the build time (primarily by not using cygwin).
I run it on an old desktop that was lying around that I converted to a dedicated builder server. Its stats are roughly:
Not horrible, but it's hard to be satisfied with that knowing I have a spiffy quad core machine and could build most of the managed assemblies in parallel. I had to try it out on my main desktop machine, which is:
I of course expected my desktop to be faster, but it was already completing the build in less than half the time of my server. On top of that, it builds the managed assemblies in 3:23 minutes.
Even with excellent parallelization, there isn't a lot of time to chop off of three and a half minutes. And we know we won't get perfect parallelization due to the bootstrap and the common assemblies that have to be built first that everything depends on (corlib, System, System.Xml, etc.). I think I could maybe cut two minutes off of that with a parallel build.
For now, I have aborted my plan to parallelize my build system. It turns out that Joel is right that sometimes its better to throw money at a problem instead of spending time to optimize it.
Of course, that's also a dangerous idea, because now I look at my copying numbers and want an SSD disk drive. :)
I run it on an old desktop that was lying around that I converted to a dedicated builder server. Its stats are roughly:
- Windows Server 2008 (32bit)
- Pentium IV 2.4ghz
- 768 MB ram
- 5400 rpm IDE HD
| Svn update mono, mcs | 2:25 min |
| Copy mono and mcs to a fresh directory | 6:38 min |
| Build Mono runtime | 4:38 min |
| Build 2.0, 3.5 assemblies (101 assemblies) | 8:01 min |
| Remove mono/mcs directory | 1:00 min |
| Total | 22:42 min |
Not horrible, but it's hard to be satisfied with that knowing I have a spiffy quad core machine and could build most of the managed assemblies in parallel. I had to try it out on my main desktop machine, which is:
- Windows 7 RC (64bit)
- Intel Core2 Quad Q6600 (2.4ghz)
- 4 GB ram
- 7200 rpm SATA HD
| Svn update mono, mcs | 1:03 min |
| Copy mono and mcs to a fresh directory | 4:29 min |
| Build Mono runtime | 1:57 min |
| Build 2.0, 3.5 assemblies (101 assemblies) | 3:23 min |
| Remove mono/mcs directory | 0:25 min |
| Total | 11:17 min |
I of course expected my desktop to be faster, but it was already completing the build in less than half the time of my server. On top of that, it builds the managed assemblies in 3:23 minutes.
Even with excellent parallelization, there isn't a lot of time to chop off of three and a half minutes. And we know we won't get perfect parallelization due to the bootstrap and the common assemblies that have to be built first that everything depends on (corlib, System, System.Xml, etc.). I think I could maybe cut two minutes off of that with a parallel build.
For now, I have aborted my plan to parallelize my build system. It turns out that Joel is right that sometimes its better to throw money at a problem instead of spending time to optimize it.
Of course, that's also a dangerous idea, because now I look at my copying numbers and want an SSD disk drive. :)
Comments
Why not use a RAMdrive in the dedicated build server. It seems AT least half of your time is spent on disk IO in one form or another. With a RAMdrive you could effectively reduce that to zero.
When looking at your figures I'd assume that you could easily go down to 5 minutes just by doing that.
8 GB of RAM cost less than 100€.
And a 7GB RAMdrive should be big enough to hold both, the working and the compiling copy.
In fact, when I started to outgrow this computer, my decision came down to "spend $600 for more ram for this computer" or "spend $600 for a better, brand new computer with more ram".
Trying a ramdisk on my desktop however if definitely something I plan on trying out.
That surprised me, so I went looking for other people's build times. Note, that I'm using ccontrol and that make is effectively aliased to 'colormake -j5'.
I'm not using ccache and distcc although it would be easy to change that, even on the fly using ccontrol.
Highly recommended, and thanks for giving me a reference 'benchmark' figure there