[tei-council] Speeding up Jinks builds

[James Cummings]

Hi Martin,

The whitepaper I read on optimizing Jenkins is: 
http://www.cloudbees.com/sites/default/files/whitepapers/7WaysToOptimizeJenkins.pdf 
but really I don't think most of it applies to us.  (Well, we 
could do a distributed set of VM nodes all in the same set of VM 
to do different tasks, but this seems overkill.)

I'm interested in whether just throwing multiple cores and more 
memory at it will give significant improvements... maybe 
something we could test but upping it for a week or two to see if 
it has any effect.

-James


On 26/10/12 16:43, Martin Holmes wrote:
> This message is mostly for Sebastian, but I'm sending it to the list
> because anyone could potentially have suggestions that would help, and
> most of us will end up as a Release Technician at some point.
>
> Yesterday was a bit exceptional in that there were lots of last-minute
> commits that held up the build process, but we did notice that waiting
> for Jinks builds to complete was very tedious, and spent some time
> thinking about ways we might speed this up. These issues crossed my mind:
>
>    - I'm not sure why P5-Documentation and P5 are two separate jobs.
> Could they be combined? I think that would be quicker in that it would
> save copying over lots of stuff from the workspace of one build to the
> other.
>
>    - A few months ago, all builds were going much more quickly on both
> Jinkses. I can't think what's changed, except for the move to XInclude
> from entities. Is it possible that makes a difference? It might be worth
> doing some tests to find out, and if it does, investigate a way of doing
> the XIncludes more quickly.
>
>    - Both Jinkses are running on virtual machines, so we can presumably
> push up the memory or change the processing for the VMs if that will
> help. I'd like to investigate whether adding more cores would be useful
> (if, for instance, separate build jobs run on separate cores), and
> whether more memory will make much difference.
>
>    - I suspect there are lots of tweaks I could make to how memory is
> allocated to ant tasks etc., and those could also be integrated into the
> Jinks server build script.
>
> Anything else anyone can think of that would help speed up the Jinkses?
> We're committed to steadily adding more and more tests to P5-Test
> (Schematron etc.), so that build is going to take progressively longer;
> anything we can do to mitigate that would be really helpful.
>
> Cheers,
> Martin
>


-- 
Dr James Cummings, researchsupport at it.ox.ac.uk
Research Support, IT Services, University of Oxford