Jenkins and Windows

[Tom Zeller]

TL;DR What (why), when, and how do we want to test on Windows via Jenkins ?

I’ve been working on the “how”, and am ready to ask “what” and “when”.

My assumption is that we want to (a) mirror some of our jobs by running them on Windows in addition to Linux, and (b) we will likely run some jobs just on Windows (like building snapshots of the Windows installer, maybe browser tests).

I see three ways to mirror (meaning run on Linux/master and Windows/slave) jobs : (1) create new jobs, (2) add a node/slave axis, and (3) add parameterization to existing jobs.

“Create new jobs” : We could create jobs which are copies of the jobs we want to run additionally on Windows, for example, java-support-windows which is restricted to running on the Windows slave. It’s easy to do, but creating new jobs obviously doesn’t scale well if we want to test on additional platforms (maybe a different flavor of Windows, Mac OS X, or Linux). Triggering would be straightforward, the *-windows jobs would be configured to run after their corresponding Linux job.

“Add a node/slave axis” : We could add a node/slave axis much like our JDK axis. The *-multi-jdk jobs are configured to run on Oracle/OpenJDK Java versions 7/8, and we would add a Windows/Linux axis. The nice thing here is that adding additional platforms to test on does not require additional jobs. One downside I see is losing the automatic configuration of triggering downstream projects. Not a big deal, we can manually configure our project/job hierarchy as we have done elsewhere (e.g. trigger-nightly-v3), which actually might be considered a positive. The Jenkins detail here is that AFAIK multi-configuration job types are different from Maven job types in that only Maven job types know how to trigger downstream projects a priori.

“Add parameterization to existing jobs” : We could add and configure plugins to Jenkins, like the Node/Label Parameter Plugin, to re-configure our existing jobs to run on Windows as well as Linux. I tried this option first, but had trouble with recursion. As with any plugin, there is a bit of conflict with built-in Jenkins functionality (“require label when building”).

At this point in the email, I’ve had a conversation with Scott, so I’ve minimally edited the above, and I’ll re-word “add a node/slave axis” as “add a platform axis” because that seems to match what we are trying to do.

As I’ve typed above, the next steps there would be to add Windows/Linux as an axis to the *-multi-jdk jobs. One difference between the *-multi-jdk jobs and the V3 Maven jobs is that while the *-multi-jdk jobs are also triggered by commits, they only look daily unlike the V3 Maven jobs which look for commits every 15m.

One potential nit with the multi-configuration/matrix job type is that the “freewheel/lightweight” job which orchestrates the running of the jobs along the axis will fail/hang if the Windows slave is offline. Not a big deal, we can install a plugin to skip offline nodes, but I think instead we should try and keep the Windows slave up if possible. 

The follow-up is whether we want certain/all jobs to pass on Windows before deploying snapshots. We could start building and deploying snapshots of the Windows installer via Jenkins on the Windows slave, much like the V3-nightly jobs do now.

If we go with “add a platform axis”, we might want to rename *-multi-jdk to something else, maybe just *-multi, but that’s not a big deal either.

So, thanks for reading, and for your feedback. I’ll head further down the “platform axis” path and see what comes up, but I’m open to suggestions.