config changes, xmldiff (was: Re: CRITICAL Shibboleth Security Advisory 20110725)

[Peter Schober]

* Chad La Joie <lajoie at itumi.biz> [2011-07-27 16:30]:
> This was the result of running the files through line ending
> normalization process which was part of us switching to SVN's "native"
> eol-style.
> 
> As Scott said, if you want to diff XML files you need to use xmldiff.
> Anything else is likely to give you too many false positives if more
> than just a single line changes.

I usually `svn merge` changes from one release to the other to a
working copy of my own configuration. With normal diff tools this of
course has the problems mentioned above (namespace declarations,
whitespace, comments, etc.) but then I usually want to track upstream
config files closely in order to easier spot changed defaults,
configration settings required to enable new functionality, etc.
So I usually also migrate comments, xmlns decls and whatnot to my own
config to keep diffs to upstream minimal.

Anyway, based on your and Scott's comment I had a look at xmldiff,
assuming you're referring to the eponymous Python tool (but possibly
you're just using it as a pars-pro-toto in a more generic sense; I
mean the Logilab tool).
I hacked up a small wrapper script based on the Subversion book[1] to
allow use of xmldiff as an external diff tool with `svn --diff-cmd`.
The wrapper also ignores all files not ending in .xml since xmldiff
would barf on those anyway.
But xmldiff's output is of course rather unpleasant to read (for
humans and probably for most programs as well) and not suitable for
merging with `svn merge` as is. I haven't yet cooked up a wrapper for
`svn merge --diff3-cmd` which would make this more/actually useful.

Anyway, with the attached script you can xmldiff the config between
all shib releases (or any other xml files, of course), e.g. for the
IdP (arbitrarily picking releases where actual differences occured):

  $ svn diff --diff-cmd sxdw \
  https://svn.shibboleth.net/java-shib-idp2/tags/2.2.1/src/installer/resources/conf-tmpl/ \
  https://svn.shibboleth.net/java-shib-idp2/tags/2.3.0/src/installer/resources/conf-tmpl/

Or for the SP:

  $ svn diff --diff-cmd sxdw \
  https://svn.shibboleth.net/cpp-sp/tags/2.4.2/configs/
  https://svn.shibboleth.net/cpp-sp/tags/2.4.3/configs/ 

With 'sxdw" being the local filename of the attached "Subversion
XmlDiff Wrapper" script, which would need be made executable and put
somewhere in your $PATH (or provide the full path to the script).
Usage would probably be easiest setting up an alias in your $SHELL
called 'svnxmldiff', e.g.:

  $ alias svnxmldiff="svn diff --diff-cmd /path/to/sxdw"

to be able to just call `svnxmldiff PATH1 PATH2`.
Note sure this is actually helpful yet, probably more so with the
`xmldiff -x` output (historic XUpdate specs), which could then be
merged using Perl's XML::Update::LibXML, for example.
-peter

[1] http://svnbook.red-bean.com/nightly/en/svn.advanced.externaldifftools.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sxdw.py
Type: text/x-python
Size: 776 bytes
Desc: not available
Url : http://shibboleth.net/pipermail/users/attachments/20110818/0d4b859b/attachment.py