Over the past few years, there has been growing interest in open standards as the only fool-proof way to facilitate interoperability in the ICT sector. But, a recent study out of the University of Illinois finds that open standards are not enough: implementations matter. (Thanks to Doug Mahugh for highlighting this paper in his blog). 
  
 The study “Lost in Translation: Interoperability Issues for Open Standards,” looked at several implementations of the ODF and OOXML standards and found a “troubling” lack of interoperability between them.  The authors conclude these interoperability issues:
  

 …suggest the need for improved interoperability testing for document formats. The results also highlight the importance of interoperability for open standards in general. Without interoperability, governments will be locked-in to the dominant implementations for either standard and in the process lose many of the benefits that might accrue from adopting an open standard in the first instance.

  
For his blog, Doug does some of his own tests with the ODF implementations in Sun’s OpenOffice and IBM’s Symphony office suite and finds similar results.  He goes on to suggest that the promise of open standards can only be achieved if implementers work together in three critical ways:

 shared stewardship of the standards, through active participation in the defined maintenance processes;
 
transparency of implementation, through published implementation notes that describe how the major implementations are handling the myriad details found in modern document format standards; and
 
collaboration between implementers in events like the the DII workshops, interoperability working groups such as the OIC TC, and direct engagement between implementers to test interoperability.

 
We generally agree with Doug’s proposed approach, but it may not be sufficient either.  It occurs to me that part of what Doug and the Illinois researcher were seeing is a challenge that has ALWAYS been inherent in collaboratively developed standards: the lack of a reference implementation. 
  
The benefits to collaborative standards are pretty obvious: collaboration leads to a better spec and a more level playing field for implementers. The drawback has always been that simultaneous implementation by competitors leads to deviations (some warranted, some not) from the collaborative standard. For years, implementations of C compilers and SQL database engines have veered quite a bit from the ANSI standards to allow for the implementation of new and experimental features. Nowhere was the MORE true than during the browser wars where the much vaunted HTML "standard" bore little or no resemblance to the web pages people were creating for Netscape and IE.  
 
Now contrast that with the evolution of the PDF specification. Adobe always made the spec available but did it simultaneously with the release of its latest version of Acrobat. The argument could be made this gave them a time to market advantage in implementing the new spec but it also provided a reference implementation so that any new entrant could be certain to build documents compatible with the market leader because the reference implementation could be used to resolve any ambiguities in the spec.
 
The simple fact is there are no easy solutions.  Mandating the use of an open standard does not guarantee interoperability.