ISO takes standards published by industry consortia and accreditates them by giving them an ISO number. How did this process become necessary, and how does it work? Trying to find out, I didn't expect to find at ISO what looks more like scandal, bitter political fights among corporations, and burgeoning activism over the internet.
Though it apparently set the technology blogosphere on fire at the time, I missed the mess that was the introduction of DIS 29500, once Microsoft's "Office Open XML", through the Fast Track process by vote in the SC34 committee of the ISO's joint effort with the IEC, JTC1. Lost me already? Good.
During this series of events, Microsoft appeared to stuff the voting commitees of different national standards bodies that vote in ISO's committees. An unprecedented number of new countries signed up to become voting members of SC34, and voted to approve the ISO certification of the consortium standard effort led by Microsoft and based on the design of its "Office" package. Many of the new members then failed or refused to vote on further business of the SC34 committee, blocking its ability to operate for many months. Microsoft paid a SC34 invited expert and popular blogger to spend several days editing areas of Wikipedia which appeared to be biased. Up to seven Microsoft bloggers were commenting on and drawing out discussion about OOXML at this time.
Manipulative, heavy-handed, yes, yet most of these efforts were just successful PR efforts, admissible by the letter, if not the spirit, of ISO agreements and the membership of national-level standards bodies. Microsoft merely used its substantial and thorough lobbying and 'turfing operation in the direction of ISO, a body somewhat like the UN or the Eurovision Song Contest, wherein each country has a vote. Such a system has a built-in liability to become subject to political horsetrading. As standards proliferate, a lot of material of quite complex specialism is pushed past a group of people whose time is mainly volunteered by large corporate entity or large public agency, though a few may volunteer "their own" time.
The debate over OOXML has featured a lot of knee-jerk MS-bashing and letter-ism, when a serious critique of the standards system is needed. Several global companies and networks of companies have the ability to influence in toto the network of consortia and national standards organisations that are created in order to "accreditate" technical standards. It is certain that they counter-lobbied and in doing so pressed Microsoft to intensify their own efforts. The DR Congo's national standards body voted unconditionally in favour of OOXML. Can anyone find the assertion that DR Congo has a functioning technical standards committee as of late 2007, at all credible?
MS can get away with ballot-stuffing in the ISO process. It can offer unclear and overgenerous assertions as to patent rights because the consortium process which creates candidate standards, lets this happen. These are the problems - in the process itself, not in the detail of Microsoft's, or any other global company's, actions.
Now, one's usual knee-jerk MS-bashing makes it hard to entertain this hypothetical: what if we could assume goodwill on the part of MS? On the question of ODF many commentators voicing the "open source community" have assumed goodwill about the behaviours of several other very large corporate entities. It is likely that IBM's counter-lobby against MS among the National Standards Bodies increased the seriousness of the committee-stuffing effort and the resources devoted to it. Sun spend up to 1/2 million dollars a year keeping OpenOffice running as a loss-leader. Many companies sell big open source platforms to defence and administration on the basis of OpenOffice's support for open standards (ODF) rather than on the technical or economic benefits of its 'openess'.
Yet a couple of groups of professional activists on the Internet, have raised campaigns against OOXML by picking away at the fine details, both in the events during the influence of process in ISO, and in the technical details of the proposed ISO standard. There is in some quarters a desire to "punish" Microsoft by agitating against their format becoming an ISO standard.
These same activists are promoting ODF, the preferred format of IBM, Sun and other well known open source and open standards pioneers (remember the 'Java Community License'? No?). This incomplete work was ushered through SC34 by large corporate partners for marketing advantage - this is the way that standards bodies work. Of course a consortium may not be representing all enterprises in an industry - it quite possibly omits one very large company that may have acquired a proprietary-standards-influenced monopoly on that industry. In ODF's case the effort carries along with it the meme of MS-bashing as well as that of documented public format.
MS-bashing is tiring, especially when it is being spurred on by marketing from large corporate entities. It is not MS's fault that the ISO process has been compromisable by a very large corporate entity. It is not MS's fault that specifications defined as "open standards" are coming out of apparently credible industry bodies containing clauses that may render the user liable to pay for patent rights without clearly informing them about it. They allegedly did some fucked up stuff during the SC34 votes but that is no reason to punish MS for publishing what is essentially "their" format, giving away "rights" to extract money from its use. The role of other business entities, the status of other specifications, is just as important, and is overlooked in the simplification of blaming MS.
The "Open Specifications Promise is an interesting document which basically says, "we promise never to bring an action against you seeking payment for use of things that are fully described in this specification and on which we, Microsoft, also hold some patent. If you sue us first, we're released from this promise."
Taking this approach is offering an interesting construct, a kind of open patent license, with clauses constraining re-use just as CC applies for copyright licenses; "we won't sue you over use of this" "unless you sue us over it". I see one effort-in-progress to create such a license for an open source world. For many practitioners patents are anathema, enforcing one predatory.
The NoOOXML site claims that Microsoft's "OSP" isn't "legally valid". How, in what jurisdictions, after whose dictation, do we get "legally valid" from? The GPL isn't legally valid either. Nor are the Creative Commons licenses. There just isn't enough case-law, anywhere. These things are neither-legal-or-illegal. So we take all these declarations on trust, along with declarations like the OSI Open Source Definition, as coming from a viable authority. But though there's not much real law involved, there's plenty of imaginary law.
To get sucked into letterism, to go for the technical details this late in the process and accept the process's role in correcting them; a tactical error. Debate on the detail will grow in volume, making it harder to find the hard numbers and the used rules. All this debate on technical merits will necessarily also provide design decisions and business benefit to Microsoft, to Sun and IBM (Though if the standard is to be open that is for the best). It will also appear to justify dealing with formats at this stage of development, with a standards body of this structure and nature.
Letterism can be critically useful sometimes, ideally when it's asked for; usually a lot earlier in the process than before a vote in which it's already past one vote and at a meta-level. If there are shortfalls in the way in which consortia provide information about their projects, plans and action, then these kind of clashes will arise where standardisation efforts go a long way without being visible to a full interest community, only to paying parts of it. The place for letterism is when substantially overlapping efforts are caught and turned into collaboration and merge projects or split up into more modular ones so that the overlap is less.
As pointed out by Miguel de Icaza, many companies have been selling OpenOffice based desktop platforms and integration based on the fact that OO supports the "open standard" document format ODF, an item from the archive that you'll always have rights to use and never lose the ability to read. People have been marketing Open Source principally on the basis of support for Open Standards. Suddenly the format in future versions of the proprietary Office will be open standard. Yours is an ISO number? Ours is too. Er, hang on...
We've seen in the geospatial industry an ongoing faith in the ISO process and extensive selling on an Open Standards / Open Source basis. Increasing perception of viability of open source platforms had the effect of starting to squeeze out medium-to-large size proprietary companies, or send them in an open source direction and invoke other business changes, such as a move towards services.
Our geospatial industry standards consortium has been significantly supported by US defence and intelligence agencies. Now Google and Microsoft head its sponsor list. OGC is lucky enough to have A-Liaison status with ISO, allowing it to drop its specifications nearer the end of the vetting procedure, to have a delegation at its Working Groups. When Google get their KML standard, version 2.2 through the OGC "Mass Market" process, it will likely be up for ISO accreditation. It'll be easier to sell KML-based "solutions" to government, and the reach of Google Earth client will widen, the competition will see less added value to their efforts. GML may have an ISO stamp too, but it will lose "market share" of public implementation.
This is all done for the purposes of getting government business. It must be lucrative, hey? To have your contract preferred by the addition of ISO number, assisted by the volume of your partner businesses with a voice in standards efforts. Consortia are being used to compete indirectly by businesses, distorting reactions and encouraging the pseudo "standardisation" of technical detail too soon in the process of invention.
The increasing volume of efforts applying for standardisation is beyond the full attention of those people really participating inside the process. So one-project corporate efforts (OOXML, KML) get pushed through working groups inside standards-setting consortia. Everyone inside sees the marketing benefits of a limited "openess". They change the "standard" enough to guarantee that now no-one implements it completely, and pass it upwards for international accreditation by bodies like ISO.
The scandal is not that one very large company was able easily and legally to influence the ISO process (and that this failed, perhaps because of the acceleration of the counter-lobby by other companies). The scandal is how this issue concerning 90% of computer desktops get so far through the process without wide community awareness, of the kind that developed when OOXML entered the ISO process.
This competition is also disabling the "higher organisation" of enterprise of which these entities are supposed to be a part.
I collected a pageful of links to commentary on the standards process in IT during the OOXML/DIS 29500 discussion.
I also tried to write down some statements about standards, how they work and how they can get formalised based on what i have experienced and inferred.
Thorstein Veblen had some incisive thoughts about industry standards and consortia a century ago.