[tei-council] DRAFT: Backward Compatibility and the Maintenance of the Text Encoding Initiative Guidelines

[David J Birnbaum]

Dear Council,

I append a draft discussion of backward compatibility and the 
maintenance of the TEI. The proposal (section III) reflects my first 
attempt at a semi-formal statement of when Council should or should not 
feel licensed to break backward compatibility, which I tried to 
construct as a compromise between taking advantage of new technologies 
and maintaining the stability that users expect. Comments and 
suggestions welcome, of course, with an eye toward discussing these 
issues and agreeing on a final version at our next conference call.

Best,

David
djbpitt+xml at pitt.edu
________

DRAFT: Backward Compatibility and the Maintenance of the Text Encoding 
Initiative Guidelines

David J. Birnbaum
djbpitt+tei at pitt.edu
Draft of 2006-08-02

I. Background

At the May 2006 TEI Council Meeting in Kyoto, Council expressed an 
interest in elaborating principles for further development of the TEI 
guidelines. In particular, Council expressed an interest in reconciling 
two potentially conflicting principles:

1) As new technologies permit us to improve the ability of the TEI 
guidelines to serve the research needs of users, those new capabilities 
should be made available to users. The TEI should not eschew better ways 
of doing things for fear of breaking backward compatibility.

2) Because breaking backward compatibility is unpopular with users, and 
for good reason, the TEI should avoid doing so unless the benefits are 
compelling.

Some Council members argued in Kyoto that our license to break backward 
compatibility would be revoked once we signed off on P5; others felt 
that Council (especially new Council members) would appreciate the 
availability of guiding and stable principles that did not simply 
prohibit breaking backward compatibility, but that indicated how and 
when it should be permitted.

It should be noted that documents developed with a particular schema 
remain valid against that schema in perpetuity, and in this sense no 
change to the TEI Guidelines breaks backward compatibility. On the other 
hand, a user may wish to integrate new, emerging features into an 
existing project without rendering existing encoded information invalid, 
and may need to produce a new schema for that purpose. It is in this 
context that backward compatibility with existing features of the user's 
original schema becomes particularly important.

Finally, the TEI currently seems to make no use of the concept of 
deprecated features, although such a concept may provide a compromise 
that allows the TEI to adopt new structures without invalidating old ones.

II. History

Relevant sources for determining a suitable policy include:

TEI PCP1. The Preparation of Text Encoding Guidelines. Closing Statement 
of the Vassar Planning Conference. Version of 13 November 1987. 
http://www.tei-c.org/Vault/SC/teipcp1.gml or 
http://www.tei-c.org/Vault/SC/teipcp1.txt

TEI EDP1. Design Principles for Text Encoding Guidelines. Version of 9 
January 1990. http://www.tei-c.org/Vault/ED/edp01.gml

TEI EDW57. Procedures for Correcting Errors in the TEI Guidelines. 
Version of July 23, 1994 (Revised July 27, 1994). 
http://www.tei-c.org/Vault/ED/edw57.htm

TEI EDW48. Procedures for Maintenance and Extension of the TEI 
Guidelines. Version of 21 May 1995. http://www.tei-c.org/Vault/ED/edw48.htm

None of these documents prohibits breaking backward compatibility, but 
TEI EDW57 distinguishes corrigible errors from those that are not 
corrigibile according to (among other things) whether correction would 
break backward compatibility. If not, the editors are empowered to 
implement the necessary correction themselves. If so, the proposed 
correction must be "referred to TC" ("Technical [Review] Committee," 
which would now be Council?) for action. The history would thus seem to 
suggest an awareness that breaking backward compatibility might confer 
sufficient benefit that it should not be prohibited, but it also should 
not be undertaken without careful review.

III. Proposed Guidelines

    A. Council should not be constrained by a policy that bluntly
    prohibits breaking backward compatibility after the publication of
    P5. Council should instead feel authorized to break backward
    compatibility when Council concludes that the advantages of doing so
    outweigh the disadvantages.

    B. Wherever possible, Council should implement changes so that new
    schemas are supersets of old ones, and do not render invalid any
    documents that would have been valid before the implementation of
    the changes. The addition of an element to a class creates such a
    superset; removing an element from a class has the opposite effect.

    C. In extreme situations, where Council concludes that the utility
    of a component would be improved substantially by a revision that
    could break backward compatibility, Council should:

        1. Create the necessary new structure (class, content model,
        etc.) under a new name.

        2. Deprecate but retain the structure it is intended to replace.

            a. Deprecated structures determined to be in wide use might
            be maintained in perpetuity. Such a decision would reflect
            Council's expert opinion that eliminating them would be too
            disruptive to existing projects to be entertained, but
            employing them in new projects would not conform to best
            practice.

            b. Deprecated structures determined not to be in wide use
            might eventually (as discussed immediately below) be removed
            from the Guidelines.


        3. Council may break backward compatibility in a more
        substantial and comprehensive way (as happened with the
        transition to P4 to P5) only with the transition to a new major
        version (e.g., P6, but not P5.99999). Such transitions should
        happen only in response to the emergence of new technologies
        (e.g., the transition from SGML to XML) or the development of a
        new architecture that conveys substantial benefit (e.g., the
        development of the new class system). Historically, new major
        versions have emerged approximately every five years.