TEI in the wild: The Perseus Project

I was so unbelievably delighted to see the Perseus Project (http://www.perseus.tufts.edu/hopper/) on the TEI website's list of projects that use the TEI. Perseus! Perseus uses the TEI! I love Perseus! I used the website extensively throughout my undergrad in Classical Studies. I found their linguistic features very useful, but it was the convenience of their open access philosophy and the website's search functions that made me a regular user. I couldn't not write about the Perseus Project once I saw it on that list. However, the website is not very clear at all about its use of TEI or even XML, so it may not actually be the best example for this blog question. I’ll do my best!

Perseus states that their mission is “to help make the full record for humanity as intellectually accessible as possible to every human being, providing information adapted to as many linguistic and cultural backgrounds as possible” (they also admit that actually achieving this is impossible) ("Research"). What a wonderful mission statement! They have many collections of texts in their original languages and/or in English translation, such as Arabic materials, Germanic materials, Renaissance materials, and a huge collection of Greek and Roman texts. 

These texts are encoded with TEI-conforming XML, which Perseus uses together with something they call the Perseus Java Hopper to provide services for interacting with the XML collections. Perseus doesn’t explain very clearly how this works or what it does, but instead talks broadly about their overarching strategies and principles. They focus on three categories of access: Human readable information, Machine actionable knowledge, and Machine generated knowledge ("Research"). They want to achieve three main features ("Open Source"):

• Linguistic Support, to link inflected words to their dictionary entries and provide morphological analysis (so that clicking tulit in a Latin text will go to the dictionary entry for fero, and tell you that tulit is the 3rd person singular perfect active indicative form).
• Contextualized Reading, providing services derived from the interaction of all the texts rather than the texts in isolation.
• Searching, so that for example users are able to search by canonical abbreviations or check the frequency with which a word appears in a text for all its inflected forms.

Finally they note a desire to facilitate extensibility, by making the Perseus Hopper available and designing it with modularity in mind. Most if not all of these features and strategies relate not to the encoding of the XML, but instead to the strategies behind the Perseus Hopper. 

Figure 1: A section of one of Cicero's letters to Atticus in Latin through the Perseus Project. Source: http://data.perseus.org/citations/urn:cts:latinLit:phi0474.phi057.perseus-lat1:1.1.1

Perseus's XML code is also all available online for download by anyone. There is an "XML" button at the bottom of every page of every text (figure 1). Their source code is also available ("Open Source: Download"), although the download page notes that the source code is deprecated and cannot offer support for installation. 

As I noted above, Perseus hardly mentions the TEI. Looking at the XML of the texts reveals that they are TEI-compliant, with TEI headers and tags, but that’s basically it. The XML often seems fairly basic, with paragraphs, line breaks, and section numbers marked, but with no tags included for to mark things like names or places, for example (see figure 2). I assume Perseus’s language and search features are completely facilitated by the Perseus Hopper. Although Perseus does not clearly state their encoding strategies for their XML, it seems that their aim was to provide a basic digitization of the texts in TEI-compliant XML that could then be fed into this main program. 

This lack of communication about encoding strategies can make looking at Perseus's code confusing. For example, the XML for the collection of letters form Cicero to Atticus (figure 2) is full of the <reg> tag. This tag is for marking a word that has been regularized, for example because of variant forms or nonstandard spellings (TEI Guidelines 3.4.2). There is no way to tell why these particular words are marked in the Cicero letter, or why the encoders decided that this was something they wanted to indicate.

Julia Flanders (2012) argues that the TEI provides a functional mechanism not for collaboration, but for negotiating dissent, noting that the digital humanities don't just need a common way to represent data, they also need a common mechanism to negotiate about the representation of data.

It seems to me that projects should help facilitate this negotiation by providing information not just about their mission and goals (as Perseus Project does), but also about the more specific encoding choices they've made (like what we're doing in the encoding project, I guess). In order to make dialogue about choice and disagreements possible, those choices and the reasoning behind them needs to be clear and easy to access. I guess I just can't see how Perseus provides enough information for its code to be functional in the way that Flanders means.

Figure 2: The TEI-compliant XML for the text shown in figure 1. Source: downloaded from http://data.perseus.org/citations/urn:cts:latinLit:phi0474.phi057.perseus-lat1:1.1.1

References

Flanders, J. (2012). Collaboration and Dissent: Challenges of Collaborative Standards for Digital Humanities. In M. Deegan & W. McCarty (Eds.), Collaborative Research in the Digital Humanities. Burlington, VT: Ashgate.

“Research.” Perseus Digital Library. http://www.perseus.tufts.edu/hopper/research

“Open Source.” Perseus Digital Library. http://www.perseus.tufts.edu/hopper/research

"Open Source: Download." Perseus Digital Library. http://www.perseus.tufts.edu/hopper/opensource/download

TEI Guidelines 3.4.2: Regularization and Normalization. http://www.tei-c.org/release/doc/tei-p5-doc/en/html/CO.html#COEDREG