Perseus Digital Library Updates

Gregory Crane
November 15, 2024

A great deal of work towards the new Perseus (Perseus 6) has been going on behind the scenes and that work is now beginning to become visible. James Tauber of Signum University has led the development, with collaboration from Charles Pletcher and Clifford Wulfman. In most cases, the “we” in this narrative means “James Tauber did the work with the rest of us making comments from the peanut gallery.”

We are still not in the final stage of implementation but we are finishing up a major, and probably the biggest, task: creating a sustainable backend to manage a growing range of machine actionable textual data from a widening set of open data projects in multiple countries. As we refine the backend data, we can begin to move to the final stage of work: implementing within the Scaife Viewer the frontend services that support treebanks, grammatical annotations, aligned translations, metrical analyses, automatic mapping, links from edition to manuscripts and that were developed in Beyond Translation.

For now I point out work done to make the existing Scaife/Perseus Greek and Latin accessible in the ATLAS architecture.

Every Greek and Latin text available in Scaife is now available in the new ATLAS architecture. If you start with the Scaife ATLAS homepage (https://atlas.perseus.tufts.edu/) drill down into the CTS library (https://atlas.perseus.tufts.edu/library/), you can find your way into a list of all editions currently available in Scaife (a dozen versions of Thucydides in various languages are available https://atlas.perseus.tufts.edu/library/urn:cts:greekLit:tlg0003.tlg001/), and then down to each of the lowermost citable notes (e.g., https://atlas.perseus.tufts.edu/library/passage/urn:cts:greekLit:tlg0003.tlg001.perseus-grc2:1.1.1/). At this point, you can call for either plain text or xml by adjusting the final argument: https://atlas.perseus.tufts.edu/library/passage/urn:cts:greekLit:tlg0003.tlg001.perseus-grc2:1.1.1/text/ vs. https://atlas.perseus.tufts.edu/library/passage/urn:cts:greekLit:tlg0003.tlg001.perseus-grc2:1.1.1/xml/.

If you explore the Github repository, you will not not only the full text library stored in the simpler ATLAS form (identifier+textchunk) but also complete morphosyntactic analyses for all 40 million words of Greek (using GreCy) and 16 million words of Latin (using LatinCy). Navigating to particular authors is a bit cumbersome because we had too much data and it was necessary to split the data into multiple Github repositories. The information you need to find the right repository for any given Scaife Greek or Latin author is at https://github.com/scaife-viewer/tagging-pipeline/tree/main/data.

We now have three layers of morpho-syntactic data.

1. Actively curated Treebanks (e.g., Francesco Mambrini’s Daphne).
2. The growing repository of curated and automatically generated treebanks from Alek Keersmaeker’s Glaux Trees (currently 20 million words of Greek and including some works that are not yet in Perseus)
3. The comprehensive morpho-syntactic data generated for any texts added to Perseus.

The opening sections of Thucydides (source here):

The identifier+textchunk format is all that we need to add new texts into Perseus. While the greater structure of the TEI XML format that we have used in Scaife has advantages, we can now add much more content much more quickly and work with larger corpora that has been practical in the more demanding XML framework. The goal is not to abandon XML but to provide the XML as time allows while also being able to work with larger, less structured collections.

The morpho-syntactic analysis (link here):

A draft of a fuller write-up for this work is available in Zenodo: The Sixth Generation of the Perseus Digital Library and a Workflow for Open Philology.

Gregory Crane

As far as I can tell, things have settled down on Perseus 4. As I understand it, there were two issues.

First, Tufts Technology Services moved Perseus 4 to a new hardware platform. In the older platform, various checks had been added so that we would automatically reboot Perseus servers if they hung. This automatic rebooting feature was added a long time ago — probably at least ten years in the past. Replacing this automatic feature seems to have slipped our attention (and I should have checked that). It looks like we have made progress on this. It is possible we will need to tweak this reboot feature but for now I am finding that the site works (at least for me). We monitor email to the Perseus webmaster (and try to respond in a timely fashion).

Second (as noted before), we were hit with scraping and of high volume use that differed from previous patterns over the many years that Perseus 4 has been running. Measures have been put in place to mitigate against these behaviors. We will monitor this situation and respond as needed.

I would like to thank Tufts Technology Services for their on-going help.

Gregory Crane

May 4, 2024

Updates on the conversion to Perseus 6 are available here. The update itself is published on the PubPub platform and the shift from WordPress (which is the engine behind the Perseus Updates blog) to PubPub reflects a major component of the overall change.

Gregory Crane
Editor-in-Chief, Perseus Digital Library
Update as of January 19, 2024.

We have now completed work on Beyond Translation (a draft white paper on this work that has been submitted to the NEH Office of Digital Humanities is available here) and are focused on using the Beyond Translation work as part of an update to the Scaife Viewer. The resulting system will finally allow us to replace Perseus 4. We are calling the new version Perseus 6 (rather than 5) to reflect the amount of work embedded in the Scaife Viewer and now Beyond Translation (which we view collectively as Perseus 5). A grant from the NEH Collections and References program in the NEH Division of Preservation and Access for Perseus on the Web — preparing for the next thirty years provides the primary support for this phase of work, with additional support from the Tufts Data Intensive Studies Center, the School of Arts and Sciences, Tufts Technology Services and Google.

Our main collaborator in this phase of development is James Tauber, who is now working with Signum University. We also are waiting on Tufts administrative paperwork to finalize a contract with another group to help us reorganize the Perseus home (and associated sub pages) and replace this WordPress-based blog with the Pubpub Publishing Platform (which we already began using in documenting Beyond Translation.

A draft outline of the work that we are doing is now available here.

For now, the focus of work is to fold the services visible on Beyond Translation into the Scaife Viewer. The first results from that work will probably documentation, with changes to the Scaife Viewer following.

GREGORY CRANE

The Office of Digital Humanities at the National Endowment for the Humanities asks its projects to submit white papers after completing a project. We have posted a draft of the White Paper for Beyond Translation here and invite suggestions as we finalize our work.

dNext up: a formal description of plans to replace Perseus 4. We secured the last chunk of funding needed shortly before the winter break. Once we finalize this report on what we have done, we will provide more details on what we are doing.