Perseus Digital Library Updates

Gregory Crane
February 26, 2025

On behalf of myself and my co-authors, Alison Babeu and Farnoosh Shamsian, I am pleased to announce that our article on “Greek, Latin and Augmented Intelligence: the other AI” has appeared online as part of the Classical Review. The DOI is https://doi.org/10.1017/S0009840X25000356. The online publication occurred today. The article has not yet been assigned a volume number but it is available as part of the Cambridge Core.

The article is, unfortunately, not available under an open access license. We felt that, by publishing in a well-established journal, we would reach a different audience. By publishing in a traditional journal such as Classical Review, we also place this paper in a space that has its own mechanisms for long term preservation. The description of treebanks and aligned translations as they appear in the early 2020s may be of interest to future readers as they reflect on the technological evolution of scholarship.

The opening and concluding paragraphs from the piece, quoted below, serve as an abstract.

“This Profile looks at two technologies that were developed to make source texts in the original Greek, Latin and, indeed, any language directly accessible to audiences who have not yet studied – and may never study – the language itself: (1) translations aligned at the word and phrase level with the original text and (2) rich linguistic annotations explaining the part of speech, regularised dictionary form and syntactic function of each word in a corpus (typically called treebanks, because the syntactic structure is commonly visualised as an inverted tree).”

CONCLUSION – TOWARDS THE OTHER AI, AUGMENTED INTELLIGENCE

An enormous amount of work remains to be done improving our ability to generate alignments and treebanks automatically, refining the results of automatic methods for manageable amounts, and on using new refined data to generate better models for automatic analysis. Nevertheless, we finally have in place the basic services that enable new forms of reading and make source texts in the original language intellectually accessible to new audiences. This addresses a critical challenge for linguistically constrained subjects such as ancient Greek and Roman culture where only a relative handful will develop mastery of ancient Greek or Latin, much less mastery of the research publications in languages such as French, German, Italian and Spanish (to mention only traditional European languages of publication). But if we have the basic tools, we are only now starting to understand how these tools are used and how users do and do not benefit. The most important research area for students of the past might not be how to produce new articles and monographs but to learn how we can exploit digital methods to make the results of our work advance the intellectual life of society as a whole.

Gregory Crane
November 15, 2024

A great deal of work towards the new Perseus (Perseus 6) has been going on behind the scenes and that work is now beginning to become visible. James Tauber of Signum University has led the development, with collaboration from Charles Pletcher and Clifford Wulfman. In most cases, the “we” in this narrative means “James Tauber did the work with the rest of us making comments from the peanut gallery.”

We are still not in the final stage of implementation but we are finishing up a major, and probably the biggest, task: creating a sustainable backend to manage a growing range of machine actionable textual data from a widening set of open data projects in multiple countries. As we refine the backend data, we can begin to move to the final stage of work: implementing within the Scaife Viewer the frontend services that support treebanks, grammatical annotations, aligned translations, metrical analyses, automatic mapping, links from edition to manuscripts and that were developed in Beyond Translation.

For now I point out work done to make the existing Scaife/Perseus Greek and Latin accessible in the ATLAS architecture.

Every Greek and Latin text available in Scaife is now available in the new ATLAS architecture. If you start with the Scaife ATLAS homepage (https://atlas.perseus.tufts.edu/) drill down into the CTS library (https://atlas.perseus.tufts.edu/library/), you can find your way into a list of all editions currently available in Scaife (a dozen versions of Thucydides in various languages are available https://atlas.perseus.tufts.edu/library/urn:cts:greekLit:tlg0003.tlg001/), and then down to each of the lowermost citable notes (e.g., https://atlas.perseus.tufts.edu/library/passage/urn:cts:greekLit:tlg0003.tlg001.perseus-grc2:1.1.1/). At this point, you can call for either plain text or xml by adjusting the final argument: https://atlas.perseus.tufts.edu/library/passage/urn:cts:greekLit:tlg0003.tlg001.perseus-grc2:1.1.1/text/ vs. https://atlas.perseus.tufts.edu/library/passage/urn:cts:greekLit:tlg0003.tlg001.perseus-grc2:1.1.1/xml/.

If you explore the Github repository, you will not not only the full text library stored in the simpler ATLAS form (identifier+textchunk) but also complete morphosyntactic analyses for all 40 million words of Greek (using GreCy) and 16 million words of Latin (using LatinCy). Navigating to particular authors is a bit cumbersome because we had too much data and it was necessary to split the data into multiple Github repositories. The information you need to find the right repository for any given Scaife Greek or Latin author is at https://github.com/scaife-viewer/tagging-pipeline/tree/main/data.

We now have three layers of morpho-syntactic data.

1. Actively curated Treebanks (e.g., Francesco Mambrini’s Daphne).
2. The growing repository of curated and automatically generated treebanks from Alek Keersmaeker’s Glaux Trees (currently 20 million words of Greek and including some works that are not yet in Perseus)
3. The comprehensive morpho-syntactic data generated for any texts added to Perseus.

The opening sections of Thucydides (source here):

The identifier+textchunk format is all that we need to add new texts into Perseus. While the greater structure of the TEI XML format that we have used in Scaife has advantages, we can now add much more content much more quickly and work with larger corpora that has been practical in the more demanding XML framework. The goal is not to abandon XML but to provide the XML as time allows while also being able to work with larger, less structured collections.

The morpho-syntactic analysis (link here):

A draft of a fuller write-up for this work is available in Zenodo: The Sixth Generation of the Perseus Digital Library and a Workflow for Open Philology.

Gregory Crane

As far as I can tell, things have settled down on Perseus 4. As I understand it, there were two issues.

First, Tufts Technology Services moved Perseus 4 to a new hardware platform. In the older platform, various checks had been added so that we would automatically reboot Perseus servers if they hung. This automatic rebooting feature was added a long time ago — probably at least ten years in the past. Replacing this automatic feature seems to have slipped our attention (and I should have checked that). It looks like we have made progress on this. It is possible we will need to tweak this reboot feature but for now I am finding that the site works (at least for me). We monitor email to the Perseus webmaster (and try to respond in a timely fashion).

Second (as noted before), we were hit with scraping and of high volume use that differed from previous patterns over the many years that Perseus 4 has been running. Measures have been put in place to mitigate against these behaviors. We will monitor this situation and respond as needed.

I would like to thank Tufts Technology Services for their on-going help.

Gregory Crane

May 4, 2024

Updates on the conversion to Perseus 6 are available here. The update itself is published on the PubPub platform and the shift from WordPress (which is the engine behind the Perseus Updates blog) to PubPub reflects a major component of the overall change.

Gregory Crane
Editor-in-Chief, Perseus Digital Library
Update as of January 19, 2024.

We have now completed work on Beyond Translation (a draft white paper on this work that has been submitted to the NEH Office of Digital Humanities is available here) and are focused on using the Beyond Translation work as part of an update to the Scaife Viewer. The resulting system will finally allow us to replace Perseus 4. We are calling the new version Perseus 6 (rather than 5) to reflect the amount of work embedded in the Scaife Viewer and now Beyond Translation (which we view collectively as Perseus 5). A grant from the NEH Collections and References program in the NEH Division of Preservation and Access for Perseus on the Web — preparing for the next thirty years provides the primary support for this phase of work, with additional support from the Tufts Data Intensive Studies Center, the School of Arts and Sciences, Tufts Technology Services and Google.

Our main collaborator in this phase of development is James Tauber, who is now working with Signum University. We also are waiting on Tufts administrative paperwork to finalize a contract with another group to help us reorganize the Perseus home (and associated sub pages) and replace this WordPress-based blog with the Pubpub Publishing Platform (which we already began using in documenting Beyond Translation.

A draft outline of the work that we are doing is now available here.

For now, the focus of work is to fold the services visible on Beyond Translation into the Scaife Viewer. The first results from that work will probably documentation, with changes to the Scaife Viewer following.

GREGORY CRANE

The Office of Digital Humanities at the National Endowment for the Humanities asks its projects to submit white papers after completing a project. We have posted a draft of the White Paper for Beyond Translation here and invite suggestions as we finalize our work.

dNext up: a formal description of plans to replace Perseus 4. We secured the last chunk of funding needed shortly before the winter break. Once we finalize this report on what we have done, we will provide more details on what we are doing.