Tufts University will introduce a new course in spring 2023: “Natural Language Processing and the Human Record.” Students at Boston College and Boston University can already cross-register to take this course for credit but, insofar as space allows, it will be open to others in person and to a wider potential audience participating online. This project-based course will not only provide opportunities for students of Greek and Latin, but also for students of other historical languages. It also addresses a major gap between the curricula to which most students of historical languages have access and the realities of doing research in a digital age.
Language technologies allow students of the Greco-Roman world to address all three of the intellectual goals that this job posting requests. In particular, students of Ancient Greek and Latin who take advantage of contemporary digital methods will be positioned to work with a variety of languages. The figure below illustrates how we can, for example, now offer dense linguistic annotations for a growing number of sources in historical (and contemporary) languages.
A reading environment such as the one above depends upon a hybrid environment that integrates automated analysis based not only on both machine learning and traditional procedural programming but also on contributions by human experts on the particular source. Knowledge of the language (whether in the form of annotated training data or heuristic rules) provides the starting point for computation and the computation can improve based on expert feedback. We need participants with strengths on both the computational and the content sides (and ideally some participants who can contribute to both sides of this process).
Few programs (if any) are, however, designed to provide students of Greco-Roman culture with the skills that they need to apply digital methods. Those students who do acquire such skills often do so as undergraduates in Computer Science, working in a job involving computation before they begin graduate school, or as something they pick up on the side during graduate school.
In the Tufts Department of Classical Studies, we will be teaching a new course with the title “Natural Language Processing and the Human Record.” It will be taught for the first time in spring 2023 as CLS 191 (and will, in subsequent years, appear with its own number as CLS 162).
This class will be taught on Monday nights from 6:00-8:30 pm so that people who are not regularly on the Tufts campus (such as in-service teachers and students from other local institutions) would be able to participate in person. An existing agreement would allow students from Boston University and Boston College to cross-register, but, if space allows, others are welcome to join. The course is officially listed as in person but we would work to make it accessible remotely as well.
Any intellectually determined student could profitably take this class. First, those who wish to focus on the computational side would need to acquire core skills in programming with Python and in working with Jupyter Notebooks but that is certainly doable, given the variety of online tutorials available, between the end of class in fall 2022 and the beginning of this class. The work required would be non-trivial but effective computational scholarship has long required, and probably long will require, a great deal of informal, on-going, self-directed study.
Second, although ability to apply a range of Python based libraries will always be a big help and extend the intellectual range, students with an interest in how to organize philological data could take this class and focus on topics such as the application and assessment of methods such as morpho-syntactic analysis (particularly the Universal Dependencies Framework), co-reference resolution, translation alignment, named entity recognition and linking, topic modeling, sentiment analysis, and ontology development. Technical terms such as these may be unfamiliar to most practitioners (and do evoke blank stares from most senior scholars), but they represent foundational new building blocks upon which the study of historical languages must be based in a digital age. Early career researchers, as well as students who wish to use their study of the past to prepare them to flourish in the modern world, face very different challenges and opportunities than those who were fashioned by the limitations and assumptions of late print culture.
The course description specifically mentions Ancient Greek and Latin because those are two languages that we know we can support but students are welcome to focus on any language where we have independent expertise to help guide and evaluate their work.
Strong contributions from the course will have an opportunity to be published, with credits for individual authors and/or each member of the team, both in the Tufts Dataverse and (as appropriate) in the new Perseus.
CLS 191 01 Seminar on Current Topics in Digital Humanities: Natural Language Processing and the Human Record Cross listed with GRK 191 01 and LAT 191 01 G. Crane 3 SHU In Person (although we will make remote participation possible0 Mondays, 6:00 – 8:30 pm (local time in Boston, MA, USA)
This class explores the application of natural language processing to the study of the human record and serves two complementary audiences. First, students who are familiar with, or able quickly to develop familiarity with, Python and related technologies can use these skills to develop course projects. Second, students who do not yet have this technical background but who wish to focus on how to publish born-digital versions of historical sources can take his class to develop new ways of reading, organizing and analyzing texts. Students of Greek or Latin who wish to focus on the language can take this class as GRK 191 or LAT 191. Students who wish to focus on another language (including sources in English) are welcome but should consult the instructor. We will cover recent publications and examine current applications that define the state of the art in digital humanities and digital publication. While students will be able to work on their own, we will particularly support the development of collaborative projects in which students with complementary skill sets work together.
Recommendations: CLS 162 – CLS 161, CS 10, CS 11;
For those enrolling in GRK 191 or LAT 191: three or more semesters of study recommended.
This paper announces the creation of a version of the Homeric Iliad and Odyssey that links each line of each poem with those other lines in the Iliad and the Odyssey that share the most significant vocabulary. Each line has at least one parallel. The line with the most parallels (Od. 2.569) has 227 parallels but that is exceptional. The average line has 24.4 parallels. Forty-eight files, one for each book in the Iliad and Odyssey, are available on GitHub and I expect to add them to other repositories in the future. This paper describes how similarity is calculated.
I had created this dataset for the sake of curiosity and I had not intended to publish it. I am publishing this now because, every time I wish to read a passage in Homer carefully, I find myself consulting this hypertextual version of the the poems. I constantly see new connections or simply marvel at the flexibility of Homeric formulaic composition as it reveals itself. I expect that most readers may use this to explore, according to various different approaches, Homeric formulaic composition. The algorithm for ranking lines, known by the abbreviation tf/idf, is (as I will explain) both venerable (not considered a good thing in computer science) and simple.
There are other ways by which to find patterns of traditional composition in Homeric poetry. Each method is quite tractable, given the openly licensed data that we have at our disposal about Homeric epic.
Word embeddings: Word embeddings are a relatively recent technique, made possible by advances in machine learning that have been, in turn, made possible by advances in computing hardware. Essentially, word embeddings can provide the context for particular words to identify related terms (e.g., “rock” and “stone” are similar in meaning). The most recent strategies can capture word senses (e.g., distinguish contexts were the English term “bank” designates a financial institution vs. the side of a river). Word embeddings work best when we have much larger collections than Homeric epic but we could probably make up for lack of data by using the Perseus Treebanks to show which words depend on which and in what function. Embeddings may allow us to measure the degree to which different words are actually synonymous and only appear as different options because they allow the tradition to express an identical idea in different metrical slots of the hexameter.
Collocations: We can measure the extent to which two or more words co-occuring exceeds random chance. This method can allow us to identify significant phrases (e.g., “Peleides Achilles”) even when those phrases are composed of very common words.
Metrical position: We can examine the tendencies of particular words to appear in one or more metrical slots in the line (as has Sansom 2021). If a word is not only in two different lines but in the same metrical position, we can increase the weight that we assign to that shared similarity.
The first and second halves of lines: Repeated phrases tend to cluster before or after the caesura (the main break that divides virtually every Homeric hexameter into two pars) and it would make sense to compare the first and second halves of lines. Anyone looking at this dataset will see, over and over, cases where two lines with relatively modest similarity scores have identical first or second halves (especially when the shared words are themselves common individually).
Each of the techniques above could retrieve more clearly formulaic expressions. In the terms of information retrieval, this would improve the precision, i.e., reducing false positives but at the expense of missing some valid results. Using tf/idf to compare whole lines sacrifices precision to find potentially interesting collocations, turning up related lines that more demanding algorithms would miss. The more general method also reveals the fuzzy line between formulaic composition and the semantics of language.
Consider, for example, the lines most similar to Od. 5.409:
Ζεύς , καὶ δὴ τόδελαῖτμα διατμήξας ἐπέρησα ,
… and indeed, I have cut my way through and crossed this gulf,
νηχόμενος τόδε λαῖτμα διέτμαγον , ὄφρα με γαίῃ
by swimming I cut my way through this gulf …
ἥ με κέλεαι σχεδίῃ περάαν μέγα λαῖτμα θαλάσσης ,
[you, Calypso] who urge me to cross the great gulf of the sea.
The variation τόδελαῖτμα διατμήξας/διέτμαγον, of course, illustrates a single formula that starts in the same slot but varies in the number of subsequent slots it occupies (˘˘| ˉ˘˘ |ˉˉ|ˉ vs. ˘˘ |ˉ˘˘ |ˉ˘˘). What particularly caught my eye was the fact that linking these three lines immediately answered a question that I had in my own mind as a reader. What verb governs λαῖτμα? The translation above (adapted from A. T. Murray’s Loeb edition) takes the noun with both verbs. The two linked lines immediately support this decision. The comparanda are there in front of me as I read. The form διατμήξας may well most directly govern the noun λαῖτμα, but surely the Greek audience would understand that both verbs governed λαῖτμα.
If I want to understand the semantics of the verb peraô, “to cross,” I need to consider the fact that this verb governs the noun laitma, “gulf,” not only in Od. 5.174 but also, at least indirectly, at Od. 5.409. As a side note, this also highlights a fundamental weakness of the Perseus Treebanks – and, indeed, of the use of trees to represent syntax. Trees are a class of graphs in which each node can have one, and only one, ancestor. We cannot have a single word depend on two other words if we use a tree. We can, of course, augment what we store with the tree by adding another layer of annotation easily enough — anyone working with linguistic annotation will have multiple layers of annotation. But the fact is none of the Greek and Latin Treebanks with which I am familiar have added such a layer. On the other hand, our print reference works are often no better: Cunliffe’s Homeric Lexicon notes that perâo governs laitma in Od. 5.174 but does not mention its connection to laitma in Od. 5.409.
It would — and will ultimately — be easy to build an exploratory environment in which readers can modify the parameters to suit their needs and choose among the various linking techniques that I listed above. Such interactive environments are, however, difficult to maintain over time — there is, in fact, no way to be confident that any piece of code will function decades or generations from now. Modern scholarship on Homeric poetry goes back at least to Friedrich Wolf’s 1795 Prolegomena ad Homerum. In my own case, I have (to take three examples that happen to come to mind) recently looked at scholarship about Homeric scholarship published in 1872 (Düntzer), 1928 (Parry), and 1968 (Hainsworth) — even those three works represent a span of nearly one century and extend back more than half a century from the present. If I am consulting scholarship published a century and a half ago, what can I do to maximize the chance that my contributions would be useful in a century and a half?
It may well be more likely than not that the field changes so much that contemporary articles and monographs play little role in the future study of Homeric epic. Nevertheless, we still participate in a conversation that extends over decades, generations and centuries. We should, I believe, do what we can to maximize the survivability of what we do. To do so, I have chosen the following:
Open license: By publishing under an open license, I make it possible for others to make and preserve copies of what I produce. The more use this work finds, the more copies will be made and the better the chances that at least one will survive.
Publication in long term repositories. The most widely used repository for Digital Philology is surely GitHub but GitHub is a corporate project and no one can be sure of its future (although it is likely that a successor platform would take on content in the future if GitHub falters). We also deposit the results in the Tufts University Digital Collections and Archive, which is tasked with long term preservation, and the European Zenodo archive.
Very basic formatting. The tables, pictures, and paragraphs in this description should be supportable for the foreseeable future.
A tab-delimited file as data format. The core results of this work are published in tab-delimited files that should be easily understood. The technical terms tf/idf and edit distance (described below) are not currently well understood among students of Greek and Latin but they are very basic concepts from computer science and readers should be able to learn what they mean even without the narrative explanation offered here. Otherwise, the data includes citations to, and accompanying lines from, the Homeric epics with a list of dictionary entries (lexemes) that each line pair share.
The work described here depends upon two data sources. First, machine readable versions of the Iliad and Odyssey as edited by T. W. Allen more than a century ago (Allen 1920) provides the starting point for this work. Second, the Perseus Dependency Treebank Project used this text to create a linguistically annotated edition of the two epics, that includes part of speech, syntactic function, and a lexeme (the dictionary form) for each inflected form of the two epics. In this analysis, we want to capture different inflected forms of the same word because Homeric formulaic composition will change the form to suit the syntactic needs of the moment. We look for shared lexemes in different lines. The more shared lexemes, the higher the similarity score for any two lines. The tf/idf metric allows us to dampen the weight given to more common lexemes and emphasize less common lexemes.
Anyone with a list of mapping from the inflected forms in Homeric poetry to lexemes could apply this method. Results will differ, though probably not in ways that are significant for most readers of Homeric epic. Different lexica differ slightly in how they map forms to lexica — some lump very different word senses under a single lexeme, others split a forms into two different lexemes. Some editions choose different readings. In an ideal world, we would include variant readings and/or different critical editions — this would be particularly useful for those who are interested in different variants and see the Homeric epics as dynamic parts of a living tradition (multitexts, on which, see the Homer Multitext Project). Results would thus vary based on different schemes of lemmatization and different editions. Those studying Homeric epic as a multitext should create a version of this work that includes a range of different readings. The results published with only a single version of the text, however, already demonstrate the flexibility of Homeric formulaic composition.
After assigning a score to the shared vocabulary in each line of the Iliad and the Odyssey, the algorithm sorts the most highly scored lines first and then prints out the results until the similarity score drops below a preset cutoff point. Each output line contains the following eight tab-delimited fields.
index: Each line of the Iliad and Odyssey is paired with at least one other line based on shared vocabulary.
base citation: An abbreviated version of the citation of the current line that is being compared to every other line in the Iliad and Odyssey. Citations that begin “12-1:” are from the Iliad and “12-2:” from the Odyssey (this follows the convention that the original Thesaurus Linguae Graecae (TLG) established giving Homer the serial number 12 and the Iliad and Odyssey 1 and 2 respectively). Book and line numbers are padded with extra zeroes so that users can easily sort them in a spreadsheet. Otherwise, we would jump from “12-1:1.1” to “12-1:10.1,” with book 2 only showing up after we had completed book 19. Thus, we use “12-1:01.001,” rather than “12-1:1.1”.
base line: the text of the line that the system is comparing to all other lines of the Iliad and the Odyssey.
tf/idf: This figure, rounded to one decimal point, provides a score for the similarity between base line and the line to which it is compared. More extensive discussion follows in a separate section. The key point is that tf/idf provides a way to give more weight to less common words and less to more common words — we do not want to give kai, “and,” the same weight as less common terms when we compare the similarity of two lines of Homer.
Edit Distance: This figure quantifies the similarity of two lines by measuring how many operations it would take to convert one line into the other. We use a standard Python to compute edit distance and the score varies from 0 (for completely different strings) to 100 (for strings that are identical. Very high values for edit distance can pick up small editorial changes that might otherwise not be noticed. Thus, the edit distance between the following lines is 99, rather than 100:
In his edition of the Iliad (3rd edition 1920), T. W. Allen used the simple form onomaze, “she called him by name.” In his edition of the Odyssey (2nd edition, 1917), Allen chose to add the “nu-movable,” an extra consonant that can be added to the end of some forms to keep a short vowel from being elided before another vowel. The “nu-movable” has no effect on the meaning — and is irrelevant at the end of the line. The difference is small but could matter if researchers were studying linguistic usage. In this particular detail, the two editions are simply inconsistent. I only noticed the difference because the edit distance was 99 rather than 100 and I noticed the variation. There will surely be other places where a score just below 100 will reveal more significant changes.
Comparison citation and comparison line: These designate the citation and text of the line that is being compared to the base text line.
Lexemes: This lists shared dictionary words (lexemes) and provides information about their relative weight. Achilles, for example, shows up 382 lines in Homeric Epic and its tf/idf weight is 4.28, while common Peleides, which appears in 59 separate lines, has a tf/idf weight 6.15, a higher value than that assigned to Achilles because it is substantially less common. Both Achilles and Peleides appear once in both lines.
Deciding which lines to link:tf/idf
The tf/idf metric is, by computational standards, a very old method: the British Computer Scientist, Karen Spärck Jones (1935-2007) published the algorithm in 1972, fifty years ago as I write this. Nevertheless, this measure remains widely used and provides a simple mechanism by which to compare search for a query in a collection of documents.
In the case of the work described here, an algorithm uses the normalized dictionary forms for each word in each line of the Iliad and the Odyssey as a query, follows the tf/idf metric to determine the similarity of this query line against every other line in the two poems, and then returns a list with the most highly scored lines at the top. If our query contains a word such as de, the word that shows up in the most lines of Homeric Epic (12,138), we do not want to return all the other 12,137 lines. So we assign a cutoff score (rather arbitrarily set at 10 for now). Every time we return a line, we check to see if we have dropped below the cutoff score. If so, we end the list. We return one entry before checking the cutoff score because we want to have at least one example for each line in the Iliad and the Odyssey.
The lines that have the least similarity to the rest of Homeric Epic both appear in the catalogue of ships:
Τροιζῆν’ Ἠϊόνας τε καὶ ἀμπελόεντ’ Ἐπίδαυρον
Troezen and Eïonae and vine-clad Epidaurus,
Λίνδον Ἰηλυσόν τε καὶ ἀργινόεντα Κάμειρον,
Lindos and Ialysus and Cameirus, white with chalk.
The only words that these two lines share with any other lines in the Iliad and Odyssey are the connectors te and kai, which together are conventionally translated “both and.” These lines are both equally similar to every other line in the Iliad and Odyssey that have this common combination.
The two lines above also demonstrate another point: different methods detect different kinds of patterns. Readers familiar with Homeric formulaic composition will, of course, immediately see that the two lines above are very similar in form and clearly share much the same formulaic pattern. Each line contains, besides the connectors, only proper names and none of the proper names (surprisingly) shows up elsewhere in the epic poems. The two lines least similar to the rest of the epic poems when that similarity is measured by vocabulary are almost identical in terms of formulaic structure. The Perseus Treebanks do allow us to search for similarity based on the parts of speech or the syntactic labels and dependencies in each line. The work described here focuses strictly on vocabulary.
The expression tf/idf contains two acronyms: term frequency (tf) and inverse document frequency (idf). The measure assumes that we have a collection broken up into documents. Documents can, in turn, be journal articles, news stories, or chapters of a book. But documents can also be individual sentences — or, as in the case of the work described here, individual lines of poetry.
When we compare two lines, we want to give more weight to a word such as mênis (which is conventionally translated as “wrath” or “anger and which appears in 16 lines of Homeric poetry) than we do to the connector de (which is roughly equivalent to “and” and shows up in 12,138 lines), while occurrences of the name Achilles (which shows up in 382 lines) should be somewhere in between. The tf/idf metric yields weights of 0.92, 4.28, and 7.45 for de, Achilles, and mênis respectively. We calculate the similarity of two lines by adding the tf/idf values for each shared dictionary entry. Consider the opening line of the Odyssey and its closest match, which comes from the Catalogue of Ships in the Iliad.
Each word is accompanied by the number of different lines of the Iliad and the Odyssey in which it occurs, the corresponding tf/idf score based on that frequency, and then number of times it appears in both lines (i.e., if it occurs 2x in one line and 3x in the other, we count it as 2x). If we add the tf/idf scores for egô (“I'”), enepô (“to utter”) and Mousa (“Muse”), we get a score of 16.3 (rounded to one digit).
Traditionally, tf/idf is applied to collections where the documents are long enough that the word frequency within documents matters. Consider a collection of 100 newspaper articles in which two words appear 20 times. The first of these words may be a typical, medium frequency word such as “president” that could apply to a number of figures while the other may be more tightly bound to a particular topic (“Macron”) and cluster in two or three documents. The tf/idf measure is designed to capture the fact that, in such a case, “Macron” tells us more about a document than does “president,” even though each appears the same number of times in the corpus.
Lines of Homer are much shorter than newspaper articles. At one extreme, 4 lines contain only 3 words: ranging from 3 words (4x: Il. 2.706, 11.427, Od. 10.137:
αὐτοκασίγνητος μεγαθύμου Πρωτεσιλάου
αὐτοκασίγνητον εὐηφενέος Σώκοιο
αὐτοκασιγνήτη ὀλοόφρονος Αἰήταο·
κολλητὸν βλήτροισι δυωκαιεικοσίπηχυ
Three begin with with forms of αὐτοκασίγνητος. The fourth, 15.678, contains a single token that contains four distinct words (δυωκαιεικοσίπηχυ including kai, “and”).
On the long end, the treebank counts 13 tokens in only two lines, and in each case that count is two words higher than most would list because, in adding syntactic annotation, we split οὔτε into οὔ τε.
The other two lines are both variations on a formula that begins ἂψ δʼ ὅ γʼ ἐπʼ οὐδὸν ἰὼν κατʼ ἄρʼ ἕζετο. The second occurs relatively quickly after the first (a gap of 250 lines, < 1% of the 27,000 lines of Homeric poetry).
Three quarters of all lines are 6, 7, or 8 words long (22%, 30%, and 23% respectively). Almost 99% are 4 to 10 words long.
The tight clustering of line lengths stands out from bar chart.
Sentences with 3, 12, 13 and 14 words are so few in number that they are not even visible in the figure above.
Such lines obviously contain many fewer words than documents such as even short news articles and most words will only occur once in a line. Nevertheless, of the 27,793 lines in the Allen edition of the Homeric Epics used by the Perseus Treebanks, 3,989 (14%) have two or more instances of the same lexeme. The majority of these lexemes are, however, very high frequency words.
in lines 2+x
in lines 1+x
The three connectors, de (which is the neutral particle indicating a new sentence) and te, and kai (which are often equivalent to “and”) account for more than 2,000 lines — more than half of all duplicates. Such repetitions are very useful but it is not clear that the line linking strategy described here is the most useful way to study such syntactic repetition. The tf/idf measure is most fully effective when documents are relatively consistent in size (as the lines of Homer are) but the lines are so short that most repetitions involves function words rather than content. The repetition of function words can reveal syntactic patterns (especially when we combine them with metrical information and with syntactic data from the treebank) but repetition of words within a line usually does not tell us much about shared semantic content.
There are, however, cases where repetition captures formulaic repetition across lines.
Il. 2.489 appears during the dramatic opening to the Catalog of Ships The poet is describing various limitations (including not having enough tongues and mouths) that make it impossible to describe the combatants at Troy. Il 23.851, by contrast, lists the prize that Achilles sets for the contest in archery.
The two lines above clearly share the same formulaic pattern with five words starting in identical metrical slots but the differing syntactic structure clearly brings out the flexibility of Homeric formulaic composition. In Il. 2.489, the tongues and mouths are the subjects of the verb eien, a form of the verb “to be,” and this verb appears at the end of the line. In Il. 23.851, the ten double-axes and ten single-axes are the objects of the main verb, tithei, “he set down” (taking κὰδ, “down,” as modifying the verb).
Two of the three shared lexemes, the connector de and the particle men have very little weight.
found in lines of Homer
The repetition of deka doubles its weight to 14.684 — more than enough for the line to exceed the cutoff score of 10.
Likewise, in the following example, the repetition of heneka, “because of”:
εἵνεκ‘ ἐμῆς ἔριδος καὶἈλεξάνδρουἕνεκ‘ ἀρχῆς·
because of my quarrel andbecause of the beginning of Alexandros
εἵνεκ‘ ἐμεῖο κυνὸς καὶ Ἀλεξάνδρουἕνεκ‘ ἄτης,
because of me, the bitch, and because of the madness of Alexandros
Each instance of heneka adds 5.733 to the score — the single repetition is enough by itself to create a link between the two lines. The two lines are clearly variations on one another. First, we can point out that the choice of two different forms of the preposition, heineka and then heneka, allows the same word to fill two differently shaped metrical slots (long-short in the first case, short-short in the second).
Second, each line is slightly different in meaning and even in structure. In Il. 3.100, Menelaus is speaking to the Greek and arguing that a proposed one-on-one combat should be between himself and Alexandros. He refers to “my quarrel,” because the war is ostensibly being fought to return Helen. The form emês is a possessive adjective that means “my.” The line ends with the word archês, “beginning,” because it was the Alexandros who began the war.
In Il. 6.356, Helen is speaking to Hector on his visit to Troy. She speaks of herself and of Alexandros is very harsh terms because they are responsible for the war. She, however, uses emeio, a genitive form of the first person pronoun (rather than the adjective “my”). The phrase means “of me, the dog” (properly translated by English “bitch”). Here the noun kunos, modifies emeio whereas in the earlier line emês, “my,” modifies eridos, “quarrel.” And Helen refers to the “idiocy” (atês) rather than simply the more matter-of-fact “beginning” of Alexandros that has caused the war. The information conveyed in the two lines differs somewhat. The second variation is far sharpe, even brutal. This offers an excellent example of Homeric traditional composition.
I have, as I will describe below, found this format useful for tasks such as computing similarities among different books of the Iliad and the Odyssey and for tracking the varying degree to which different lines in different parts of the epics share more or less vocabulary. I try to avoid stating that lines with fewer attested parallels are not formulaic and to point out, rather, the extent to which we can observe parallels in the 200,000 words of our Iliad and Odyssey.
Using the files
Each file contains one book of the Iliad or the Odyssey. Readers can simply scroll through and examine the parallels for each line. I use these files as a kind of dynamically generated commentary. Whenever I read a passage closely, I look to see how many — or how few — linked lines there are and what the parallels reveal.
μῆνιν ἄειδε θεὰ Πηληϊάδεω Ἀχιλῆος
Sing, goddess, the wrath of Achilles, son of Peleus,
The first line of the Iliad shares two lexemes with sixteen other lines of Homeric epic. To make the exposition clearer, I have optimized the list below a bit. I don’t present all the information that each line in the data set includes. I have highlighted the shared lines. I have also resorted the results to make the word order and different spellings stand out. The changes only make more noticeable patterns that a careful viewer would quickly detect.
εἰ μὲν δὴ θεράπων Πηληϊάδεω Ἀχιλῆος
Μυρμιδόνες ἕταροι Πηληϊάδεω Ἀχιλῆος
εὗρον δὲ ψυχὴν Πηληϊάδεω Ἀχιλῆος
ὄφρ’ ἠῢς θεράπων Πηληϊάδεω Ἀχιλῆος
ἔρχεσθον κλισίην Πηληϊάδεω Ἀχιλῆος·
ἔλθωσ’ ἐς κλισίην Πηληϊάδεω Ἀχιλῆος.
νεῖκος Ὀδυσσῆος καὶ Πηλεΐδεω Ἀχιλῆος,
Πηλεΐδεω Ἀχιλῆος ἐναντίβιον πολεμίξειν;
Πηλεΐδεω Ἀχιλῆος· ὃ δ’ ἀνστήσει ὃν ἑταῖρον
Πηλεΐδην Ἀχιλῆα δίκῃ ἠμείψατ’ ἀναστάς·
Πηλεΐδῃ Ἀχιλῆϊ, ποσὶν δ’ ἀπὸ τείχεος ἄλλῃ
Πηλεΐδῃ Ἀχιλῆϊ· ὃ δὲ μελίην εὔχαλκον
Πηλεΐδῃ Ἀχιλῆϊ· κακῶν δέ κε φέρτατον εἴη.
Πηλεΐδῃ Ἀχιλῆϊ κακὸν ἔπος ἀγγελέοντα.
Πηλεΐδῃ Ἀχιλῆϊ δαμήμεναι, ἐσθλὸν ἐόντα.
αὐτὰρ Ἀχιλλεύς ἐστι θεᾶς γόνος , ἣν ἐγὼ αὐτὴ
The last line in the long list above would probably be considered noise to someone exploring the workings of formulaic poetry. Achilles shows up in both but the other shared word, theâ, “goddess,” refers to the Muse in Il. 1.1 and to Achilles’ mother, Thetis, in Il. 24.59. Observers with no knowledge of Greek would see this if they consulted the translations. All readers, whether they know Greek or not, would, of course, have to understand the context of Il. 1.1 and Il. 24.59 and thus to conclude that the pairing of Il. 1.1 and Il. 24.59 probably reflects a random collocation.
But even this line reveals an important feature of the formulaic system: in this, the last of sixteen examples, the name Achilles is spelled, for the first time, with two lambdas (-λλ- vs. -λ-). Homeric tradition exploits the ability to alter the spelling because the -i- in the name Achilles is short (in Greek, the letters a, i, and u can be long or short) but by doubling the following lambda, the short syllable is lengthened. The tradition offers two spellings for Achilles and the name can thus fit into different metrical slots in the line.
Most observers, with or without knowledge of ancient Greek, would immediately see the dominant feature of “Achilles the son of Peleus”: this pair of words always appears either at end or the beginning of a line.
Readers who look closer will notice that the words are not spelled identically. Ancient Greek is highly inflected and we have the same formula not only in different positions (line beginning and end) but also in different cases: genitive (PêlêiadeôAchilêos, “of Achilles the son of Peleus”), accusative (Pêlêiadên Achilêa, “Achilles the son of Peleus as the object of a verb), and dative (Pêlêiadêi Achilêi, “to/for Achilles the son of Peleus). But even without knowing the details of the case system, we can see that there are both continuities and variations in this same formulaic expression.
The tab delimited files can easily be loaded into a spreadsheet and resorted. You could, for example, look at those lines of Iliad 1 (for example) that have the highest similarity scores. This will identify lines with identical vocabulary, with the least common shared vocabulary generating this highest scores.
Lines that are not quite identical but that share less common vocabulary will sometimes score higher than identical lines with common lexemes (e.g., Il. 1.481 and Il. 1.480).
I cited the two lines that share the lowest similarity score to the rest of Homeric epic above. The lines with the highest similarity score in Homeric epic both have scores of 49.8:
ὥρῃ ἐν εἰαρινῇ ὅτε τε γλάγος ἄγγεα δεύει,
ὥρῃ ἐν εἰαρινῇ, ὅτε τε γλάγος ἄγγεα δεύει·
ῥίμφά ἑ γοῦνα φέρει μετά τ’ ἤθεα καὶ νομὸν ἵππων·
ῥίμφά ἑ γοῦνα φέρει μετά τ’ ἤθεα καὶ νομὸν ἵππων·
I also published two files with summary data. The first, booktots.tsv, publishes the average number of linked lines for each book of the Iliad and the Odyssey.
More linked lines reflects more attested repetition and says something about the visibly formulaic nature of each book. I add a graph that illustrates which books of the Iliad and the Odyssey have the most links per line. The range is quite wide — from under 19 links per line in Iliad 12 to almost 29 links per line in Odyssey 16. A range of 50% strikes me as large and surprising.
The figure reveals several features. First, there is more attested repetition in the Odyssey than in the Iliad — a much bigger difference than I would have expected (although some who see the Odyssey as derivative from the Iliad may claim not to be surprised). Second, the book of the Odyssey with the least attested repetition is book 9, which tells the story of Polyphemus. That suggests to me that this story draws from traditions that are unusually distinct from the rest of the Odyssey. Third, the book of the Iliad with the most repetition is Iliad 10 — which many have seen as both an outlier and as more Odyssean. Dué and Hackney (2010) have made strong arguments that Iliad 10 stands out because it draws on traditions of ambush that are not widely attested elsewhere in the Iliad and Odyssey and that that distinctive content makes this book appear as an outlier. Their argument makes the fact that Iliad 10 has by far the most attested repetitions of any book in the Iliad all the more striking.
Second, I include a file measuring how many linked lines connect each book in the Iliad and the Odyssey with each other. I have added the number of shared words in all linked lines for any two books and, to make up for the fact that different books vary so much in size, then divided by the sum of the number of lines in the two books combined. Each book then has a number ranking its closeness to the other 47 books. There is more to be done here but I think that the preliminary data is worth sharing. It is available as book2book.tsv. The following are the 10 links that, by this metric, have the lowest scores and thus are least similar.
Iliad 12 and Odyssey 7 have the least in common. Note that the score from Iliad 12 to Odyssey 7 is not quite the same as going from Odyssey 7 to Iliad 12. That results from the way we manage uncommon lines but the variation is not large.
The most closely connected books follows:
Odyssey 16 and 17 are the most similar books by this metric and Odyssey books are more like each other than Iliad books are. That may reflect the fact that the Odyssey does not need a stream of otherwise unknown figures who appear only to die in battle at the hands of better known heroes. But I suspect that it says something about the composition of the Odyssey.
The full file contains 2256 different links and others can compare them at greater length.
Hainsworth, J. B (1968). The Flexibility of the Homeric Formula. Clarendon P., 1968, https://catalog.hathitrust.org/Record/001223030.
Parry, Milman (1928). “L’Épithète Traditionnelle dans Homère: Essai sur un problème de style Homérique.” The Center for Hellenic Studies, 1928, https://chs.harvard.edu/book/parry-milman-lepithete-traditionnelle-dans-homere-essai-sur-un-probleme-de-style-homerique/.
Parry, Milman (1928b). “Les formules et la métrique d’Homère.” The Center for Hellenic Studies, https://chs.harvard.edu/book/parry-milman-les-formules-et-la-metrique-dhomere/.
Sansom, Stephen A. “Sedes as Style in Greek Hexameter: A Computational Approach.” TAPA (Society for Classical Studies), vol. 151, no. 2, 2021, pp. 439–467, https://doi.org/10.1353/apa.2021.0017.
Spärck Jones, K. (1972). “A Statistical Interpretation of Term Specificity and Its Application in Retrieval”. Journal of Documentation. 28: 11–21.
[Preprint of the abstract for Digital Humanities 2020: Responding to Asian Diversity – https://dh2022.adho.org/]
Our work explores the hypothesis that a new mode of reading is taking shape, one in which dense, machine actionable annotations allow readers to work directly and effectively with sources in languages that they do not know – a new middle space between reliance on translation and mastery of the source text (Crane et al. 2019, Crane 2019). This hypothesis has substantial potential importance for our ability to use source texts to explore cultural diversity in general and the diversity of Asian cultures in particular. Our particular work focuses on two challenges for a traditionally Eurocentric subject, Classics (or Classical Studies), which is still used to describe the study of Greco-Roman culture. On the one hand, university students without training in Greek and Latin in secondary school have difficulty mastering the languages and learning about the subject. In spring 2021, the Princeton Classics Department provoked controversy when it made it possible for majors to study Greco-Roman antiquity without learning any Greek or Latin — too few students, especially students of color, had access to Latin, much less Greek, before college (Wood 2021). At the same time, Classics and Classical Studies are far too narrow – we must include other classical languages – Sanskrit, Classical Chinese, Classical Arabic, etc. – if we are to continue using these terms. We report on work that addresses both challenges.
In order to explore this broad topic, we chose to focus on two complementary corpora in two Classical languages: the Homeric epics in Ancient Greek and the Shahnameh in early-modern Persian (Firdawsī 1430, 1988). The goal is both to support Persian speakers who wish to work directly with Homeric Epic and English speakers who wish to engage directly with the Shahnameh. In the case of Persian culture, the links with Greco-Roman culture are deep, the information in Greek sources about ancient Persian history is extensive, and the influence of Greek philosophy, medicine and science are extensive. At the same time, few institutions in Europe and North America, for example, teach modern Persian, much less the early modern Persian of the Shahnameh. We hope to increase the role that the Persian epic, in Persian as well as in translation, plays beyond the Persian speaking world.
The use of dense linguistic annotation to make sources accessible to a broader audience is, of course, hardly new. In his late seventeenth-century description of Ottoman Turkish, Arabic, and Persian, for example, Franciszek Meniński (1680) introduced Persian poetry to a European audience by transliterating a passage from a poem by Hafez, providing a word-by-word translation, and providing detailed explanations of the metrical, morphological, and syntactic function of each word. Contemporary linguists depend upon exhaustively annotated text to work across sources from the thousands of languages, ancient as well as modern, in the human record (Werning 2009).
Digital methods, however, fundamentally change our ability (Berti 2019, Schulz 2021). First, we can use natural language processing pipelines such as Stanza and Spacy (Papantoniou and Tzitzikas 2020), multilingual language models such as BERT (Bamman and Burns 2020), machine translation (Kontogianni et al. 2020), most effective for now between modern languages (Bowker 2021), and similar openly licensed resources. In the work that we present, we document the categories which we have found as starting points to augment machine readable texts. The Homeric epics have provided a useful starting point because a particularly rich set of preexisting, open digital resources are available upon which to build. The work with Homeric epic provides us with the framework upon which we are building work with the Shahnameh. We will report on the work with Homeric Greek and summarize progress with pre-modern Persian (building on Pizzi 1881).
Exhaustive Metrical Analysis and accompanying sound: Machine-actionable metrical analyses (Schoisswohl and Papakitsos 2020) for every syllable in every line of the Iliad and the Odyssey and readings for a substantial portion of the epics are available under an open license (Chamberlain 2021). From the time they learn the Greek alphabet learners engage immediately with the Homeric epic as performed poetry, following metrical diagram and performance.
Treebanks document features such as the dictionary entry, part of speech, and syntactic function of every word in a source (Keersmakers, 2019). Treebanks are available for both the Iliad and Odyssey and for more than 1 million words of Greek and Latin (Bamman and Crane, 2011; Celano 2019). We can use these to identify and quantify grammatical structures that students will encounter in the corpus that they will learn.
Paradigm information identifies the morphemes within each individual word and allows learners to see which morphological patterns are most common and to prioritize their learning.
Born-digital aligned translations are created from the start to expose the linguistic structures of a source. From the first lessons, learners can explore the meaning of vocabulary by seeing passages where these words appear (Palladino 2020, Palladino et al. 2021). They can focus on words introduced in a lesson but, in using the aligned translation, they gain constant incidental exposure to words that they have not learned. With translation, the problem of the learner language becomes far more pressing and we can report on the varying challenges of articulating Greek language with translations into English and Persian (Foradi 2020).
Grammatical explanations explain the patterns that learners encounter in the annotations (Mugelli 2021) described above (e.g., the various uses of the dative in Greek or the way we translate the imperfect tense). Grammatical explanations build upon the aligned translations. Grammatical explanations cannot simply be translated but must be adapted to bridge the gap between the target language and the learner language. We report on the differences that arise for speakers of Persian and English.
We will report upon experiences of learners from both Iran and the United States, upon our experiences opening historical students to new audiences (e.g., Classics majors who do not know Greek or Latin) and cross-cultural explanation of content (e.g., Homeric Epic for Persian speakers and Persian poetry such as the Shahnameh for English speakers). Bibliography
Bamman, D., Burns, P. (2020). “Latin BERT: A Contextual Language Model for Classical Philology.” https://arxiv.org/abs/2009.10053
Bamman, D., Crane, G. (2011). “The Ancient Greek and Latin Dependency Treebanks,” in Language Technology for Cultural Heritage, ed. Caroline Sporleder, Antal van den Bosch, and Kalliopi Zervanou, Theory and Applications of Natural Language Processing (2011), pp. 79–98.
Berti, Monica, ed. (2019). Classical Philology. Ancient Greek and Latin in the Digital Revolution. Berlin: De Gruyter Saur.
Bowker, Lynne. (2021). “Digital humanities and translation studies.” Handbook of Translation Studies: Volume 5 5 (2021): 37-.
Celano, Giuseppe. (2019). “The Dependency Treebanks for Ancient Greek and Latin”. In Digital Classical Philology. Ancient Greek and Latin in the Digital Revolution, edited by Monica Berti, 279–298. Berlin: De Gruyter Saur.
Chamberlain, David. (2021). “A Reading of Homer (work in progress).” Greek and Roman Verse, https://hypotactic.com/my-reading-of-homer-work-in-progress/. Accessed 8 December 2021.
Crane, G. R; Shamsian, F.; et al . (2019). “Confronting Complexity of Babel in a Global and Digital Age.” DH2019: Digital Humanities Conference, Book of Abstracts (2019), pp. 127–138. https://dev.clariah.nl/files/dh2019/boa/0611.html
Crane, Gregory. (2019). “Beyond Translation: Language Hacking and Philology.” Harvard Data Science Review 1, no. 2. https://doi.org/10.1162/99608f92.282ad764.
Firdawsī, A. & Khaleghi-Motlagh, D. (1988). The Shahnameh: Book of kings. New York: Bibliotheca Persica.
Firdawsī, A. & Ja’Far, P. C. (1430) The Book of Kings. Tehran: Cultural Heritage Organization. [Pdf] Retrieved from the Library of Congress, https://www.loc.gov/item/2021667287/.
Foradi, Maryam. (2020). Engagement with Classical Literature in the Framework of a Citizen Science Project Using Translation Alignment: Date Accuracy and Pedagogical Effectiveness, [Doctoral dissertation, University of Leipzig]
Keersmaekers, Alek. (2019). “Creating, Enriching, and Valorizing Treebanks of Ancient Greek.” 18th International Workshop on Treebanks and Linguistic Theories (TLT, SyntaxFest 2019), https://doi.org/10.18653/v1/W19-7812
Kontogianni, A., et al. (2020). “Computer-Assisted Translation of Egyptian-Coptic into Greek.” Journal of Integrated Information Management, http://ejournals.uniwa.gr/index.php/JIIM/article/view/4470
Meniński, F. (1680). Thesaurus Linguarum Orientalium Turcicae, Arabicae, Persicae, Praecipuas earum opes a Turcis peculariter usurpatas continens, Vienna: Franciscus a Mesgnien Meninski.
Mugelli, Gloria et al. (2021). “Learning Greek and Latin Through Digital Annotation: The EuporiaEDU System.” In: Teaching Classics in the Digital Age. Kiel: Universitätsverlag Kiel | Kiel University Publishing. S. 25–36. (= Think! Historically: Teaching History and the Dialogue of Disciplines). Online unter: https://macau.uni-kiel.de/receive/macau_mods_00001367.
Palladino, Chiara (2020). ”Reading Texts in Digital Environments: Applications of Translation Alignment for Classical Language Learning.” The Journal of Interactive Technology Pedagogy, Issue 18, December 10, 2020, https://jitp.commons.gc.cuny.edu/reading-texts-in-digital-environments-applications-of-translation-alignment-for-classical-language-learning/
Palladino, C., Foradi, M., Yousef, T. (2021). “Translation Alignment for Historical Language Learning: A Case Study.” Digital Humanities Quarterly, 15 (3), http://digitalhumanities.org/dhq/vol/15/3/000563/000563.html
Papantoniou, K., Tzitzikas, Y. (2020). “NLP for the Greek Language: a Brief Survey.” SETN 2020: 11th Hellenic Conference on Artificial Intelligence, pp. 101-109. https://doi.org/10.1145/3411408.3411410
Pizzi, I. Manuale della lingua persiana. Grammatica, antologia e vocabolario, Leipzig, W. Gerhard (1881). Available at https://archive.org/details/manualedellalin00pizzgoog/
Schoisswohl, O., Papakitsos , E. C. (2020). “Automated metric profiling and comparison of Ancient Greek verse epics in Hexameter.” Linguistik Online, 103(3), 159–177. https://doi.org/10.13092/lo.103.719
Schulz, Konstantin (2021). “Natural Language Processing for Teaching Ancient Languages.” In: Teaching Classics in the Digital Age. Kiel: Universitätsverlag Kiel | Kiel University Publishing. S. 37–48. (= Think! Historically: Teaching History and the Dialogue of Disciplines). Online unter: https://macau.uni-kiel.de/receive/macau_mods_00001368.
Werning, Daniel A. (2009) “Glossing Ancient Egyptian. Suggestions for Adapting the Leipzig Glossing Rules.” Lingua Aegyptia. Journal of Egyptian Language Studies, https://www.academia.edu/1484975/Glossing_Ancient_Egyptian_Suggestions_for_Adapting_the_Leipzig_Glossing_Rules.
Wood, G. (2021, June 9). Princeton Cancels Latin and Greek. The Atlantic. Retrieved December 10, 2021, from https://www.theatlantic.com/ideas/archive/2021/06/princeton-greek-latin-requirement/619136/
In part one, I discussed the frequency of different vocabulary items and ways by which learners could track how many words they knew in a particular corpus as they went through some sequence of vocabulary acquisition. I focused upon the first book of the Homeric Iliad as an immediate target corpus and on how learning the vocabulary of this chunk would transfer when readers moved to other sections of the Iliad and Odyssey.
I assume the ability to visualize aggregate changes in vocabulary knowledge as in Figure 1 (below):
Figure 1 shows how learners working with Clyde Pharr’s Homeric Greek encounter 100% of all vocabulary items in Iliad 1 over the course of the book’s 77 chapters. Running words belonging to known vocabulary items are represented as green, new words as blue, unseen and common words as red, and unseen but uncommon words as orange.
Figure 2 illustrates how known, new, and unknown words are distributed throughout Iliad 1. I should emphasize that the visualization is itself a general tool designed to show the distribution of different phenomena across the 48 books of Homeric Epic with a particular focus on one book at a time. It could track any variables — e.g., dialogue vs. narrative, references to Achilles vs. Hector, or to optatives or particular clusters of repeated words. In this case, we just happen to be tracking and prioritizing what words learners have and have not seen.
Figure 2 (above) provides a visualization of the Homeric Epic, with a focus on book 1 of the Iliad. The visualization is a simple mockup to explore features that a more mature and technologically sophisticated visualization would exhibit. It consists of a SVG figure in an HTML file. It offers limited interactivity — mousing over the representations of individual words and books of the Iliad and Odyssey causes information to appear (in figure 1, there is information about the word boulê, “will, plan” (as in the “plan of Zeus”) that appears 8th (and as the final) word in line 5 of Iliad 1.
The visualization contains two major parts. The left part, in a combination of black and grey, provides an overview of the Iliad and the Odyssey.
The left side of the figure above provides a representation of each book of the Iliad and of the Odyssey, 48 books in all. The length of each column reflects the relative number of lines in each book. The book illustrated in the main part of the visualization is highlighted as black, with the other 47 books as gray.
Colors were chosen based on the recommendations of Bang Wong [“Points of View: Color Blindness,” Nature Methods 8 (2011)] to minimize the impact of different forms of color blindness to make the color palette of visualization as accessible as possible. Colors describe six different features:
Vocabulary items that students have seen so far (in figure 1, the first 10 chapters of Pharr’s Homeric Greek) are lime green.
Vocabulary items introduced in the most recent lesson (chapter 10 of Pharr’s Homeric Greek) that appear more than 3 times in Iliad 1 appear as blue.
Vocabulary items introduced in the most recent lesson that appear 3 or less times appear as light sky-blue. The goal is to help learners prioritize the vocabulary on which they focus, emphasizing more active command for more common words.
Vocabulary items that learners have not encountered and that will appear more than 3 times in Iliad 1 appear as vermilion (a darker reddish color). Learners can skip ahead and see which frequent terms appear.
Vocabulary items that learners have not encountered and that will appear 3 times or less in Iliad 1 appear as orange.
Punctuation marks are listed as black. This allows readers to get an overview of sense breaks in the poetry (insofar as editors have chosen to represent these sense breaks with punctuation). At present, every form of punctuation is marked. It may well make more sense to include only full stops (and not commas) as a default.
The main section of the visualization presents one small box for each word in the Iliad 1. The length of each row reflects the number of words in each line. There are breaks between every 10 lines and each column contains up to 100 lines.
Figure 5 shows a close-up from figure 1. Almost every word in lines 1-5 is green (a known vocabulary item) or black (punctuation): Pharr begins with the start of Iliad 1 and moves through the poem a few lines at a time in each chapter. The one word that is unseen (word 3 in line 5) is a form of the adjective pas, “all, each.” The grammar has not introduced third declension adjectives yet and thus this word is not yet in the vocabulary (it was covered with a note to the text)
Figure 6 illustrates information about a particular word. The reader has moused over the 4th word in Iliad 1.109, sees the text of this line, discovers that it is a form of the name Danaans (one of several words to describe the Greeks) and that this word shows up 10 times in Iliad 1. A gloss from Pharr, the inflected form and a code for the part of speech (noun, plural, masculine, dative) follows.
Figures 7, 8, and 9 (below) show how the learners knowledge of vocabulary changes as they move through the text book, with snapshots showing where they stand.
Figure 7 shows the state after chapter 20 of Pharr. This chapter covers Il. 1.28-32. A cluster of dark and light blue dots (depending on their frequency) in these lines shows how new vocabulary presented in this chapter was designed to fill in the gaps for these new lines. Places where the vocabulary presented in chapter 20 appears later in book 1 also appear as light or dark blue dots.
Even at the relatively low resolution of the image above, a solid block of green now appears at 1.371-379 and that block shows, not only to those of us who have worked on Homeric poetry for years but also to those learning Homeric Greek, how the oral tradition works. Iliad 1.372-375 repeats 1.13-16; 1.376-379 repeats 1.22-25. Iliad 1.371 does not repeat 1.12 but every word in 1.371 has already occurred. We can also note that 1.371-379 repeats two different chunks that are separated earlier in book 1 by a speech: Chryses’ request that the sons of Atreus accept ransom for his daughter (1.17-21). We can see at work here how the tradition can change the length of an account by selectively adding or subtracting lines.
Chapter 30 covers Il. 1.81-85. Almost all of the words through line 82 appear as green. New vocabulary clusters in the lines covered in this chapter.
Figure 9 shows the state in chapter 40, which covers Il. 1.158-164. Blue dots cluster around the seven newly introduced lines. At the same time, the share of green dots, representing known vocabulary, has increased dramatically since figure 1.
At some point, learners will presumably want to move on another text beyond the first book of the Iliad. At the moment, we are able to recalculate what vocabulary readers will have seen if they move to a new book of Homer after any given chapter of Pharr.
Work to be done
The images above are, as noted earlier, only mockups to help think through what a more finished system for learners to track what they (should) have learned and what they will need to learn going forward. I list below some of the more obvious topics.
Support for word senses: the visualizations above assume that once learners have encountered the piles of glosses and synonyms that Pharr offers in his glosses, they can recognize the meaning of that word in any subsequent passage. Since words have multiple meanings –some words have very many meanings — that assumption is clearly false. In fact, we have several machine readable Greek-English dictionaries such as Liddell-Scott Jones, Cunliffe’s Homeric Dictionaries (including his dictionary of people and places), and the new Cambridge Greek Lexicon (which will hopefully appear on the Scaife Viewer in 2022). We spent a fair amount of time capturing the structure of the dictionary entries and giving each sense that had a label in the print lexicon a unique digital identifier. We could experiment with the benefits of linking to particular word senses and not just to the dictionary headwords.
Support for inflection: visualizations above assume that once learners have encountered a dictionary entry, they can understand all of its forms. Greek verbs, however, have many different inflectional pattern and traditional learners of Greek spend much of their time learning the many ways Greek verb forms can be generated. We actually have segmented words (as appropriate) into preverbs, augments, stems, and endings, with labels for different inflection categories (e.g., first vs. second declension). We could identify which forms of a new verb learners could already parse and which forms follow paradigms that they have not yet learned. In an earlier version of this work, I tracked which forms learners could parse at any given time but decided that the added complexity was not worthwhile (at least until we have a more customizable system). As it is, we have the inflectional analyses for every word in the Homeric Epics. Learners develop practice using lookup tools to parse unknown forms.
Relative frequency in each of the 48 books of Homeric Epic: The left-hand side macro-view should show how many words readers would know for each of the 48 books of Homeric Epic. Readers could see at a glance if some books stand out because they share more or fewer vocabulary items. Such a visualization requires normalization — the longest books of the Iliad are twice as long as the shortest books in the Odyssey.
Implementation as a fully-interactive, browser based visualization: The SVG implementation shown above does support some interaction but a more mature implementation would allow for many different use cases and could be customized in many different ways.
A great deal of work in digital philology goes into the backend processing of textual data and into the production of new conclusions. We also need a great deal of work on the design of very basic visualizations that can reveal through our highly developed visual abilities linguistic patterns that our ears will not catch and that our eyes will not see as they scan text a word at a time. We need — and we will ultimately have — new dashboards — aggregations with multiple visualizations at a glance — that allow us to work with our textual sources at scale. Once we happen upon visualizations that reach a certain level of functionality, the field will cluster around them. Soon they will be taken for granted as if they had always been there. We are, however, a long way from that state. My goal here is to push the process along, whether that leads to improving what I have presented or to a wholly different, and more effective, approach.
This paper is designed to be the first of two that explore the degree to which learners can track how much of the vocabulary as a whole in a target corpus they have encountered and to see the frequency in the rest of the corpus of each newly encountered vocabulary item. We focus here upon the Ancient Greek Iliad and the Odyssey, a corpus of just over 200,000 running words. Homeric Epic provides a useful starting point because a growing cluster of openly-licensed, digital resources for this corpus are available, including links from each form in the epic to a dictionary entry, the starting point for vocabulary analysis.
In other work, as part of this larger, NEH-funded effort that we call Beyond Translation, we are developing ways for learners to see any and all other passages where a new Greek word appears and thus to see for themselves what the word means, rather than depending upon textbook glosses — from the very first day, learners should be able to begin studying words in context by using translations aligned at the word and phrase level to the original as well as the linguistic analyses explaining the function of each word. This paper, however, focuses simply on how learners can organize vocabulary acquisition by corpus frequency and then how they can track their progress.
This paper also builds upon Clyde Pharr’s Homeric Greek (D.C. Heath & Co., 1925), a brilliant and innovative introduction to Ancient Greek, produced almost a century ago. Pharr introduces learners to the vocabulary of Iliad 1.1-5 on page 1 of his grammar and begins including small, but gradually growing chunks, of Iliad 1 from lesson 13 on, ultimately covering all 611 lines in this book by the end of lesson 77. Book 1 of the Iliad thus serves as the target corpus and leaners acquire a basic knowledge of Greek by learning as much as possible about the 4,563 running words in this book.
Researchers studying modern languages commonly report that readers need to understand at least 95% and preferably 98% of the running words in a text to gain adequate comprehension (e.g., Schmitt et al., 2011). The 95% threshold is often cited by professional teachers of Greek and Latin. Homeric Epic may be different and more amenable to reading before readers approach 95% vocabulary knowledge. Students of Greek commonly report that they can quickly begin to read Homeric Greek with speed and facility — that was certainly my personal experience when I read Homer intensively in the 1970s, long before any digital tools were available. It may well be that the formulaic nature of Homeric poetry means that it is easier to skip words when reading — there are certainly many common epithets for which the conventional translations are (in my view) guesses based on context.
Furthermore, when we explore Ancient Greek literature as a whole, we shift from genre to genre — from Homer to Attic Tragedy, from historians such as Herodotus (who write in the Ionic dialect) to Thucydides (who writes in Attic), from the comedies of Aristophanes to the dialogues of Plato. The various authors and genres differ in style and vocabulary. It is difficult to learn enough Ancient Greek from so many sources for any but the most experienced to expect they would understand 95-98% of the words in a text they had not seen. Most readers of Ancient Greek will depend upon using online dictionaries. My goal is to help learners to internalize as much knowledge as possible and to understand these sources with as much fluency, precision and pleasure as possible.
While Pharr brings his readers into contact with Homer as soon as he feels he can, he does not confine himself to authentic Greek text. Pharr includes exercises composed in artificial, textbook Greek and asks learners to produce exercises in prose, but the textbook vocabulary is almost entirely drawn from Iliad 1 and designed to help learners acquire both active command and passive recognition of Iliad 1.
Such exercises, of course, carefully track what learners have and have not learned in any given lesson. While John Wright and Paula Debnar completely rewrote the grammatical explanation in Pharr, they carefully preserved Pharr’s exercises (Pharr 2012). Ultimately, we should be able to generate such exercises automatically from an annotated corpus — we could almost certainly do so now, given the state of natural language processing — but for now we will build upon the manually constructed exercises that Pharr constructed a century ago.
I assume that learners are working to master an existing corpus, rather than to produce and understand an effectively infinite number of utterances in a living language. This use case can, of course, describe use cases with modern language where a learner wishes to understand a particular corpus — e.g., the screenplay of a Kurosawa movie or the lyrics of the Grateful Dead. They are able to measure their progress against a corpus that is, at least for the moment, relatively fixed.
Manually curated annotations listing the dictionary form, part of speech, and syntactic function for more than 1 million running words of Ancient Greek are available on GitHub from various openly licensed treebanks. This includes the c. 200,000 running words in the Iliad and the Odyssey. We can thus begin to measure how factors such as how many dictionary words and how many inflected forms we find in Homeric Epic.
Vocabulary in the Homeric Epics
The precise counts will vary according to the edition chosen (the Perseus Treebank for Homer uses Allen’s 1920 Oxford Classical Text), to the way words are counted (the Perseus Treebank treats words such as mête (“neither”) as two words: mê (“not”) and te (“also, and”)) and to decisions as to whether particular words get assigned to one or to two or more dictionary entries. These factors are unlikely to change the picture that emerges in the figure below.
Figure 1 plots the contribution of each dictionary entry and inflected form to an understanding of Homeric Epic as a whole. It starts with the most common dictionary forms and inflected forms, and then moves down the lists, adding the frequency of each new item and measuring how many words in Homeric epic for which we can now account. Figure 1 drives home the fact that Greek is a highly inflected language, where each dictionary entry shows up — on the average — in 3.6 different forms.
The table below shows most common forms in Homeric epic and how much each contributes to the overall corpus of 200,518 running words. Most of the forms are indeclinable particles and prepositions. Only a few of this top 20 are, strictly speaking, inflected. Note that no attempt was made to normalize forms: thus, kai (“and”) with a grave accent, for example, appears as a distinct form of kai with a grave accent, although the two words are identical in meaning. Normalization might reduce the number of forms but would not significantly change the relationship between dictionary entries and forms in Figure 1.
on the one hand
I (first person pronoun)
[interactional particle: S&H on same page]
From the table above, we can see that most of the most frequent forms in the corpus are not inflected and thus the ratio of 3.6 forms per dictionary entry underestimates how many different forms of many words audiences actually encounter. (When, in summer 2019, Ethan Yates generated the forms that Clyde Pharr listed in the paradigms for Homeric Greek textbook, he produced a list of almost 14,000 different inflected forms, illustrating the various paradigms. If he had included every inflected participle, the list would have been substantially longer).
The following table shows the counts for the 20 most common dictionary words in Homeric Epic.
I (first person pronoun)
you (personal pronoun)
on the one hand
unemph. 3rd pers.pronoun; -self; [the] same
The figure above reports that 8,792 different dictionary words account for all 200,518 running words counted in the Perseus Treebank for Homeric Epic. For those who do not know what a power law is, the figures above provide a good example. The top 20 dictionary words — less than 1 out of 450 or 0.23% — account for 56,285 running words — more than a quarter of the corpus as a whole (28%).
The top thousand dictionary entries (11% of the whole) account for 165,922 words (82% of the whole). The table below lists every 50th entry for perspective and to illustrate how much progress learners could make if they based the vocabulary on frequency. (Of course, they would probably use a modified strategy where they started with the most common nouns, verbs etc. rather than the most common forms).
personal and (ind.) reflexive pronoun
indef. relative or indirect interrogative
to divide among themselves
to the ground
The model above makes at least one assumption that would be problematic in print culture: once a dictionary word is encountered, it assumes that all forms of that word will be understood. For traditional learners, that is a major problem with Greek verbs because much of first year Greek is usually devoted to the many arcane ways Greek verbs can be produced.
In a digital environment, however, readers can see the analysis of each and every verb form long before they choose to internalize the ability to generate those forms. In this scenario, learners focus early on the outlines of Greek grammar (e.g., system of tenses, moods and voices) so that they have an understanding of what, for example, an aorist optative contributes. Full annotation of source texts would include links to a grammar explaining the function of each and every word.
The Homer treebank will, for example, tell you that the Greek words “spear” and “shoulder” are both datives and modify the verb ballô, “to strike with a thrown object” (a verb that shows up 469 times in Homer), but it will not tell you that the first dative reflects instrumentality (you strike someone “with a spear”) while the second designates location (you strike someone “on the shoulder”). We need to use either more complex tags (such as my Tufts colleague Matthew Harrington has developed) or you can create a separate annotation layer (as my collaborator Farnoosh Shamsian and I are doing).
The case explored below is based upon Clyde Pharr’s Homeric Greek (D.C. Heath & Co., 1925), a brilliant and innovative introduction to Ancient Greek, produced a century ago. It took a data-driven approach in its design when data had to be collected by hand. Pharr had two fundamental principals: (1) design the grammar so that learners engage directly with the corpus of interest as soon as possible and (2) use frequency data so that learners can engage with more frequent phenomena first.
There is, however, tension between the principles of engaging as quickly as possible with the source texts and the of focusing on the most common words. On the one hand, Pharr’s first year grammar introduces Iliad 1.1-5 in lesson 13 — the beginning of week 5 if learners move through the book 3 chapters each week. Pharr has, however, been preparing learners for this from the first page of the textbook. Learners are asked to learn the Greek alphabet by spelling and pronouncing the dictionary entries for each word in Il. 1.1-5 , each of which is offered along with a definition.
Pharr introduces the vocabulary for the first five lines of the Iliad on page 1. The immediate goal is for learners to use these words to practice reading the Greek alphabet. At the same time, they have an opportunity for incidental learning, as they are exposed to the definitions and to words that they will study in more detail much later (such as tithêmi, “put, place, cause”, the complex morphology of which is introduced in lesson 32).
Before addressing the question of word frequencies and vocabulary acquisition, I want to point out in passing that we could take a very different approach to having learners begin by sounding out words from Iliad 1.1-5 on day one. In a digital environment, for example, we easily add sound as a guideline that learners can use as a starting point. We can certainly link a simple reading of the Greek words and definition. We can, however, also provide metrical analysis and reading as well.
Figures 3 and 4 illustrate how data published under an open license can be available in multiple venues. The use of either venue would make possible a different approach for the beginning learner, one that made the sound of the dactylic hexameter an object of study from day 1. That would certainly add to the complexity of what can be a daunting start, but it would also enable learners to begin approaching Homeric Epic as poetry immediately. In a world of smart phones, learners can also listen to recordings anywhere and there is no reason why we cannot make sound the sound of Homeric poetry a fundamental part of their experiences. Many of us listen to music online in languages that we do not know and do not wait to study the grammar before listening to songs that capture our imagination.
Returning to the question of vocabulary acquisition, the focus on exhaustive coverage of particular passages does, however, come at a price. Any substantial passage in Homeric epic — and any regular text — will contain words that rarely appear. Those who follow Pharr’s exercises will find themselves practicing an active command of words such as proiaptô, “hurl forward, send forth,” and helôrion, “booty, prey, spoil,” even though, in the rest of the Iliad and Odyssey, proiaptô appears only 3 times elsewhere and helôrion is never seen again.
We can, however, now easily indicate to the learner how often each of the words presented in this list actually occurs in the Iliad and Odyssey.
wrath, fury, madness, rage.
sing (of), hymn, chant.
son of Peleus, Achilles.
accursed, destructive, deadly.
his, her(s), its (own).
grief, pain, woe, trouble.
put, place, cause.
much, many, numerous,
post. conj. but, and, so, for.
mighty, valiant, stout-hearted, brave.
soul, breath, life, spirit.
Hades, god of the lower world.
hurl forward, send forth.
HERO, mighty warrior, protector, savior.
self, him(self), her(self), it(self), same. (one), he, she, it.
booty, spoils, prey.
We could easily identify dictionary words that fall below some threshold of frequency in Homeric Epic as a whole. Learners could prioritize active mastery of dictionary words that are above that threshold and passive mastery of uncommon words (such as helôrion). Figure 5 shows the progress that learners would make as they moved through the 77 chapters of Pharr.
Figure 5 divides words into four classes. The y-axis show the number of running words in Iliad 1 while the x-axis tracks the 77 chapters in Pharr. The orange band at the top represents unseen words that fall below the selected threshold (here 20, so that orange words occur 19 times or less in Homeric Epic). The red band designates unseen vocabulary that meets the cutoff for common words (20 or more occurrences in Homeric Epic). The blue band shows how many new running words in Iliad 1 learners can understand after mastering the vocabulary in the current chapter. The green bars then allow learners to see how many words they should be able to recognize. Following Pharr’s plan, they achieve 100% coverage for all words in Iliad 1 at the end of the final chapter.
The 4563 running words in Iliad 1 contain 1118 separate dictionary entries. If, however, we focus on dictionary words that occur 20 or more times in the Homeric epics, we reduce that total to 655 dictionary entries for active mastery. The other 463 dictionary entries only account for 593 running words (pictured in orange above), i.e., the vast majority of them only occur once in Iliad 1. Learners can concentrate on recognizing these lesson common words in context rather than on actively mastering them.
If we were to increase the cutoff point to dictionary words that appear 30 or more times in Homeric epic, we would reduce the number of words for active recall from 655 to 565 — 90 dictionary entries from Iliad 1 occur between 21 and 30 times in Homeric Epic. Of the 4563 running words in Iliad 1, 730 would now fall into the less common category and could be recognized passively in context.
In increasing the cutoff from 20 to 30, we increase the number of words that we do not try to actively master, but Figure 6 shows that the change is not drastic — most readers will have to look carefully to see the difference from Figure 5.
The problem with focusing on Iliad 1 (or on any one book of the Iliad or the Odyssey) is that we will miss many words that are common in Homeric Epic as a whole but do not appear in our chosen book.
Where the green region in Figures 5 and 6 ultimately cover every single word in Iliad 1, being able to recognize every vocabulary item in Iliad 1 only prepares learners to recognize 145,026 of the 200,518 (72%) running words in the Homeric Epics as a whole.
If we return to the cutoff of 20 occurrences in Homer, 655 lemmas fall into this category. Adding this extra 90 dictionary entries for active mastery raises the the number of running words in Homer Epic that learners can understand from 145,026 to 150,848 (from 72% to 75%).
The problem is that many dictionary words that are common in Homer do not appear in Iliad 1. The following list shows the 20 most common such words in Homer that do not appear in the first book of the Iliad (with Telemachus, not surprisingly, at the top of the list).
Short Gloss (after Chicago Lemmas)
to urge, command
of one another
a house; a course of stone
to stir up
that one over there
people; (originally) a country-district
each of two
Unsurprisingly, we do not find suitors (mnêstêres) or guests (xenos) — these are, like the name Telemachus, more typical of the Odyssey. But Iliad 1 does not have common words for spears (enchos) or weapons (teuchos) that appear frequently in the Iliad.
One could adopt a different approach and learn each common dictionary word in Homeric Epic, roughly starting from the top and working downwards. (You probably would start not only with the most common words but also with the most common regular nouns and verbs.) This would require looking at examples of Homeric Greek from outside of Iliad 1 to see how words such as mnêstêres and xenos are used. That would take away from the focus on Iliad 1 and/or require additional work. But it could produce a more balanced result.
Figure 8 illustrates two approaches to learning Homeric vocabulary. The top and bottom lines show vocabulary acquisition for learners working through the vocabulary in the 77 lessons of Pharr. If learners mastered every single dictionary word in Iliad 1 (without setting aside words of lower frequency), the would have seen 76% of all words in Homeric Epic. The top line represents 100% coverage for Iliad 1.
The two middle lines reflect results from the top-down approach, where we learn words purely by frequency (hence these lines have very smooth curves). Instead of learning the 1118 separate dictionary entries covered in Pharr, we learn the most common 1000 dictionary entries in Homer. These two lines are much closer. Learners would have only mastered 85.5% of the words in Iliad 1 but they would have seen 83% of all words in Homeric Epic. A stricter adherence to the overall frequencies of Homeric vocabulary and less focus on the vocabulary of a particular passage might prove more satisfying and effective for learners. They would learn to recognize the 15% of vocabulary that was not on their list.
Overall, the point is that learners can now track their progress and the future value of new vocabulary in real time. Our hypothesis is that this will increase motivation and satisfaction, keeping learners more fully engaged and for a longer period of time. A next step will be to do design ways to test that hypothesis. First, however, we need to present, in the second part of this paper, another method by which to track seen and unseen vocabulary more precisely.
This paper makes a simple, but significant, observation. Vocabulary keeps growing in any corpus — there is no final, fixed set of words. That phenomenon appears with any natural language corpus. Here I emphasize the significance for students of Homeric epic. The Homeric Iliad and Odyssey contain about 200,000 running words and we can see how the number of dictionary entries (e.g, anêr, ‘man’) and of inflected forms derived from dictionary entries (e.g, andros, ‘of a man,’ andri) increase slowly but continuously: 8,792 dictionaries appearing as 31,664 different account for the 200,581 running words that appears in the Perseus Dependency Treebanks of the two epics.
First, readers should pause and take note of the fact that two curves in Figure 1, smooth though they are, were not produced by a mathematical formula. These are individual data points. A program sorted the dictionary entries and inflected forms in the Homeric Epic by frequency, started with the most frequent, then worked its way down the list, adding the frequency of each new term to the running total. There is an underlying pattern to vocabulary frequencies such as the we generate curves that exhibit no outliers at this resolution. This is not a scatterplot that we have fitted to a curve. This figure shows two series of linguistic data that follow an extraordinarily regular pattern.
Heaps’ Law is an empirical law that provides a mathematical model for the growth of vocabulary in a corpus and for the growth of dictionary entries and of forms through the epics. The mathematical model offered by Heaps’ law is based on the observation that vocabulary growth follows a power law, i.e., a few very common entries account for a disproportionate percentage of total words in a corpus. As we move through the corpus, the number of new dictionary entires and inflected forms slows but it does keep growing. If we had 500,000 running words, we would have more dictionary entires and inflected forms, if we had 1,000,000 running words, more still and so on. Many words that appear only once in the first 200,000 words would reappear.
Figure 2 illustrates the number of new dictionary entries encountered in each 5,000 word chunk of the Iliad and the Odyssey. The sequence above departs from the traditional ordering of books. It scans the books of the Iliad and Odyssey in this order: books 1, 10-19, 2, 20-24. Chunk 12 above contains an unexpected spike. Students of Homer may well guess that chunk 12 contains the catalogue of ships (hence, the unexpectedly large new vocabulary reflects a surge in the number of new nouns).
Chunk 10, which exhibits a smaller, but sudden, spike covers Iliad 18.28-19.107 and, thus, the description of Hephaestus’ workshop and Achilles’ shield, both of which would plausibly introduce new vocabulary, including more verbs as well as nouns.
By contrast, chunk 9 drops precipitously after the first 8 chunks had declined according to a fairly clear pattern. This contains Iliad 16.259-17.95, most of the aristeia and death of Patroclus. At this point, the narrative may be reprising, and pointing backwards towards, language and formulae that it had cultivated in the preceding battles books.
If we reverse the order and start with the Odyssey, the basic picture remains the same.
In figure 2, the bulge caused by the catalogue of ships in Iliad 2 is less pronounced, both because it is counted later (when some of the place names have been encountered) and because it is split between two of the 5,000 word chunks (29 and 30). The point of both figures 2 and 3 is to show that we continue to encounter new vocabulary in the last 5,000 word chunk, even after analyzing the previous 195,000 running words.
What are the new words that we encounter after reading through 195,000 words of Homeric poetry? If we start with the Iliad and follow the order of books as described above, the final chunk begins with Odyssey 8.399 and the first 20 (of 41) unseen words are:
to fit on; fix; prepare
to cut off from
to pass over from one place to another
the upper city
to fall around
to lead up into
to touch on the surface
Neritus (a personal name)
Of these twenty words, only one is a proper name, i.e., new vocabulary does not mainly consist of unfamiliar people or places.
If we scan the Odyssey first, then the first 20 unseen words in our final chunk are:
with gray hair on the temples
urged on by the Κῆρες
to divide into tribes
to lift up
without hearth and home
the reception of a guest
Deiopites (a personal name)
without share of booty
This chunk begins at Iliad 8.508. It also contains only one unseen proper name.
We cannot talk about what words and expressions were active in the oral tradition. We cannot say that lines with no parallels are not formulaic, i.e., that they were not learned and repeated in multiple contexts. We can only say that we have no parallels. We can, if we can combine a good understanding of probability with a solid grounding in how poetry works, begin to build more sophisticated models about what might appear if we had more epic on more subjects or more epic poetry on subjects that are not common in the current corpus (e.g., night raids such as Iliad 10). But then we are making our assumptions and our models explicit.
If you are an accomplished performer, you should be able to begin performing Greek poetry and prose, with an understanding of every syllable of what you are reading, by the end of one semester. I base that on the preliminary results that my collaborator Farnoosh Shamsian observed after 30 hours of instructing Persian speaking students Homeric Greek. We desperately need passionate and compelling performances of Greek and other languages to bring these sources to life. We can use podcasts and YouTube videos to reach a global audience. We have compelling sources. We need performances in different voices by people from different backgrounds.
Performances would be published in the new version of “beyond-translation,” version of Perseus, alongside exhaustively annotated editions of the source text. Ideally, performers would create performances in both Greek and English, with viewers able to go back and forth over the same statements in each language, comparing performances in both languages at the word and phrase level.
We could start with the Antigone — we have a lot of annotation for this play and are hoping to make it possible for learners to internalize all the grammar that they need to understand every last word in the play (insofar as anyone understands it). But we also have a lot of poetry and prose with dense linguistic annotation (treebanking) that would allow readers to get past a translation and quickly (one semester) begin to see how the Greek works.
In print culture, we trained people to work with inert editions available on paper pages that could not interact or answer questions. Now the dominant use case must be to train learners how to use a growing network of annotations to go back and forth between one or more translations and the source text (in multiple editions if that should be relevant). Learning how to perform Sophocles or Sappho is doable. Those who wish to do do or who wish to go on to study ancient languages professionally can internalize more.
Will people learn less over time? Will they have to go back and relearn things more thoroughly if they start pragmatically? Or will the incidental learning that they acquire from interacting with Greek early and often provide a broader foundation in which more active mastery of traditional paradigms and production will be more firmly grasped? Or will we simply have more people who get started and don’t quit after plodding through made-up exercises and learning paradigms and vocabulary that they may rarely see in their actual reading?
At Tufts University, the course on Classical Historians (Classics 141 — details in the departmental course booklet) will focus on Classical Arabic sources composed in, and about, pre-colonial West Africa. While we will consider Arabic sources produced outside of West Africa and accounts of European travelers, we will focus primarily on two different historical sources from West Africa istself: the Tārīkh al-Sūdān and (what has traditionally been called) Tārīkh al-Fattāsh. Our goal is not just to learn about the Mali and Songhai empires but to use what we learn to create openly licensed, digital sources of various kinds that will help others explore a major historical period that has attracted far too attention in the teaching and research.
Students will have an opportunity to explore emerging, digitally enabled methods by which global audiences can begin exploring the human record. In particular, we will exploit techniques by which we can begin to make the Arabic source text itself accessible to a general audience. We will begin publishing sections of these sources in the new version of the Perseus Digital Library that we are developing with support from the NEH. The development site for this is Beyond Translation and will be augmented between now and the fall semester.
The course itself will meet during Tufts’ fall semester Monday evenings from 6:00-8:30. Space allowing, we hope to see students from other institutions participate, whether by direct cross-registration or by getting credit through a directed study authorized by a faculty member at their own institution.
We will also offer a weekly reading group for those who wish to go over sections of the Arabic. This will can be taken as an optional addition to the Monday class or as a separate class. The Arabic reading group would be 1 credit (vs. 3 for the Monday class). Any students taking both would receive 4 credits.
During the summer, I will also be working on the digital edition of these two histories and of other sources. If others are interested learning more and in possibly contributing, they should contact me. There are a number of ways to contribute that match a range of skillsets. The basic requirement would be an ability to read English carefully but there are also clearly opportunities for those with knowledge of French, of various aspects of Computer and Data Science, and of Classical Arabic.
I am hoping this summer to resuscitate my own Arabic and to see how far that helps me with the language of these Islamic scholars from Timbuktu. I will be using tools such as the suite of Arabic natural language processing tools developed by the CAMeL Lab at New York University Abu Dhabi to extend my (not very advanced) knowledge of Arabic. The goals are to create exhaustive annotations, included translations aligned at the word and phrase levels, for (1) a small but extensible set of passages and (2) sets of sentences that allow readers to trace the meaning of Arabic words which cannot properly be translated.
The larger goal of this class and the larger project that it represents is to create openly licensed digital materials that are not only of immediate use but that also can be modified and, wherever possible, reused under a Creative Commons CC-BY license. Such a license is not suitable for publications that seek to represent a particular scholarly voice at a particular time. We are, however, managing the sources in Github and so each particular contribution is recorded in the versioning history. We are supporting a collaborative model of authorship that may be familiar to readers from Wikipedia. That said, individuals will be able to use the Github versioning history to document what they have done and to create hybrid publications that contain their own accounts of what they did and what critical decisions they needed to take.
The Tārīkh al-Sūdān and the Tārīkh al-Fattāsh
The fall course will focus primarily on two histories — tarikh, pl. tewarikh — composed by Islamic scholars in West Africa.
(1) The Tārīkh al-Sūdān (TS) by al-Sa’di (1594-1655+ CE) focuses on the history of the Songhay Empire from the mid-fifteenth century until 1591 and then the Moroccan invasions and subsequent administration down to 1655. Houdas (Sa’dī 1900 and 1900a) published the Arabic text and a French translation. In 1999, John Hunwick published a scholarly translation with notes that covered the 28 of 35 chapters most relevant to Timbuktu and the Songhay Empire.
(2) Thirteen years later, Delafosse (1913 and 1913a) published the Arabic text and a French translation for a far more challenging source that has been known as the Tārīkh al-Fattāsh (TF), the “Chronicle of the Researcher,” an account of the Songhay Empire through 1599 and thus includes the early years of the Moroccan occupation. Almost a century later, Wise and Taleb (2011) published an English translation of the Tārīkh al-fattāsh based on Delafosse’s French translation and the Arabic text. Tārīkh al-fattāsh is a novel chronicle written in the 19th century, and not the effort of three generations of scholars who worked on it starting from the early 16th century and eventually interpolated in the 19th century, as previously advanced by most scholars. This 19th century TF was composed by a substantial rework of a 17th century anonymous work. With support from the NEH Translation Program, Mauro Nobili and Ali Diakite are publishing a new edition of this work that contains an English translation, the Arabic text and clear indications of how the 16th and 19th century texts relate to each other.
We will, however, also consider other sources. We have a reasonably accurate transcription for the Arabic of Ibn Battuta’s description of West Africa and an accompanying French translation, long in the public domain. We can use DeepL or Google Translate to create a quick first English version and then edit this as we align it to the Arabic original.
Where the project stands.
Work on this project begin in summer 2021, when Ayah Aboelela (UMass Boston CS ’21) led preliminary work. We found not only PDF versions of the public domain Arabic/French editions of the TS and TF but also text automatically generated by Optical Character Recognition.
The French OCR-generated text was good enough as a base for further work. We applied DeepL and Google Translate to convert the French into English. The results were surprisingly good: we did find ourselves making occasional changes to the English but such correction did not materially add to the overall work of adding base TEI XML tags, marking footnote markers in the text and footnotes on the bottom of the page, adding occasional Arabic words in the notes, and adding Arabic numbers in the translation that pointed to the corresponding pages in the Arabic edition. Readers can examine a sample of such work, chapters 21-22 of the TS on Github.
The Arabic text was more problematic and, in two different OCR-workflows, had a character error rate of c. 5%. The text is still sufficiently accurate to support a range of text mining (such as topic modeling and text reuse detection). It is also good enough as a starting text if the goal is to add extensive annotation to relatively short passages and to create an initial reader. Ayah produced initial versions of curated passages where she took the time to correct errors in the OCR-generated text.
A few thousand words of carefully edited Arabic with aligned translation and annotation would be a useful start and allow readers at an intermediate level to familiarize themselves with the style and content of these sources before moving to passages without curated annotations and translations.
Ayah published exploratory work using natural language processing tools for Arabic available from Spacy and CAMel on Github. She tested services for morphological analysis and disambiguation and dependency parsing for the Arabic.
For named entity recognition (NER) (determining whether names are people, places or groups), we decided to apply tools from Spacy to English generated from the French with machine translation. A relatively modest amount of training improved the accuracy of NER from 60% to 82%.
Not only do these two histories constantly refer to places with which most readers in the US are probably not familiar, but there are many personal names, often complex in form, and rarely familiar.` Identifying these names and creating links between related characters, and linking this information directly to the source text will, we hope, make it easier for readers to see who is who and to identify characters on whom they should focus their attention.
Figure 5 illustrates social networks derived from primary sources about Roman history. We use simple collocation to posit connections. Our hope is to include more complex information about relationships (son-of, occupation, etc.). Zach Sowerby was working with a much larger text corpus than the two histories and his relatively rapid prototyping already reveals basic patterns of who is important and which characters are connected. We can get useful information starting with this fairly rapid work.
In fall 2021, I taught the first iteration of this class. We spent much of our time reading Michael Gomez’s 2018 book African Dominion, now the standard English account of early and medieval West Africa and then examining the accounts in the two histories on which Gomez bases much of his work (and which he carefully cites). The map in Figure 2 (above) was produced for this class by members of the Tufts Data Lab as a model for additional student work. I will adding results from student projects in this class during the summer.
Let’s see what we can do in the summer and then in the fall!
The study of Greco-Roman culture can exert a purposeful and transformative role in Europe’s development of a more just multinational and multiethnic society. This is a topic about which I have thought and on which I have spoken for years. Technology has begun to change the ways in which we can relate to sources in different languages from different cultural contexts. Those nascent changes are by no means deterministic – the technology can develop in more and less helpful directions and its development can move under very different influences. Likewise, the study of Greece and Rome can be appropriated by groups such as white supremacists or more narrowly Eurocentric nationalists (for whom whiteness is a necessary but far less sufficient condition). Therefore my own work seeks for constructive ways by which emerging technologies for reading and the study of Greco-Roman culture can together help foster a society that is more just and affords greater happiness to its citizens.
The rest is published on the Classical Continuum, a publication venue supported by the New Alexandria Foundation and the Harvard Comparative Literature Department.
Gregory Crane Tufts University March 25 Gregory.Crane@tufts.edu
Tufts University will offer two different sections of introductory Ancient Greek in fall 2022, each of which takes a complementary approach. Both sections of the class have been designed to exploit increasingly powerful digital tools for understanding Ancient Greek and other languages — the skills that you learn will also help you exploit, and go far beyond, what you can do with translation, whether those are literary translations by human beings or the product of systems such as Google Translate or DeepL. Both sections build directly on an emerging new version of the Perseus Digital Library. Neither section has any prerequisites.
The first section will follow a textbook and will teach you to produce, as well as to understand, ancient Greek. It will, however, also give students far more exposure to ancient Greek source texts from the opening weeks of the semester. The second section, which will be online at a time to be determined, will focus on exploiting increasingly sophisticated digital tools to analyze ancient Greek sources.
The first section follows a traditional textbook but exploits a range of digital methods to enhance the experience of learning Ancient Greek, providing substantial immediate feedback as you practice traditional exercises. Instead of translating Greek into English or English into Greek and then waiting days for correction, you will be able to receive substantial feedback. We will also spend as much time possible seeing how the vocabulary and grammar are used in actual Greek sources and minimize use of artificial textbook Greek. The goal is to give you active as well as passive command of the Greek. This section is better suited to your needs if you feel you may wish to go beyond first year Greek. It will meet Mondays and Wednesdays 1:30-2:45 PM local time.
This section will be primarily in person but will be open to those who wish to participate remotely. If you are at an institution where you can cross-register with Tufts (such as Boston College or Brandeis), you would not have to travel across town — scheduling may prevent you from taking your local introduction to Greek or you may wish to participate in this novel approach. Those seeking credit should be able do so through Tufts’ University College.
The second section will meet online at a time to be determined. It will focus entirely on reading and is designed for those who may have only one year — or even one semester — to study Ancient Greek. This second section represents a more radical departure from traditional approaches as it focuses on annotated texts themselves and could be applied to any corpus with sufficient annotation. After one semester, practice with digitally enabled tools will allow you to compare a translation of Homeric epic to the original Greek, to explore what the words really mean in Homeric Greek (end not just how they are translated), and to engage with the epics on your own. In the second semester, you will be able to move on to more syntactically complex sources such as Plato.
If space allows, we would particularly encourage participation in this online section by students from outside of Tufts. We want to understand how to apply this more radical departure from traditional pedagogy. We are building on work done by Farnoosh Shamsian, Phd student at the University of Leipzig and participants in this class will be contributing not only to her research but to an ongoing reimagining of how we work with historical languages.
We are aware of no modern language programs that will provide such transferable skills. You will not only learn how to work with sources in Ancient Greek but will have tools to analyze Latin as well as modern languages such as French, German, and Italian but also Croatian and Latvian, Arabic and Mandarin. Our goal is not to help you check into a hotel or order dinner. Our goal is to allow you to work directly and quickly with not only Ancient Greek primary sources but with scholarship about these sources in a variety of modern languages. Our goal is to transform who can participate in traditional scholarship about the Greco-Roman world and then to enable new forms of scholarship and new intellectual communities that were never possible in print culture.
Description of this version of Greek 1 as it appears in the course book for the Department of Classical Studies at Tufts University.
Greek 1: Fall 2022 Introduction to Ancient Greek Section 1: Monday/Wednesday 1:30-2:45 Section 2: To be scheduled. Gregory Crane, Professor of Classical Studies, Editor-in-Chief, Perseus Digital Library Christopher Petrik, Tufts ’24 Farnoosh Shamsian, Phd Candidate, Leipzig University
The rise of digital methods and, increasingly, of machine learning has begun to enable a transformation in the study of Ancient Greek. What you can learn in an introduction to Ancient Greek can be far greater now than was ever possible before. At the same time, what you can do with what you learn will take you much farther now than was possible before. Tufts University has been at the forefront of this transformation. In taking Ancient Greek, you not only can benefit from this work but will have an opportunity to contribute yourself, creating during the course of first year Greek materials that will serve other language learners and advanced researchers alike.
You will have more exposure to authentic Greek in this introductory class than has ever been. Exhaustive annotation exists explaining the function of more than a million words of Ancient Greek while a new generation of translations, designed to clarify the working of Greek for those who do not know the language makes it possible to see how grammar and vocabulary actually work in some of the most famous works of Greek literature, from the time you learn your first words. The very same methods that you learn to begin working with Ancient Greek have been applied to dozens of other languages
A major barrier to learning historical languages has been the slow pace and limited reach of the feedback that you receive. You do an assignment one day, hand it in the next, and then see how you did in the next class, two days or more later. When you practice what you have learned, you will often be able to get immediate feedback and then be able to practice what you have learned until you have mastered it.
We offer two different sections, each with a complementary approach aimed to serve different audiences. The first section builds off of a traditional textbook, offering all exercises online with immediate feedback. Class time will be devoted to questions that you cannot resolve on your own and to seeing how what we have learned in class helps us begin to understand real texts. Students will also begin working with short passages from the Iliad and Odyssey, Sophocles, Thucydides, Plato, and the New Testament. The second section is designed to support those who may be able to devote only a year or even a semester to the study of Ancient Greek. You will learn enough of the grammar to understand the basic working of highly inflected languages such as Ancient Greek (and Latin and Russian and many other languages) but you will spend most of your time learning how to apply the rich set of tools available to help you read Ancient Greek – and many other languages. If you do choose to continue your study beyond the first year, we will provide you with a framework by which you can do that.