Assessing Treebanks

One of the strengths of treebanks is the ability to give specific quantitative feedback on a tree. There are many ways to grade these choices, but we offer specific tools designed to visualize this feedback.

On this page we provide instructions for viewing the difference between trees in Arethusa, as well as a link to the Inter-Annotator Agreement tool, Also known as Dolphin.

You can use Robert Gorman’s tool to visualize the planarity of treebanked sentences. This tool visualizes non-planar nodes in dependency trees. It is helpful in teaching students about problems presented by Greek and Latin word order.

If you are using a text that is part of the Ancient Language Dependency Treebank that you use those trees as your “gold standard” for grading. When working on a text that isn’t part of the ALDT or the JMH git hub repo you will need to create your own gold standard trees.

If you are working with the students to treebank a text for the first time, you will not have a gold standard to compare to. As long as the metrics for grading are made transparent and quantitative, you should be able to produce the same kind of feedback that the gold standard review would be able to produce. This quantitative form of grading allows an educator to target weaknesses in a student’s translation skills, and directly address systemic misunderstandings.

Available here is an example rubric for assessing treebank assignments. This rubric is designed to focus on clauses and basic dependency. When used in conjunction with the gold standard reviewing tools, you should be able to give direct targeted feedback on the treebanks your students produce.

Instructors have had some success with response papers. The important element of the response paper is to force students to address specific choices in their treebank. This exchange can also take place during in class analysis of treebanks which was discussed in the Classroom section.