Glossary:
Resources
Overview
Availability and Usage
Plugins
Video Tutorials
Feedback
Acknowledgements
Citation
Liability Notice
Publication Resources

Resources

News:
GailBot paper published in Dialogue and Discourse. 
News Coverage by Tufts University.
GailBot AMLAP 2020 Presentation and Abstract

Documentation:
Gailbot User Manual

Forms
GailBot App. Request Form

Overview

Take a second to listen to the following conversation:

Most speech-to-text (STT) systems would produce a transcript that looks something like this:

This transcript is useful, but falls short of truly capturing the conversation. There are a few obvious omissions. Perhaps most importantly, there is a 4.1 second silence between lines 5 and 6. This gap does some heavy lifting: it shows that Speaker 2 expected Speaker 1 to understand their joke. In addition, STT systems typically ignore laughter. On line 2, Speaker 2 laughs, an early cue that they are joking. On line 7, Speaker 1 laughs, indicating that they finally understand the joke. To really understand the sequence, what we really need is a transcript more like the one below, which marks paralanguage – everything that comes along with the words.

The system to transcribe paralinguistic features – Jeffersonian transcription – was created by Gail Jefferson. Jeffersonian transcription is slow going. On one hand, the process of re-listening to the audio to add more and more details helps the researcher understand the mechanisms underlying the interaction. On the other hand, the laborious process limits the amount of data researchers can analyze. For example, it would be close to impossible to generate enough Jeffersonian transcripts to train a deep learning language model.

Enter: GailBot. We designed GailBot to create first-pass transcriptions of some paralinguistic features (speech rate, silences, overlaps and laughter). It interfaces with existing STT algorithms, and then applies post-processing modules that insert Jeffersonian symbols. All of this is useful, but GailBot’s most valuable characteristic is that it allows researchers to create, insert, and adjust plugins, which are a standardized interface for applying customized plugins to the GailBot pipeline.

Availability and Usage

GailBot is available to use both as a Graphical User Interface (GUI) based app for non-technical users and as a python package for developers. The GailBot app is currently only supported by Max OSX and can be installed directly using .pkg bundles.

Use the app. request form to submit a request to use GailBot. Thank you!

GailBot is also available as a python package that can be installed using pip and allows users to directly use GailBot within their python applications.

Please note that we are a small development team with limited resources, which means that there may be bugs and issues with GailBot. We appreciate your patience as we resolve these.

Plugins

GailBot provides a framework for users to add custom algorithms to identify specific paralinguistic features. Plugins are wrapper objects (implemented as Python classes) that provide a standard API for these algorithms to interact with the core pipeline. This means that customizing and changing plugins does not require modifying GailBot’s source code. Plugins may or may not be dependent on each other.

While users can implement custom plugins, we support a number of ‘official’ plugins as listed below.

• HiLabSuite [Documentation]

Video Tutorials

In addition to written documentation (linked above), we provide a number of tutorial videos for users to get started with GailBot. Note that video tutorials may use concepts defined in the user manual (also linked above).

Feedback

Users can reach out with questions, feedback, or concerns to hilab-dev@elist.tufts.edu

Acknowledgements

GailBot has been made possible by a small but dedicated team of researchers and research assistants in the Human Interaction Lab at Tufts University.

Faculty, post-docs, and graduate student contributors:

Muhammad Umair

Dr. Julia Mertens

Undergraduate Contributors:

Jason Wu ( June 2023 – Sept. 2023)

Siara Small (Sept. 2022 – June 2023)

Annika Tanner (Jan 2022. – June 2022)

Muyin Yao (Jan. 2022 – June 2022)

Rosanna Vitiello (Jan. 2021 – June 2021)

Eva Denman (Sept. 2020 – Jan. 2020)

Citation

Please cite GailBot using the following bibtex:

@article{umair2022gailbot,
  title={GailBot: An automatic transcription system for Conversation Analysis},
  author={Umair, Muhammad and Mertens, Julia Beret and Albert, Saul and de Ruiter, Jan P},
  journal={Dialogue \& Discourse},
  volume={13},
  number={1},
  pages={63--95},
  year={2022}
} 

Liability Notice

Gailbot is a tool to be used to generate specialized transcripts. However, it is not responsible for output quality. Generated transcripts are meant to be first drafts that can be manually improved. They are not meant to replace manual transcription.

GailBot may use external Speech-to-Text systems or third-party services. The development team is not responsible for any transactions between users and these services. Additionally, the development team does not guarantee the accuracy or correctness of any plugin. Plugins have been developed in good faith and we hope that they are accurate. However, users should always verify results.

By using GailBot, users agree to cite Gailbot and the Tufts Human Interaction Lab in any publications or results as a direct or indirect result of using Gailbot.

Publication Resources

This section contains resources that were included as part of various publications.

Media Files

• Excerpts 1 and 2
• Excerpts 3 and 4
• Excerpts 5 and 6
• Excerpts 7 and 8
• Excerpt 9
• Excerpt 10
• Excerpt 11
• Excerpts 12 and 13
• Excerpt 14 and 15