NSF SOLSTICE Training Grant

The SOLution-oriented, STudent-Initiated, Computationally-Enriched (SOLSTICE) approach is a teaching method for improving graduate student training in data intensive fields. The approach seeks to improve students' knowledge, skills, and attitudes in data sciences to solve complex problems, think critically, and effectively communicate across inter-generational, trans-disciplinary research teams. Using a data-intensive, project-based learning approach, graduate students work collaboratively to design, evaluate, and disseminate research in team environments. As data scientists, these students will learn how to pose research questions, translate information into potential actions, develop data collection, analysis and visualization schemes and protocols, and exchange information, data methods, and results in a tailored manner to various audiences. Educational resources developed during this study will then be available for guiding faculty development and training process in the future.

This grant is funded by the National Science Foundation's (NSF) Innovations in Graduate Education (IGE) Program.  For more information, please see this overview document.

The project-based design of the SOLSTICE approach centers on feedback and collaboration. This includes students providing feedback on one another's works as well as providing feedback on the feedback they are given to ensure transparent, communicative teamwork. Students will also be challenged to organize and manage their semester-long projects by creating data management flows and building and executing data analysis plans. To do this, SOLSTICE employs team role-play where students serve as Team Leaders, Collaborators, and Reviewers of their own and their peers’ works.

Team Leader
Students will serve as principal investigator to direct their own individual project. This includes defining a testable hypothesis, proposing a plan of execution, and designing a plan of evaluation for their project. Students will also perform data cleaning, analyses, and summaries for in-class presentations and a final report.

Collaborator
Students will serve as a co-principal investigator to revise the works of their partners. This includes helping to find solutions for identified challenges and evaluating whether partners' study results are communicated to a broad range of stakeholders in a suitable language.

Reviewer
Students will provide constructive criticism to partners by identifying flaws and weaknesses of presentations, manuscripts, and analysis plans. The reviewer is not meant to play an adversary role - all feedback should offer possible solutions by providing a big picture perspective that may have been missed by investigators.

The SOLSTICE approach uses survey assessments of students' knowledge, skills, and attitudes on data sciences to develop and refine educational resources for guiding graduate student studies. These three principles are the key for preparing students to become data scientists, analysts, and communicators in their professional careers. The key learning objectives for each principle are listed below:

KNOWLEDGE

1. Identify important research questions tested by secondary analysis
2. Articulate the perspectives, language, and terminology of methodological and statistical problems
3. Interpret findings from data exploration and analytics tools
4. Integrate findings from other lines of research when interpreting and drawing inferences from evidence-based research
5. Identify the challenges of data management
6. Identify the challenges, principles, and key details of data sharing
7. Apply classroom learning in own project

SKILLS

1. Prepare data for public use by collecting, cleaning, storing, and documenting data
2. Integrate data from basic, clinical, environmental, social, and population sciences
3. Manipulate complex linked data sets from multiple sources while protecting individual-level data
4. Produce high-quality tables, graphs, and visuals
5. Demonstrate interdisciplinary collaboration by interacting effectively with team members from other discipline

ATTITUDES

1. Give and receive constructive feedback
2. Participate actively by attending, reacting, and showing awareness
3. Recognize the value of responsible conduct of research

The SOLSTICE approach uses entry and exit assessments to monitor the experience of enrolling students. These  assessments are conducted electronically, are anonymous to course instructors, and do not influence student course grades. Assessments consist of two parts:

Surveys
These surveys are used to evaluate students' prior knowledge, skills, and attitudes in data sciences. Surveys inquire about students' academic background, software competency,  proficiency in performing data analysis-related tasks, challenges in conducting research, and experience collaborating on team-based projects.

Questionnaire
This questionnaire evaluates students' expertise with respect to the core competencies addressed by the teaching approach. All questions are multiple choice and cover basic principles of data science covered in introductory or intermediary STEM-related courses.

By completing the assessments above, students help SOLSTICE researchers understand the preparedness of STEM training in nutrition and public health graduate courses. Student responses inform what concepts and technical skills are most critical for improving student training. Researchers then work to create educational resources and training materials to best assist the learning process of enrolled students. Comparisons between entry and exit assessments determine in what areas the SOLSTICE approach is most effective at training students for becoming data scientists.

Previous student participants have noted the following benefits:

Enrollment is easy and incentivized!
Surveys and questionnaires combined take only 15-20 minutes, can be completed electronically to minimize student hassle, and students are eligible to receive extra credit from their coursework instructors when offered. Once completing these assessments, students have the option of consenting to become enrolled in our study. By consenting, students allow researchers to use their anonymous responses in analyses and publications. Once enrolled, students are eligible to enter the SOLSTICE Sweepstakes to earn a reward to partially cover conference abstract submission and registration fees to present their coursework!

Improvement in quantitative skills and reasoning
Students commented that the project-based design emulated a similar working environment experienced when entering a professional career after their graduate studies. This includes greater experience performing research projects in multidisciplinary teams, improved proficiency in conducting and understanding data analyses, confidence to complete data-related projects, and skill building in statistical software.

Educational resources can be reused long after class end
To best cater to diverse student audiences, SOLSTICE researchers have created a diversity of auditory and visual resources to assist students' personal learning process in statistics and data sciences. These resources follow an iterative design to gradually build on student knowledge and skills over time. This process allows students to drive their own independent data analysis workflow while becoming experienced in giving and receiving collaborative feedback from peers.

Previous faculty participants have noted the following benefits:

Providing more-detailed information on students’ backgrounds
Oftentimes, faculty are unaware of the range of knowledge, skills, and attitudes acquired by students' in their prior experiences before arriving to class. The SOLSTICE surveys and questionnaires provides greater insight on the knowledge, skills, and attitudes most and least expressed by students. This information provides a quick, easy assessment to better cater educational resources and enhance course materials. All survey results are timely processed and analyzed by SOLSTICE researchers, minimizing faculty burden.

Supporting collaboration across teachers of data-intensive courses
Faculty will take part in a workshop to learn about the SOLSTICE approach and ways for integrating this approach into their course. This collaboration can help coordinate course topics, aims, and materials within school departments and across schools to maximize faculty support and student learning. This opportunity also allows faculty to collaborate with SOLSTICE researchers on publications stemming this project.

Enrollment is as easy as 1, 2, 3!

1. Faculty members enroll their course into the study
This begins with an email to the Research Coordinator (Ryan.Simpson@tufts.edu). Once enrolled, the faculty member’s course information will be posted on the grant website and the course roster will be contacted about the grant. If your course is not enrolled but you hope to participate, contact your professor and ask them to contact us!

2. Students complete assessment surveys and questionnaires
At the beginning and end of the semester, students complete a survey and questionnaire examining their baseline knowledge, skills, and attitudes in data science. This will not be shared with instructors or classmates and has no effect on student’s grades!

3. Students enroll in the study by submitting a consent form
This form is completed after submitting the entry survey and questionnaire. By enrolling, students allow SOLSTICE researchers to use anonymous assessment responses in published works submitted after the grant concludes.

Principal Investigators (PIs)

Dr. Elena Naumova, Chair of the Department of Nutrition Epidemiology and Data Science, Tufts Friedman School of Nutrition Science and Policy

Dr. Barbara Brizuela, Dean of Academic Affairs, Tufts University School of Arts and Sciences

Dr. Remco Chang, Associate Professor in the Department of Computer Science, Tufts School of Engineering.

Research Coordinators (RC)

Ryan Simpson, PhD Candidate, Nutrition Epidemiology and Data Science, Tufts Friedman School of Nutrition Science and Policy

NUTB-0250, Statistics for Health Professionals I (Prof. Naglaa El-Abbadi & Lynne Ausman)

NUTR-0394, Advanced Data Analysis (Prof. Elena Naumova)

PH-0205, Principles of Biostatistics (Prof. Rachel Silver)

PH-0206, Intermediate Biostatistics (Prof. Ken Chui)

PH-0265, Introduction to SAS (Prof. Rachel Silver)

DATA-0201A, Introduction to Python for Data Analytics (Prof. Nirav Shah)

DATA-0201B, Python and Machine Learning (Prof. Nirav Shah)

EM-206, Introduction to Data Analytics (Prof. Kyle Monahan)

NUTB-0350, Statistical Methods for Health Professionals (Prof. Naglaa El-Abbadi, Lynne Ausman)

NUTR-0204, Principles of Epidemiology (Prof. Fang Fang Zhang)

NUTR-0231, Fundamentals of GIS (Prof. Alexandra Thorn)

NUTR-0307, Regression Analysis for Nutrition Policy, (Prof. Parke Wilde)

NUTR-0309, Statistical Methods for Health Professionals II (Prof. Naglaa El-Abbadi, Anastasia Marshak)

NUTR-0319, Intermediate Epidemiology (Prof. Fang Fang Zhang)

NUTR-0331, Environmental Life Cycle Assessment (Prof. Nicole Tichenor Blackstone)

NUTR-0375, Genetic Epidemiology and Biostatistics (Prof. Jiantao Ma)

NUTR-0393, Data Visualizations and Effective Communication (Prof. Elena Naumova & Corby Kummer)

PH-0201, Principles of Epidemiology (Prof. Mei-Chun Chung)

PH-0205, Principles of Biostatistics (Prof. Alice Tang)

PH-0205, Principles of Biostatistics (Prof. Rachel Silver)

UEP-0232, Introduction to GIS (Prof. Rebecca Shakespeare)

UEP-0235, Advanced GIS (Prof. Sumeeta Srinivasan)

UEP-0236, Spatial Statistics (Prof. Sumeeta Srinivasan)

ENV-0107, Introduction to GIS (Prof. Alexandra Thorn)

DATA-0202A, Introduction to Data Visualization (Prof. Nirav Shah)

DATA-0202B, Excel to SQL (Prof. Nirav Shah)

CEE-0189, Introduction to Remote Sensing (Prof. Magaly Koch)

NUTB-0250, Statistics for Health Professionals I (Prof. Naglaa El-Abbadi & Lynne Ausman)

NUTR-0204, Principles of Epidemiology (Prof. Gitanjali Singh & Silvina Choumenkovitch)

NUTR-0231, Fundamentals of GIS (Prof. Alexandra Thorn)

NUTR-0394, Advanced Data Analysis (Prof. Elena Naumova)

HIA-0201, Introduction to Health Informatics and Analytics (Prof. Anna Orlova)

HIA-0213, Informatics for Public Health Professionals (Prof. Anna Orlova)

CTS-0527, Biostatistics I (Prof. Angie Rodday)

CTS-0533, Advanced Topics in Biostatistics (Prof. Angie Rodday)

DHP-P207, GIS for International Applications (Prof. Patrick Florance)

UEP-0232, Introduction to GIS (Prof. Sumeeta Srinivasan)

UEP-0235, Advanced GIS (Prof. Sumeeta Srinivasan)

DATA-0201A, Introduction to Python for Data Analytics (Prof. Nirav Shah)

DATA-0201B, Python and Machine Learning (Prof. Nirav Shah)

MCM-0591, GIS for Conservation Medicine (Prof. Carolyn Talmadge)

NUTR-0393, Data Visualizations and Effective Communication (Prof. Elena Naumova & Corby Kummer)

NUTR-0394, Advanced Data Analysis (Prof. Elena Naumova)

While we hope students and faculty will join the SOLSTICE grant, enrollment is not mandatory. Students have the choice of consenting to the study when completing the IRB consent form described above.

All student responses remain anonymous to faculty and instructors of their enrolled course and principal investigators of the SOLSTICE grant. Only the Research Coordinator has access to names of student participants.

After completing surveys and assessments, the Research Coordinator will assign every student participant a study identification number. The student's name will be replaced with this identification number to protect student privacy. Even after de-identification, survey results and assessments will not be distributed to faculty and instructors. However, de-identified results will be shared with Principal Investigators on the SOLSTICE grant.

Completion of consent forms will take no more than 5 minutes. All surveys and assessments have been designed to take no more than 15 minutes each. This means that baseline information will be recorded in ~20-30 minutes, end-of-term progress information will be recorded in ~20-30 minutes, and six-month follow-up information will be recorded in ~20-30 minutes.