Sheryl Barnes' Work Blog

Using Analytics to Examine Course Design and Identify Effective Practices
Karin Readel, Director of Instructional Technology, University of Maryland, Baltimore County
readel@umbc.edu

Their public BB reports:
http://www.umbc.edu/oit/newmedia/blackboard/stats/

They wanted to know about the impact of their faculty training.

About their training program:
* Alternate Delivery Program (ADP)
* for moving from f2f to online/hybrid
* one time $2500 stipend
* 1 day “hybrid workshop”
* self-reflection on pedagogical problem
* interview with IT & FDC staff
* Create & present 2 online activities
* teach redesigned course
* evaluate adp process

Questions:
* are there differences in course measures?
* does the training make a difference?
* can we use BA4L data to identify effective practitioners?
* what other non-LMS data can be included in the analysis?

Measures looked at:
* “Average # of interactions” (in the LMS); all faculty & for trained; across f2f, hybrid, online.
* ave amount content, ave course accesses (by students), ave min/access (about the same for all faculty & trained)
* put it all together to see effect – ave content/ave accesses/ave min/access (across f2f all, f2f trained, hybrid all, hybrid trained, online all, online trained)
* % of courses using particular features (content, % accesses, grade center tools,

BB defines an “interaction” as anything a student does in a class (clicks/page view/submission) (could indicate poor design, if folder depth is bad, they don’t use the grade center, etc…)

Cross mapped projector use data with course schedule data – to better allocate the limited number of projector-enabeled classrooms.  (some rooms are monitored by GVE – web-based AV management software)

Next steps:
* look at impact of training over time on specific faculty
* look at impact of “redesign” on specific courses
* add clicker & projector use data to BBAnalytics cube so she can compare it all directly

—

Are All Mobile Devices Equal When It Comes to Teaching and Learning?
Robbie Kendall-Melton, Associate Vice Chancellor for Academic Affairs: eLearning    Tennessee Board of Regents

My Script calculator app, Toca Boca

http://emergingtech.tbr.edu/

Learning Sciences and Learning Analytics: Time for a Marriage – Sponsored by Starin
Roy Pea: http://www.stanford.edu/~roypea/index.html
David Jacks Professor of Education and the Learning Sciences

Informed by Learning Sciences: Bransford, “The Cambrigde Handbook of Learning Sciences” Learning science in informal environments, Science magazine

Journal of Learning Sciences (+ another journal)
Int’ society of learning sciences: http://www.isls.org

National Tech Plan (Pea, David Rose, others…)
http://www.ed.gov/technology/netp-2010

Calls for connected, personalized learning.

Priorities:

• Develop interconnected learning maps
• Design for social learning
• Educate for broader competencies
• Use richer pedagogical models
• Better understand learner goals
• Forge interdisciplinary teams

urgent priority to build learning maps & competencies to allow personalized learning, like the common core initiative: http://www.corestandards.org/  (Pea showed a great visal map,b ut I can’t seem to find it…)

lots of folks are working on learning maps, but they are unconnected

Shared Learning Collaborative (recently renamed “In Bloom”): https://www.inbloom.org/

most fundamental transformation is that teachers can draw on shared, interoperable platforms.

we need interdisciplinary teams > collaborator for online learning

improvement in post secondary ed will require converting teaching from a “solo sport” to a community based research activity

Enhancing Teaching and Learning through Educational Data Mining and Learning Analytics
http://www.ed.gov/edblogs/technology/files/2012/03/edm-la-brief.pdf

Recent NSF Dear Colleague letter

Int’l Educational Data Mining
Society for Learning Analytics Research

—-

Student Use of Digital Resources: Implications for Learning and Technology Support – nice story about how their research developed.  Good presentation to possibly revisit.
Glenda Morgan, Dir. ATS & eLearning Strategist, University of Illinois at Urbana-Champaign

—

Annual grant program,
http://oia.arizona.edu/content/online-education-project-rfp
funded out of student tech fee
$10,000 to faculty to develop individual online & hybrid courses
Based on student priorities (bottleneck classes, summer, winter, etc… whatever classes students want to see more of, maybe the most over-subscribed classes)
Can apply – any instructor (including adjunct, grad students)

Project leaders (aka faculty) complete a “readiness to teach” diagnostic (theirs comes from Penn State)
Assigned ID consultant (out of 5 in total, they split the 27 projects)
PL required to attend a min of 3 prof dev activities (fairly flexible)
3-day summer intensive (want opportunities to share with & learn from each other)
Course is assessed with Quality Matters criteria & rubric
Mid-semester evaluations are completed
Final course evaluations (fax & student) are conducted
Final reports are submitted (with budget component)

Leveraging similarities to overcome differences by creating communities.

—

If faculty are highly rated on these three factors, they always get a high rating over all (zero probability of a low rating)

• Facilitate learning
• Ability to communicate
• Respect & concern for students

10 commandments (but there are only 8 below) – Chuck Dziuban, rite.ucf.edu, www.if.ucf.edu

1. good, fast, cheap – you can have 2, but not 3
2. don’t let tech drive the evaluation
3. stats & measurement are great, but are not everything “if you want people to believe something that is really, really stupid, stick a number on it:
4. push assess & flexibility out (don’t sit on your data)
5. a great product makes a huge difference (it’s hard to evaluate junk)
6. just because you can, doesn’t mean you should (eg studying class size, variables can bite!)
7. progress best made in simple steps
8. have to make what you’re talking about relevant to others’ world

Scholarship of Teaching & Learning – faculty in specific disciplines build evaluations for their areas.

For the first time that I’m aware of, we are participating in ELI online as a team (paid for one registration, invited faculty to join us, which the organizers encourage) & so far, it’s going well.  Very impressive to see the investment in the effort to deliver a whole conference online.  Hashtag: #eli2013.

Key takeaways from day one:

Why Intervention and Impact Evaluation Is Critical for Improving Student Learning
J. D. Walker, Research Fellow University of Minnesota
University of Wisconsin-Milwaukee
Specialize, Learn from others, Compare, Understand mechanisms
Great resource:
http://www.oit.umn.edu/research-evaluation/selected-research/learning-environments/index.htm

Raymond Fleming, Professor, University of Wisconsin-Milwaukee

U-pace method: http://www4.uwm.edu/upace/about.html – great resource,  including free training & a syllabus to get started.

Use info about the student that’s recorded in the LMS to monitor what they are & are not doing, intervene quickly.
Leveraged Bloom’s revised taxonomy

Great talk, great quotes, I love the vision (though perhaps overstated at times, you might say). If you have thoughts, leave a comment below…

I registered for 2 more Moocs I’m sure I won’t keep up with, but it’s interesting to see how they are organized & if I don’t at least post them here, I probably won’t ever even find them again, so here there are:

Learning Design For 21st Century Curriculum (OLDS Mooc): http://www.olds.ac.uk/the-course

Intro to Databases: http://class2go.stanford.edu/db/Winter2013

Happy to be spending the day at a “Flipping the classroom” Nercomp SIG
today. Especially happy to see two of my faculty colleagues here & a full session – maybe 100 people. I have the attendee list in email, but don’t want to violate everyone’s privacy by posting it here.  Very interesting group of folks.

I’m taking notes in Notability on iPad, instead of in this post as I’ve done in the past, I don’t see any real advantage in the wifi is good.

My notes from the day.

Agenda:  http://nercomp.org/index.php?section=events&evtid=204&eventsubpage=flip_agenda

Resources:https://sites.google.com/site/invertedclass/