As we all know too well, our lives moved largely online beginning in mid-March 2020. Here at Vermont Afterschool, we adapted quickly by shifting all of our professional development offerings to Zoom (and provided them for free through June). We provided some of our regular offerings such as Basic Specialized Care, Reframing Conflict, and Adolescent Brain Development; as well as some special relevant topics like Effective Use of Zoom and Virtual Programming, and Supporting Staff Who are Working from Home.

We continued to collect online evaluations for each workshop, just as we do for regular in-person trainings. Now that we have a collection of data from the past year, it is a good time to not only summarize the annual PD evaluation data, but also try to understand what outcomes, if any, are different when learning becomes virtual.

To begin, in 2019-20 we offered a total of 72 workshops, including pre-pandemic in-person trainings and the aforementioned virtual trainings. These workshops made up 178 hours of content and were attended by 376 individuals – many of whom participated in more than one. (This was a significant increase from the previous year in which workshops were attended by 196 individuals as far as our records show).

The 376 individuals that attended represented 68 afterschool and youth-serving organizations around the state that comprised a total of 136 individual program sites. That means our PD offerings ultimately impacted youth that attended 29% of program sites around the state. That is an increase from the previous year when we impacted 21% of sites from 98 sites that were represented by workshop attendances.

From the workshops offered in 2019-20, we have evaluation data from 772 evaluations representing 67 trainings and Communities of Practice. We collect evaluations to help understand how our workshops are ultimately benefiting the field. As such we as whether participants gained new knowledge, will apply what they learned, found attending to be worth their time and effort, and most importantly, believe that attending will help them do their job better. For each item, participants respond with “Fully Agree,” “Partially,” or “Disagree.” We will look at each of these four items in more depth in terms of the percentage of full agreement attained.

“I gained new knowledge and/or perspective from attending this training.” In total, 84% of training participants in 2019-20 fully agreed with this statement. Evaluations on Communities of Practice sessions, which are less-focused on specific knowledge acquisition and more focused on idea-sharing generally received lower ratings in this area. Removing them from the analysis yields a total of 86% of evaluations that fully agreed with the statement. When we break down the workshops (excluding Communities of Practice) with those that were help in-person vs. online, we find that 85% of those who evaluated live trainings fully agreed while 89% of those who evaluated virtual trainings fully agreed.

“I will apply what I learned in this training in my program.” Participants that attended 2019-20 workshops learned a lot from them: 87% of evaluations showed full agreement with this statement. The breakdown when comparing live vs. virtual is as such: 85% of evaluations of in-person trainings fully agreed with the statement while 92% of virtual trainings fully agreed with it.

“Attending this training was well worth my time and effort.“ Across the board, 87% of evaluations fully agreed with this statement. Breaking down the results for in-person vs. online workshops reveal identical percentages of full agreement (87%) for both categories.

“I believe attending will help me do job better.“ We look at this statement as the ultimate indicator of whether our professional development opportunities have the potential to strengthen programs. Overall, 89% of evaluations submitted indicated full agreement with this statement. Of the evaluations that represented live trainings, 90% indicated full agreement, and of the evaluations that represented online trainings, 88% indicated full agreement.

It seems that for the evaluation items related to knowledge acquisition (“gained new knowledge,” and “apply what I learned”), virtual trainings were more successful at accomplishing these goals. Of course, this is not a flawless comparison. Only about one-third of all workshops (21) were offered virtually, and the list of virtual offerings was somewhat different than the list of in-person offerings in 2019-20.

There are a number of possible explanations for the discrepancy. Perhaps the end-of-the-year workshops generally score higher because by that point in the year trainers are very well-versed in their material — and all of our online workshops took place at the end of the year. Maybe the virtual trainings focused more on the programmatic content than on things like ice breakers and activities — thus leading to increased content delivery. Maybe there is a subjective confirmation bias at play as a result of the COVID-induced need for connection. It’s unclear. The following are a few comments from evaluations speak to the efficacy of the virtual format:

This training worked relatively well online. While I imagine that it is even more impactful in-person, I’m glad this was offered as a virtual training for after school staff members.

There was a lot of good information it was just unfortunate that the forum was via the web. It is certainly understandable based on the current situation. I do appreciate the opportunity to take the class.

I really enjoyed this training. Zoom meetings can be very draining if you are just sitting and listening. This training allowed for physical activities pertaining to the topics, and I feel that kept me more engaged than meetings with no activities.; I feel [trainer’s name] did a fantastic job turning a hands on workshop into a Zoom training. It is not an easy task to keep everyone engaged but I feel she stepped up to that challenge and pulled it of successfully.

There you have it! 2019-20 marked another successful year of professional development for Vermont Afterschool, pandemic notwithstanding. The next year will likely bring more virtual trainings. It’s helpful to know that in the future, whether they are necessary or not, online workshops have the capacity to produce equivalent, if not better, results than their in-person counterparts — and are able to be attended to all those with internet access. No travel, no snow cancellations, no masks required.