We are pleased to share the findings from our recent evaluation conducted by Blackstone Evaluation and Research, which assessed the effectiveness of FOUNT courselets in classroom settings. The evaluation focused on courselets incorporated into courses taught by Co-PIs Dr. Fraida Fund and Dr. David Koop during the Spring 2024 semester.

Evaluation Overview

• Participants: Students from the “Introduction to Machine Learning” course taught by Dr. Fund and the “Advanced Data Management” course taught by Dr. Koop.
• Data Collection Period: May 3, 2024, to June 20, 2024.
• Methodology: Surveys were administered to gather feedback on specific courselets and the overall courselet format.

Key Findings

1. Enhanced Learning Objectives:
   • Dr. Fund’s Courselets:
     • Most respondents reported that the courselets improved their understanding and ability to perform the learning objectives.
     • Courselets covered topics like launching Kubernetes clusters, deploying services with Flask and Docker, and implementing load balancing and dynamic scaling.
   • Dr. Koop’s Courselets:
     • Nearly all respondents indicated that the courselets significantly advanced their understanding.
     • Topics included data analysis with Pandas, data cleaning, and data integration and fusion.
2. Courselet Format Feedback:
   • Effectiveness and Preference:
     • 89% of respondents rated courselets as more effective than other online instructional formats.
     • 83% expressed a desire to learn using more courselets in the future.
   • Comparison with In-Person Instruction:
     • While in-person lectures were still the most strongly liked format (78%), courselets tied with in-person labs (56%) in terms of preference.
     • 45% felt that courselets were more effective than in-person lectures or labs.
3. Impact on Motivation:
   • Dr. Fund’s Courselets:
     • Half or more respondents indicated increased motivation to learn more and apply the knowledge to other projects.
     • Fewer respondents reported increased motivation to pursue external research projects.
   • Dr. Koop’s Courselets:
     • Nearly all respondents reported increased motivation across all areas, including pursuing research or external projects.

Evaluator Recommendations

• Content Review and Enhancement:
  • Analyze structural differences between courselets to identify elements that boost motivation and engagement.
  • Incorporate collaborative features, such as discussion forums or synchronous sessions, to foster peer interaction.
• Community Building:
  • Develop strategies to build a community around courselet use, including promotional materials and platforms for user engagement.

Moving Forward

These evaluation insights are invaluable as we continue to refine our courselets and enhance their impact on learners. We are committed to:

• Implementing Recommendations: We will explore ways to incorporate collaborative elements and community engagement strategies into our courselets.
• Expanding Evaluation Efforts: Ongoing data collection will include all active courselets to provide a more comprehensive assessment.

We extend our gratitude to the students who participated in the evaluation and to Blackstone Evaluation and Research for their thorough analysis. Together, we are advancing data systems education and empowering learners to thrive in a data-centric future.