How to evaluate a Teaching Program through the alumni’s experience: an empirical research that provide insights for teaching optimization
DOI:
https://doi.org/10.7346/SIRD-022020-P137Keywords:
higher education, teaching program, SWOT Analysis, Evaluation, Service LearningAbstract
Evaluation is a fundamental step in the optimization process of a Teaching Program, however, in order to place it in a transformative perspective, it is necessary to adopt a dynamic vision, oriented to change. A model able to respond to these needs is the developmental evaluation, according to which evaluation is a fluid process, which should go beyond the summative approach. Its aim is to decode the complexity of reality according to a multidimensional vision, that puts a heuristic perspective at the service of the evaluation, in order to explore reality from non-obvious points of view. This is the humus from which the research here presented is born, a research that investigates a Teaching Program involving unusual interlocutors, such as students who have completed the path and are now involved in professional contexts (alumni). The study, through a qualitative approach, analyzes the experience of 42 alumni who participated in the experimental phase of the Service Learning Program promoted by the University of Verona, in order
to draw from it insights useful to optimize the program itself. The tool chosen to collect the data is SWOT Analysis because of its ability to shed light on the different sides of a complex organization, such as that of a Teaching Program.
References
Abu-Jdayil, B., & Al-Attar, H. (2010). Curriculum assessment as a direct tool in ABET outcomes assessment in a chemical engineering programme. European Journal of Engineering Education, 35(5), 489-505.
Anderson, J. B., & Erickson, J. A. (2003). Service-learning in preservice teacher education. Academic Exchange Quarterly, 7(2), 111-116.
Astin, A.W., Vogelgesang, L.J., Misa, K., Anderson, J., Denson, N., Jayakumar, U., Saenz, V., Yamamura, E. (2006). Understanding the effects of service-learning: A study of students and faculty. Retrieved from University of California, Los Angeles, Higher Education Research Institute.
Baldwin, S., Buchanan, A., & Rudisill, M. (2007). What teacher candidates learned about diversity, social justice, and themselves from Service Learning experiences. Journal of Teacher Education, 58, 315-327.
Bennett, S., & Santy, J. (2009). A window on our teaching practice: Enhancing individual online teaching quality though online peer observation and support. A UK case study. Nurse Education in Practice, 9(6), 403-406.
Billig, S. H., & Waterman, A. S. (Eds.). (2014). Studying service-learning: Innovations in education research methodology. Routledge.
Carrington, S., & Saggers, B. (2007). Service-learning informing the development of an inclusive ethical framework for beginning teachers. Teaching and Teacher Education, 24(3), 795-806.
Conner, J.O. (2010). Learning to unlearn: how a Service Learning project can help teacher candidates to reframe urban students. Teacher and Teaching Education, 26, 1170-1177.
Carroll, C., & O’Loughlin, D. (2014). Peer observation of teaching: enhancing academic engagement for new participants. Innovations in Education and Teaching International, 51(4), 446-456.
Duckenfield, M., & Wright, J. (1995). Pocket Guide to Service Learning. Clemson: National Dropout Prevention Center.
Dyson, R. G. (2004). Strategic development and SWOT analysis at the University of Warwick. European journal of operational research, 152(3), 631-640.
Ellingson, L. L. (2009). Engaging crystallization in qualitative research: An introduction. Sage.
Ellingson, L. L. (2017). Crystallization. The International Encyclopedia of Communication Research Methods, 1-5.
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of advanced nursing, 62(1), 107-115.
Erikson, M., Erikson, M. G., & Punzi, E. (2018). A single-question qualitative bachelor’s programme evaluation. Assessment & Evaluation in Higher Education, 43(6), 969-978.
European Commission (2014). Initial teacher education in Europe: an overview of policy issues. http://ec.europa.eu/education/policy/strategic framework/expertgroups/documents/initial-teacher-education_en.pdf (ver. 15.12.2015).
Fagen, M. C., Redman, S. D., Stacks, J., Barrett, V.,Thullen, B., Altenor, S., & Neiger, B. L. (2011). Developmental evaluation: Building innovations in complex environments. Health Promotion Practice, 12(5), 645-650.
Felisatti, E. (2019). La valutazione all’Università: riflessioni dal passato e prospettive per il futuro. Italian Journal of Educational Research, 15-28.
Felten, P., & Clayton, P. H. (2011). Service learning. New directions for teaching and learning, 2011(128), 75-84.
Gordon, J., Hazlett, C.,Ten Cate, O., Mann, K., Kilminster, S., Prince, K., & Newble, D. (2000). Strategic planning in medical education: enhancing the learning environment for students in clinical settings. Medical education, 34(10), 841-850.
Gosling, D. (2002) Models of peer observation of teaching, Proceeding LTSNGC Peer Observation of Teaching Conference. URL: www.heacademy.ac.uk/resources
Guba, E. G., & Lincoln, Y. S. (1982). Epistemological and methodological bases of naturalistic inquiry. ECTJ, 30(4), 233-252.
Harvey, L., & Newton, J. (2004). Transforming quality evaluation. Quality in higher education, 10(2), 149-165.
Harvey, L., & Newton, J. (2007). Transforming quality evaluation: moving on. In Quality assurance in higher education (pp. 225-245). Springer: Dordrecht.
Helms, M. M., & Nixon, J. (2010). Exploring SWOT analysis–where are we now? Journal of strategy and management, 3, 215-251.
Hill, T., & Westbrook, R. (1997). SWOT analysis: it’s time for a product recall. Long range planning, 30(1), 46-52.
Hounsell, D. (2003).The evaluation of teaching. A handbook for teaching and learning in higher education: Enhancing academic practice, 200-212.
Jackson, C., Mohr Schroeder, M., Cavalcanti, M., Albers, S., Poe, K., Delaney, A., Chadd, E., Williams, M. & Roberts, T. (2018). Prospective mathematics teacher preparation: exploring the use of service learning as a field experience. Fields Mathematics Education Journal, 3(5), 1-21.
Kettunen, J. (2015). Stakeholder relationships in higher education. Tertiary Education and Management, 21(1), 56-65.
La Marca, A. (2018). La formazione universitaria per lo sviluppo delle competenze de futuri insegnanti. In A. La Marca, E. Gulbay, Didattica universitaria e sviluppo delle soft skills. Lecce-Brescia: Pensa MultiMedia.
Lee, S. F., & Ko, A. S. O. (2000). Building balanced scorecard with SWOT analysis and implementing “Sun Tzu’s The Art of Business Management Strategies” on QFD methodology. Managerial Auditing Journal.
Leonard, S. N., Fitzgerald, R. N., & Riordan, G. (2016). Using developmental evaluation as a design thinking tool for curriculum innovation in professional higher education. Higher Education Research & Development, 35(2), 309-321.
Mainardes, E. W., Alves, H., & Raposo, M. (2010). An exploratory research on the stakeholders of a university. Journal of Management and Strategy, 1(1), 76-88.
McClain, L., Gulbis, A., & Hays, D. (2018). Honesty on student evaluations of teaching: effectiveness, purpose, and timing matter!. Assessment & Evaluation in Higher Education, 43(3), 369-385.
Merriam, S. B. (2002). Introduction to qualitative research. Qualitative research in practice: Examples for discussion and analysis, 1(1), 1-17.
Moore, S., & Kuol, N. (2007). Retrospective insights on teaching: exploring teaching excellence through the eyes of the alumni. Journal of Further and Higher Education, 31(2), 133-143.
Mortari, L. (2007). Cultura della ricerca e pedagogia. Roma: Carocci.
Mortari, L. (Ed.) (2017). Service Learning. Per un apprendimento responsabile. Milano: Franco Angeli.
Mortari, L. (2018). Valutare l’università. Verona: Quiedit.
Mortari, L. (in press). Community Responsible Learning: costruire insieme un bene comune. In L. Orlandini, S. Chipa, C. Giunti (eds.), Il Service Learning per l’innovazione scolastica Le proposte del Movimento delle Avanguardie educative. Roma: Carocci.
Mortari, L., Silva, R., & Ubbiali, M. (2019). A Case of Service-Learning and Research Engagement in Preservice Teachers’ Education. Journal of Higher Education Outreach and Engagement, 23(3), 145-158.
Olson, B. D. (2012). The ontological architectures in the application of a knowledge management system for curricular assessment (Doctoral dissertation, Capella University).
Osborne, W. L., & Purkey, W. W. (1995). A model faculty peer review process for counselor education programs. Journal of Counseling & Development, 73(6), 654-658.
Panagiotou, G. (2003). Bringing SWOT into focus. Business strategy review, 14, 8-10.
Patton, M. Q. (1994). Developmental evaluation. Evaluation practice, 15(3), 311-319.
Patton, M. Q. (2006). Evaluation for the way we work. Nonprofit Quarterly, 13(1), 28-33.
Patton, M. Q. (2016). What is essential in developmental evaluation? On integrity, fidelity, adultery, abstinence, impotence, long-term commitment, integrity, and sensitivity in implementing evaluation models. American Journal of Evaluation, 37(2), 250-265.
Penny, A. R. (2003). Changing the agenda for research into students’ views about university teaching: Four shortcomings of SRT research. Teaching in higher education, 8(3), 399-411.
Pointe, C. D. (2013). A reconsideration of the faculty peer-review process for promotion and tenure. Currents in Pharmacy Teaching and Learning, 5(3), 233-235.
Ramírez-Montoya, M. S., Mena, J., & Rodríguez-Arroyo, J. A. (2017). In-service teachers’ self-perceptions of digital competence and OER use as determined by a xMOOC training course. Computers in Human Behavior, 77, 356-364.
Rivoltella, P. C. (2017). Premessa. In C. Hadji, La valutazione delle azioni educative. Brescia: La Scuola.
Reynolds, M. (2014). Equity-focused developmental evaluation using critical systems thinking. Evaluation, 20(1), 75-95.
Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598-642.
Stanton, T. K., Giles Jr, D. E., & Cruz, N. I. (1999). Service-Learning: A Movement’s Pioneers Reflect on Its Origins, Practice, and Future. Jossey-Bass Higher and Adult Education Series. Jossey-Bass Inc., Publishers, 350 Sansome St., San Francisco, CA 94104.
Stowell, M., Falahee, M., & Woolf, H. (2016). Academic standards and regulatory frameworks: necessary compromises? Assessment & Evaluation in Higher Education, 41(4), 515-531.
Swick, K.J. (2001), Nurturing decency through caring and serving during the early childhood years. Early Childhood Education Journal, 29(2), 131-137.
Taylor, C. (1994), Il disagio della modernità. Roma-Bari: Laterza.
Trinchero, R. (2018). Valutazione formante per l’attivazione cognitiva. Spunti per un uso efficace delle tecnologie per apprendere in classe. Italian Journal of Educational Technology, 26(3), 40-55.
Veltri, N. F., Webb, H. W., Matveev, A. G., & Zapatero, E. G. (2011). Curriculum mapping as a tool for continuous improvement of IS curriculum. Journal of Information Systems Education, 22(1), 31.
Verducci, S., & Pope, D. (2001). Rationales for integrating service learning in teacher education. Service-learning in teacher education: Enhancing the growth of new teachers, their students, and communities, 2-18.
Viganò, R. M. (2020). Valutare l’insegnamento nell’istruzione superiore. A cosa serve veramente? Italian journal of educational research, (24), 120-137.
Volkwein, J. F. (2010). Assessing alumni outcomes. New Directions for Institutional Research, 2010(S1), 125-139.
Wade, R. C., & Anderson, J. B. (1996). Community service-learning: A strategy for preparing human serviceoriented teachers. Teacher Education Quarterly, 59-74.
Wauck, H., Yen, Y. C., Fu, W. T., Gerber, E., Dow, S. P., & Bailey, B. P. (2017, May). From in the class or in the wild? Peers provide better design feedback than external crowds. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (5580-5591).
White, M. D., & Marsh, E. E. (2006). Content analysis: A flexible methodology. Library trends, 55(1), 22-45.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Luigina Mortari, Roberta Silva
This work is licensed under a Creative Commons Attribution 4.0 International License.
The authors who publish in this magazine accept the following conditions:
- The authors retain the rights to their work and give the magazine the right to first publish the work, simultaneously licensed under a Creative Commons License - Attribution which allows others to share the work indicating the intellectual authorship and the first publication in this magazine.
- Authors may adhere to other non-exclusive license agreements for the distribution of the version of the published work (eg deposit it in an institutional archive or publish it in a monograph), provided that the first publication took place in this magazine.
- Authors can disseminate their work online (eg in institutional repositories or on their website) before and during the submission process, as it can lead to productive exchanges and increase citations of the published work.
