2012 Feb. 29
APQN conference - Siem Reap, Cambodia
Summary of IDRC-supported QA in DE studies
Insung Jung
The study was carried out between January and December 2010 and employed three data collection steps: 1) 11 cases (10 countries and one SAR) from East, South, and Southeast Asia were selected to include those with relatively well-established QA systems, those just introducing QA systems, and others still in the process of developing QA concepts in DE; 2) formal documents published by QA agencies (research institutes and governments in the selected countries/territories) and other references were analyzed to delineate DE development, QA policies, procedures, standards, and criteria for higher education in general and DE specifically; and 3) face-to-face, email, or telephone interviews with local experts working in the QA agencies and DE institutions were conducted to verify the data obtained.
This study revealed that QA in DE is still at an early stage of development compared with QA in conventional higher education. QA in DE is still a relatively new concept in some countries/territories, and the different QA approaches described above reflect the differences in cultures, expectations, and stages of development. At the same time, it found that there are common features in the approach taken by the different QA agencies in the cases examined, including: 1) working towards promoting a culture of quality within QA agencies and DE institutions, 2) positioning QA in the pursuit of self-improvement and public accountability of DE institutions, 3) considering distinctive features of DE in QA frameworks or during evaluation processes, 4) linking QA results to direct or indirect funding, levels of autonomy, or other supports,( only applies to public funded institutions) 5) adopting both internal and external assessments, and 6) making QA results public.
In light of these findings, several suggestions are made for the improvement of QA systems in Asian DE.
Details of this study can be found in: Jung, I.S., Wong, T.M., Li, C., Baigaltugs, S., & Belawati, T. (2011). Quality assurance in Asian distance education: Diverse approaches and common culture. The International Review Of Research In Open And Distance Learning, 12(6), 63-83. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/991/1953
A survey was sent out to over 40 DE providers in Asia between January and December 2010 to collect information about institutional QA mechanisms including procedures, methods and criteria, and accreditation and quality certification. Eventually 26 institutions responded to the survey.
When asked about the meaning of quality and QA in DE, the participating DE institutions saw quality and QA differently, depending on the needs of stakeholders, their priorities and perceptions. Despite differences in the meaning of quality and QA, they share common underpinning concepts relating to quality, such as continuous improvement, quality culture and quality standards. Some see QA as a comprehensive continuous improvement effort in terms of inputs, process, outputs and outcome, and others see it as effort to maintain, reach or comply with standards externally set, normally by their governments or quality agencies. To most DE providers, a good QA system should make efforts to meet customers’ and stakeholders’ satisfactions of distance and e-learning programmes and courses, and QA effort should be applied at planning and implementation stages to ensure quality outputs. Most DE providers indicate the important role of leadership in encouraging member participation throughout the QA process and effort is important so that everyone in the institution shares common awareness, understanding and spirit of QA.
Overall, large DE and e-learning institutions in Asia have a formal QA policy framework. Generally smaller institutions and institutions with small DE and e-learning operations state that there is no formal QA policy, despite the fact that they try to implement QA systems for their DE and e-learning programmes and courses. Also, DE and e-learning institutions in Asia vary in terms of the establishment of a formal QA Unit or Office, depending on size and priorities of the institution.
Several DE institutions in Asia also employ an external QA system, one referring to adopting benchmarks and good practices worldwide, another relating to the use of external peer review. Internal QA processes in Asian DE and e-learning institutions share common practices despite the fact that they vary slightly in terms of implementation, and these include such processes as assessment, monitoring, and review of programmes and courses, feedback consultation with stakeholders and employers, managements, reports and self-assessment. Some Asian DE and e-learning institutions have developed their own guidelines, manuals, procedures and forms for their own internal QA system.
Major challenges indicated by the DE institutions include: 1) inclusion of external reviewers with knowledge and experience in DE and e-learning as this will result in fair quality assessment, 2) applying indicators and standards that take into account the special characteristics of DE institutions, 3) securing well qualified and trained manpower in QA for DE, 4) improving awareness among faculty members and administration that the QA process needs to be implemented for quality DE and e-learning, and 5) establishing reliable technology infrastructure and receiving enough funding and support from governments or other supporters in the case of smaller DE institutions.
Details of this study can be found in: Zuhairi, A., Belawati, T., Nugraheni, E., Sadjati, I.M., PKH, Y., & Isman, S.M. (2011). A provider survey on the quality of distance and e-learning in Asia. A paper presented at the 25th AAOU Annual Conference. September 28 – 30, 2011. Penang, Malaysia.
The study identified ten quality dimensions and built a QA model on three domains: supportive, pedagogical and environmental. The three domains are used to categorize and organize the ten dimensions in a meaningful and efficient manner.
This study concludes that Asian DE providers should consider learners’ perspective in establishing or improving their QA framework since their views highlights important quality criteria which are not always reflected in the providers’ QA guidelines, and that DE providers should consider the gender differences when designing a learner support system. Some gender-considerate support strategies suggested in the study include: offering flexible schedules that help distance learners, especially females, avoid time conflicts with other responsibilities in taking exams and attending face-to-face meetings, setting up virtual or face-to-face office hours to provide academic or learning content-related support to distance learners, offering both personalized counseling and tutoring services especially to female students and taking advantages of online technologies in providing such services, and providing information on values of the degree earned through DE based on policies and factual evidences specific to male students.
This book attempts to document the existing regulatory framework covering QA in higher education in a number of Asian countries and how 16 DE/e-learning providers/programmes have developed their QA systems and procedures to address the regulatory requirements. It presents good practices in QA for DE/ e-learning, analyzes challenges in assuring the quality of DE and e-learning products and services, offer possible solutions to meet those challenges and presents lessons for other DE and e-learning providers. The 16 institutions include: four mega open universities (India’s Indira Gandhi National Open University, Open University of China, Thailand’s Sukhothai Thammathirat Open University, and Indonesia’s Universitas Terbuka), two dedicated public open universities (Open University of Sri Lanka and University of Philippines Open University), four dedicated private open universities (Malaysia’s Open University of Malaysia and Wawasan Open University, Open University of Hong Kong, and Singapore’s SIM University), two online programmes offered by conventional universities (China’s Peking University and Japan’s Kumamoto University), two new virtual universities (Korea’s Hanyang Cyber University and Virtual University of Pakistan), an NGO (Mongolian e-Knowledge), and a for-profit corporation (Korea’s AutoEverSystems).
The above selection of cases is not exhaustive but ensures that a wide range of QA systems and perspectives of quality in DE/e-learning is covered in this book.
Details of the 16 cases can be found in: Jung, I.S., Wong, T.M., & Belawati, T. (Eds.). (in process). Quality assurance in distance education and e-learning: Best practices, challenges, solutions, and lessons from Asia. India, New Delhi: Sage Publications India Pvt Ltd.
Asian QA Model for DE
proposed by IDRC QA project team
The aforementioned studies have revealed that there exist social and cultural diversities in QA concepts, systems and activities. Different countries and institutions in Asia have developed and applied their own QA model for DE based on their social environment and cultural values. There are therefore variations in the QA policies, criteria and methodologies they propose. Furthermore, they are at different stages of educational and technological development and adopting new and innovative approaches to teaching and learning.
A general model of QA in DE cannot entirely apply in all countries in Asia, but previous studies, for example Jung (2004), reveal that most of the key elements do in fact apply in most countries even though some countries depart further than others from the general model. The open, flexible and distance nature of DE requires a QA system that pays attention to open access, flexible learning methods, course development, reliable infrastructure, interactive communications and support. This suggests that a common or general QA model can be used as a starting point from which to map differences and divergences, and to which each country adds own QA dimensions and elements, or modifies procedures and methods. These deviations will be determined by cultural and other attitudes toward DE and learning in general, the stage of technological development, the flexibility of the legislative system of QA and other practicalities.
We propose a general model that can be benchmarked or modified by a nation or a DE institution as a basis for adapting QA in DE to its own unique context and seeking quality DE provision and social recognition.
Quality assurance policy framework
A QA policy framework at the national level (Figure 1) should have the following features:
Figure 1: QA policy framework for DE at national level
Figure 2: QA policy framework for DE at institutional level
The QA standards for DE should reflect the various quality concerns of key stakeholders including DE providers, assessors, employers, graduates and learners.
Thus the QA standards proposed here are created based upon the findings of the aforementioned studies which identified quality concerns at national, institutional and personal levels. In addition, the following references published by QA agencies or professional bodies are used.
In order to devise the QA standards, we first propose 18 QA criteria or areas across six domains as shown in Figure 3. Environmental, Pedagogical, and Supportive domains include 9 QA criteria in total which reflect key concerns of Asian distance learners. Institutional, Cyclic and Outcomes domains include 9 QA criteria in total which are major concerns of other stakeholders including DE providers, assessors, and other stakeholders.
Environmental domain
Pedagogical domain
Course Development refers to policies and guidelines that help ensure and maintain the quality of content and course, instructional design and course materials and resources development.
Teaching &Learning refers to all teaching and learning activities and online and physical resources provision. It also includes learning activities that promote teacher-learner and learner-learner interactions.
Evaluation &Assessment refers to policies and performance in regard to learning assessment, feedback, and various stakeholders’ evaluation.
Supportive domain
Faculty Support refers to policies and procedures for training, support during course development and delivery, and faculty welfare.
Student Support refers to policies and guidelines for technical, financial, psychological, social and administrative support, flexible payment and appeal mechanisms.
Information & Publicity refers to the provision of course-related and other logistical information and clear indication of requirements.
Institutional domain
Governance and administration refers to the institution’s system of governance and management to facilitate the successful accomplishment of its mission and goals.
Vision/Mission refers to the institution’s vision, mission statements, goals and intended outcomes of DE.
Cyclic domain
Outcomes domain
Figure 3: 18 QA criteria for DE across six QA domains
KPIs are defined as a set of both quantifiable and qualitative measures that a DE institution can use to gauge performance in terms of meeting its QA standards. But more specific evidences to measure these KPIs will be different across countries and institutions.
References