National Quality Assurance Systems in Distance Education in Asia

Appendix 1:
National Quality Assurance Systems in Distance Education in Asia

January 5, 2011

(Prepared as a report to ‘Openness and Quality in Asian Distance Education’, a collaborative project of IDRC and VUP)

As one part of Sub-Project 6: Quality Assurance (QA) Models, Standards and Key Performance Indicators for ICT-supported Distance Education (DE) in Asia, this report describes and analyzes national quality assurance (QA) systems in distance education (DE) at tertiary level in Asia based on the analysis of formal documents provided by QA agencies, research institutes and governments and other references and interviews with local experts working in the QA agencies and DE institutions. In this report, DE includes various forms of technology-supported education including e-learning.

Summary and Conclusion

Insung Jung (International Christian University, Japan)

Introduction

Over the last few decades, there has been a noticeable growth in DE in Asia. Asia has the largest number of distance learners in the world, at least ten the world’s mega-universities, over 70 open universities, a growing number of conventional institutions offering DE, and the rapidly growing private and/or for-profit DE providers. Driven by these changes and other economic and social forces, Asian DE providers are now asked to respond to accountability demands from their government and other stakeholders and provide more concrete evidences for returns on investment beyond their broader contributions to society and national development. In response to the public demand for accountability, some countries/territories have developed a QA system for DE, some are trialing it, and some are in the process of the development.

Some countries/territories like Hong Kong (China), Malaysia, Indonesia, the Philippines, Singapore, and Sri Lanka consider DE/e-learning as an integrated part of higher educational delivery and thus apply same procedures and criteria to any types of educational provisions. Some show consideration toward uniqueness of DE/e-learning during the accreditation or audit process.

Indonesia adjusted instruments to accommodate the uniqueness of open and distance higher education programmes (the ‘Accreditation Instrument for Distance Education Study Programmes’) in evaluating DE programmes but does not separate the accreditation process for purely online programmes.
The Philippines specifies DE accreditation criteria in CHED Memorandum Order No. 27.
Sri Lanka encourages the use of the ‘Quality Assurance Toolkit for Distance Higher Education Institutions and Programmes’ in evaluating and improving QA systems and policies.
But Hong Kong (China), Malaysia and Singapore do not have guidelines or standards designed for DE/e-learning.

Other countries such as China, India and Korea acknowledge distinctive features of DE/e-learning and thus apply different QA procedures and criteria.

In China, a new QA system for online colleges and the Open University of China (OUC) has been applied in evaluating their online learning practices. But no QA systems exist to accredit or periodically audit OUC’s DE DE processes and outcomes.
In case of India, the Distance Education Council oversees QA for DE.
In Korea, the Korea Education and Research Information Service (KERIS) has managed the evaluation of cyber universities using a QA framework different from that for other higher education institutions. But in the future, the Korean Council for University Education (KCUE) and/or other agencies which acquire the government’s recognition as an accreditation agency will be in charge of quality audit and accreditation of DE education institutions.

Some countries like Japan and Mongolia are yet to determine their stand or are in the process of developing QA procedures and criteria to consider DE/e-learning.

QA purposes

Brennan (1999) has suggested seven purposes of QA models in higher education: (1) ensuring accountability for public funds; (2) improving the quality of provision; (3) stimulating competition within and between institutions; (4) verifying the quality of new institutions; (5) assigning institutional status; (6) underwriting transfer of authority between the state and institutions; and (7) facilitating international comparisons.

In Asian countries, the common rationale behind the adoption of a QA system for DE is to improve the quality of DE provision.
But in case of India, Indonesia and Sri Lanka where DE is publicly funded, QA is emphasized to ensure accountability for public funds.
In Malaysia, Korea and China where private DE provision or e-learning provision by conventional universities is growing, QA also focuses on verifying the quality of new institutions and stimulating competition between DE institutions.
In case of Singapore and Hong Kong (China) where the government does not accredit DE institutions or self-accreditation is adopted, QA is also to underwrite transfer of authority between the government and institutions.

QA models

Four basic QA approaches or models for DE exist in Asian countries: Accreditation, academic audits, performance-based funding and performance reporting.

Accreditation is a process of assessment and review of whether an institution (or programme) qualifies for a certain status or to be recognized or certified as meeting certain required standards. The result of accreditation is whether an institution or programme either receives or does not receive accreditation. Accreditation for DE institutions or programmes takes place in China, India, Indonesia, Korea, Malaysia and the Philippines.

audits focus on the processes that an institution conducts a critical self-analysis, and external review teams verifying the self-report, making recommendations for improvement, and monitoring progress. It asks: “how well are you doing what you say you are doing?” Academic audits use both quantitative and qualitative process performance indicators that are developed and collected at the institutional level or using standardized national performance indicators against which institutions are audited. In Asian DE, countries/territories like China, Hong Kong (China), Korea, Malaysia, Sri Lanka and Singapore conduct periodic academic audits.

Performance-based funding ties public funding to performance of an institution or programmes. In the case of Indonesia, Sri Lanka and Korea, the outcomes of accreditation or academic audits are directly or indirectly linked to governments’ funding decisions.

Performance reporting refers to a QA approach that the reports of institutional performance are open to the public and submitted to governments and/or QA authorities. The reports provide valuable information for the public and policymakers to make decisions and they reflect the customer-oriented focus of DE provision. While most Asian countries make the reports public, some countries such as Japan, Sri Lanka, the Philippines and China either disclose only the final outcome on the status of accreditation or audits or share the reports within institutions and QA authorities. However, the trend of moving towards public disclosure of more information to the public is observed.

Regulatory framework

Accreditation and/or academic audit approaches in DE/e-learning can be either mandatory or voluntary. In Hong Kong (China), Indonesia, India and Mongolia, accreditation and audit are conducted on an institution’s voluntary basis. In these countries, the outcomes of QA processes are not directly linked to government funding. However in case of India and Mongolia, special development funds or government scholarships are given only to accredited institutions.

In other countries including China, Japan, Korea, Malaysia and Singapore, accreditation and periodic audits are mandatory. In China, those online institutions which fail to pass the annual academic audit are not allowed to recruit students in the following year. In case of Korea, the outcomes of QA activities are directly linked to financial and administrative supports from the government. In Japan, no funding or administrative sanctions or rewards are given to the institutions. Malaysia and Singapore do not link the QA results to governmental funding decisions but Malaysia links the outcome to levels of institutional autonomy.

As seen in Table 1, there exist different types of regulatory frameworks for QA in DE.

In China and Singapore, the government (Ministry of Education) directly regulates QA measures for DE institutions or programmes.
In Hong Kong (China), India, Korea, Malaysia, Mongolia, and Sri Lanka, a QA agency regulates QA in DE.
Some QA agencies (Korea’s KERIS and Indonesia’s BAN-PT) are governmental initiatives and others are quasi-governmental structures where the QA agency has a close relationship with the government but is administered by autonomous governing structures as in India. In Indonesia, QA in higher education is enforced through both self-evaluation monitored directly by the Ministry of National Education and accreditation processes by an independent accreditation agency (BAN-PT).
In the Philippines, both technical panels organized by a government operating QA agency and membership-based agencies or professional associations are responsible for QA.
In case of Japan, three non-governmental membership-based agencies regulate QA in DE.

Table 1: QA/accreditation agencies in 11 Asian countries

Country	QA Agency	URL	Note	Guidelines for DE	Resource
China	the Higher Education Department (HED) of MOE	http://www.moe.edu.cn/publicfiles/business/htmlfiles/moe/moe_2792/index.html	Compulsory, every year	Yes	(QA Procedure)
Hong Kong	Hong Kong Council for Academic and Vocational Qualifications	http://www.hkcaavq.edu.hk	Voluntary	No
India	The Distance Education Council (DEC)	http://www.indiaedu.com/education-india/distance-education-council.html	Voluntary	Yes	DEC Handbook on Assessment and Accreditation of Open & Distance Learning Institutions (HAAODLI)
Indonesia	the National Accreditation Board of Higher Education (Badan Akreditasi Nasional Perguruan Tinggi or BAN-PT)	http://ban-pt.depdiknas.go.id/	Voluntary	Yes	Accreditation Instrument for Distance Education Study Programmes
Japan	the National Institute of Academic Degrees and University Evaluation	http://www.niad.ac.jp/english/index.html	Compulsory, every 7 years	No
	Japanese University Accreditation Association	http://www.juaa.or.jp/en/index.html	Compulsory, every 7 years	No
	Japanese Institute for Higher Education Evaluation	http://www.jihee.or.jp/english.html	Compulsory, every 7 years	No
Korea	the Korean Council for University Education	http://english.kcue.or.kr/	Compulsory, every 5 years (Self evaluation: Compulsory, every 2 years)	Yes	Cyber University Evaluation Handbook (for self-evaluation)
Korea	The Korean Council for University College Education (for 2 year college)	http://english.kcue.or.kr/sub_new_01_4.html	Compulsory, every 5 years	No
Malaysia	The Malaysian Qualifications Agency	http://www.mqa.gov.my	Compulsory, every 5 years	No
Mongolia	The Mongolian National Council for Education Accreditation	http://www.accmon.mn/english	Voluntary	No
Philippines	the Accrediting Association of Chartered Colleges and Universities of the Philippines	http://www.aaccupqa.org.ph/	Voluntary	No
	the Federation of Accrediting Agencies of the Philippines	http://en.wikipilipinas.org/index.php?title=Federation_of_Accrediting_Agencies_of_the_Philippines	Compulsory	No
	the Commission on Higher Education (the CHED Technical Panel on Distance Education)	http://202.57.63.198/chedwww/index.php	Compulsory	Yes	CHED QA System for DE
Singapore	the Ministry of Education’s (MOE) Quality Assurance Framework for Universities (QAFU)	http://www.moe.gov.sg/	Compulsory, every 4 years	No
Singapore	the Council for Private Education (for private institutions)	http://www.cpe.gov.sg/	Compulsory, every 4 years	No
Sri Lanka	the Quality Assurance and Accreditation Council (QAAC) of the UGC/Ministry of Education	http://www.qaacouncil.lk/	Not known	Yes	Quality Assurance Toolkit for Distance Higher Education Institutions and Programmes

QA methods and procedures

The QA systems of selected Asian countries adopt the following common methods:

Review based on pre-determined QA criteria: A set of QA standards and criteria determined by the government or the QA agency are applied to all institutions or programmes. In developing the standards and criteria, nation-wide consultations with DE experts are often sought.
Self-assessment (self-study; self-evaluation): The institution (or programme) undergoing the QA process is required to do a self-assessment and report on how it meets the pre-determined standards or criteria.
External review (peer review): A team of external peers constituted by the QA agency analyses the self-assessment report of the institution/programme and validates the claims made in the report, generally by visiting the institution.
Final decision by the QA/accreditation agency: Based on the results of the self-assessment and the external review, the QA agency makes the final decision.

Some QA agencies (e.g. Japan’s NIAD-UE and Mongolia’s MNCEA) assist the institutions by providing training for the preparation of a good self-report. Many agencies provide training for external reviewers. In case of India’s DEC, before going through the formal QA process, the institution’s readiness is assessed.

QA standards and criteria

Where there is a QA system for DE, QA criteria, guidelines or performance indicators both for self-assessment and external review are often specified covering input, process and output variables. In case of Sri Lanka, the Quality Assurance Toolkit for Distance Higher Education Institutions and Programmes offer the performance indicators for distance higher education institutions under ten QA criteria and those of programmes under six criteria representing the various dimensions of DE practice. These performance indicators are designed to enable institutions to: 1) conduct a self-assessment of the performance of their processes in order to make necessary adjustments and changes for quality improvement; and 2) monitor the processes for continuous learning and ongoing improvement.

Table 2 summarizes key QA criteria in Asian countries. Common criteria are:

Vision, mission and/or goals
Assessment & Evaluation
Educational resources
Leadership, governance & administration
Finance
IT infrastructure
Teaching and learning
Course development
Student support
Faculty & Staff
Internal QA system
Research

Table 2: Key QA criteria for DE in 11 Asian countries

Criteria	Countries
	China	Hong Kong	India	Indonesia	Japan	Korea	Malaysia	Mongolia	Philippines	Singapore	Sri Lanka
Vision, Mission, Goals	√	√	√	√	√	√	√	√	√	√	√
Assessment/Evaluation	√	√	√	√	√	√	√	√	√	√	√
Educational Resources	√	√	√	√	√	√	√	√	√	√	√
Leadership, Governance & Administration		√	√	√	√	√	√	√	√	√	√
Financial Resources		√	√	√	√	√	√	√		√	√
IT Infrastructure		√	√	√		√
Teaching & Learning	√	√	√	√		√	√		√	√
Course Development	√	√	√	√	√	√	√	√	√	√	√
Student Support	√	√	√	√	√	√	√	√	√	√	√
Faculty & Staff	√	√	√	√	√	√	√	√		√	√
Internal QA System		√		√	√				√		√
Research		√	√	√	√	√	√	√		√	√

International accreditation

DE institutions in two countries have obtained international accreditations. Several online education colleges in China and Indonesia’s UT have obtained ISO 9001. UT had been reviewed by a Panel of the International Council for Distance Education (ICDE). Details of ICDE Quality Review are explained in the Indonesia’s country report.

Conclusion

The level of QA policy integration in an overall national QA in higher education policy framework varies across the eleven countries examined. The experience of these countries at different stages of QA system development shows that the QA purpose, methods and instruments are tailored to each country's particular circumstances. But at the same time, it reveals that there are some commonalities that connect these different QA efforts of all countries examined including:

positioning QA in the pursuit of public accountability and self-improvement of DE institutions,
considering distinctive features of DE in QA frameworks,
linking QA results to direct or indirect funding, levels of autonomy or other supports,
adopting both internal and external assessments, and
making QA results public.

Overall, QA in DE is still at the initial stage of development compared with QA in conventional higher education. QA in DE is still a relatively new concept in some countries/territories, and the different QA approaches described above reflect the differences in cultures, expectations and stages of development. Each of these approaches has its particular strengths and weaknesses and so it would be invidious to prescribe any single approach. However, in the light of these findings, it is suggested that the following principles should underpin national QA policy directions and that without these, there will be a considerable waste of resources and human potential in Asian DE.

All nations should see QA in DE as an integral part of broader national, regional and international QA frameworks. In a cross-border DE context, learners can be distributed anywhere and education can be delivered to them wherever they are. In the not too far future, learners can take some part of their course from one university, and some part of their course from another university, and some part of their course from a different university within their own nation or beyond the nation. To protect learners from the risks of low quality programmes and education of limited national and international validity, QA policies in DE in a nation should be linked to the broader national, regional and international QA frameworks.

Concerted effort is needed from governments, national and regional QA bodies and institutions regarding cross-border accreditation and QA in DE. Charmonman (2005) observes that while all ASEAN countries are promoting e-learning, there are problems of cross accreditation that need to be resolved for the sake of educational development and free flow of skilled manpower. He suggests that all ASEAN governments should either grant recognition to all online degree programmes accredited by another ASEAN government or establish a commission to do this.

With the convergence of on- and off-campus educational methods, there should be no distinctions between QA in DE and conventional higher education or e-learning and face-to-face teaching. However, having said that, specific QA guidelines, criteria or methods are needed for the various delivery modes.

National QA frameworks should help DE institutions or programmes make QA an integral part of the institutional missions to teach and research, and promote the quality culture in the institutions. Koul and Kanwar (2006) suggests that what is called for is a ‘culture of quality’ that is shared willingly by all members of staff who both draw from it and nurture it, links internal and external accountability, builds capacities in QA and involves open and transparent management and communication. Such a culture entails asking awkward questions, admitting to things that go wrong, acknowledging the changes needed and implementing these changes. Such behavior may well represent a considerable paradigm shift in some of Asia’s more hierarchical and bureaucratic institutions that currently only pay lip-service to the idea of QA, but is essential if DE is to deliver on its promises and to be held in high regard.

Quality key performance indicators (Q-KPIs) will help DE institutions monitor their performance against institutional objectives and key principles of their plan. DE institutions can use these Q-KPIs in self-assessment for continuous quality improvement. In particular new DE institutions will benefit from using the indicators during formative evaluation and thus to correct their initial actions.

Country Reports

China
Chen Li (Beijing Normal University, China)

Overview

There are over 2,300 universities and colleges in China. Since China kicked off the ICT-supported DE pilot project in 1999, ICT-supported DE has achieved remarkable developments. Two major DE provider groups are 68 online education colleges in the conventional universities and the Open University of China (OUC) system (http://en.crtvu.edu.cn/) and have attracted over eight million students by the end of 2008. In 2008, the number of active students was 3,560,000 (1,310,000 studying in the online education colleges and 2,250,000 in the OUC system), which accounts for 12% of the total number of students receiving higher education in China.

The 68 online education colleges offer 1,674 majors in 11 disciplines and 300 specialties at three different programs: undergraduate programs for high school graduates, junior college programs for high school graduates, and undergraduate programs for junior college graduates. OUC, together with China's 44 provincial open universities, 949 municipal open universities and experimental schools of provincial open universities and 1,823 county-level open universities, forms the largest DE system in the world with 74 majors - 38 majors for junior college education, 20 for undergraduate education and 16 for the “one undergraduate student per village” project.

China does not have a system to assure the quality of DE institutions and programs at national policy level. However it has established a QA procedure for the establishment and management of online education colleges operated by 68 conventional universities and OUC.

QA methods and procedures

The establishment of online education colleges in the conventional universities is under direct administration of the Ministry of Education of the People's Republic of China (MOE). From 1999 to 2003, MOE approved 68 conventional universities to take part in ICT-supported DE project. OUC was also chosen to take part in the pilot project. MOE has stopped the approval of setting up new pilot colleges since 2003.

The approval process of ICT-supported DE project began with each university’s submission of application documents in which the university should explain its new program’s mission, organization and management, teaching and learning models, equipment, teaching and support staff, off-campus learning centers, online instructional resources, technology infrastructure, QA system, and the benefit analysis. Then the university presented its case in a meeting with officials of the Higher Education Department (HED) of MOE (http://www.moe.edu.cn/publicfiles/business/htmlfiles/moe/moe_2792/index.html) and in another discussion meeting with external reviewers or experts. Based on the results of all these evaluations, the university was either approved or disapproved to participate in the ICT-supported DE project.

Once established, the online education colleges have to take two QA measures: following nationally standardized syllabi and examinations, and following Annual Reporting and Censorship (ARC) procedure. Also the online colleges are encouraged to share their experiences through a consortium and obtain external reviews.

Standardized syllabi and examinations: To enhance the quality of online education, MOE introduced nationally standardized syllabi and examinations for selected subjects including literature, mathematics, English, introduction to computer application offered by the online colleges. Those who want to get the undergraduate diploma from these online colleges have to take the national examinations in those subject areas. Two online computerized exams are implemented in English and introduction to computer application.

Annual Reporting and Censorship (ARC): MOE has begun to implement the ARC system since 2001 to assure and improve the quality of DE and better manage and regulate its decision making process. All online education colleges, the OUC system and public service systems are the subjects of the ARC. The ARC system requires each online college to follow four steps: self-report, annual reporting to MOE, site visit, and annual censorship. Provincial education administrations are in charge of the evaluation of learning centers. Self-report requires each college to evaluate its own online education and management. Once the self-report is prepared, each college needs to submit a report to MOE annually. The online system was developed in 2003 to more efficiently carry out the annual reporting to MOE. After the site visit by the evaluators (both government officials and experts), the censorship decision is made. Once an online college or OUC gets censored, then it cannot recruit students next one year. So far only one college got censored but successfully passed the re-evaluation next year.

Consortium and external reviews: The National Higher ICT-supported Distance Education Consortium, consisted of deans in charge, instructors and technology officers of the pilot online colleges, was founded in 2000 to encourage its members to exchange experiences in online education, explore possible solutions to common problems and foster the development of DE in China. In addition to participation in the Consortium, many online colleges have adopted periodic reviews of their instructional processes and some - Peking University Medicine online education college, Northeast University of Finance online education college, China University of Petroleum (East China) College of Distance and Continuing Education College, China University of Petroleum (Beijing) Institute of Distance Education College - have obtained ISO090001 quality management system certification for their internal QA system.

QA criteria

QA criteria for initial approval include:

Mission and course design: vision, course objectives, instructional design of content, learning activities and assessment
Teachers: academic standards, teacher competencies, teaching team structure, teaching experience and research evidences
Content and learning resources: Knowledge coverage, applicability and scalability of teaching content, interface and interaction design, media/technology selection, multimedia use, following the CELTS (China ELearning Technology Standardization) series and other relevant standards
Organization and implementation of teaching activities: Use of a variety of teaching-learning strategies, Evaluation tools, feedback and record of learning process, practice and guidance
Learning supports: Learning and technical support, humanistic care, support staff, means of learning support
Teaching effectiveness: student feedback, evaluation of student assignments, peer and self-evaluation
Policies: Institution’s effective policies to promote e-learning quality

It is emphasized that online courses should promote application-oriented personnel training objectives, be suitable for adult practitioners who study part-time, and use online technology effectively. During the annual reporting process, an online education college is subject to assess several criteria including:

Mission, management policy and organization’s conditions
Management of admission and enrollment
Management of examination
Instructional management
Development and sharing of instructional resources
Learning support services
Off-campus learning center management
Graduation and degree award of qualifications
Non-degree education
Inclusion of students from Western China and rural areas
Innovative features

Hong Kong Special Administrative Region, China
Tat Meng Wong (Wawasan Open University, Malaysia)

Overview

There are 9 universities and a number of colleges and vocational institutes in Hong Kong SAR, China. The Hong Kong Council for Academic Accreditation (HKCAA) was established in 1990 to advice the Government on the quality of educational programmes through accreditation for higher education providers (HEP) which have not been conferred self accrediting/university status. The HKCAA also conducts institutional audits leading to the award of self accreditation status. In Hong Kong, only institutions with self-accreditation status can apply to the Government to be conferred the title of ‘University’

In Oct. 2007, with the enactment of the ‘Accreditation of Academic and Vocational Qualifications Ordinance (AAVQ Ord.), the HKCAA was re-titled to the Hong Kong Council for Academic and Vocational Qualifications (HKCAAVQ: http://www.hkcaavq.edu.hk ).

The HKCAAVQ acts as the Accreditation Authority as well as the Quality Review Authority under a 7 level Qualifications Framework (QF) where each level is formulated as outcome-based Generic Level Descriptors (GLD) which describes the common features of qualifications awarded at that level of the QF. Qualifications which are quality assured are listed under the appropriate level in the Qualifications Registrar (QR)

Distance education ‘arrive’ in Hong Kong with the establishment of the Open Learning Institute of Hong Kong (OLIHK) in 1989. The OLIHK was audited by a Panel from the UK based Council for National Academic Accreditation (CNAA) and found ‘fit’ to offer degree level education prior to its 1st intake of students in October 1989. In 1993, the first batch of OLIHK programmes successfully undertook the accreditation exercise conducted by a Panel set up by the Hong Kong Council for Academic Accreditation (HKCAA) which was established in 1990.

In 1995, the HKCAA conducted an institutional audit on OLIHK and after a period of consolidation during which a number of additional QA processes were put in place, OLIHK was granted self-accrediting status in 1996. In 1997, the OLIHK was conferred the title of University by the Government and was re-titled as the Open University of Hong Kong (OUHK). As a University with self-accrediting status, programmes offered by OUHK were no longer required to be subjected to external accreditation. However, the institution is required to undertake periodic institutional audits. OUHK successfully undertook subsequent Institutional Reviews (Audits) in 2002 and again in 2007.

Other DE/online education institutions in Hong Kong include the Cyber University of the Hong Kong Polytechnic University (http://www.hkcyberu.com) and the Hong Kong Virtual University consortium (http://hkvu.ust.hk/hkvu/programme.html).

QA methods and procedures

The HKCAAVQ is the accreditation authority for the Quality Framework in Hong Kong. It uses a 4 stage QA process as follows:

Initial Evaluation (IE): All institutions who wish to have their qualifications listed on the QR must apply to undertake the IE. This accreditation exercise enables the HKCAAVQ to assess whether the institution has the competency to effectively manage and provide adequate resources to develop, deliver assess and quality assure their learning programmes and educational services.
Programme Validation and Revalidation: Programme Validation is based of document submission followed by the usual site visit and the overall evaluation is based on the assessment of programme planning and management, curriculum design and assessment methods, delivery arrangements and learning outcomes to ensure that they are appropriate for the qualification awarded. Validated programmes qualify to be entered into the Qualifications Registry.
Revalidation takes place towards the end of the initial validation period and re-evaluates whether the findings of the 1st validation are still valid based on the assessment of the same parameters. Validity of the Revalidation period is also specified.
Programme Area Accreditation: Institutions who has successfully completed two cycles of Programme revalidation may apply to be considered for Programme Area Accreditation (PAA). Those awarded the PAA status can develop and offer new programmes within the same defined scope of programme area and enter these qualifications into the QR without being subjected to external quality assurance by the HKCAAVQ during the PAA validity period.
Periodic Review (Audit): All institutions with PAA status are required to undertake Periodic Review (Audit). Those who are successful will have the validity of their PAA extended for another fixed period.

QA criteria

Criteria for Initial Evaluation (IE) cover:

Organizational Management: Operators’ management, including structure, processes and quality assurance arrangements are sufficient to manage their operations to meet the stated objectives.
Staffing and Staff Development: Operators have adequate teaching and support staff with qualifications and experience necessary for the effective delivery of their learning programmes at the intended Qualifications Framework (QF) Level(s).
Financial and Physical Resources: Operators have adequate financial and physical resources for the delivery of their learning programmes.
Quality Assurance (including Programme Development andManagement): Operators develop learning programmes by addressing the needs of the community, employers and employees and aligning them with the generic skills and knowledge specified in Generic Level Descriptors. Operators also monitor and review the performance of all their learning programmes on an ongoing basis to ensure that the programmes remain current and valid and that the content and design, teaching and learning activities and learner assessments are effective.

In evaluating the above (based on submission documents including a self –evaluation report and a site visit) the Evaluation Panel is guided by four guiding principles including peer review, fitness for purpose, evidence-based and threshold standards.

Threshold standards covering the above areas are set up for different levels of awards within the Qualifications Framework. These guides are available from the HKCAAVQ (http://www.hkcaavq.edu.hk/en/publication.asp). For example, 10 QA criteria for programme validation at QF levels 1 – 3 include:

Programme Objectives and Learning Outcomes: The programme has competency-based learning outcomes which: are consistent with the Generic Level Descriptors (GLD) of the relevant QF level and reflect the stated programme objectives.
Programme Content and Structure: The programme enables learners to achieve the stated learning outcomes and the required QF standards and provides up-to-date contents to meet the programme objectives.
Training and Learning: Training methodology is compatible with the programme objectives and appropriate to the learners’ capabilities and learning needs. And workplace attachment, if applicable, is structured and managed to meet the programme objectives.
Learner Assessment: Learner assessment is valid and reliable to assess the learner’s attainment of the intended learning outcomes.
Admission Requirements and Learner Satisfaction: Admission requirements and selection criteria ensure learners have the knowledge and skills to undertake the training activities and achieve the intended learning outcomes.
Learner Support Services: Learners are effectively guided and supported throughout their course of training to ensure the successful completion of the programme.
Staffing and Staff Development: Programme management and teaching staff have relevant qualifications, industry experience, and training experience, and understanding of QF levels and QF standards. And the skills and knowledge of training staff are up-to-date.
Programme Development, Management and Review: The operator develops programmes addressing community/industry needs. The programme is continuously evaluated to ensure its content, learning outcomes and assessments are current, valid and effective in meeting the programme objectives. And roles of staff in developing, managing and reviewing of the programme are clearly defined.
Financial and Physical Resources: The programme is financially viable. And training materials, equipment and facilities are adequate and appropriate.

Learner Records and Information Management: Learner records are safely protected based on written procedures. And learners are informed of the arrangements of issuance of QF-recognized qualifications.

India
Tat Meng Wong (Wawasan Open University, Malaysia)

Overview

India has 42 central universities, 259 state universities, 130 deemed universities, 65 private universities established by State Legislature, 39 institutions at state or national level, and 16885 colleges, including over 1800 exclusive women's colleges, functioning under these universities and institutions.

Maintenance of standards and quality in higher education is the responsibility of the University Grants Commission (UGC). Standards and quality of education in professional areas such as Engineering, medical, Nursing, Accountancy, etc. are maintained by specific agencies established by the Parliament of India. Quality in distance education is the responsibility of the Distance Education Council (DEC).There are three main bodies responsible for the accreditation of qualification awarded by colleges and universities.

The National Assessment and Accreditation Council (NAAC) is the Quality Assurance arm of the UGC which oversees all public colleges and universities. NAAC assesses the quality of higher education institutions on a voluntary basis and awards them a rating on a 4-point scale (A= Very Good, B= Good, C= Satisfactory, and D= Unsatisfactory)
The All India Council for Technical Education (AICTE) was set up by the Ministry of Human Resource Development in 1985 to regulate the standards of technical education. AICTE has also set up its own Quality Assurance body named the National Board of Accreditation (NBA) which accredits programmes offered in the field of technical education (defined to include Engineering and Technology, Management Education, Hotel management, Catering technology, Pharmacy, Architecture etc.
The Distance Education Council (DEC) (http://www.ignou.ac.in/ignou/aboutignou/icc/dec) was set up by IGNOU in 1991 to provide oversight on the Open and Distance Learning sector including maintenance of standards, as IGNOU is empowered to maintain quality in distance education by virtue of IGNOU Act, 1985.

To qualify for employment by the Government sector holders of DE-based degree and certificates must come from institutions that are approved by the DEC. In furtherance of it standards maintenance role, DEC has more recently introduced accreditation of institutions as a mark of quality. The criteria used by DEC to audit the quality of ODL institutions vary from those used by NAAC as well as the NBA.

DE first made its appearance in India in the 1960’s in the form of correspondence education. Traditional ODE made its appearance in the 1980s and was given a huge boost with the setting up of the Indira Gandhi National Open University (IGNOU) in 1985 while the last decade saw the appearance of a number of institutions offering Virtual and Online education. The DEC was set up within IGNOU in 1991. Among the functions assigned to DEC was ‘the determination of standards of teaching, evaluation and research’. In pursuit of its standards maintenance role, DEC has used the approach of accrediting institutions (instead of academic programmes). A major development in this direction took place in 2009 with the publication of the DEC Handbook on Assessment and Accreditation of Open & Distance Learning Institutions (HAAODLI) accompanied by the establishment of a National Assessment & Accreditation Board (NAAB). A copy of the comprehensive HAAODLI can be downloaded at http://www.ignou.ac.in/upload/Publications.doc.

According to the Handbook, currently ‘DEC will only be considering Institutional Accreditation while the Programme accreditation will be taken up at a later stage’ No time frame was indicated.

QA methods and procedures

According to the HAAODLI, there are 6 stages in the assessment process as follows:

Submission of Application and Letter of Intent: This is initiated by the institution and must include Part 1 of Self Appraisal Portfolio (SAP)
Readiness Assessment: The NAAB will screen the submitted SAP to ascertain the institution’s ‘readiness’
Preparation and Submission of full SAP: On being assessed as ‘Ready’, the institution prepares and submit the full SAP (Parts I, II and III) including all the supporting documents.
Peer team Visit: The Peer team of 5-6 suitably qualified professionals assembled by the NAAB will study the submitted documents and conduct a site visit (3-4) days to audit the institution
Assessment Outcome: On completion of the Peer Team visit an audit report highlighting its findings is prepared and submitted to the NAAB of DEC
Grading: DEC after a review of the Peer Team Report decides on the grant of accreditation and the validity period as well as the declaration of the institutional grade.

QA criteria

As explained in the HAAODLI there are 6 main criteria used for assessing the institution for Institution Accreditation. They are as follows:

Infrastructure and Human Resource Capabilities
Academic Programmes and Learning Resources
Learner Support Services
Research and Consultancy
Governance
Innovative practices

For detailed criteria, see http://www.ignou.ac.in/ignou/aboutignou/icc/dec

Indonesia
Tian Belawati (Universitas Terbuka, Indonesia)
Aminudin Zuhairi (Universitas Terbuka, Indonesia)

Overview

There are currently 85 state/public universities (including Universitas Terbuka or UT), 26 state/public polytechnics (engineering, commerce and agriculture) and 2,987 private higher education institutions (including academies, polytechnics and teacher training institutions), serving over 240 million people. Besides, there are also Islamic higher education institutions (both private and state/public) which are under the control of the Ministry of Religious Affairs. Several other government Ministries also operate their own higher education institutions to meet the specific needs of qualified human resources for their own Ministries.

All (except for those under the Ministry of Religious Affairs and other government Ministries) higher education institutions in Indonesia operate with government’s permit issued by the Ministry of National Education. In order for any higher education institution to maintain the permit of any educational program offering, every semester they have to submit an online report to the Directorate General of Higher Education, Ministry of National Education (http://evaluasi.dikti.go.id/). This online and self-evaluation report, known as Evaluasi Program Studi melalui Evaluasi Diri or EPSBED) is intended to assure that all minimum required standards for offering university level academic programs are safe-guarded. Submission of the EPSBED is compulsory and failure of submission may result in the terminaton of government permit to admit students. Therefore, EPSBED is the tool used by the government to assure the quality of higher education in Indonesia.

Another strategy for quality assurance is conducted through accreditation by the National Accreditation Board of Higher Education (Badan Akreditasi Nasional Perguruan Tinggi or BAN-PT), which is an independent body established by the Ministry of National Education (then was the Department of Education) in 1994. BAN-PT is responsible for accreditation of higher education institutions (http://ban-pt.depdiknas.go.id/), including Universitas Terbuka (UT).
Even though BAN-PT accreditation focuses at Study Program level, the assessment also takes into account the QA policy and practice at the institutional level. Different levels of programs use different sets of accreditation instruments. Since 2009, BAN-PT has introduced the most recent updated instruments. Then, since 2010, BAN PT has also applied the most recently updated version of accreditation instrument for Sarjana Program through open and distance education. The list of instruments is as the following.

Accreditation Instrument for Sarjana (Bachelor) Program
Accreditation Instrument for Magister (Master) Program
Accreditation Instrument for Doktor (Doctoral) Program
Accreditation Instrument for Diploma Program
Accreditation Instrument for Sarjana (Bachelor) Program through open and distance education.

UT, established in 1984, is the only higher education institution that is entirely employing open and distance education system. . UT has a total student body of more than 650,000, which consists of fresh senior high school graduates as well as working adults. UT has been accredited by BAN-PT and other international organizations such as the International Council for Open and Distance Education (ICDE) and the International Organisation for Standardisation/ISO for ISO 9001:2000/2008.

UT QA system has been developed based on the draft QA framework developed by the Asian Association of Open Universities (AAOU), which was modified and contextualized to suit the specific needs of UT for the Indonesian context. An actual step to put QA approach into action was through the establishment of a Quality Assurance System Committee in 2001 to work thoroughly on the design, development, implementation, and evaluation of QA system. The Committee worked with people from various units across the university to develop QA policy and procedures. A document called “Quality Assurance System for Universitas Terbuka” was produced in 2002 after numerous consultations and meetings. When ideas about QA have been more widely accepted, a Quality Assurance Centre was founded in 2003 to plan, organize, coordinate and facilitate the quality assurance mechanism and continuous improvement effort of the institution’s distance education services.

The UT QA framework comprises nine components:

policy and planning,
human resource provision and development,
management and administration,
learners,
program design and development,
course design and development,
learner support,
learner assessment, and
media for learning.

Each statement is further delineated into indicators and methods of achievement, such as what sort of policies, systems, procedures, manuals, methods, work instructions, and records are needed to improve quality. This document serves as the foundation for implementing QA system, supplemented with an instrument for self-evaluation and priority setting, and a list of units within UT responsible for assuring the quality of their respective work activities. QA implementation required the writing of procedures, work instructions, and records of work activities. Working Committees were assigned to write these manuals, involving members consisting of managers and staff from different units related. QA system documents work activities, and ensures that systems and procedures are implemented accordingly. The writing of QA procedures required a great deal of time for discussion and consultation to ensure that clear and precise systems, procedures and work instructions could be understood by staff. The writing of procedures involved tryout and continuous revision for improvement.

UT has decided to invite both BAN-PT and ICDE to perform the quality assessment. This is because both agencies focus on slightly differenct emphasis on their assessment processes.

The BAN-PT accreditation system is designed to be at Study Program level, even though the assessment is comprehensive involving both the institutional and faculty levels. The accreditation status is valid for three years.The area of assessment include inputs, process and outputs of the whole instructional system.
The UT external quality assessment has also involved Quality Review by the International Council for Open and Distance Education (ICDE). The purpose of the ICDE Quality Review is to promote public confidence that the quality of standards and provision of services to students in Open and Distance Learning are being assured. The quality review does not involve a review of academic standards of courses or qualifications. It assumes that institutions will be operating within the overall national and/or state legislation and guidelines which are applicable to them at any given time. Thus the aims of an ICDE review are to meet the public interest in knowing that the institution is:
providing services to students of an acceptable quality and an appropriate standard in the context of national and regional standards;
exercising its legal powers in a proper manner.

QA methods and procedures

BAN-PT accreditation begins with filling out instruments (known as Borang) developed by BAN-PT.

Based on the completed and submitted Borang, appointed assessor teams (usually consists of two people for every Study Programme) will conduct the desk-evaluation using standardized evaluation schemes.
A 2-3 days field visit (to both the Head Quarter and all 37 Regional Offices) intended to verify the submitted data and information would follow the desk-evaluation process. The visit will usually involves meetings and interviews with management of the University, the Faculty, and Study Programme as well as with tutors, exam supervisors, students and partners who are involved in UT’s educational processes. At the end of the visit, the assessor team will make a preliminary visit report to the Rector.
The final result of the accreditation process is announced through BAN-PT website, which is open for public.

Although it is hihgly encouraged, accrediation through BAN-PT has not been made compulsory by the government, and thus it does not correlate to government funding. Nevertheless, accreditation status significantly influences public trust in the university and therefore universities are mostly voluntarily requesting BAN-PT to accredit them.

Similar to BAN-PT, ICDE quality review also begins with self-evaluation report based on several variables related to aims and objectives; curriculum design including materials distribution; teaching, learning and assessment; learning support and guidance; learning resources; quality assurance and enhancement; research; and public communication/relation. The submitted self-evaluation report was assessed by a Review Team consists of three people, who then also conduct a one-week field visit. The visit includes observations as well as discussion and interview with staff, students, alumnies, partners including course authors, tutors and exam supervisors. The preliminary result of visit is also presented at the end-meeting of the visit. The final result of the review process is presented in a qualitative report and a Quality Certificate for the university. It is important to mention that the final report also contain a set of recommendations for further institutional improvement.

QA criteria

BAN-PT applied 7 criteria/standards since 2009/2010. These criteria are basically the same as those applied to face-to-face higher education institutions. However, some indicators have been adjusted to be more appropriate and suitable to the open and distance education system. The following is an excerpt from BAN PT “Accreditation of Open and Distance Education Study Program Book 1 Academic Paper” (2010).

Vision, Mission, Objective, Target, and Achievement Strategy:

This standard is reference to quality excellence in the management and strategy of open and and distance education study program to achieve its future vision. The strategy and effort in materialising vision, implementation of missions, and achievement of goals is to be understood and supported with commitment and involves participation of stakeholders. The formulation of vision, missions, goals and adjectives is to be easily understood and logically presented in terms of sequence and stages of their achievement.

Governance, Leadership, Management System, and Quality Assurance:

This standard is reference to quality excellence in governance, leadership, management system, and quality assurance system of open and distance education study program as an integrated unity. This is to be seen as key success for an open and distance education study program to achieve its vision, implement mission, and achieve desired goals.

Students and Graduates:

This standard is reference to quality excellence for students and graduates. Prospective students in open and distance education must have senior high school level certificate. There is no selection in terms of entry, age limit, year of high school graduation, time flexibility for registration, and frequency limit in taking semester examination, and duration of study. In this standard, an open and distance education study program must have focus and commitment to instructional process and activity to develop competencies of the students to become quality graduates. Students are provided with quality services, including academic and administrative services, monitoring and evaluation of student progress needs analysis of students and stakeholders so that quality competent graduates can be produced in accordance with the needs of stakeholders.

Human Resources:

This standard is a reference to quality excellence of human resources, and how an open and distance education study program recruit and develop quality human resources to achieve vision, missions and defined goals. Human resources in an open and distance education program includes lecturers, tutors, as well as other educational staff, such as librarian, technical staff, and managers.

Curriculum, Instruction, and Academic Atmosphere:

This standard is reference to quality excellence in the instructional system of open and distance education study program. The curriculum is viewed as all instructional activities of students as reference for an open and distance education study program in planning, implementing, monitoring, and evaluating teaching and learning activities in achieving the goals of the study program. The curriculum must be constructed based on in-depth study of the nature of the knowledge, the needs of the stakeholders, and assurance for the achievement of competencies of the graduates.

Funding, Infrastructures and Facilities, and Information System:

This standard is reference for quality excellence to provide supporting resources in the implementation of quality academic process, including the acquisition and management of funding, infrastructures and facilities, and information system needed to materialise vision, implement mission, and achieve the goals of the open and distance study program.

Research, Community Services, and Partnership:

This standard is reference to quality excellence in research, community services, and cooperation in connection with the quality development of the open and distance education study program.

The ICDE quality review is more focused on the institutional level, and it has to be updated every five years. The components of ICDE quality review are the followings:

Aims and Objectives

Curriculum Design including Materials Distribution

Teaching, Learning and Assessment

Learning Support and Guidance

Learning Resources

Quality Assurance and Enhancement

Research

Communicating with the Community

In addition, ICDE quality review also puts a significant importance on:

Institutional policy to openness and flexibility
All aspects of student recruitment including marketing and general publicity – including advice offered through staff, printed materials and all electronic means;
All aspects of admission – including fee payments and the contractual arrangements between the individual and the institution;
Learner support – including requirements on the student where provision is ICT based;
The integration and maintenance of systems and services – including record keeping and links with other students;
A commitment to educational values, good customer relationship and business practices.

In general, the main concern of ICDE is that the university implements the system which potrays the name that it carries: open university. Therefore, the assessment is very much interested to see whether the institution values and maintains the system flexiblity, openess, as well as reach-out capability, and whether the institution controls the quality throughout the whole system implementation. At the heart of the assessment is that whether or not students are provided with the learning supports that they need.

UT has also gone through the ISO 9001:2000/2008 quality management audits for its core business processes, such as the development and distribution of learning materials and examination materials, distance learning services, and academic administration services. The ISO quality certification lasts up to three years prior to renewal, subject to meeting satisfactory requirements of surveillance audit every six months.

Japan
Aya Fukuda (International Christian University, Japan)

Overview

Serving a population of about 128 million, Japan’s higher education system comprises 778 universities including the Open University of Japan (OUJ), 76.7% of which are private, and 395 two-year colleges, 93% of which are private as of 2010.

OUJ first offered bachelors’ programmes in the greater Tokyo area through terrestrial TV and radio and correspondence in 1985. In 1998 it went nationwide, using satellite digital broadcast and a network of study centres and today it serves almost 100,000 students. Since 2001, OUJ has also provided graduate programmes but e-learning is not yet mainstreamed into OUJ’s repertoire.

Forty-two Japanese universities are classified as ‘correspondence institutions’. These correspondence institutions may offer entire distance education courses without any face-to-face component. And the ‘conventional universities’ can now offer up to 60 credits of synchronous and/or asynchronous distance study. As of 2010, there are a few cyber universities/graduate schools (i.e., a company-owned Cyber University which provides bachelor’s degree program in information technology and World Heritage only through the Internet without any face to face sessions, another company-owned Business Breakthrough University for bachelor’s degree in business administration and the Kenichi Omae Graduate School of Business, and the Graduate School of Instructional Systems at Kumamoto University) and several online programs within conventional universities.

The establishment of a new higher education institution including distance and online education institutions should be approved by the Council for Establishment of University and School Incorporation of the Ministry of Education, Culture, Sports, Science and Technology (MEXT) based on the ministerial ordinance of University Establishment Standards.

Once established, a higher education institution is subject to be periodically reviewed and accredited every seven years by one of three QA agencies approved by the MEXT. This accreditation system is based upon School and Education Law No. 109 enacted in April 2004.
Three QA agencies responsible for evaluating universities in Japan are the National Institute of Academic Degrees and University Evaluation (NIAD-UE) (http://www.niad.ac.jp/english/index.html), Japanese University Accreditation Association (JUAA) (http://www.juaa.or.jp/en/index.html), and Japanese Institute for Higher Education Evaluation (JIHEE) (http://www.jihee.or.jp/english.html). NIAD-UE evaluates mainly public institutions including universities, junior colleges, colleges of technology and professional graduate schools for Law. NIAD puts more emphasis on education contents, means to deliver education or student support than facilities/equipments or administrative operation. JUAA is in charge of mainly private institutions including universities, junior colleges, professional graduate schools for Law and Business. JUAA was originally the association of the universities aiming at improving quality of higher education and contributing to international cooperation of higher education. JUAA puts emphasis on education contents and means to deliver education as well as facilities/equipments or administrative operation. JIHEE is also mainly for private institutions and in charge of evaluation for universities, junior colleges. These agencies apply the same QA procedures and criteria in evaluating both conventional and DE institutions (OUJ and forty-two correspondence institutions). The focus of JIHEE is similar to NIAD. No specific QA or accreditation criteria for the DE programs have been developed.

QA methods and procedures

Periodic review (academic audit) process generally follows three steps: 1) self-assessment by each institution, 2) field audit by an evaluation organization and 3) issuing the evaluation result by an evaluation organization. Procedures are slightly different among three accreditation agencies.

In the NIAD-UE’s case, the process begins with briefing by NIAD-UE before each institution’s application. Then NIAD-UE provides training to the institution’s internal assessors before the institution’s self-evaluation takes place. There are eight steps:

Briefing by NIAD-UE in June to July,
Application sent by the targeted institution in the end of September,
Training program for internal assessors by NIAD-UE in November to December,
Submission of self-assessment report by the targeted institution in the end of June,
Document analysis and site visit by NIAD-UE in July to January,
Notification of draft report by NIAD-UE in the end of January,
Statement of objections by the targeted institution in February,
Finalization and publication of evaluation results by NIAD-UE in March.

In case of JUAA, there is no briefing or training session provided to the institution. In case of JIHEE, a briefing session is offered to the institution.

QA criteria

Key criteria for initial establishment of a higher education institution include:

Enough and needed classes and systematic curriculum to achieve expected educational purposes
Secure appropriate full-time faculty members and enough space for campus
Allocate competent faculty members properly to each class
Prepare funding for education and research, research rooms for faculty and a library
Estimate proper budget based on the scale of departments
Develop proper administrative and management structure
Estimate proper revenue and expenditure for its opening first year
Set debt or debt rate under the standard

NIAD-UE applies the same criteria for evaluation of conventional and DE institutions. In it’s document, it is mentioned that ‘where distance learning programs are delivered, modes of teaching (i.e. document-based lessons, broadcast classes and other media-based classes) are well selected and appropriate guidance is provided’. Key areas of university evaluation include:

Purpose of the university
Education and research structure
Academic staff and support staff
Student admission
Academic programs (undergraduate, graduate, professional degree programs)
Effectiveness of institutional performance
Student support
Facilities
Internal quality assurance system
Finance
Management

JUAA also applies the same criteria in reviewing both conventional and DE institutions and does not have separate considerations for DE programs. But evaluators are asked to consider such questions as:

Does an institution provide orientation or introductory training to students?
Does an institution provide ICT training and support for students?
Does an institution offer guidance to improve writing skills for students?
Does an institution prepare helpdesk service to support students utilizing PC?
Does an institution implement means to prevent students from dropping out?
Does an institution implement ways to enhance students-students/students-teachers interaction?
Does the institution utilize ICT effectively for classes and students interaction?
Does the institution have the special administrative unit only dealing with correspondent education?
Does the institution provide training opportunities to its staff?

JUAA’s general evaluation criteria cover:

Mission and goals
Educational and research structure
Educational program and instruction
Student admissions
Student services
Research conditions
Social contribution
Faculty
Administration staff
Facilities and equipment
Library and electronic media
Administration
Financial affairs
Self-study
Accountability

Korea
Insung Jung (International Christian University, Japan)

Overview

As of 2009, the Republic of Korea, with a population of 47.5 million, has 212 universities including the Korea National Open University (KNOU), 78% of which are private, 177 two-year colleges, 90% of which are private, and 19 cyber universities, all of which are private.

The establishment of a conventional university is required by the Higher Education Law to be approved by the Ministry of Education, Science and Technology (MEST) in Korea. Creating a new degree-awarding program within a university after being approved to be established must also be approved anew. By prescribing standards and quality criteria in the process of initial establishment of a university, the MEST is seeking the minimum level of the quality.

Once a university is established, it is subject to be accredited every 5 years and its selected programs to be evaluated annually by the Korean Council for University Education (KCUE) (http://english.kcue.or.kr/). KCUE, since its foundation in 1982, has conducted university accreditation and program evaluations using pre-described standards and procedures. It also provides training to universities’ faculty and staff to improve overall education quality and offers the Academic Records Verification Service (ARVS) which verifies information related to post-secondary academic qualifications and institutions on behalf of organizations wishing to confirm the credentials of prospective or existing staff and students. The results of KCUE’s institutional accreditation and program evaluation are used by the MEST to determine administrative and financial support to each university while each university uses the results to improve its systems.

In Korea, ‘cyber institutions’ have been more strategically targeted than in most countries and virtual learning might be seen as entering the mass adoption stage. In 2001, the MEST legislated for the creation of cyber universities, inter-institutional and international collaboration, and private sector involvement with the aim of achieving synergy in expertise and provision and spreading the costs of expanded postsecondary education. Until 2008, cyber colleges and universities were regulated under the Lifelong Education Law. From 2009, these cyber colleges and universities could apply for a change of status to be operated under the Higher Education Act, which places them on an even more competitive footing with the conventional universities and allow them to establish graduate schools as well as offer bachelor and associate bachelor degree programs. As of 2010, 12 cyber colleges and universities are now operated by the Higher Education Act and 6 remaining cyber colleges and universities are under the Lifelong Education Law. The establishment of a distance teaching university including a cyber university is now required by the Higher Education Act to be approved by the MEST.

In October 2007, the Higher Education Act was revised to introduce a new evaluation system for Korea’ higher education institutions (including KNOU and cyber universities). The new system focuses on encouraging each institution’s voluntary QA activities and at the same time strengthening the institution’s public accountability. In 2008, related ordinances were developed to provide detailed procedures for the university accreditation and evaluation. From 2008, by the Higher Education Act, all higher education institutions are required to publish and update university information on selected areas at their homepage and the Academy Information site ( http://www.academyinfo.go.kr/mainAction.do?process=load; Korean only) managed by the Academy Information Center at the Korean Educational Development Institute (KEDI). In 2009, information on student enrollments, the number of full-time faculty, graduates’ employment rate, new students’ admission rate, drop-out rate, and full-time faculty’s publication record (in case of 2-year colleges, returns on College-Industry collaboration) of the higher education institutions was open to the public.

Beginning 2009, all 4-year universities regardless the delivery mode must conduct a self-evaluation at least once every two years. Areas for self-evaluation can be decided by each institution but should include areas to be uploaded to the Academy Information system. The full self-evaluation report and a 10 page summary should be published at the university homepage and the Academy Information site. The University Self-Evaluation Monitoring System (http://selfeval.kcue.or.kr/) is created and managed by KCUE to collaboratively solve problems faced by the universities during self-evaluation, elaborate evaluation criteria and methods, link the evaluation results to quality enhancement and improve the self-evaluation system itself.

Once the self-evaluation is completed, the universities must submit the application to an accreditation agency recognized by the government to be accredited and/or audited. As of 2010, KUCE is the only agency with the government recognition that can accredit 4-year universities. The Korean Council for University College Education (KCCE) is recognized as an accreditation agency for 2-year colleges. From 2011, universities (including KNOU, but not Cyber Universities) and colleges which completed the self-evaluation begin to apply for formal accreditation either to KUCE or KCCE.

KNOU, launched in 1972 in response to a spike in school leavers and adults seeking university entry, offers TV, radio, Web-based and multimedia lectures across 22 programmes to its over 270,000 students at the undergraduate level. Its graduate school programmes are operated totally online. KNOU’s reputation and funding had depended mainly upon internal QA reports to the MEST until the new evaluation system was first applied in 2009. With the introduction of the new evaluation system explained above, KNOU completed its first self-evaluation in 2009 in the areas of:

Management (vision, strategies, leadership, finance, operations, and social contribution),
Education (educational planning, curriculum, and learning outcomes),
Students (new and transfer students, current students and graduates),
Faculty and Staff (teaching, research and publications, staff),
Educational environment and supports (environments for teaching and research, learning supports, contents development, facilities, lab/practicum facilities, computer systems).

Also as core areas for self-evaluation, it assessed areas of:

Accomplishment of university vision
Educational quality
Student satisfaction
Attracting new and transfer students and enrollment rate
Competency development of faculty and staff
Improvement of educational environment and student supports.

QA methods and procedures

The Higher Education Act and related ordinances state procedures and detailed criteria for establishing a cyber university and guidance for operating the university. In addition, they also specify procedures and criteria for establishing online degree programs within conventional universities. Procedures for initial accreditation are specified as follows:

An institution which seeks the accreditation and the approval of the establishment of a cyber university should first submit a detailed development plan to be undergone scrutiny by a Cyber University Establishment Evaluation Committee commissioned by the MEST.
Once the plan has been reviewed, the second round evaluation consisting of extensive reviews of Learning Management System, online courses, and at least two years of experiences in e-learning provision, site visits to assure the optimal preparation of hardware and network facilities, and interviews with management and academic groups of the institution is conducted.
The decision to approve or disapprove the establishment of a cyber university is made based on the results of these evaluations.

Once established, all cyber universities have to follow procedures for annual academic audit.

A cyber university should first conduct self-evaluation following guidelines provided in “Cyber University Evaluation Handbook” and submit a self-evaluation report to be undergone scrutiny by a Cyber University Evaluation Committee commissioned by the MEST and managed by KERIS.
Then the Cyber University Evaluation Committee will review the performance of the cyber university based on the self-evaluation report and give a grade on a five-level scale (A, B, C, D and E) on each evaluation item.
Afterwards, the Cyber University Evaluation Committee will visit the site and conduct interviews with the university staff and students to confirm its evaluation of the report.
Then the Cyber University Evaluation Committee will give a final grade on a four-level scale (excellent, good, average and improvement required) using all the findings from the review of the self-evaluation report, site visit and interviews.
The final report will be submitted to the MEST and published online for public disclosure.
This annual evaluation is to support and encourage cyber universities to make an effort for self-improvement and gain public trust.

While KNOU must conduct a self-evaluation at least once every two years and make its full report open to the public beginning 2009 and after 2011, it must apply for accreditation to KUCE, in case of the Cyber Universities, as of 2010, no agencies including KUCE and KCCE have been recognized by the government as accreditation agencies for cyber universities. Thus it is likely that KERIS will continue to evaluating and monitoring the quality of those 19 Cyber Universities until a new QA/accreditation agency for cyber education is formally recognized.

QA criteria

Criteria for initial accreditation for a cyber university have been elaborated and updated every year. In early years, the evaluation focused on hardware, network and course development systems as shown in Appendix 1. Six QA areas included: hardware and network establishment, course development system, quality assurance mechanism, student support services, vision and missions, and administration.

Since 2008, revised QA criteria have been applied to evaluate the establishment of a new cyber university. 5 key QA areas cover: essential conditions, facilities and equipment for education and research, management of academic affairs, human resources and organizational structure, and quality management. Under each area, detailed evaluation criteria are specified as seen in Appendix 2.

Criteria for establishing online graduate schools within conventional universities are found to be similar to those for cyber universities except tutor and student ratio of 1:20 is recommended to maintain highly interactive quality of graduate studies. Once established, a cyber university is subject to annual external reviews by the Korea Education and Research Information Service (KERIS) (http://english.keris.or.kr/es_main/index.jsp) and the findings of these reviews are used to shape MEST’s cyber university policies and guidelines. KERIS publishes these findings on its website.

The criteria to assess the quality of the cyber universities have been refined every year and explained in detail in the “Cyber University Evaluation Handbook” published by KERIS. As of 2008, there are 95 specific criteria in 16 dimensions across six areas. Appendix 3 shows 95 specific criteria in 16 dimensions. Six areas are: educational planning, instruction, human resources, physical resources, management and administration, and outcomes. Total points are 500. Maximum scores assigned to each area vary: 45 points (9%) for educational planning; 155 (31%) for instruction; 100 (20%) for human resources; 75 (15%) for physical resources; 80 (16%) for management & administration; and 45 (9%) for outcomes.

As of 2010, KUCE does not have specific QA guidelines or criteria for DE/e-learning evaluation.

Malaysia
Tat Meng Wong (Wawasan Open University, Malaysia)

Overview

There are 20 public universities, 27 polytechnics, over 60 community colleges and 48 private universities/ university colleges including branches of foreign universities. There are three dedicated ODL/E-Learning universities (Open University Malaysia, Wawasan Open University, Asian e-University).

Prior to the1980s, all higher education providers were set up, funded and controlled by the Government. The qualifications they award were recognized by the government and the quality of the programmes was ‘assumed’ on the basis that they were ‘Government institutions’.

The realization that the Government will be increasingly hard pressed to fund and support the explosive demand for higher education consequent to the shift towards a knowledge economy provided the impetus for the Malaysian Government to revise its education provision strategy to one where the private sector was invited to share in the provision of higher education. This was done by allowing privately funded Higher Education Providers (HEP) to be set up. Due to a pent-up demand caused by a short supply of places in public institutions of higher learning and the application of a quota system as well as the business potential of the educational sector, many private HEPs were established during the late 1980’s and early 1990’s This brought along with it a number of institutions that were offering educational experiences and credentials of questionable standards. The need for a quality assurance regulatory body soon became critical. In 1996, the National Accreditation Board (Lembaga Akreditasi Negara or ‘LAN’) was established to oversee quality assurance and accreditation of programmes offered by private HEPs. Until 2007, LAN was involved mainly in the accreditation of Academic Programmes offered by private HEPs. Programmes that met the prescribed educational quality standards were usually accredited for a period of 5 years before the next accreditation exercise was due. All programmes offered by private HEPs irrespective of the mode of delivery (F2F, ODL or E-Learning) fell under the jurisdiction of LAN.

Besides LAN, two units of the Ministry of Higher Education oversaw QA in public higher education providers: the Quality Assurance Division (QAD) for universities and separately the QAD for polytechnics and community colleges. Both were set up in 2002 and in the main they operate through periodic institution audits.

The Malaysian Qualifications Agency (MQA) (http://www.mqa.gov.my), consolidating and replacing these three QA bodies that existed previously was established on 1st November 2007. It is charged with the responsibility of monitoring and overseeing the quality assurance practices of all Higher Education Providers (HEP) in the country as well as the accreditation of their programmes. All ODL/E-Learning institutions come under the purview of the MQA. With the establishment of the MQA, a number of major initiatives were introduced in addition to their programme accreditation role. These include:

the introduction of the Malaysian Qualification Framework and its related Malaysian Qualifications Registry which list all qualifications offered by Malaysian Registered HEPs (public and private) that are accredited by the MQA

the introduction of a scheme whereby HEPs progress through three stages as they evolve towards achieving self-accreditating status ie:

Stage 1: newly set up institutions and all new programmes must attain provisionally accredited status before students are recruited.
Stage 2: more mature institutions which have programmes that are fully accredited
Stage 3: achievement of self accreditation status

the conduct of Academic Performance Audits for ALL HEPs in the country. This massive exercise was undertaken in 2009-2010 by the MQA under instructions from the Minister of Higher Education as a step towards improving the quality assurance systems and practice of HEPs in Malaysia. Prior to this no HEP in Malaysia had undertaken a comprehensive Institutional Audit of their academic processes. As a sub set of this exercise, 8 established local and branch campuses of foreign institutions were also evaluated on their ‘readiness’ to be conferred Self Accreditation status. ( The award of ‘self accreditation’ was duly announced by the Minister of Higher Education during Q3 of 2010)

ODL/E-Learning institutions: All major ODL institutions in Malaysia namely, Open University Malaysia (OUM : http:// www.oum.edu.my), Wawasan Open University (WOU : http://www.wou.edu.my) and Asian e University (AeU:http://www.aeu.edu.my) were also required to undertake this exercise. Meanwhile, all programmes offered by WOU and most programmes offered by OUM and AeU are also accredited by the MQA (both OUM and AeU offer a number of ‘executive’ programmes that are not MQA accredited. Details can be seen at their respective websites.

In the case of programme accreditation, there was also a shift towards the use of an outcome based approachbesides increasing the evaluation of criteria and standards from 5 areas to 9 areas and a greater emphasis on self review accompanied by the development of a continuous monitoring and improvement culture.

QA methods and procedures

Programme accreditation: In Malaysia, all new programmes to be offered are submitted to the MQA for provisional accreditation evaluation. The documents must be submitted using a prescribed proforma. These are evaluated by a specially appointed MQA Panel that consists of Academic peers from other HEPs. This is basically a desk exercise and involved no site visit. The preliminary report of the Panel is presented via the MQA to the HEP for comments and response. Specific conditions that are laid down must also be addressed. The HEP’s response is incorporated into the final report of the Panel and subject to an overall satisfactory report by the Panel, a recommendation is made to the MQA Board for approval (or otherwise) and the award of ‘Provisional Accreditation’ status. On receipt of this, the HEP can apply to the MOHE for approval to recruit students into the programme.

All programmes that are provisionally accredited must seek full accreditation in the semester prior to the appearance of the 1st batch of graduates. The evaluation criteria are similar but the documentation required is much more comprehensive since it now involves evaluating the performance of the programme itself. Evaluation normally includes a site visit during which teaching support facilities are inspected, documents verified and interviews are conducted with all major stake holders. A significant policy change by the MQA (compared to LAN) is the fact that once full accreditation is awarded to a programme no further periodic accreditation of the programme is required (cf LAN) so long as the QA of the HEP is found to be of good standing based on the Institutional Audit report.

Institution Audit: All Malaysian HEPs are required to undertake periodic audit of their academic performance in addition to the need for their programmes to achieve full accreditation status. After a few cycles of programme accreditation and institutional audit, a HEP that is deemed to have demonstrated the appropriate level of QA standards may be invited by the Minister to undertake a special institutional audit for purpose of being conferred ‘Self Accrediting Institution Certification (SAIC). Institutions that are awarded this SAIC status will no longer need to have their programmes accredited by the MQA. They will however still need to undertake periodic institutional reviews.

QA criteria

The MQA has developed a code of practice on criteria and standards for higher education in Malaysia. This code of practice is benchmarked against international good practices and is nationally accepted by stakeholders through numerous consultations prior to implementation. The practices for quality assurance adopted by the MQA is based on clearly defined, transparent and fair criteria and standards that serve as references for evaluations and reports of programmes offered by higher education providers.

The MQA uses the same nine areas for both evaluating the accreditation of programmes as well as the auditing of academic performance of institutions. These are as follows:

Vision, mission, goals and learning outcomes;
Curriculum design and delivery;
Assessment of students;
Student selection and support services;
Academic staff;
Educational resources;
Programme monitoring and review;
Leadership, governance and administration; and
Total continual quality improvement.

Details explaining the basis and spelling out the criteria and standards set for each of the nine areas as well the type of information required are clearly explained in two comprehensive documents published by MQA:

The Code of Practice for Programme Accreditation (COPPA) (1st Edition, 2008) (http://www.mqa.gov.my/en/garispanduan_coppa.cfm)
The Code of Practice for Institutional Audit (COPIA) (2nd Edition, 2009) (http://www.mqa.gov.my/en/garispanduan_copia.cfm)

Mongolia
S. Baigaltugs (Mongolia University of Science and Technology, Mongolia)

Overview

Over the past two years, Mongolia has reduced its number of higher education institutions from 214 (2008/2009) to 114 (2009/2010), among which 86 institutions (10 universities, 44 technical and vocational schools and 32 colleges) are public and the rest are private.

DE programmes are offered in a few institutions including four public universities (Mongolian University of Science and Technology (MUST), National University of Mongolia, University of Health Science, and Educational University ) and four private institutions. Among these universities and institutions, MUST has most actively developed and implemented e-learning programmes using the University Management Information System (UNIMIS). Between 2007 and 2010, MUST has offered 16 master degree online programmes and integrated ICT in its 1,560 undergraduate courses. On January 2010, e-Open University was established within MUST, as the first formal DE programme. MUST hopes to develop this e-Open University as an independent Cyber University in the future. (see http://202.5.195.17/emust/web)

The Mongolian National Council for Education Accreditation (MNCEA) (http://www.accmon.mn/english) was established as a government MNCEA initiative in 1998 to address public concern over quality of higher education, and to evaluate and accredit universities and colleges and in 2002 it began to accredit vocational and technical institutions. It is responsible for accreditation and QA of higher education institutions in Mongolia and for providing management consulting to Mongolian higher education insitutions.

The MNCEA provides accreditation at the institutional level but piloted program accreditation with selected programs in co-operation with professional associations such as the Consortium of Management Development Institutions and Professional Association of Engineering and Technology.

The pilot program accreditation project, began in 2002 , has been carried out in the fields of telecommunications engineering, dentistry, veterinary medicine, and accounting, history and social studies, pre-school teaching, and solo musicianship.

Since 1990, several DE projects and programs have been implemented in non-formal education settings with supports from international organizations. For example, a non formal DE project titled “Gobi Women”, was jointly implemented by the Government of Mongolia and UNESCO, funded mostly by DANIDA (Danish International Development Assistance) between 1992-1996. As the second phase of the Gobi Women project, another DE project called “Learning for Life” was implemented between 1997 – 2001 involving more than 37.0 thousand people. In the beginning of these projects, teaching/learning materials were a combination of print, radio as well as local learning groups and visiting tutors, and family-based education for the rural nomadic population, which were relevant to the learners' needs, knowledge and skills. Then TV, tele-video lessons and currently more ICT-based materials such as CDs and VCD materials were added. Another project named DREAM-IT supported by IDRC implemented the curriculum development of university Computing and ICT education, the blended technology education project (BTEP) using technology to improve education and online psychological service for health professionals in Mongolia.

The first distance education center was established in 1997 in Mongolia. The centre was extended as the National Centre for Non-formal and Distance Education (NCNFDE) in 2002 and now serves as the responsible agency for non-formal and distance education. The Centre oversees non-formal education policy development, non-formal education surveys, the provision of information and consultancies, materials development and human capacity building at the national level. NCNFDE offers various DE progammes and has served over 8 thousand people. Nowadays there are 12 Distance education centres in Mongolia. They offer web-based training courses teaching English, ICT education, gender education, mathematics, pedagogy, medical education and others. DE materials have been distributed through these distance education centers across the country and they are available free of charge.

QA methods and procedures

The institutional accreditation process may take 1.5 – 2 years to complete its three stops.

institutional self- study,
site visit or peer evaluation,
approval by the MNCEA through secret ballots

The process starts two years before the final decision is expected, and it is initiated by MNCEA’s Officers who serve as the MNCEA’s staff liaison to the HE institutions. The officer(s) meets with various constituencies (the president and senior administrators, faculty, students, trustees, and most importantly, the self-evaluation team) at the institution and explains the role of each in the self- study process. The self-evaluation team of the college or university is appointed at the very early stage of the accreditation process, because it has the actual responsibility of coordinating the self-study review process on campus and writing the final self-study report. It is usually led by a chair or co-chairs appointed by the president and chief academic officers and faculty leaders from various programs are often appointed as team members to represent a total campus community. The role of the self-evaluation team is to identify key issues in QA, establish working groups or subcommittees, compile data across the institution, prepare and analyze interim reports, and finally to assemble, edit, and prepare the final self-study report. The institution submits the self-study report approximately 2-3 weeks prior to the visit. The MNCEA’s staff forwards the report, relevant publications and other materials to the external evaluation team for review before the visit.

The site visit follows the review of the self-study report. The external evaluation team discusses QA issues found in the report and observation with the institution’s self-evaluation team members and conducts pre-arranged interviews with members of the community in various QA areas. Then the final report is prepared by the external team and sent to the institution for comment.

The external evaluation team’s report with recommendations and the institutional written response are submitted to the MNCEA. The MNCEA Board members discuss the case and make a final decision on accreditation by secret ballot.

QA criteria

Accreditation is a voluntary process which includes both institutional and program accreditation. The institutional evaluation covers such areas as:

Purpose and goals of the institution;
Building and material base;
Finance and economy;
Curriculum;
Faculty;
Students;
Library;
Management and organization and
Scientific research.

Philippines
Patricia Arinto (University of the Philippines Open University, the Philippines)

Overview

The higher education system of the Philippines consists of 2,180 colleges and universities, of which 607 are publicly funded and 1,710 are privately owned. The public higher education institutions (HEIs) include state universities and colleges (SUCs) funded by the national government, local universities and colleges (LUCs) funded by local government units, and other institutions operated or supervised by the government. Both public and private HEIs are under the supervision of the Commission on Higher Education (CHED) (http://202.57.63.198/chedwww/index.php).

SUCs have their own charters, and have the autonomy to develop their own curricula, institute programs, and award degrees. CHED ensures their compliance with the Higher Education Modernization Act (Republic Act 8292) by serving as chair of each SUC’s governing board. In addition, most SUCs undergo voluntary accreditation by the Accrediting Association of Chartered Colleges and Universities of the Philippines (AACCUP). Private HEIs are required by CHED to be certified by the Federation of Accrediting Agencies of the Philippines (FAAP), which includes the Philippine Accrediting Association of Schools, Colleges and Universities (PAASCU) and the Philippine Association of Colleges and Universities' Commission on Accreditation (PACUCOA).

Accreditation is done at both institutional and program levels. There are four levels of program accreditation. Level I status is granted after a formal survey by the accrediting agency and is effective for three years. Level II status is awarded to re-accredited programs and is effective for three to five years based on the appraisal of the accrediting network. Re-accredited Level II programs that meet additional criteria set by the accrediting network are given Level III status. Level IV status is awarded to re-accredited programs recognized to be of very high quality in the Philippines and as prestigious as similar programs in excellent foreign universities. These programs will have excellent outcomes in research and publications, teaching and learning, and community or extension service, as well as international linkages and a well developed internal quality assurance mechanisms.

Following a system of progressive deregulation, CHED grants autonomous status to HEIs with consistent Level III accreditation for its programs, outstanding performance of graduates in licensure examinations, and a reputation for high quality tertiary education provision. The University of the Philippines (UP), the national university, has autonomous status, as does some 40 private HEIs. Institutions on their way to being granted autonomous status are given deregulated status and as of 2003, there were 44 deregulated HEIs.

As an archipelago of 7,100 islands the Philippines would be an ideal place for distance education (DE) and e-learning. However, only a handful of HEIs — numbering 17 in 2005 — offer DE programs. Among the standalone DE providers are the UP Open University (UPOU), which is part of the University of the Philippines System, CAP College, the Asian Institute for Distance Education (AIDE), and the Southeast Asia Interdisciplinary Development Institute (SAIDI). The rest are conventional universities offering a few of its programs by DE mode. Most of the DE provision is at the graduate level, which would perhaps account for the low DE student enrolments nationwide. UPOU, the most comprehensive DE institution in the country with 24 degree programs, offers only two undergraduate programs and has a total enrolment of about 2,500 students per semester.

Quality methods and procedures

CHED’s policies for DE programs are laid out in CHED Memorandum Order No. 27 Series of 2005, or CMO 27. A DE program is defined as one where at least 25% of all courses are offered via distance mode. CMO 27 stipulates that only graduate-level programs with Level III accreditation can be offered by DE mode. Section 13 of CMO 27 specifically states that undergraduate degree programs should not be offered by DE mode “because undergraduate students need face-to-face interaction with mentors and peers as part the academic environment for optimal learning.” However, some DE institutions, including UPOU, offer one or two undergraduate Associate and Bachelor programs, and the degrees granted are recognized by CHED.

As a constituent unit of the UP System, UPOU has autonomous status and is not subject to normal monitoring and evaluation by CHED, and it is not obliged to undergo accreditation. UPOU’s programs comply with internal quality assurance policies and procedures governing all UP units. For example, all program proposals are closely reviewed by various UPOU and System-wide committees (composed of representatives of UP’s seven constituent universities), before finally being approved by UPOU’s University Council and then by the UP Board of Regents, which is chaired by the CHED Chairperson and whose members include representatives of the Philippine Senate and House of Representatives, sectoral representatives, and three members at large appointed by the President of the Philippines. UPOU was declared by CHED as the Center of Excellence in Open and Distance Learning in 2003, and it is a member of CHED’s Technical Panel on Distance Education.

All other DE providers must comply with the CHED QA System for DE, which requires them to get government authority to offer DE programs from CHED, and then undergo periodic monitoring and evaluation by the CHED Technical Panel on Distance Education. An application for government authority to offer a DE program is reviewed first by the CHED Regional Office (CHEDRO) in the area where the HEI is located, and then evaluated by the CHED Office of Programs and Standards (OPS), and then finally deliberated upon by the Commission en banc.

Quality criteria

CMO 27 stipulates adherence to the following principles for all courses offered by DE mode:

Learner-centeredness
Rigorous and sound instructional design
Transparency and peer review to guide informed choice
Public responsibility and accountability
Quality and continuous improvement

HEIs seeking government authority to offer DE programs and/or government recognition of its DE programs are assessed in terms of the following criteria:

Institutional qualification
Institutional management and commitment
Curriculum development and approval
Instructional materials development
Delivery mode/strategies
Student assessment
Student support services

Institutional qualification refers to whether the HEI has Level III accreditation for the program it intends to offer by DE mode, or is a recognized CHED Center of Excellence in that program.

Institutional management and commitment is measured in terms of the mission and policy statement of the HEI’s DE unit, the financial or budgetary allocation for DE operations, the organizational structure and procedures for managing the DE unit, the qualifications of the DE unit head or manager, and a plan for continuing self-evaluation for DE program improvement.

Curriculum development and approval is evidenced by the complete program curriculum and detailed syllabi defining appropriate learning outcomes and assessment methods for each course.

Instructional materials development refers to the availability of self-instructional learning packages that have been developed by a team of qualified subject matter specialists, instructional designers, and production design specialists. In addition, these learning packages must have undergone testing, and they must be compliant with copyright laws. Moreover, the HEI must have defined procedures or guidelines and policies for periodic review and updating of the learning packages.

Specific requirements for delivery mode/strategies include a core of qualified teaching faculty; policies and procedures for course delivery; mechanisms for enabling interaction between teachers and students and among students; a validated system of student assessment and evaluation; and resources for learning such as libraries and learning center facilities.

Student support services include providing students with complete and clear program information, including registration advice; study and technology skills training; access to grievance procedures; regular feedback on academic progress; and clear admission and retention policies and procedures.

Singapore
Tat Meng Wong (Wawasan Open University, Malaysia)

Overview
The Government of Singapore currently does not accredit programmes or higher education institutions. It confers powers to two public universities (National University of Singapore: NUS and Nanyang Technological University: NTU) to grant their own degrees. These universities are audited under the Ministry of Education’s (MOE) Quality Assurance Framework for Universities (QAFU). Also Singapore Management University (SMU), the first private university which has received funding from the government has been assessed by MOE as being of sufficient quality to be accorded university status and to award degrees in its own name. Besides SMU, there are 9 private higher education institutions in Singapore including the Singapore Institute of Management (SIM).

The Higher Education Quality Assurance Section of the MOE (http://www.moe.gov.sg/about/org-structure/hed/) established the QAFU in 2004, the Polytechnic Quality Assurance Framework in 2006, the Arts Quality Assurance Framework in 2008 and the ITE Quality Assurance Framework in 2009 to oversee quality assurance in various post secondary education institutions and conduct benchmarking of higher education systems in other countries.

Private educational institutions come under the regulatory purview of the Council for Private Education through the implementation of the Private Education Act 2009. The Council for Private Education (http://www.cpe.gov.sg/) spells out the regulatory frameworks under which private institutions are allowed to operate but conducts neither programme accreditation nor institutional audits.

The main DE provider in Singapore UniSIM uses a blended approach for delivering education where e-Learning is used to supplement face-to-face classes. Its ‘birth’ could be traced back to 1992 when MOE appointed the SIM to collaborate with the Open University of United Kingdom (OUUK) to offer the Open University Degree Programme (OUDP). In 2002 OUDP was granted accreditation status by the UKOU and renamed SIM Open University Centre (SIM-OUC). SIM-OUC was granted full university status in 2005 and renamed UniSIM. UniSIM comes under QAFU in terms of its institutional QA audit.

QA methods and procedures

The MOE currently audits universities in Singapore on a 4 year cycle. Basically, it involves the following:

Institution’s self-assessment: The audited university prepares a submission document stating its institutional goals in the 5 areas of evaluation (could be reduced for new institutions) and a list of self-selected performance indicators
On-site evaluation visit: This is followed by an external validation exercise conducted by an external review panel
Feedback and development: The evaluation review panel provides recommendations to the institution. MOE monitors the universities' follow up on the panel's recommendations.

QA criteria

The QAFU used by the Singapore MOE is based on 5 areas of evaluation. These are:

Governance & Leadership: Refers to the role of the Council or Board of Directors for setting broad strategic directions and guiding the institution in reviewing its organizational performance.
Management & Strategic Planning: Refers to the institution’s management, accountable for the Council or Board of Directors for formulating and implementing strategic plans and monitoring their outcomes
Teaching & Learning: Assesses the quality of the institution’s faculty, its curriculum, teaching processes and support systems. International benchmarking is encouraged.
Service: Refers to both internal and external services including contributions to the community
Research: International and regional research standing

Sri Lanka
S. Baigaltugs (Mongolia University of Science and Technology, Mongolia)

Overview

University education in Sri Lanka is offered mainly by 15 conventional public universities with an enrolment of 40,000 students and an Open University Sri Lanka (OUSL) with an enrolment of about 25,000 students. These universities are accredited by the Quality Assurance and Accreditation Council (QAAC) of the UGC/Ministry of Education (http://www.qaacouncil.lk/). QAAC was established in 2003 with the support of World Bank’s IQRUE (Improving the Quality and Relevance of Undergraduate Education) project. The mission of the QAAC is to ensure quality, continuous development and efficient performance of Sri Lankan higher education institutions, and to gain the confidence of the community in their graduates in accordance with internationally recognized evaluation mechanism.

OUSL is the only university in the country that provides DE programs and courses. However with the implementation of the Distance Education Modernization Project (DEMP) (http://www.depp.lk/demp.php) funded by the Asian Development Bank, several universities have developed DE programs and materials. The DEMP was initiated in 2003 with the goal of significantly increasing access to higher education throughout the country with quality, technology-enhanced DE. The objectives of DEMP are to build the capacity among the institutions and organizations of Sri Lanka; to design and deliver effective open and distance education programming; and, to identify and establish an enduring, sustainable system of distance education that will continue beyond the duration of the project. To achieve these objectives, DEMP operates three sub-projects: the OUSL Capacity-Enhancement Project (OUSL-CE); the Distance Education Partnership Programme (DEPP); and, the Public Private Partnership Programme (PPP). With the co-sponsoring of COL, the DEMP produced the ‘Quality Assurance Toolkit for Distance Higher Education Institutions and Programmes’ in 2009.

QA methods and procedures

At present, the QAAC conducts two types of reviews, namely Institutional Review(IR) and Subject Review(SR).

Purposes of these reviews are:

To install confidence in an institution/programme’s capacity to safeguard standards ,both internally and externally, through a transparent process
To achieve accountability through external review and public report of an institution/programme’s evidence of its own attentiveness to quality and standards, and of actions take to improve and be responsive to feedback.
To provide systematic, clear and accessible information on the standards and quality claimed by an institution/programme so as to inform the choices and decisions of potential students, employers, funding bodies and other users.
To promote improvement by identifying and sharing through peer review, good practice and encouraging innovation and active use of national and international standards and benchmarks

In evaluating institutions/programmes, the emphasis is placed on sharing views, ideas and expertise of both external reviewers and staff and students of an institution/programme. Both IR and SR adopt the following steps:

An institution/programme’s analytical self-evaluation
Peer review/Review visit to the institution/programme
The report and discussion of the report and other quality issues with academic and administrative staff, managers and students
Final decision and publication

Specific details of these steps can be found at The QA Handbook for higher education http://www.qaacouncil.lk/publications/QA_handbook.pdf.

QA criteria

Institutional Review focuses on the powers and responsibilities, which universities hold for quality and standards. It is concerned with how a university assures itself and the wider public, that the quality and standards it sets for itself are being achieved. It analyses and tests the effectiveness of an institution's processes for managing and assuring the quality of academic activities undertaken by the institution. And it also evaluates the extent to which internal QA schemes can be relied on to maintain the quality of provision over time. 8 QA criteria include:

University Goals and Corporate Planning
Financial Resources and Management
Research

Research goals and strategy
Funding
Staff support and opportunities
Research infrastructure
Monitoring and evaluation

Quality Management and Administration

Quality policies and strategies
Learning infrastructure
Administrative support for quality assurance

Quality Assurance

Regulation of awards and qualifications
Entry standards and policy (including pre-entry information)
Programme design and approval
Programme/subject monitoring and review
Assessment procedures
Recruitment, reward and development of teaching staff
Feedback and verification mechanisms

Learning Resources and Student Support
External Degree Programmes
University/Industry/Community/ Other Extension Activities

Subject Review evaluates the quality of education within specific subject or program. It is focused on the quality of the student learning experience and on student achievements. It is designed to evaluate the quality of both undergraduate and taught postgraduate programmes. 7 QA criteria include:

Curriculum design, content and review
Teaching, learning and assessment methods
Quality of students, including student progress and achievement

Suitably qualified students on entry
Satisfactory progression through the programme
Achievement that matches learning outcomes

The extent of student feedback, qualitative and quantitative
Postgraduate studies

Appropriate support and resources for postgraduates
Critical mass of permanent research-active academic staff
Availability of training in research methods and other areas

Peer observation
Skills development
Academic guidance and counseling

The ‘Quality Assurance Toolkit for Distance Higher Education Institutions and Programmes’ by DEMP lists 10 QA criteria for DE institutions as follow:

Vision, mission and planning
Management, leadership and organizational culture
The learners 4 Human resource and development
Programme design and development
Course design and development
Learner support
Learner assessment
Infrastructure and learning resources
Research consultancy and extension services

It also presents 6 QA criteria for DE programmes:

Institutional planning and Management
Programme design and development
Course design and development
Infrastructure and learning resources
Learner support and Progression
Learner assessment and Evaluation

Then the Tookkit specifies the performance indicators covering the most relevant domains of quality in distance higher education institutions across inputs, processes and outcomes. For more details of the performance indicators, visit http://www.col.org/SiteCollectionDocuments/HE_QA_Toolkit_web.pdf

Appendices

Appendix 1: Original evaluation criteria for new establishment of a cyber university, Korea.
Source: MEST (2003). Guidelines for virtual university operations. Ministry of Education, Science and Technology, Korea.

Areas/
Dimensions

Detailed evaluation criteria

Hardware and Network
1. Hardware

2. Software

3. Network

4. Physical spaces

* A virtual university can use hardware and network systems in a commercial IDC. But the specs specified in the criteria should be met.

1.1. Web server
CPU-700MHz X 2 above;
Memory-1GB above;
HDD-SCSI 36GB above
1.2. VOD server
CPU-700MHz X 2 above (512K cache per processor)
Memory-ECC SDRAM 2G above
HDD-SCSI 72GB above
1.3. DB server
CPU-700MHz X 2 above (512K cache per processor)
Memory-ECC SDRAM 1G above
HDD-SCSI 72GB above
1.4. Academic Administration server
same as Web server
1.5. Backup DB server
same as Web server
1.6. Firewall server
CPU-300MHz X 1 above
HDD-10GB above
Memory-256MB above
1.7. Mail and Community server
can be used with other servers, but encourage to set up independent server
1.8. Storage (disk array)
500GB above (7TB maximum)
1.9. Auxiliary memory
Magnetic tapes, and others

2.1. software for Web server
Web engine
2.2. software for VOD server
50 streams above
2.3. DB management system
20 concurrent users
include search engine
2.4. Learning Management System (LMS)
Modules for course delivery, academic management, authoring, and others
2.5. Firewall software
for security

Intranet

FDDI, Fast Ethernet or ATM
3.2. Internet
T3 above

4.1. Minimum space required
660m2
4.2. other spaces
laboratories if necessary

Course Development

Multimedia production systems

Course Preparation

LMS functions

4. LMS operation experience

Hardware

Video editing system
Audio editing system
Graphic editing system

Software

Web editor
Audio editing software
Video and graphic editing software (2D/3D and others)

2.1. Syllabus
Quality and quantity
2.2. Web contents development
Content quality
Instructional design strategies

3.1. Menu structure and functions
3.2. Authoring tools’ functions
3.3. Interactive functions
Assessments, Assignments, Discussions

4.1. Accumulative e-learning experience
2 years of institutional level of e-learning experience required

Quality Assurance (QA)

Internal quality assurance mechanism

Interaction

Faculty qualification

Detailed Course design/development process
Full time faculty and Assistant

At least one full time faculty and one assistant per programme

QA functions integrated
Effective feedback system

Student feedback
External expert feedback

2.1. tutor-student ratio
1: 50 advised

3.1. faculty qualification
same as in conventional universities

Student Support Services

Technical support

Digital library

Academic support

Administrative support

356/24/7 support system advised
Emergency support system

2.1. Independent digital library advised
2.2. Links to other digital resources

3.1. Q & A system
3.2. Tutor support system

4.1. One-stop online service system

Administration

Appropriate policy development

Rigorous student assessment and evaluations

Open admission policy
Tuition and fee, refund policy
Enrollment and course registration policy

2.1 Computerized grading system
2.2. Registrar system

Vision and Mission

Reflecting market needs
Developing concrete short-and long-term investment plan
Contributing to working adults’ education

Appendix 2: Revised evaluation criteria for new establishment of a cyber university, Korea

Source: MEST (2008). A study on the improvement of accreditation criteria for the establishment of a cyber university. Ministry of Education, Science and Technology, Korea.

Areas/
Dimensions

Detailed evaluation criteria

Essential Conditions
1. Goals and appellation

Strategic plan

Analysis of target learners and their needs

Plan for employment of instructors and tutors

Plan for employment of faculty and staff

Plan for property acquirement

Appropriateness of goals as a university under the Higher Education Law
Appropriateness of its appellation as a cyber university

-------

Concreteness of founding philosophy
Practicality and relevance of curriculum to meet learners’ need
Feasibility of long-term and short-term development plans

-------

Founding philosophy being valid and specialized to meet needs of online learners
Adequate and accurate analysis of target service groups

-------

Fair and valid plan for employment of instructors and tutors

-------

Fair and valid plan for employment of faculty and academic staff

-------

Fair and valid acquirement plan for property required for university operation

Facilities and equipment for education and research

Network and server infrastructure

Software

Information security system

Identity recognition and authentication system

Valid plan to establish network and server infrastructure to adequately serve its students

-------

Valid plan to equip with an operating system adequate for online education
Valid plan to construct an online administration management system
Valid plan to install security software

-------

Valid plan to build an information security system

-------

Valid plan to establish an students’ identity recognition and authentication system

Management of academic affairs

Curriculum

Quality assurance plan

Legislation of school charter and regulations

Clear curriculum plan
Specialized curriculum plan
Valid plan for creation of majors
Valid plan for creation of courses within each major

-------

Appropriate instructor-student ration in a course
Valid learning assessment and lecture plan

-------

Satisfactory and adequate school charter
Satisfactory and adequate school regulations

Human resources and organizational structure

Staffing and responsibilities

Valid plan for departmental structure
Valid plan for assigning responsibilities to staff
Valid plan for employment of general staff
Valid plan for employment of management staff

Quality management

Content development plan

Content development

Plan for content quality assurance

Plan for content maintenance

Valid plan of content development for first 8 weeks

-------

Appropriate content development model
Systematic and detailed process for content development

-------

Arrangement of content development team and quality assurance measures
Documentation plan for content development and quality management
Detailed plan for creating environment suitable for quality content development
Detailed plan to develop or adopt an Learning Management System

-------

Detailed plan to manage departments and staff for content maintenance
Detailed procedures for content maintenance
Detailed plan for institutionalizing content maintenance system

Appendix 3: 95 performance indicators for annual evaluation of a cyber university, Korea

Source: MEST (2008). A report on comprehensive evaluation of cyber universities. Ministry of Education, Science and Technology, Korea.

Areas/
Dimensions

Detailed evaluation criteria

Educational Planning

Goals

Curriculum

Appropriateness of organizational visions as a cyber university
Appropriateness of educational goals for cyber education
Fair goal setting procedures
Results of goal achievement
Members` awareness of educational goals

-------

Appropriateness of curriculum
Effective curriculum development and specialization
Policies and guidelines for curriculum implementation
Results of curriculum development and implementation

Instruction

Instructional Design

Content

Management of instruction

Teaching & Learning

Appropriateness and concreteness of instructional design processes
Clear course plans
Clear lesson plans
Clear content development plans
Structured management of instructional design
Proper documentation of instructional design activities

-------

Clear content development procedure
Learning content valuable to learners
Content types relevant to learning needs
Appropriate allocation of learning time
Structured management of content development
Proper documentation of content development activities

-------

Fair management of attendance
Appropriateness of learning guidance and facilitation
Fair management of assignments
Quality class management
Interactive teaching & learning strategies
Effective interactions between instructors and students
Effective interactions among students
Adequate teaching supports
Adequate learning supports
Good quality materials for teaching and learning supports

-------

Utilization of results of course evaluation
Appropriate learning evaluation
Fair learning evaluation

Human Resources

Students

Faculty

Staff

Allocation of adequate learning time
Re-enrollment rate
Operation of students` association
Outcomes of students clubs’ activities

-------

Ratio between full- and part-time faculty
Percentages of Dept assistants
Percentages of faculty with doctoral degree
Publication of full-time faculty
Faculty welfare
Faculty workload
Fair treatment of faculty

-------

Adequate number of general administrative staff
Adequate number of content developers
Adequate number of system management staff
Adequate staff training
Adequate training opportunities for staff
Reasonable task allocation
Fair treatment of staff
Adequate employee benefits
Optimal use of part-time employment system

Physical Resources

Educational buildings

Facilities

ICT system

Adequate number of school buildings
Optimal use of school buildings

-------

Sufficient facilities
Good quality facilities
Clear management plan for facilities
Adequate number of software
Satisfactory functionality of software

-------

Sufficient functions to support learning activities
Sufficient functional to support teaching activities
Sufficient functions to support system managers
Effectiveness of management system for academic affairs
Adequacy of data storage and management
System stability and security
Adequacy of system maintenance

Management & Administration

Organizational management

Administration

Clear decision-making process
Efficient management of various committees
Leadership of the president
Adequate strategic plan
Evaluation of progress of organizational development
Reasonable educational expense per student
Rate of foundational GDP
Credible university finance
Transparent accounting

-------

Adequate financial support for students
Adequate support for students` club activities
Sufficient support for students` services
Follow-up services after graduation
Satisfactory management of enrollment
Satisfactory management of course registration and academic affairs
Proper documentation of academic affairs
Satisfactory faculty management
Satisfactory staff management
Good budget planning and execution
Satisfactory management of school buildings and facilities

Outcomes

Satisfaction

Social recognition

Students` satisfaction with education
Graduates` satisfaction with education
Students` satisfaction with school status within the society
Graduates` satisfaction with school status within the society

-------

Good results of MEXT evaluation
Contribution to lifelong education
Strong collaborative partnerships with industries

References

Australasian Council on Open, Distance and E-Learning Benchmarks: Retrieved from http://www.acode.edu.au/resources/ACODE_benchmarks.pdf
Australian Universities Quality Agency (2006). Enhancement of quality assurance wystems in higher education in APEC member economies. Retrieved from http://aei.gov.au/AEI/CmsTemplates/DocumentTemplates/Document.aspx?NRMODE=Published&NRNODEGUID=%7B40323A51-D35B-487F-BC2C-C6BCDB379CB2%7D&NRORIGINALURL=%2FAEI%2FGovernmentActivities%2FQAAustralianEducationAndTrainingSystem%2FAPEC_QA_HE_pdf.pdf&NRCACHEHINT=ModifyGue
Brennan, J. (1999). Evaluation of higher education in Europe. In M. HENKEL, & B. LITTLE (Eds.), Changing relationships between higher education and the State. London: Jessica Kingsley.
Bonk, C.J. (2004). The perfect e-storm: emerging technology, enormous demand, enhanced pedagogy and erased budgets. The Observatory on Borderless Higher Education, June. Retrieved from http://www.obhe.ac.uk/products/reports/publicaccesspdf/Bonk.pdf COL, Quality assurance toolkit for distance higher education institutions and programmes. Retrieved from http://www.col.org/SiteCollectionDocuments/HE_QA_Toolkit_web.pdf
Commission on Higher Education. (2005). CHED Memorandum Order No. 27 – Policies and Guidelines on Distance Education. Retrieved from http://202.57.63.198/chedwww/index.php/eng/Information/CHED-Memorandum-Orders/2005-CHED-Memorandum-Orders
Commission on Higher Education. (2010). Information on Higher Education System. Philippines. Retrieved from http://202.57.63.198/chedwww/index.php/eng/Information
Japan, University Evaluation Information Portal. Retrieved from http://portal.niad.ac.jp/library/1179798_1415.html
Jung, I. S. (2001). Korea. In O. Jegede and G. Shive (eds.), Open and distance education in the Asia Pacific Region (pp.103-130). Hong Kong: Open University of Hong Kong Press.
Jung, I.S. (2005). Quality assurance survey of mega universities. In C. McIntosh, & V. Zeynep (Eds.), Perspectives on distance education: Lifelong learning and distance higher education (pp. 79-98). Vancouver: The Commonwealth of Learning and Paris: UNESCO.
Kim, S., & Lee, J.H. (2006). Changing facets of Korean higher education: market competition and the role of the state. Higher Education, 52(3), 557-587. Retrieved from http://www.springerlink.com/content/584348373553kk56
KNOU (2009, December). A self-evaluation report summary. KNOU.
Korea.net (2008, August 11). Cyber universities to be established under the Higher Education Act. Retrieved from http://www.korea.net/news/news/NewsView.asp?serial_no=20080808014&part=101&SearchDay=&source=Ministry%20of%20Knowledge%20Economy
Koul, B.N., & Kanwar, A.S. (Eds.) (2006). Towards a culture of quality: From quality assurance to a culture of quality in opendistance education. Vancouver: The Commonwealth of Learning.
Lee, H. C. (2003, January 17 - 18). University accreditation system in Korea. Paper presented at the International Network for Quality Assurance Agencies in Higher Education (INQAAHE) Asia Pacific Sub-Network Forum. Hong Kong, China.
MEST (2003). Guidelines for virtual university operations. Ministry of Education, Science and Technology, Korea.
MEST (2008a). A study on the improvement of accreditation criteria for the establishment of a cyber university. Ministry of Education, Science and Technology, Korea.
MEST (2008b). A report on comprehensive evaluation of cyber universities. Ministry of Education, Science and Technology, Korea.
MEST (2010, February 1). Policy implementation of higher education institutions’ accreditation and evaluation. Report distributed to the press. Seoul, Korea.
Middlehurst, R., & Woodfield, S. (2004). International quality review and distance learning: lessons from five countries. Report prepared for CHEA Institute for Research and Study of Accreditation and Quality Assurance. Washington, DC. Retrieved from http://www.editlib.org/d/27931/proceeding_27931.pdf
Ricafort, N. (n.d.) The Philippine Higher Education System: Current Trends and Developments. Retrieved from http://www.docstoc.com/docs/11381774/Philippine-Educational-System
Saito, T. (2009). Quality assurance of distance education/ e-learning. Report of Project Group 3: Asia Pacific Quality Network. Retrieved from http://www.apqn.org/files/virtual_library/project_reports/pg3_project_report_february_2009.pdf
Souda, Y., & Funado, T. (2007). Understanding of university accreditation and evaluation. Tokyo: Aidell Research institute.
Stella, A., & Gnanam, A. (2004). Quality assurance in distance education: The challenges to be addressed. Higher Education, 47, 143-160.

Social commentary Cackle