Assessing the Impact of an Instructional Design Course on Arabian Gulf University Distance Learning Students’ Instructional Design Competencies ()
1. Introduction
Instructional design is the practice of creating instructional experiences which make the acquisition of knowledge and skills more efficient, effective, and appealing [1] . The process consists broadly of determining the state and needs of the learner, defining the end goal of instruction, and creating some intervention to assist in the transition. It is a systematic process by which the instructional materials are designed, developed, and delivered. The terms instructional design, instructional technology, learning experience design, educational technology, curriculum design, and instructional systems design (ISD) are often used interchangeably. It is a way of planning instruction considering the learner, end goal or product, and evaluation. Many times, multimedia tools are used to improve instruction and increase student engagement [2] .
When designing instructional material for open and distance learning the concept of “deep design” is highly recommended. The term “deep design” is used intentionally to distinguish student and learning centered lesson planning from the classroom-centred, activity-oriented planning that is common among beginning teachers [3] . Deep design work is not directly visible to students or to anyone else who is not part of it. The teacher and classroom-centered instructional design models focus on activities, teacher performance, classroom events and experiences burning question: “What will students be doing today?” and planning addresses only the teacher’s time with students. On the other hand; the student and learning-centered instructional design models focus on what kinds of thinking students do, the intellectual skills students develop burning question: “What will students be learning today?”, planning addresses long-term outcomes and what students take away from the classroom events and experiences.
To achieve effective learning outcomes, the science of instruction and instructional design models are used to guide the development of instructional design strategies that elicit appropriate cognitive processes. [4] explored the competencies required for an instructional design manager to be effective in higher education settings; they used a Delphi study surveying managers and leaders of instructional design through an anonymous consensus-building process consisting of two rounds of surveys. Results identified eight major categories with 64 competencies as relevant for leading and managing instructional design in higher education. Competencies specific and beneficial to online high school teachers who are modifying their own courses were identified by [5] . He stressed out that existing instructional design standards available to guide online teachers are not only too numerous, they are also inconsistent. Moreover, a lack of clarity exists about which specific standards benefit this emerging professional group in the process of developing and revising their courses.
The instructional designer’s competencies essential for the context of online higher education were investigated by [6] who have selected an instructional design unit in a university research course as a case of investigation. To identify and compare competencies at organizational and individual levels, their study employed a mixed method to collect and analyze data based on a validated instructional design (ID) competency model by the IBSTPI as a framework. Throughout the study Instructional design competencies expected jobs/tasks and currently performed jobs/tasks were systematically analyzed, and the applicability of the IBSTPI model in this specific context of online higher education was verified. Based on the empirical findings, the study proposed a refined competency model to improve the instructional design competencies performance in human resources development and management practice.
Critical discussions within the field of instructional design have addressed the roles and competencies of designers, as well as the nature of design work per se. In this concern, [7] presented an overarching metaphor—namely, instructional design as a journey into the unfamiliar—that views design as a two-fold learning enterprise (i.e., innovative and maintenance learning) and characterizes designers as sojourner learners. The metaphor placed instructional design in a narrative context and considered designers, rather than formalisms, as the primary drivers of the design process. The study presented several implications of his metaphor related to the identity and innovative practices of designers in the field and finally, suggested that this metaphor could serve as a framework for inquiries into everyday instructional design work, examinations of innovative design practices, and further discussion of the respective roles of instructional designers and design formalisms.
In their study [8] investigated developing competency of teachers in basic education schools. The research instruments included the semi-structured in-depth interview form, questionnaire, program developing competency, and evaluation competency form. Data analysis revealed that factors of competency were: 1) the persistence in learning management and work practice, 2) competency in planning, goal setting, learning management and work practice, 3) competency in Information and Communications Technology (ICT) use in learning management and work practice, 4) creativity of learning management, 5) competency in following up and evaluation in knowledge management and work practice, and 6) competency in improving and developing the learning management and work practice. Program for developing competency included 9 factors and 7 learning units; total of 200 hours.
The original set of instructional design competencies was developed in 1986 as a result of more than a year of research, discussion, and validation by a group of instructional design professionals and academicians [9] . The level of proficiency described in the 1986 competencies was taken to represent an instructional designer who would probably have at least three years of experience in the field beyond entry-level training. Since the first set of ID competencies, much has changed in the landscape of practice, technology and developments in the major theories that underpin the field of instructional design.
In response to this changing environment, the IBSTPI set out to review and revise the ID competencies in the year 2000 and, more recently, in 2012. With every revision, more elements have been added to the process including:
1) The influence of advanced technologies, team-based design, and business management skills;
2) The professional foundations of design, as well as planning and analysis, design and development, and implementation and management skills;
3) A categorization of competencies as essential or advanced;
4) A larger representation of professionals around the world including directors, consulted experts and working professionals participating in the validation studies.
Based on the above reviewed literature, the instructional designer was defined as a person who designs instruction, a person who knows how people learn and have ideas on how to help them learn better. If you are looking for engaging learning activities or ways to make practice closer to real life skills, that’s when an instructional designer is who you need. The job of an instructional designer can be summarized in the following points:
1) Work with subject matter experts to identify what students need to learn
2) Develop objectives and ensure content matches those objectives
3) Revise and rewrite content to shape it for learning needs
4) Structure content and activities for student learning
5) Create media to support learning (e.g., visual aids for face-to-face, various multimedia for e-learning and online)
6) Develop assessments (note that this does not only mean tests)
7) Adapt instructional materials created for one format to another format (usually this is adapting materials from face-to-face to e-learning)
Aim of the Study
The present study aimed to investigate the impact of an instructional design graduate course (module) on the Arabian Gulf University graduate students’ learning outcomes represented in the instructional design general competencies generated by the IBSTPI 1986, and to explore satisfaction of the students with the learning experience.
2. The International Board of Standards for Training, Performance and Instruction (IBSTPI) Generic Competency Development Model
According to [10] [11] a competency model refers to “the organization of identified competencies into a conceptual framework that enables the people in an organization to understand, talk about, and apply the competencies in an organizing scheme”. With the advent of performance-based educational techquices, competencies have served as the nucleus of program design and the development efforts [9] . According to [12] the basis of this approach was the demand for clearly definable measures of program effectiveness in teacher education programs. Another origin was that competency-based education applied the innovative systems design techniques and elements of mastery learning [13] .
Competency based education program design is not a new approach; it was widely used in both teacher education and K-12 education during the 1970s of the past century. Today, competencies continue to be used in many of these same activities in higher education and business and training environments. As such, competency based education is applied in the distance teaching and training program at the Arabian Gulf University. In institutions which utilize and make use of distance education, teaching responsibilities are usually divided into two phases; that of course development in which course materials are prepared in advance, and tutoring in which instructional support is provided to learners as they are using the materials. Course development tends to be subdivided further into two aspects; that of providing subject matter expertise, and that of providing expertise in the area of instructional techniques appropriate to distance education. Similar competencies are required for both aspects of course development, with some specific competencies required for subject matter specialists and instructional design specialists. The generic IBSTPI competency development model is shown in Figure 1.
The effectiveness of courses delivered over a distance, like face-to-face instruction, depends on the planning of the course, class activities and the instructional materials used. The use of systematic instructional design in course planning can help to make any instruction more successful in promoting learning. However, there are significant differences between face-to-face instruction and distance learning. [14] reviewed applicable instructional design models for distance education, the unique characteristics of distance education in higher education via interactive television (ITV) are discussed, and a nine-step instructional design model, designed specifically for college course delivery via interactive television, is presented. Their model is an adaptation of the Dick and Carey model which is discussed in the article. Implementation of this model in situations which do not have large support staff and budgets for ITV is discussed.
In the distance teaching and training program at the Arabian Gulf University, these competencies are addressed in many courses [15] taught in the first semester of the first academic year for both postgraduate diploma and master students. These courses include Learning Theory (DLRDT531); Materials, Media and Development, Tools (DLRDT522); Writing the Materials (DLRDT533); Design and Presentation (DLRDT524) and Designing e-Learning Materials (DLRDT525). All these courses are program required and prerequisites for studying the instructional design course.
![]()
Figure 1. The generic IBSTPI competency development model.
The aforementioned courses in addition to the instructional design course (the topic of the present investigation) are developed to help the distance teaching and training post graduate candidate to develop a wide range of knowledge, skills and build the essential competencies needed for designing and developing distance learning projects, modules, and learning materials. The activities of the present study are only limited to the instructional design general competencies proposed by the IBSTPI in 1986 and covered in the Arabian Gulf University instructional design course.
3. The Study
3.1. Statement of the Problem
Researchers in the field of educational technology and distance learning and training assert that instructional design competencies play a critical role in the success in developing effective learning materials. Therefore, the present study is conducted to investigate the impact of an instructional design graduate course (module) on the Arabian Gulf University graduate students’ learning outcomes represented in the instructional design general competencies generated by the IBSTPI 1986 and to explore satisfaction of the students with the learning experience.
3.2. Research Questions
The main question of the present study was “what was the impact of an instructional design graduate course (module) on the Arabian Gulf University graduate students’ learning outcomes represented in the instructional design knowledge competencies and instructional design general competencies generated by the IBSTPI 1986?, and how did Arabian Gulf University students judge their satisfaction with the learning experience?
The following sub questions emerged from the main question of the study;
1) What were the basic (general) instructional design competencies according to the Association for Educational Communications and Technology (AECT)?
2) What were the objectives and components of the Arabian Gulf University instructional Design course for distance learning students?
3) What was the proposed mastery learning strategy used for teaching the instructional design course like?
4) What was the impact of the instructional design teaching strategy on Arabian Gulf University Diploma and Master Students’ mastery level of the instructional design course knowledge competencies?
5) What was the impact of the instructional design course on developing Arabian Gulf University Diploma and Master Students’ instructional design competencies?
6) How did the Arabian Gulf University students judge their satisfaction with the instructional design course?
3.3. Research Hypotheses
The present study sought to test the following research hypotheses:
1) Instructional design course learning materials and teaching strategy would help students to master the instructional design course knowledge competencies to a level of 85%.
2) Instructional design course would significantly develop Arabian Gulf University Diploma and Master Students’ instructional design competencies.
3) Instructional design course materials and teaching strategy would lead to a high level of satisfaction with learning among Arabian Gulf University students.
3.4. Importance of the Study
Studying the development of instructional design competencies by distance learning specialists is very important for many reasons: Firstly, most of the instructional design competencies are needed by all instructional designers and distance learning writers; Secondly, based on their learning and training, distance teaching and training program candidates must demonstrate that they have acquired the basic and the general instructional design competencies in order to perform all the specialized tasks related to distance learning design and learning material development; Thirdly, a distance learning designer and learning material writer needs to master how to create learning contents with information that can be used and applied in solving a real distance learning problem; Fourthly, a systematic evaluation of an instructional design course can help in the determination of merit, worth, and significance of the learning and training processes by comparing criteria against a set of proposed standards which represent the AECT instructional design competencies in this study.
3.5. Limitations
The sample of the present study was selected from the Arabian Gulf University distance teaching and training master and postgraduate diploma students who were studying the instructional design course. Instructional design competencies investigated were adapted from the general instructional competencies published by the IBSTPI in 1986, in addition to the knowledge component of these competencies generated from conducting content analysis of the instructional design course. Therefore, generalization of the results would be limited to instructional design courses and training contexts and learning communities which possess similar characteristics and study the same course contents.
4. Method and Procedures
This is a developmental research study in which a mixed research method was utilized. The developmental research can be defined as the systematic study of designing, developing, and evaluating instructional programs, processes, and products that must meet criteria of internal consistency and effectiveness [15] . In the developmental research mixed research methods may be applied. Furthermore, Richey (1994) classified three approaches of developmental research; the first type involves situations in which the product-development process is analyzed and described, and the final product is evaluated. The second type of developmental research focuses more on the impact of the product on the learner or the organization. The third type is oriented toward a general analysis of design, development or evaluation processes as a whole or as components. In the present study a mixed research method combined three research approaches:
1) A descriptive research approach was implemented for analyzing the students’ characteristics, instructional design module content analysis, previous research and resources analysis to establish a list of instructional design general competencies and develop the course online component.
2) A systems development approach by utilizing the ADDIE instructional design model in guiding the course development starting by the analysis phase and ending with the evaluation phase which intended to assess the impact of the proposed course on developing distance teaching and training program master students’ instructional design competencies.
3) An experimental research approach with quasi research design (one group design) to assess the impact of the developed course on the students’ instructional design competencies which were assessed at the beginning and towards the end of the course after successfully completing the course materials.
4.1. Instruments
Three research instruments were used in the study; an instructional design closed book final exam to test the knowledge component of the course, instructional design general competencies list (scale) (IBSTPI, 1986), and a satisfaction with learning scale prepared by the AGU Quality Assurance and Excellence Center.
4.1.1. Instructional Design Mastery Learning Assessment Methods
These methods included the following:
1) Instructional design reading assignment which aimed to provide students with basic knowledge of various instructional design models and their applications in designing and developing distance learning materials.
2) A course main assignment. The purpose of this assignment was to help students apply the instructional design standards in developing distance learning materials.
3) Six formative objective tests in instructional design aimed to help students master the cognitive competencies of each of the six learning units.
4) A closed book final exam in instructional design used to assess the students’ mastery level of the cognitive instructional design competencies.
4.1.2. Instructional Design General Competencies List (Scale)
The instructional design general competencies scale was adapted from the IBSTPI 1986 list of competencies. The list of the instructional design general competencies was composed of 16 basic competencies; each main competency has its own performance statements (indicators). The rating used for the assessment is a 5-point Likert-type scale, where 5 meant “a very high level of mastery” and 1 referred to a weak level of mastery of the nominated competency. The students were asked to judge their mastery of a competency before and after studying the instructional design course. Reliability coefficient of the instrument (Cronbach’s Alpha) was calculated and found to be around 0.950 which meant that the instrument was appropriate to be used (Table 1).
The above scale was administrated at the beginning and at the end of studying the instructional design course material. The study asked the student to rate her/his level of mastery of each of the 16 competencies, then compare the means and the gain in these competencies to test the impact of studying the course materials on developing the students’ ID general competencies.
4.1.3. The Satisfaction with Learning Scale
The satisfaction with learning scale was prepared by the AGU quality and excellence center which was developed and validated by the National Commission for Academic Accreditation and Assessment—Saudi Arabia. The scale was composed of three parts; part one aimed to collect information about the course (title and code, program/department information and the academic year, in addition to the instructions on how to respond to the questions; part two aimed to assess students’ degree of satisfaction with the course and its contents. This part consisted of 26 items (questions) which addressed formed students’ satisfaction with the course. These questions were distributed among 4 dimensions: the study plan of the course, the teaching activities, judging the course value and the overall evaluation of the course. Table 2 shows scale of satisfaction with learning dimensions and number of items in each dimension.
![]()
Table 1. AECT ID general competencies scale based on IBSTPI, 1986.
![]()
Table 2. Scale of satisfaction with learning dimensions and number of items in each dimension.
The third part of the scale is open-ended questions (questions 27, 28 and 29) which enabled the student to report on her/his observations related to her/his satisfaction with the course as well as the learning experience and included the following questions:
1) What was the best part of the course you liked most?
2) What was the part of the course that you did not like?
3) What suggestions did you have for improving the course?
5. Results
The main purpose of the study aimed to answer the main question of the study stated as: What was the impact of an instructional design graduate course (module) on the Arabian Gulf University graduate students’ learning outcomes represented in the instructional design general competencies generated by the IBSTPI 1986? and how did Arabian Gulf University students judge their satisfaction with the learning experience? Discussions of results related to the sub questions of the study are presented in the following sections.
5.1. Results Related to Instructional Design General Competencies
Results related to instructional design general competencies were generated from answering the first research question stating: [What are the basic (general) instructional design competencies according to the Association for Educational Communications and Technology like?
In 1986 the IBSTPI published the first edition of “Instructional Design Competencies: The Standards”. It was the result of work that began in 1978. In its third edition, IBSTPI presented the latest view of the competencies of the instructional designers. It was a greatly expanded view that reflected the complexity of current practice and technology, theoretical advancements, and the social tenor of the times [9] .
The original set of the instructional design competencies was developed in 1986 and was the results of more than a year of research, discussion, and validation by a group of instructional design professionals and academicians. In recognition of this IBSTPI Board set out to review and revise these competencies. The 1986 ID competencies set contained 16 competencies and 70 performance statements (Table 1 shows the 16 main ID competencies). The 1986 competencies and the associated performance statement are given in the table below. The complete list of the competencies and its updates are found in Instructional Design Competencies: The Standards are available online from http://www.aboutlearning.com.
5.2. Results Related to Topics and Contents of the Instructional Design Course
Instructional Design course (DLRDT536) is a three (3 Cr) hours graduate diploma and master required course. The prerequisites for DLRDT536 are all diploma and master courses taught in the first semester of each academic year. The aim of this course is to help the students to master the skills needed for analyzing open and distance learning courses according to learners’ characteristics analysis, learning content analysis, communications and information technologies and enable them to acquire the skills needed for designing and building course materials that suit open and distance learning. The course covered the following topics: Classical and modern learning theories; the basic elements/ components included in the instructional systems and courses design and development process; designing instructional materials according to behavioural, cognitive, cognitive and social constructivist learning theories; designing instructional processes according to the principle of learning driven from different schools of learning. For more information see DTTP website, 2013 available online at: http://www.agu.edu.bh/english/colleges/grad_elearning14.aspx.
The course consents covered the basic phases of instructional design and showed in detail the sub-phases included in each phase. In the combined workbook the course units describe the basic tasks for each stage of the ADDIE ID model and offer sufficient learning activities to help learners understand and practice instructional design techniques (AGU-Distance Learning Booklet, 2014). The course contents and topics are:
Introduction to the course: highlights the course objectives and topics, teaching strategy, and assessment.
Unit 1: Overview of Instructional Design. The unit covers topics such as; instruction and instructional systems, the need for instructional design, types of Open and Distance Learning (ODL) instructional design, models and theories of instructional design, instructional design and learning theory, the phases of the instructional design process, the main steps for designing instructional systems, and Dick & Carey model for the systematic design of instruction.
Unit 2: Instructional Design: Analysis Phase. The analysis phase topics are; identifying instructional goals, conducting instructional analysis, analyzing learners and context, and writing performance objectives.
Unit 3: Instructional Design: Design Phase. This includes writing assessment instruments, planning instructional strategy, choosing delivery method, writing instructional design documents (writing the teacher package, writing the student package), reviewing instructional design documents, and revising instructional design documents.
Unit 4: Instructional Design: Development Phase. the unit covers topics such as developing instructional materials, developing the student materials: (handouts, study guide, homework), development of the evaluation materials: (test and survey items), and developing instructor materials: (producing a teaching assistance guide).
Unit 5: Instructional Design: Implementation Phase. The unit explores the following topics: specifying the technique of instruction (f2f, blended or completely online), selecting the modes of instruction: (lectures, multimedia presentation, guest speaker or students’ class activities), media selection, lunching the course, and follow- up to add enrichment and remediation activities.
Unit 6: Instructional Design: Evaluation Phase. The final unit of the course covers; designing and conducting formative evaluation: gathering the feedback that can be used to modify and improve the course through: (gathering feedback from learners, measuring improvement and gabs, and measuring business results), revising the instructional materials, and designing and conducting summative evaluation which usually takes place after the course delivery so as to answer the big question “was the course effective?”.
5.3. Results Related to the Proposed Mastery Learning Strategy Used for Teaching the Instructional Design Course
Mastery learning is an instructional strategy and educational philosophy, first formally proposed by Benjamin Bloom in 1968. Mastery learning maintains that students must achieve a level of mastery (e.g., 90% on a knowledge test) in prerequisite knowledge before moving forward to learn subsequent information. If a student does not achieve mastery on the test, they are given additional support in learning and reviewing the information and then tested again. This cycle continues until the learner accomplishes mastery, and she/he may then move on to the next stage. Mastery learning is a method of instruction where the focus is on the role of feedback in learning. Furthermore, mastery learning refers to a category of instructional methods which establishes a level of performance that all students must master before moving on to the next unit [16] .
Adeniji and his colleagues [17] examined the effect of mastery learning on senior secondary school students’ achievement and retention in circle geometry. The study was conducted in Ilorin, Kwara state, Nigeria using a quasi-experimental, non-randomized, pre-test, post-test control group design. A sample of 172 senior school II students was drawn from four co-educational schools using multi-stage sampling technique. Instrument for data collection was Circle Geometry Achievement Test (CGAT) which was validated by experts and reliability index of 0.82 was obtained using test-retest method. The result showed that senior school students’ achievement in Geometry improved significantly when taught circle geometry using mastery learning approach. There was no gender difference found as well as no difference in the achievement of low, medium and high scoring students when taught with mastery learning approach. There was also a statistically significant difference in the post test mean score and retention score of students taught circle geometry using mastery learning approach.
In 2017, [18] reported on study aimed to know the improvement of the students’ achievement in poetry mastery and their perception regarding to the semiotic method in teaching and learning poetry in English Education Department, Languages and Art Faculty of State University of Medan. The research method used is the Classroom Action Research (CAR). The subjects of the research were 37 students of the first semester of the 2013/2014 academic year. The data were collected by test in order to know the students’ achievement in poetry mastery. The students’ perception toward the implementation of the semiotic method for teaching and learning poetry was collected by questionnaire. The data were analyzed by descriptive analysis. The result of the study showed that: 1) the average score of the students’ achievement of poetry mastery was (71.75) which increased (11.59) point from pre-test (60.16), and 2) in general, the students’ perception toward the semiotic method for teaching and learning poetry were good and positive.
Hussain and Suleiman [19] study investigated the effect of Bloom’s mastery learning approach on 9th grade students’ academic achievement and retention in English. A sample of forty students of 9th grade randomly selected from Government Boys High School Khurram Karak was used. Due to experimental nature of the study, sample students were divided into two equal groups on the basis of pre-test scores. Pre-test post-test equivalent groups designed were used for data collection. Descriptive statistics i.e., mean, standard deviation and inferential statistics i.e., t-test were used for statistical analysis. Data analysis revealed that Bloom’s mastery learning approach had a positive effect on students’ academic achievement and retention.
In the present study the authors attempted to assess the impact of the course proposed learning materials and teaching strategy by comparing the students’ mean in the final achievement exam covering the cognitive components of the instructional design course competencies to a mastery level of 85% and the shape of the grades’ distribution to the normal achievement curve and report the final results.
Bloom [20] when first proposed his mastery learning strategy in 1968, was convinced that most students could attain a high level of learning capability if the following conditions were available:
1) Instruction is approached sensitively and systematically
2) Students are helped when and where they have learning difficulties
3) Students are given sufficient time to achieve mastery
4) There is some clear criterion of what constitutes mastery.
The proposed mastery learning approach for studying a learning unit used with students enrolled in the instructional design course is shown in Figure 2.
5.4. Results Related to the Impact of the Instructional Design Teaching Strategy on Arabian Gulf University Diploma and Master Students’ Mastery Level of the Instructional Design Course Knowledge Competencies
To test the impact of the instructional design teaching strategy on Arabian Gulf University Diploma and Master Students’ mastery level of the instructional design course knowledge competencies, a closed book final exam was administered at the end of the course (last day of week 4). The final test was answered by 21 out of 22 students. Descriptive statistics results revealed that the sample’ lowest score was 28 out of 40 and the highest score was 38 (i.e., with a range equal to 10). The sample mean of the final exam was equal to 35.5238 with standard deviation equal to 2.452. One sample t-test was used to compare the sample mean to a mean of 34 out 40 repressing the proposed level of mastery (85%). Table 3 shows one-sample t-test results based on a test value 34 (the estimate mean).
The mean difference between the sample mean in the final exam and the estimated mean found to be 1.52381. It was found that a p-value of 2.747; which is significant at alpha level of 0.012. Thus, the mean difference between the mean of the sample (35.5238) is statistically significantly different from the estimated mean of 34 (proposed level of mastery 85%) and the study reject the null hypothesis. The teaching strategy used to the instructional design course positively led to a high level of mastery of learning among Arabian Gulf University Diploma and Master Students and helped them to master the instructional design course knowledge competencies. Table 4 shows the sample grades’ distribution as well as their histogram graph.
The results above showed that the final exam grades were negatively-skewed where the distribution has a greater number of higher values of students’ grades with the tail heading off to the left. Such increase could be attributed to the adoption of the mastery learning strategy used for teaching the course.
![]()
Figure 2. Instructional design course mastery learning strategy.
![]()
Table 4. The sample grades’ distribution as well as their histogram graph.
5.5. Results Related to the Impact of the Course on Developing Arabian Gulf University Diploma and Master Students Instructional Design Competencies
To test the impact of the instructional design module on developing students’ instructional design general competencies, the instructional design general competencies scale was administrated twice: at the beginning of the course (first day of week 1) and at the end of the course (last day of the 4th week); after the students successfully completed the course requirements. The general instructional design competencies scale was pre and post answered by 19 out of 22 students enrolled in the course. Table 5 shows mean scores, standard deviations with regards to students’ pre and post assessment of their instructional design general competencies scale.
Table 5 provides the means, standard deviations of students’ own ratings for each instructional design competency. The table also displays the ranking of the competencies based on the degree of judgment. From Table 5 researchers could conclude the following:
1) Students’ evaluation of their instructional design general competencies before studying the instructional design course ranged between below average and above average. The average of the 16 sub competencies ranged between (m = 2.8947, SD = 1.10024) for the statement #1 (I can determine projects that are appropriate for instructional design) and (m = 3.9474, SD = 1.07877) for the statement # 6 (I can write statements of performance objectives).
2) As indicated in Table 5, the statement # 6 (I can write statements of performance objectives) scored the highest mean (m = 3.9474, Sd. = 1.07877), then came the statements # 8 (sequence the performance objectives), 15 (interact effectively with other people) and 16 (promote the use of instructional design), (m = 3.9853, Sd. = 0.4780). Determine projects are appropriate for instructional design scored the lowest mean (m = 2.8947, Sd. = 1.10024).
![]()
Table 5. Mean scores, standard deviations and students’ judgment with regards to pre-assessment of instructional design competencies.
3) The overall mean score of the students’ instructional design general competencies is above average (m = 3.4112, Sd. = 0.83679) which is judged as possessing an average level of ID competencies.
In the post ID general competencies survey a noticeable development in students’ competencies was revealed. From the post ID general competencies survey results (Table 6) one can conclude that students’ ID general competencies in all dimensions scored a mean (≥4.00) and judged as high to very high, (i.e. the 16 dimensions of ID general competencies scale means range between high and very high). The statement # 2 (conduct needs assessment) scored the highest mean (M = 4.7857, Sd. = 0.42582), then came the statement # 15: interact effectively with other people (M = 4.7143, Sd. = 0.61125), then the statement # 5: perform job, task and/or content analysis (M = 4.6429, Sd. = 0.49725) and statement #16 (promote the use of instructional design) (i.e. M = 4.6429, SD = 0.63332). The statement # 7: develop the performance measurements scored the lowest mean (M = 4.1429, Sd. = 0.86444).
The overall mean score of the students’ ID general competencies judgment in the post survey is very high (M = 4.4821, Sd. = 0.47436) which is judged as possessing a high level of ID competencies.
![]()
Table 6. Mean scores, standard deviations and degree of students’ judgment with regards to post assessment of ID general competencies.
To test the impact of the proposed instructional design course on developing students` ID general competencies, a paired samples t-test was conducted to evaluate whether students’ ID general competencies were affected by the course or not as presented in Table 7.
Notes: IDGCpr refers to Instructional Design General Competencies pre assessment, and IDGCpo refers to Instructional Design General Competencies post assessment.
From Table 7 paired samples t-test results of the instructional design general competencies pre and post assessments it could be concluded that the overall means of pre and post IDGCs were statistically significant at P = 0.05. Furthermore, about 10 out of the 16 IDGCs scale components (statements) paired samples t-test results showed significant test results P = 0.05, and 5 out of the 16 statements indicated no statistically significant differences between pre and post assessment.
5.6. Results Related to Satisfaction with the Instructional Design Course
Seventeen (17) out of the 22 students who participated in the study answered the satisfaction with the learning scale. Data analysis showed a very high degree of satisfaction with the course (i.e. overall satisfaction is 92%) as revealed in Figure 3.
With regards to the open-ended questions of the scale of satisfaction with learning, Table 8 summarizes comments made by the students.
![]()
Figure 3. Students’ satisfaction with the instructional design course.
![]()
Table 7. Paired samples t-test results of the Pre and Post ID general competencies measures.
![]()
Table 8. Comments of the students on the open-ended questions of the scale of satisfaction with learning.
6. Discussion of Results and Conclusion
One of the reasons that the quality of much instructional material is poor is because it is not carefully planned and well designed. This fact led the Arabian Gulf University distance teaching and training program curriculum developers to consider a course in instructional design to help the diploma and master degree graduates to deal with the essential principles for designing instructional materials and developing essential competencies of instructional design.
The present study aimed to investigate the impact of an instructional design graduate course (module) on the Arabian Gulf University graduate students’ learning outcomes represented in the instructional design general competencies generated by the IBSTPI 1986, and to explore satisfaction of the students with the learning experience.
The researchers developed a course in instructional design as a core program requirement, taught the course using a mastery learning strategy developed for enhancing the participants’ mastery of the course competencies and then assessed its impact on the AGU students’ learning outcomes including achievement of knowledge competencies, instructional design general competencies proposed by AECT and satisfaction with learning. AGU distance learning students’ instructional design competencies were assessed twice; at the beginning of the course and the end of course by administrating a competency scale proposed by AECT, 1986, and a satisfaction with course scale administrated at the end of the course.
Data analysis revealed that the proposed mastery learning strategy used for teaching the course content proved to help the participants reach the estimated level of mastery (85% out of 40 = 34). Thus, the mean difference between the mean of the sample (35.5238) is statistically significantly different from the estimated mean of 34 (proposed level of mastery 85).
Data analysis revealed statistically significant differences (P ≤ 0.05) in the students’ instructional design basic competencies between means of pre and post-survey. Students’ ratings of their instructional design competencies were found to be higher in the post survey and ranged from high to very high (i.e. all statements’ means scored more than 4.00 out of 5.00 in the post assessment). This significant development in students’ instructional competencies is due to studding the course and the learning material, the strategy used for developing the course as a kind of deep design approach. This result asserted thoughts proposed by [3] which ensured that “deep design” was a term used by purposefully to distinguish student- and learning-centred lesson planning from the classroom-centred activity-oriented planning that is common among beginning teachers. These results were also supported by [8] who found that factors of competency were: 1) the persistence in learning management and work practice, 2) competency in planning, goal setting, learning management and work practice, 3) competency in ICT use in learning management and work practice, 4) creativity of learning management, 5) competency in following up and evaluation of knowledge management and work practice, and 6) competency in improving and developing the learning management and work practice.
The study also used a satisfaction instrument developed by the AGU quality and excellence center administrated at the end of the course. Analysis of data revealed a high level of satisfaction with the course material i.e. (around 92% overall satisfaction level). The students reported that the course content was realistic, clear and useful and links the theory of instructional design with its real applications, contributes to developing useful knowledge, skills and competencies for graduate students and helps them to deal with how to carry out learners’ needs, learning environments and learning context. Moreover, the course helped them to understand how systematic design of instruction was guided by ADDIE, or Dick and Carey instructional design model works which in turn led to students’ realization that “instructional design” be the core of the field of distance teaching and training.
Besides these advantages of the course students highlighted some difficulties related to course contents and the language barriers stating that the content was difficult and heavy compared to other courses, more time was needed, so that the student can learn the material, do the activities and master the learning objectives, and language of the course was difficult and needed translation into Arabic.
To overcome these difficulties students suggested activation of the broken links or replacing them with active ones, planning for a field visit to do real analysis of learners and environments and contexts, taking into account all students’ learning styles when developing the course learning materials and reducing the load of the course to help students learn the material better.
The development of the course and the assessment of its impact on AGU distance learning graduate students revealed that competencies were critical for instructional designers working in the field of distance learning and teacher education. Developing such competencies can be a research topic for future studies in training and certification of instructional designers and distance learning professionals to determine which competencies have the greatest impact on the success of an instructional design course and efforts. Based on these findings, the researchers suggested further training in the field of instructional design and further studies with a large group of students to assess the impact of graduate courses in developing educational competencies in graduate and postgraduate programs. This study used the competencies statements without the performance statements. Therefore, further studies can use the full statements with their performance indicators for postgraduate students to undertake a self-assessment to determine areas in which they may need to focus on for future professional development.