Leeward Community College

RESEARCH REPORT

on

Distance Education Vs On-Campus Classes

Prepared by

Marilyn Bauer,

Andy Rossi, &

Irwin Yamamoto

September 18, 2000

Table of Contents

Abstract……………………………………………………………… iii

Introduction

Background…………..…………………………………………. 1

Purpose and Research Question..……………………………….. 1

Theories/Hypotheses….…….…………………………………… 2

Definitions……………………………………………………… 3

Research Procedures/Methodology

Design…………………………………………………………… 3

Population and Sample Size……………………………………. 3

Data Excluded from the Study………………………..………… 4

Data Collection Methods………………………………..……… 5

Validity/Reliability…...………………………………………… 5

Method of Analysis……………………………………………… 6

Findings/Results…………………………………………………….. 7

Conclusions/Recommendations…………………………………….. 12

Raw Data……….……………………………………………………. 13

Abstract

Leeward Community College began offering distance education classes in the Fall of 1992. Since that time, the number of distance education classes and the number of students enrolled in these classes have steadily increased. This alternative method of learning, while providing additional opportunities for students to take classes, has raised some questions about its effectiveness. Concerns have been voiced as to whether students actually learn as much through distance education classes as they do in regular classroom settings. This research attempted to assess student achievement using student grades as the unit of analysis.

The study involved a quantitative approach testing the relationships between distance education classes and regular on-campus classes. Descriptive Statistics and Analysis of Variance (ANOVA) were the statistical procedures used to compare the data from these classes. The data used in this study were historical data spanning nine semesters and were retrieved from the Leeward Community College’s central computer “Aldrich” system.

The results of this study suggest that except for a slightly higher rate (4%) of withdrawals, students were as successful in distance education classes as they were in regular on-campus classes. The analysis indicated that there were no significant differences in the passing or failing grades, in the “N” grades, or in the percentage of incomplete grades between students enrolled in distance education classes, and on-campus classes.

Research Report on

Distance Education Vs On-Campus Classes

Introduction

Background

Leeward Community College (LCC) began offering distance education classes in the Fall of 1992. Since that time, the demand for distance education courses steadily increased and enrollments more than quadrupled. It appears that this mode of educational delivery has become very popular. While this mode of educational delivery provides an added opportunity for students to take classes, concerns have been raised about the effectiveness of distance education courses. Questions have been raised as to whether students actually learn as much through distance education classes as they do in regular classroom settings. Some evidence exists which suggests that the development of general cognitive skills is linked to the students’ interactions with faculty and peers[1]. This evidence challenges the notion that students learn as much in distance education classes when the dynamics associated with the interactive classroom are removed. While this study did not attempt to assess cognitive development, the questions regarding student achievement seemed viable and remained unanswered. Until this time, the College had not attempted to assess the affect distance education courses had on student achievement.

The following are the Headcount Enrollments for Distance Education classes over the past six academic years.

Headcount Enrollment

(Source: LCC, Media Center)

	94/95	95/96	96/97	97/98	98/99	99/00
Headcount	157	352	320	538	502	867

Purpose and Research Question

The purpose of this research was to determine whether students were as successful in distance education classes as they were in regular on-campus classes. While student success is very difficult to evaluate, one measure of student achievement is the grades that students receive, and grades were readily available. As a result, grades and in particular passing and failing grades became the unit of analysis for this research study. In addition, because there was some speculation about distance education classes having a larger number of students who didn’t complete the various courses, withdrawal rates, incomplete grades, and “N” grades (no grades) were also analyzed in relation to their effect on student success.

Theories/Hypotheses

The College had several theories about the students taking distance education classes. The first of these theories was that there was no significant difference in the percentage of passing and failing grades students received in distance education classes, from the percentage of passing and failing grades students received in regular on-campus classes. The second hypothesis was that there was no significant difference in the percentage of incomplete grades given in distance education classes, from the percentage given in regular on-campus classes. Thirdly, it was felt that there was no significant difference in the percentage of no-grades (“N” grades) given in distance education classes, from the percentage given in regular on-campus classes. Lastly, it was theorized that there was a difference in the rate of student withdrawals between distance education classes and regular on-campus classes, and that this difference was significant.

The reason the College felt that there was a difference in the rate of student withdrawals between distance education classes and regular on-campus classes was because the Educational Media Center perceived that more students were withdrawing from the distance education classes. Although the College recognized a difference might have existed, enrollment trends were not analyzed to either prove or disprove this observation.

In response to this perception of seemingly higher rates of withdrawal, beginning in the Fall 1999 semester, the Educational Media Center started holding an orientation meeting at the beginning of each semester for distance education students. The purpose of this orientation was to educate students about the differences associated with distance education delivery in hopes that a better understanding would lead to fewer withdrawals. Aside from the misgivings associated with the withdrawal rates, the College remained positive that the degree of student learning attained through distance education courses was similar to the amount gained through on-campus classes.

Definitions: The following are the various models of educational experiences that were analyzed in this study, and the grading criteria that student achievement was based upon.

1. Model TV: The televised playback of class instruction that originated from the taping of an actual live class, or from the taping of instruction, which was recorded without a live audience.

2. Model INT: An instruction class conduct via the Internet.

3. Model OC: An on-campus class that had no association with distance education except that the same instructor also taught a similar class via distance education, or an on-campus class that was also videotaped and later used for playback as a distance education class.

4. Passing Grades: For the purpose of this study, passing grades were considered to be “A”, “B”, and “C” grades.

5. Failing Grades: Both “D” and “F” grades were considered to be failing.

Research Procedures/Methodology

Design:

This study involved a quantitative approach to testing the relationships between distance education classes and regular on-campus classes. In addition to utilizing Descriptive Statistics, Analysis of Variance (ANOVA) was the statistical procedure used to compare the data from these classes. ANOVA testing was used because it is the preferred method for determining significant differences between group data.

The data used in this study were historical data spanning nine semesters and was retrieved from the Leeward Community College’s central computer “Aldrich” system.

Population and Sample Size:

The population included all distance education classes originating at Leeward Community College. Courses originating at other campuses such as Honolulu Community College or Kapiolani Community College and offered to LCC students were not included in the population.

The sample was selected based upon a purposeful sampling technique whereby a criterion was established in choosing the classes to be compared. Classes were selected based upon the same instructor having taught both the distance education course as well as the regular on-campus class. Only those distance education and on-campus classes that had, in essence, an exact counterpart were included in the sample. An example of an exact counterpart would be a Botany 100 class, which was taught by the same instructor and offered as both a distance education class and a regular on-campus class, regardless of the semesters offered. A Math 100 class, although offered through both distance education and on-campus modes, was excluded from the sample if the same instructor did not teach both these classes.

This purposeful sampling selection procedure was chosen to eliminate as many extraneous variables as possible, which might unduly influence the study results. It was felt that true comparisons between student achievement could only be achieved if the variability associated with the grading standards of different instructors could be controlled for. Previous research suggested that while grades reflected learning, grades were influenced by other factors such as college policies, and instructor’s personality[2]. As a result, classes included in the sample were selected based upon the same instructor having taught the same curriculum via these different modes of instructional delivery.

All of the classes that met this criterion were selected as part of the sample.

The sample time frame included all classes taught in the nine semesters ranging from Spring 95 to Spring 99. Data prior to Spring 1995 were not available, and the Fall 1999 data were considered unstable because at the time of data collection instructors were still changing grades based upon students making-up work.

Data Excluded from the Study

Although initially analyzed and compared to regular on-campus classes, courses taught via the Internet were ultimately excluded from analysis. The reason for this was a matter of validity. There were only seven Internet classes that met the established criterion for determining sample size. These seven classes when compared to the 183 regular on-campus classes produced unreliable and invalid statistical results. The disparity between the number of classes was too great. Comparing data from seven Internet classes to 183 on-campus classes violates acceptable statistical standards. Therefore, the data associated with Internet classes were excluded from analysis.

Data Collection Methods

A list of all distance education classes and on-campus counterparts meeting the criterion established in the purposeful sampling strategy was prepared by the LCC Educational Media Center. This list included 232 classes (42 TV, 7 Internet, and 183 on-campus classes). The list was then used by the LCC Institutional Researcher to extract data from the LCC Aldrich System utilizing a data extraction grade report that was developed by the LCC Computer Center. The data were then downloaded via ASCII files to the Researcher’s desktop computer and subsequently imported into Excel, and Statistical Package for Social Science (SPSS) software programs. These programs were then used to analyze the data.

As part of the process, the Institutional Researcher recoded the data to remove as many identifying codes or nomenclature as possible, which would indicate times or locations of classes and instructors. This recoding was done in order to protect the confidentiality of the faculty.

Validity/Reliability

The data used in the analysis spanned four and a half academic years, or nine semesters (Spring 95 through Spring 99).

Not counting the seven Internet classes that were excluded from the study, 100% of the sample was used in the statistical analysis. These data were all historical files and no changes to the database were made. All mathematical calculations and statistical procedures were completed utilizing computer programs.

Sample selection was based on a purposeful sampling technique whereby a criterion was established in order to reduce the possible bias effect associated with different grading standards. The grading standards of different instructors were controlled for to reduce the effect this variable might have on student achievement. Student achievement reflected in the distance education and regular on-campus class grades, could be better compared if the selection of classes was restricted to classes having the same instructor who taught both modes of educational delivery.

The highest Standard Error of the Mean from all the data comparisons was .02, which was based upon a 95% confidence level (two tailed test). This standard error of .02 (2%) occurred with several of the group data. An example of how this level of error impacts the validity of the data can be interpreted by using the data associated with the average percent of students passing distance education courses, as an example. The standard error of the mean for the average percent of students passing distance education classes was 2% and the mean was 66%. This suggest that the true (population) mean for this data would fall somewhere between 62% and 70%. The Findings/Results section of this report contains the standard errors for all the data along with the mean and standard deviation results.

Method of Analysis

Descriptive Statistics were first completed on all the data in order to determine frequency counts and the percentages of grades. The use of Descriptive Statistics also allowed for the opportunity to check for errors that might have occurred as a result of transferring the data from one program to another.