Running
Head: MAILBOX MANAGEMENT: A BEHAVIORIST-COGNITIVE
Mailbox Management
A Behaviorist-Cognitive Instructional Design Project
by
Anthony Paul Niemann
ELFH 672-50
April 19, 2006
Mailbox Management:
A Behaviorist-Cognitive Instructional Design Project
A
behaviorist-cognitive approach is well-suited for this lesson plan on Mailbox
Management. This approach is appropriate for the goals outlined for this
training. Duebel (2003) tells us that there is not a single theoretical model
that can be used for all Instructional Design applications. In the case of this
particular Mailbox Management lesson plan, I have chosen a combined
behaviorist-cognitive approach. As Deubel states, a combined approach allows a
great deal of flexibility in the lesson plan.
The
lesson is designed to change behavior of Cabinet for Health and Family Services
(CHFS) employees. These employees are predominately novice computer users, even
though many have worked for CHFS with computers for many years. Because they
are novice computer users, instruction will be delivered synchronously in a
face-to-face class. Goals for this lesson are to reduce computer lockups and
slowdowns due to full mailboxes. The user’s behavior must be changed in order
to accomplish this goal. Duebel tells us that a strict behaviorist approach
alone may not work as well as a combined method because it would neglect
unobservable mental states called Gestalts. This can be more readily addressed
in a face to face class using a combined behaviorist-cognitive approach. By
using a combined approach I have the best opportunity to effect change in the
student’s behavior. I also have the best opportunity to affect change in the
student’s attitude, engendering internal motivation of my adult students
whenever possible. Students will learn about an application they need to perform
their daily duties.
A
list of documents is presented in an arbitrary order that I developed as I
worked on the lesson plan. A number was assigned to each of the documents that
were created. Although documents were listed in a specified order, the actual
development of these forms was constantly evolving throughout the process by
using a systematic approach to design and development. The ADDIE model guided
creation of documents including necessary visual supports, instructional
management devices, and evaluations, as discussed in an American Society for
Training and Development (ASTD) Info-line issue (Hoddell, 2002). I started with
analysis and moved toward evaluation using the ADDIE model. Although the ADDIE
model represents a single process, I found myself moving back and forth through
the five stages of the ADDIE model repetitively in order to fine-tune the
entire product. By using a systematic approach any component of the instruction
can be fine-tuned independently of other components, based on feedback. By any component, I am speaking about such
things as the learners, content, instructors, materials, methods, learning, and
evaluation environments. Because the individual components support each other,
I can improve the entire system by adjusting one or two components.
My
goals for the course were intuitive, due to my understanding of the problem,
students, and subject matter. I used Bloom’s taxonomy for action verbs (Miller,
2006b) combined with a class handout of an article by Kathy Waller (2006) to
develop my instructional objectives. The goals and objectives are described in
document 3. Specific steps in the lesson plan were developed only after a
detailed task analysis handout was developed (document 8). I also used the
systems approach block diagram chart in our textbook (Dick, Carey, and Carey,
2005, p.1) in the development stage of my documents. The documents are numbered
1 through 14. The ADDIE Instructional Design Model (Hoddell, 2002) coupled with
behaviorist-cognitive learning theory guided creation of the following
documents:
1. Mailbox
Management Needs Analysis
2. Mailbox
Management Audience Analysis
3. Mailbox
Management Goals and Objectives Paper (Handout B)
4. Mailbox
Management Level 0 Evaluation - Sign-In Sheet
5. Standards
for Mailbox Management Lesson Plan
6. Mailbox
Management Lesson Plan
7. Mailbox
Management OIT Policy Statement (Handout A)
8. Mailbox
Management Task Analysis (Handout C)
9. Mailbox
Management Level 2 Evaluation - Objective Test (Handout D)
10. Mailbox
Management Level 2 Scoring Key
11. Mailbox
Management Level 1 Evaluation - Smiley Sheet
(Handout E)
12. Mailbox
Management Scoring Rubric
13. Mailbox
Management Level 3 & 4 Evaluation - Post Mortem Evaluations and ROI Report
14. Design
Paper for Mailbox Management Lesson Plan
The
first step when using a system approach to instructional design is to identify
instructional goals (Dick, Carey, & Carey, 2005). CHFS employees experience
a decline in productivity due to computers that do not operate correctly. My
position at CHFS as a Network Analyst qualifies me as a Subject Matter Expert
(SME). As an SME, I know that computer users at CHFS experience slow computers
and lockups. Many of the problems are caused by full Outlook mailboxes. Problems
associated with full mailboxes result in calls placed to technical support,
causing productivity for users and technicians to drop. Goals for this lesson
were obvious to me from the beginning: Reduction of the frequency of computer
lockups would reduce the need for technical assistance and increase employee
productivity. In order to clarify the goals and objectives, I conducted a needs
analysis and an audience analysis.
Please
refer to the Needs Analysis (document 1) and Audience Analysis (document 2), which
appear in my list of documents above. Both the needs analysis and the audience
analysis were conducted with the advantage of two perspectives. First these two
documents were developed with a trainer’s perspective of what the needs are as
well as a trainer’s knowledge of the target audience. Second, the two documents
were developed with an added viewpoint of an SME. An extensive needs analysis was
performed using data extracted from the Remedy
database. Results of data extracted from this database indicate that CHFS
employees need training in using specific features of the Microsoft Outlook
application. The Remedy database contains data that allows tracking of computer
problems associated with mailboxes that have reached size limits set by the
CHFS Office of Information Technology (OIT). An audience analysis indicates two
reasons why CHFS employees do not move email from Outlook mailboxes to their
Personal Folders:
1. Employees
have not learned the proper method to move email from their Outlook mailbox and
into their Personal Folders.
2. Employees
are adult learners who are working in a state government position. They want,
and need, to understand why there is a limit on the amount of mail they can
store in their mailbox.
Item
one indicates that learning involved with this lesson is associated with
training in the Microsoft Outlook application. Training “is usually focused at a narrow set of knowledge and
skills” (Miller, 2005a, p. 3). Training often focuses on changing learner
behavior, or performance improvement. It takes on a more behaviorist character
than education. Education takes on a
broader focus, and usually involves a more extensive scope. Thus, it might
profit by a more cognitive, or constructivist approach. The constructive
approach often focuses on changing the learner’s thought processes, rather than
behavior.
Item two indicates that students are adult learners.
Adults may learn differently than children due to the context of lessons they
take. Methodologies used in elementary schools to present lessons may not be as
effective on adult learners. In the book Tools
for Teaching it is suggested that content may be more effectively presented
to adult students by first presenting theory and then presenting application (
It may not be sufficient to simply tell the adult
learner that the way they used the Microsoft application for the past several
years needs to be altered. Explaining why
they need to change the way they use Outlook is a critical component of
preparing the student to learn a new way of doing familiar tasks. Initial
training using Microsoft applications was never provided to these learners. As
new employees were hired, bad habits were passed on from user to user. Many
users do not realize that a full mailbox can prevent them from sending email, even
though they can receive email. Other users may not realize how to tell when a
mailbox is full. When the mailbox fills the computer may run slowly, or the
user may experience a wide variety of problems when using applications other
than Outlook. Many users simply choose to delete email once they received a
message from the System Administrator notifying them that their mailbox is
full. Even after deleting email from the Inbox
or Sent Items sub-folders of their
mailbox, many users do not realize that these items remain in their mailbox
inside the Deleted Items sub-folder.
If the user closes Microsoft’s Outlook application and answers the prompt that
s/he receives incorrectly, email items remain in the Deleted Items sub-folder, causing the mailbox to reach limits set
by the administrator quickly. The Remedy database proved that some users
reported problems associated with a full mailbox periodically throughout the
year because they did not know about some of the above problems indicated above.
Behaviors associated with mailbox management needed
to be changed. After creating the needs analysis and audience analysis, objectives
listed on the goals and objectives form were re-analyzed to insure that they
were satisfactory. The goals and objectives form (document 3) is included in
the lesson plan in Handout B. For the reader’s convenience, the goals and
objectives are also listed below:
Goals:
Reduce computer lockups and slowdowns due to full
mailboxes. Reduce requirements for technician intervention.
Objectives:
Given
a computer attached to the CHFS network with the Outlook application
open,
by the end of this class, learners will be able to:
1. Identify their mailbox Folders and Personal Folders
2. Create new folders inside their Personal Folders
3. Move email from the mailbox folders Inbox and Sent Items to the Personal Folders
4. Delete email from the mailbox Deleted Items folder
with
no errors. Handouts and job aids will be provided.
In
order to assist the instructor and the CHFS organization in tracking who
attends the Mailbox Management lesson, a Level 0 evaluation is performed as the
students enter the classroom (Waagen, 2002a). This form (document 4) follows
the goals and objectives form and is designed with the target audience in mind.
For example, there is no place for the student’s email address on this form
because these addresses are readily available through the state global address
list. One critical inclusion on the sign in sheet is a place for time that
students arrive for class. If the student is late s/he will miss a portion of
the class. It is critical that the instructor has a record of which students arrive
late for class. The portion of the class that is missed may affect the learning
process, and therefore should be tracked.
Standards for this lesson plan follow the Level 0 Evaluation document. This lesson follows the 2003 standards proposed by the National Council for Accreditation of Teacher Education (NCATE) and the International Society for Technology in Education (ISTE). Standards that correlate to this lesson are marked in document 5, which accompanies this lesson.
The
lesson plan appears next on the list (document 6), followed by documents that
will be used concurrently as the lesson is presented. The lesson plan was
altered countless times while creating documents. The documents that follow the
lesson plan are a crucial part of it, and many of them will be used once the
lesson gets underway. One of these documents is the OIT Policy Statement (document 7), or Handout A. Handout A itemizes the Office of Information Technology
(OIT) policy for Outlook mailboxes. This policy places responsibility on the
employee to clean out their mailbox as soon as possible after they receive a
System Administrator message. It is important for the adult learner to not only
meet the objectives indicated in Handout B (Goals and Objectives), but also to
understand the goals for the course. The methodology involved in Handout A
relies on the instructor verbalizing negative reinforcement to the student.
However, it does so in a distinctively cognitive manner. By sharing the goals
with the learner, and then sharing the OIT policy, cognitive leaning theory is
applied. Students hear instructors talk about the problems that result from
mailboxes that are not cleaned out. This includes problems such as the expense
for the state to hire additional OIT staff and budget for a tape-backup system
that would allow them to store email for 35,000 computer users. The policy
informs CHFS employees that they can expect no computer assistance until this
policy is followed. The student is being effectively conditioned to accept
responsibility for keeping their mailbox below the limits established by OIT. This
concept is similar to Skinner’s operant
conditioning, as opposed to the classical
conditioning of many of his predecessors (Mergel, 1998). However, the
document also utilizes Malcolm Knowles’ principle of andragogy (Smith, 1999) by
explaining to the student why their
past practices of cleaning out mailboxes should be changed.
The
next document the student will refer to in this lesson plan is Handout C, the
Task Analysis (document 8), which is a systematic identification of specific
skills, knowledge tools, conditions, and requirements the student will perform (Waagen,
2002b) while going through the 45 minute lesson plan. Note how Waagen’s
description of task analysis sounds much like the definition of an objective. First,
the employee must be conditioned to move mail out of their mailbox when they
receive an automated message from the System Administrator telling them that
their mailbox is full. Mailboxes will fill up if the employee does not
periodically delete some of their received, sent, and deleted email from their
mailbox. It is not sufficient to delete email, but you must also delete your
deleted items. Microsoft chose to use default settings for the Outlook
application that most Administrators of the Exchange server would support. Once
users receive a message from the administrator telling them that their mailbox
is full, they should immediately clean out their mailbox. If they delay, the
computer will run slowly, or stop sending out email completely. Failure to clean
out the mailbox can prevent other applications from working properly. The task
analysis was developed to change the student’s behavior in regard to these
issues.
The task analysis includes elements of a cognitive
approach. By telling the student that many problems with applications are
caused by full Outlook mailboxes, providing OIT policy regarding mailbox management,
and telling the user it is their responsibility to comply with this policy, the
lesson relies on learning theory that is outside the scope of a strict behaviorist
approach. The student is required to cognitively link problems associated with
mainframe applications or slow computers with the Outlook email application. As
Mergel (1998) indicates in differentiating Behaviorism and Cognitivism, the
student is not merely learning how to perform actions that can be measured
quantitatively, but are also learning why it is necessary to perform these
actions. The adult learner who works for state government has experienced
mandated requirements that seem to have no apparent logic. In order to change
adult behavior, a strict behaviorist approach may not work.
Although
Handout C, the task analysis, employs a behaviorist approach, it also includes
steps that are distinctively cognitive, as exemplified in step 21. Presently,
about 50% of CHFS employees do not use Personal Folders. The students discover
a technique for creating new folders inside their Personal Folders in this
lesson plan. Step 21 asks the student to recall steps 5 and 6. Instead of
performing steps similar to 7 and 11 where a folder with a specific name is
created, students are told that they can create a folder with any name they
wish. They are told that performing step 21 will allow them to organize email
in a way that will clean out their mailbox as well as allow email to be easily
located and organized.
Evaluation is a critical component of the Mailbox
Management behaviorist-cognitive lesson plan. Actually, evaluation is a
critical component of any lesson plan
(Waagen, 2002a). There are five documents that accompany this lesson plan that
assist instructors in evaluating whether the learning has taken place. I
followed Kirkpatrick’s Four Level model for evaluating training programs
(Kirkpatrick, 1998), adding a Level 0 evaluation (sign-in sheet) which was explained earlier, deciding not to add a
Level 5 evaluation (training’s effect on society), as discussed in Rosssi,
Freeman, and Lipsey’s book Evaluation: A Systematic Approach
(1999).
The second evaluation discussed in this paper is the
Level 2 evaluation (Handout D, or document 9), which is a summative test of
student learning. It includes a ten-item quiz that is administered at the end
of the Mailbox Management lesson. This quiz, like all other documents presented
in the Mailbox Management lesson plan, will be scrutinized for content validity
and reliability on an ongoing basis. Careful revision of this document is expected
over time. Nevertheless, questions in this document were designed to have a
clear, easy-to-read format. The test begins with five alternate choice test
items, followed by five multiple choice items. Each test item is worth ten
points, for a total of 100 points. Test item analysis was performed to insure
the content was appropriate for the objectives listed in the Goals and
Objectives document. Fifteen minutes are adequate to answer five alternate
choice test items and five multiple choice items. Response items were
alphabetized after the test was created on the five multiple choice test
questions to increase validity due of students trying to “guess” the letter for
the correct response. Item validity and reliability will also be analyzed as
test results are accumulated. An instructor scoring key follows the Level 2
evaluation and is listed as document 10.
The third evaluation used in the Mailbox Management
lesson is the Level 1 Evaluation (Handout E, or document 11), sometimes
referred to as a smiley sheet. One
critical component included in the selection of twenty items in this
questionnaire is to insure that all questions are appropriate for this specific
lesson plan. Responses to each of the twenty questions asked on this Level 1
evaluation might be helpful in revising the Mailbox Management lesson in the
future. Questions the students are asked to complete apply to the content that
is presented.
The fourth evaluation is a rubric developed to
combine the results of the summative test in Handout D with an instructor
observation of student classroom performance in completing the four objectives
listed on the Goals and Objectives sheet in Handout B. The rubric (document 12)
is included in the lesson plan and results will be retained for future research.
Results will also be used to insure that the learning experience is effectively
presented by comparing results recorded on the scoring rubric with actual student
mailbox size over the following six months. Information about student mailbox
size is available from the Microsoft Exchange servers in
The fifth evaluation is referred to as Level 3 and 4
evaluations. The Level 3 evaluation measures transfer of learning while the
Level 4 evaluation evaluates measurable results of the learning. It is document
13 in the list of lesson plan documents. It compares results recorded on the
rubric, student mailbox size as measured on the Exchange server reports, and
Remedy database reports, for the purposes of conducting a detailed
Return-On-Investment report. Cross references with the Remedy database will be
conducted by searching for selected users who have experienced Microsoft
Outlook related problems. If problems are evident, the size of the student’s
mailbox will be recorded in order to check for positive or negative correlation.
The results recorded on the rubric coupled with data extracted from Microsoft
Exchange servers and Remedy database will allow an ROI report within the first
year that should prove to be very accurate.
The final document in the presentation of the Mailbox
Management lesson plan is this Design Paper (document 14). It tells others why design
decisions were made. It also causes me to reflect on the decisions that were
made, allowing an analysis of decisions occurring in each step of this process.
References
Deubel,
P. (2003). An investigation of behaviorist and cognitive approaches to
instructional multimedia design.
Journal of Educational Multimedia and
Hypermedia,12(1), 63-90. Retrieved on
www.ct4me.net/ multimedia_design.htm#top
Dick, W., Carey, L., and Carey, J. O. (2005). The
Systematic Design of Instruction (6th
ed.).
Hoddell,
C. (2002). Basics of instructional systems development. In S. Sussan (Ed.),
Infoline: Instructional systems development issues
(pp.1-18).
Kirkpatrick,
D. L. (1998). Evaluating Training
Programs: The Four Levels. San
Francisco: Berrett-Koehler Publishers, Inc.
Mergel,
B. (1998). Instructional design and learning theory.
Blackboard. Retrieved on
webapps/ portal/frameset.jsp?tab=courses&url=/bin/common/
course.pl?course_id=_109083_1
Miller,
K. (2005a). Lesson II 1 Commentary: Characteristics of a behaviorist
instructional design. Retrieved
Miller,
K. (2006b). Bloom’s taxonomy-action verbs worksheet.
Blackboard, Retrieved March 8, 2006, from https://blackboardic.louisville.edu/
webapps/portal/frameset.jsp?tab=courses&url=/bin/common/
course.pl?course_id=_109083_1
Rossi,
P. H., Freeman, H. E., and Lipsey, M. W. (1999). Evaluation: A Systematic
Approach.
Smith, M. K. (1999) 'Andragogy', the encyclopaedia of
informal education. The Informal
Education Homepage (infed.org). Retrieved on
www.infed.org/lifelonglearning/b-andra.htm
Waagen,
A. K. (2002a). Essentials for Evaluation. In
Instructional systems development issues
(pp.251-288).
Waagen,
A. K. (2002b). Task Analysis. In S. Sussan (Ed.),Infoline: Instructional
systems
development issues (pp.132-148).
Waller,
K. V. (n.d.). Writing instructional objectives.
Retrieved
frameset.jsp?tab=courses&url=/bin/common/course.pl?course_id=_109083_1