Running Head: MGO NEEDS ASSESSMENT
MGO Needs Assessment
by
Anthony P. Niemann
ELFH 662-50
Spring, 2007
MGO Needs Assessment
Introduction and Context
MGO, a large
government organization in mid-eastern
A performance-based training NA was designed and implemented for MGO, one of twelve cabinets that make up state government. This cabinet is responsible for providing health and family services to state residents (service receivers). This NA provided a performance-based method to measure the gap between the current and ideal situations for a MGO. As such, it uncovered unmet needs and latent needs. When a NA is done for a service provider group it is most often done to assess training needs (Witkin and Altschuld, 1995). Approximately 40% of those working for state government are employees (service providers) of this cabinet. There are approximately 35,000 MGO employees at offices located in 120 counties within the boundaries of MGO.
MGO includes
multiple sub-organizations, each having responsibilities to provide a wide
array of services to state residents. Some of the larger groups in MGO include:
Protection and Permanency; Family Support; Child Support; Commission for
Children with Special Health Care Needs; Inspector General; and Office of
Information Technology. Each office has a director/manager and additional bureaucratic
layers beneath it, resulting in a hierarchically structured organization. Bates
(2000) saw problems in the NA he implemented for
All employees at MGO are given a computer and an account that allow them to connect to the state network and the Internet. Technology expertise levels of employees vary widely, but most employees are considered novice computer users. Employees are expected to use a variety of Microsoft applications, including Word, Excel, and Outlook. Employees use Outlook to access an email account which allows them to communicate with their clients, co-workers, supervisors, and other state agencies. Employee time sheets and travel vouchers are stored on Excel documents. Employees must also use specialized mainframe applications in order to enter and store client information into a database, file applications, and provide clients with paper documents. Microsoft Word and Adobe Photoshop documents are maintained on several hundred servers located throughout the state. Critical client data is backed up daily for employees if they choose to store data in the designated location. Employees use Internet Explorer to locate information associated with client services, such as travel directions to a client’s home and adoption information about a child.
Responsibility for training employees is assigned to consultants who primarily provide training for the following mainframe application databases: TWIST; KAMES; KASES; and FORMSGEN. This NA will focus on the following questions: To what extent does current technology training fail to prepare MGO employees to meet organizational goals? To what extent does current technology training fail to prepare MGO employees to provide required services to clients? To what extent does a lack of general computer knowledge, skills and attitude negatively affect services employees provide to the client?
One complaint that is associated with a NA is that it often develops a list of needs people think, or “feel”, will improve performance. This type of NA has had limited success in improving performance. Performance analyses (PA) are the preferred method used to improve performance, but they have not often been used in a large government organization such as MGO because of the costs incurred and time required in analyzing each of the hundreds of individual sub-offices. It is for that reason that we chose to use this hybrid model we refer to as a performance-driven training NA.
Methodology
Description
of Situation
A description of the MGO has been presented in the previous section and so we refer the reader to that area for information about the structure of this organization. We also refer the reader to our NA Proposal Outline, which appears in Appendix A. As a result of our first meeting we determined that this NA was commissioned to identify performance-improvement training needs for MGO employees which were related to computer use. We obtained client centered mission statements from MGO. The director of MGO related that informal observations by supervisors of virtually all departments in MGO indicated that computer training was needed. MGO wanted to discover if computer-related training needs affected services provided to clients now, or might affect client services in the future.
Analysis
Framework
In this performance-based
training NA we used a three phase plan outlined by Witkin and Altschuld (1995) to
assess needs. Performance-based issues can involve non-training issues but in
this NA we were only specifically assessing training issues. The plan consisted
of: Phase 1 – Preassessment; Phase 2 – Assessment; and Phase 3 –
Postassessment. In using this framework we conducted each of the three main phases
of this NA with two concurrent sub-phases: Phase A – Strategic NA; and Phase B
- Employee perception of performance improvement as a consequence of additional
training. Not only did Phases A and B run concurrently, but some of the
timelines for Phases 1, 2, and 3 overlapped. We based the design of this NA on
a study of
Methodology
and Procedures
In the Preassessment Phase we met with the director and staff of MGO in order to obtain agreements on the NA Proposal (Appendix A) concerning costs, time required to perform the NA, and commitments by MGO to use results generated by the NA. We also obtained existing data related to technology training needs. MGO requested a four-month time frame and had allotted a maximum of $50,000 to perform the NA. Time schedule is outlined in Appendix A. We agreed to the $50,000 fee and MGO agreed to the following revisions: 1. The timeframe was extended from 20 weeks to 26 weeks; 2. MGO supplied NA contractors with computers and email accounts connected to their network to facilitate gathering NA data; 3. MGO provided use of copying machines and conference rooms 4. MGO provided a vehicle from the motor pool for the duration of the NA; 5. MGO provided clearance to use an Information Technology (IT) call-tracking database for analysis of pre-existing data; 6. MGO allowed employees a suitable amount of overtime or compensatory time off work for time required to fill out questionnaires. This last concession helped to ensure completion of questionnaires and, consequently, reliability of data. Standards were set for identified goals and a plan to collect data was developed. Data collection instruments for managers and employees are in appendixes C, D, and E. These instruments were developed because there were none that fit the situation at MGO. The NA committee members (NAC) were introduced to the MGO staff and Assessment (Phase 2) began.
In the Assessment Phase, the scope of the NA focused on Protection and Permanency (P&P) and Family Support (FS) offices. Eighty percent of the employees who work directly with clients in MGO are either employed by the P&P office or the FS office. These two offices offer services to clients throughout the territory served by MGO. The territory is divided into six regional offices, each office having managers in charge of P&P and FS in that region. All hiring in the region is approved at the regional level. We determined that measurable gaps in computer-related expertise that negatively affected service to clients would best be measured by analyzing data associated with front-line workers and supervisors. Refer to Appendix B for an organizational chart for P&P and FS. All blocks listed in the chart are senior management positions.
In the Assessment Phase we gathered data about needs discussed in the Preassessment stage in relation to both Phase A and B, and prioritized the resulting needs list from data we gathered. In Phase A we used multiple data collection methods to measure performance which included supervisor/management focus groups, structured interviews, sampling of worker’s completed reports and forms, and work observation. Structured interviews were conducted with 25 randomly selected P&P unit managers and 25 randomly selected FS unit managers. Senior managers listed in Appendix B received results of structured interviews filled out by unit managers. Interviews were to aid in identifying high priority goals, performance indicators, and barriers. An example of the questions asked in the structured interviews is located in Appendix C. If a performance outcome could not be identified by the unit managers, the goal was not included on the list we forwarded to senior managers. Senior managers then prioritized the list into high, medium, and low priorities. Medium and low priority problems were removed from the list, and the list was forwarded to the MGO Director to ensure accuracy and prioritization placements met with the Cabinet mission and goals. Focus groups consisting of senior managers were assembled in order to gain a better understanding of the director’s prioritized goals, which gave us a better understanding of issues connected with items on the list. Samples of workers’ completed forms and work observations were used to triangulate issues that were identified from the data we had examined.
In Phase B we
relied chiefly on two surveys which were completed by approximately 1000 randomly
selected employees. We relied on surveys because of the high number of
questionnaires that we needed to send out. They included 20% (500 employees) of
P&P and 20% (500 employees) of FS workforce. Management encouraged
employees to participate by informing them that compensatory time off or overtime
pay would be allowed for the time it took to fill out the surveys. Surveys were
kept brief to focus on the topic. The first employee survey (Appendix D) asked
respondents to identify three to five job-related problems that negatively affect
their performance. They were also asked to identify training solutions and
non-training solutions for the problems listed. Data identifying problems that
required non-training solutions could still be used by MGO management, but was
not a focus of this NA. In the second questionnaire (Appendix E) a list containing
training solutions suggested by workers was developed from the initial
questionnaire. It required respondents of the previous questionnaire to select a
Likert style rating predicting how much their performance would improve if they
completed training listed.
In the Assessment Phase we prioritized problem/training lists which were developed by data collected during Phase A and B. Then we performed a causal analysis of all service provider training needs that impacted service receivers (now or in the future) to discover why they occurred (or will occur). We looked for contributing factors, whether they were facilities or the environment outside the system. We also examined the IT “Call Report” database mentioned at the beginning of this report. It allowed verification of computer-related perceived needs or issues related to problems reported in the questionnaires. We agree with Casey and Doverspike (2005) when they say that analyzing help desk calls placed to the IT department can provide an often-overlooked, fast, and low-cost method for performing or verifying the conclusions of a training NA in an organization. Verification of user problems was possible in this NA as a result of this rich data source. It provided quantitative and qualitative data during the Assessment Phase. Triangulation of data was achieved by researching the Call Report log for problem reports matching areas where training was indicated as necessary. A list of calls that computer users placed to the helpdesk allowed us to verify that training needs were impacting performance of employees, and is included in Appendix F. Only the most frequently occurring problems were listed. After we verified that no issues other than training contributed to our listed problems, we combined the results of the Phase A and B lists. The prioritized items on the B list had to be compatible with high priority goals, performance indicators, and barriers on the A list created by management, or we did not accept them. Using this methodology, the combination of these two lists resulted in a NA that provided verifiable data that performance was being affected as a result of needs associated with training. It also assured that needs that were listed were compatible with the overall goals of MGO.
As we began the Postassessment Phase we prioritized our needs based on: First, the cost to implement the training; Second, the magnitude of the gap between the current and desired status of training needs; Third, on the effect it would have on the system if we did not meet our needs. The list of prioritized needs is in Appendix G. Our findings reflect that training needs at MGO affect the service that is currently being provided to the clients, and that problems will only be exacerbated with time. Training in the following areas is advisable: 1. General (and basic) computer use; 2. Microsoft Outlook application; 3. Windows Explorer; 4. Internet Explorer.
As we said in our
opening Proposal (Appendix A), technology changes over the past fifteen years
have impacted virtually every type of organization. Mainframe training is
allotted 80% of the training budget at MGO. At the majority of companies across
the
Frontline workers are overburdened, and some carry client caseloads above the maximum limits set by MGO. Services provided to the customer are being delayed, and case files are occasionally lost. Both of these problems could be reduced if workers were better trained, but skilled workers are hard to find in today’s marketplace. There are four strategies used to develop technology skills in the workplace: hire and then train employees; transfer skilled workers where they are needed; relocate the work; or create an educational infrastructure within a community (Sleezer and Denny, 2004). Considering the pay scale of non-supervisory workers, transferring workers is an option that will have only limited success. Relocating work is not possible as offices are located in each community, where services are required. Creating an education infrastructure in the community is a long term solution which is already being addressed by primary, secondary, and post secondary training institutions. Hiring and providing training for employees in the four areas where training is needed is the best solution for MGO. Training in four required areas itemized in this NA could be delivered more cost-effectively online than by using face-to-face classes. MGO should also consider an option to create an in-house training staff to deliver training in these four areas. Online training is as effective as face-to-face classes, and can be delivered more economically. It solves geographic problems associated with travel, and can be delivered when you need it (DeRouin, Fritzsche, and Salas, 2004). There are many additional benefits to online training, and specific delivery variations are possible. The effectiveness of online training can readily be verified by quantifying problems reported in the existing Call Report system. The number of calls placed before and after training can easily be quantified for the specified categories. Return-On-Investment (ROI) is easy to calculate using this method, and additional training areas can also be identified.
Frequent meetings were arranged with top management, policymakers, and stakeholders throughout this NA in order to ensure success. Final reports were generated and presented verbally and in writing after completion of this NA in order to itemize the following: important aspects of the plan; major outcomes; priorities; action plan; and recommendations. The criteria that was used for the above itemizations were also presented, in accordance with recommendations made by Witkin and Altschuld (1995). We ensured that reports that were generated and presented were modified to ensure that they could be readily understood by the various audiences they were presented to. In analyzing this NA we took an objective look at the job we performed. We feel there are ways the NA process could be improved and take a look at lessons that we learned during the NA process in the next section.
Lessons Learned
There are several lessons that were learned in this NA. The culture of government organizations is a significant factor in a NA that targets performance improvement. While no government organization will claim that performance improvement is not an important consideration for top management, the fact is that the culture of public organizations are more focused on complying with regulations and mandates than in improving performance. The following lessons could be considered critical areas where additional focus should be placed in a performance-based NA that is carried out for a large public organization.
First: More focus
should have been placed on the Memorandum of Agreement. The purpose of this
memorandum “attests to the commitment of top management to support the NA and
to follow through with the use of its findings” (Witkin and Altschuld, 1995,
p.28). I would ensure that the spirit of the
Second: Public sector government and private sector organizations often have missions that are different. I would ensure that performance-based measurements in this training NA were closely aligned with the mission of MGO. Although this training NA is necessarily focused on service providers, the results of this training should ultimately be tightly aligned with the mission statement, which relates to their clients, the service receivers.
Third, more time and money to conduct the NA would improve the process. The hybrid design of this performance based training NA is unusual. The design was a result of an effort to reduce the costs associated with conducting multiple performance analyses for workgroups within MGO and, at the same time, maintain a level of performance-based measurements. There are few NAs of this type that have been that have been carried out, and fewer yet with large government organizations such as MGO. Additional time and money to carry out the NA would improve the process.
Fourth: In applying performance-based measurements to this NA we have “gone against the grain” of organizational culture within MGO. The culture of a hierarchically structured public sector organization such as MGO is completely different than a private organization that has a more flat organizational structure. A culture survey could have been used in a NA where focus was on performance improvement (Swanson, 1996). It is extremely difficult to change the cultural identity of any organization (Allen and Thatcher, 1995; McManus, 2003; Sims, 2005; Wilkins and Bristow, 1987). Although it was anticipated that there would be significant resistance to change in thought and action required for performance improvement, the level of resistance was not fully appreciated at the outset. Had this been considered more fully, development of specific plans could have been created to address these issues. Some sort of organizational change training could have been included in the initial proposal that would have assisted management in implementing the necessary processes. For example, twelve principles of organizational transformation were highlighted in an article by Trahant, Burke, and Koonce (1997). Some of these could have been included in change management training given to MGO employees. The principles are as follows: understand the external environment; obtain employee commitment to support the mission; steady and consistent leadership; a change in organizational culture requires change in employee behavior; service receivers determine organizational structure; support employees in their work; managers must assist (nor prohibit) employees from improving the process; teamwork; productive employees; ensure employee skill levels match talent levels; consider the human factor in employee relations; beware of false indicators of success. These lessons learned will be helpful in planning projects with MGO in the future. Additional work will need to be done to change the organizational culture of MGO, which is beyond the scope of this project.
References
Baton Rough, LA:
Ethics and Integrity. Retrieved March 15, 2007, from http://www.ahrd.org/publications/
ethics/ethics_standards.PDF
Allen, R. & Thatcher, J. (1995).
Achieving cultural change: A practical case study. Leadership
and Organizational Development Journal, 16(2), 16-23.
Bates, R. A. (2000). Large-scale performance driven training needs assessment: A case study.
Public Personnel Management. Retrieved March 9, 2007, from http:// www.allbusiness.com/human-resources/ employee-development/580932-1.html
Casey, M. S. & Doverspike, D. (2005). Training needs analysis and evaluation for new
technologies
through the use of problem-based inquiry. Performance
Improvement
Quarterly, 18(1), 110-124.
Czaja, S. J. (1997). Computer
technology and the older adult. In
and P. Prauhu (Eds.), Handbook of human-computer interaction (2nd ed.; pp. 797-812).
DeRouin, R. E., Fritzsche, B. A., & Salas, E. (2004). Optimizing e-learning: Research-based
guidelines for learner-controlled training. Human Resource Management, 43(2 & 3), 147-
162.
training in organizations.
McManus, K. (2003). The challenge of changing culture. Industrial Engineer, 35(1), 18.
Ryan, N. R., Whitener, E. M., & Philcox, H. H. (1995). The assessment of end-user training
needs. Association of Computing Machinery. Communications of the ACM, 38(7), 27-38.
Sims, R. R. (2005). Changing an
organizations culture under new leadership. Journal
of Business
Ethics, 25(1), 65-78.
Sleezer, C. M. & Denny, D. (2004). Strategies for developing a high-skilled workforce.
Performance Improvement Quarterly, 17(1), 41-55.
Swanson, R. A. (1996). Analysis for improving performance: Tools for
diagnosing organizations
&
documenting workplace experiences.
Trahant, B., Burke, W. W., & Koonce, R. (1997). 12 principles of organizational
transformation. Management Review, 86(8), 17-21.
Wilkins, A. L. & Bristow, N. J. (1987). For successful organizational culture, honor your past.
The
Witkin, B. R. & Altschuld, J.
W. (1995). Planning and conducting needs
assessments: A
practical guide.
Appendix A - Needs Assessment Outline Proposal
It
is crucial that organizations have skilled workforces in order to compete in
today’s marketplace and efficiently provide product to clients. Technology
changes over the past fifteen years have impacted virtually every type of
organization. “The major purpose of a needs assessment (NA) is to gather information
for setting priorities on needs of people in relation to a system of interest”
(Witkin & Altschuld, 1995, p. 18). This NA Proposal will examine a Major
Government Organization (MGO) to determine whether additional technology
training could improve job performance of employees (service providers) in
delivering services to the citizens of
Government organizations tend to be hierarchically structured groups with a bureaucratic chain of command. Services are provided throughout a geographic area that falls within the territorial jurisdiction of the government organization. MGO employs 35,000 workers who provide services which are available to over 4 million people within a 40,385 square mile geographical boundary. Approximately 80% of employees are female and range in age from 18 years to 65 years of age. Education level varies widely, and all workers are required to use a computer.
Training is available for new employees on selected mainframe software applications and for all employees who use this software when it is revised. MGO uses training contractors who provide specified application training to all employees who are required to use it. Contractors provide training on five specific mainframe software applications. An individual worker would typically use only one or two of the specified mainframe applications, so all employees are not trained on every mainframe application. Training on Microsoft applications is not provided to employees of MGO. Approximately 97% of all work performed by the MGO employees involve mainframe applications, Microsoft Word, Microsoft Excel, Microsoft Outlook, or Adobe documents. Training records are maintained by MGO. Some articles covering common technology related questions are published on an MGO Intranet website listing created and maintained by technical support personnel within MGO.
In a study of a Louisiana government organization (Bates, 2000), cost and time were cited as important factors in rejecting more typical methodologies to assess gaps between the current and desired state of performance in an organization in favor of a hybrid approach. In the Bates study involving 72,000 government workers, a performance-based training needs assessment was selected. The MGO proposal will closely mirror this strategy by conducting two concurrent phases while using the Three Phase Plan highlighted by Witkin and Altschuld (1995): Phase A consists of a strategic needs assessment; and Phase B assesses employee perception of performance improvement as a consequence of additional training. Acceptance of performance-based training is more difficult in government organizations than in many private organizations due to environmental differences.
The time frame for the initial planning stage to delivery of analyzed results if this NA is expected to be four months: Week 1-4: Create and modify questionnaires, determination of data sources; Week 5-9: Identify recipients of questionnaires surveys and interviews; determine sources for additional data (existing databases, etc.); Week 10-14: Delivery, completion, and return of surveys; telephone interviews to validate findings or obtain additional qualitative information; Week 15-19: Analyses of feedback; Week 20: Generation and delivery of report. Three interim reports will be delivered at Weeks 5, 10, and 15. Some of the timelines above could overlap, but the proposed project completion date will be maintained.
References
Bates, R. A. (2000). Large-scale performance driven training needs assessment: A case study.
Public Personnel Management, Summer 2000. Retrieved March 9, 2007, from http:// www.allbusiness.com/human-resources/ employee-development/580932-1.html
Witkin, B. R., & Altschuld, J.
W. (1995). Planning and conducting needs
assessments: A
practical guide.
Appendix B – Senior Managers
P&P and FS Organizational Chart
Appendix C – Phase A: Senior Management Survey
High Priority Goals, Performance Indicators, and Barriers
Please use a separate sheet for each performance area listed in Question 3
- What is the mission of your work unit?
- What are the key goals and objectives that support this mission? (Prioritize)
- Name some areas within your work unit that has the greatest performance improvement potential.
- Why do you choose these?
- What are the goals/objectives for this area within your workgroup?
- What are some major process steps that are taken in this area?
- What are some major outputs of this area?
- What are some indicators of performance problems in this area?
- How do you know performance in this area is not optimum?
- What are some obstacles to performance in this area?
- Are there system problems that impede unit performance?
- Are there system problems that impede individual performance?
- How does the organization support performance in this area?
- Could organizational support be better?
- How do work processes support performance in this area.
- Would training solve the issue?
- What makes you think the problem can be solved with additional knowledge, skills, or attitudes.
- How did this training need come to exist?
- If training is provided, are most employees capable of learning?
- Are most employees motivated to learn?
- Are there solutions of a non-training nature that would help solve the problem?
s. How could improving performance in this area move the agency closer to its goal?
t. How can this performance improvement be measured?
Appendix D – Phase B: First Employee Survey
Job Related Problems – Solutions
Please list three (3) to five (5) of the biggest problems you face in your job that prevents you from being as productive as you can be. Rank them in order of importance.
- ____________________________________________
- ____________________________________________
- ____________________________________________
- ____________________________________________
- ____________________________________________
Please list training solutions (if there is one) for each problem you listed.
- ____________________________________________
- ____________________________________________
- ____________________________________________
- ____________________________________________
- ____________________________________________
Please list any non-training solutions for each problem you listed.
- ____________________________________________
- ____________________________________________
- ____________________________________________
- ____________________________________________
- ____________________________________________
Appendix E – Phase B: Second Employee Survey
Appendix F – Training Needs / Performance Issues Causality Analysis
Compilation of the Call Report Log - highest frequency (monthly average) calls on top
- Can’t logon (see item 11)
- Failure to organize personal folders
- Connecting to the Internet
- Performing basic computer file management tasks
- Backing up personal folders
- Storing data on the desktop
- Failure to reboot computer for slowness issues
- Understanding how to reboot the computer
- Understanding the concept of a network drive
- Understanding how to use Favorites inside Internet Explorer
- Lack of understanding how to change passwords (See item 1)
- Failure to clean out the Outlook mailbox
- Differences between turning a computer off and logging off
Appendix G – Final List of Prioritized Training Needs
- General Computer Training – Logging on, restarting, difference between a computer and a monitor, cleaning off the desktop
- Microsoft Outlook – Personal Folders, sending, receiving, storing, backing up, attaching files, file types, etc.
- Windows Explorer - How to copy, backup, and browse files, and use keyboard shortcuts. File management and manipulation
- Internet Explorer – Favorites, browsing Internet, difference between Internet and Intranet