What is an in-basket exercise?
An in-basket exercise assesses a candidate's ability to perform a manager’s
job from an administrative perspective. In the exercise, the candidate is
confronted with issues and problems that have accumulated in the manager’s
“in-basket” after returning to work from an extended absence. A sample of
in-basket items might include memos, correspondence, e-mails, directives,
requests, reports, forms, messages, minutes, hand-written notes, etc., from
management, supervisors, staff members, inmates, and other stakeholders. The
candidate's task is to review the in-basket items and then take action on these
varied issues and problems using action forms to record notes, comments, and
responses. These actions are then assessed and rated based on job related
competencies through a formal question and answer session by a group of trained
raters. Standardized criteria and predefined rating scales are used to assess
the candidate.
Return to page top
What does an in-basket exercise measure?
The in-basket exercise measures administrative skills that are critical for
effective performance in supervisory and managerial jobs.
The in-basket is a standardized, behaviorally based exercise. Multiple
trained raters assess and score candidates according to defined competencies of
performance specific to the in-basket exercise. After each candidate completes
the in-basket exercise, a group of trained raters conduct a question and answer
session with the candidate to assess performance on the exercise using a set of
predefined rating scales based on job related competencies (e.g., planning and
evaluating, problem solving and decision making, etc.). The end product of the
in-basket exercise is that each candidate receives a combined score from
multiple raters based on standardized criteria.
The in-basket exercise is designed to test the candidate’s adaptive thinking,
problem analysis, judgment, administrative abilities, planning, organizing,
delegating, and integrative skills while under pressure dealing with memos,
e-mails, requests, messages, handwritten notes, etc.
Return to page top
What competencies are measured in the in-basket
exercise?
Like all tests, the in-basket exercise measure certain attributes or
qualities. These attributes or qualities are commonly referred to as
competencies.
To provide an example, all competencies have both a title (e.g., oral
communication) and a definition (e.g., ability to communicate orally in clear,
understandable, and polite tones at the level of the listener giving clear
instructions and detailed information). The definition will list behaviors
appropriate for that competency based on a job analysis. Some examples of
common oral communication behaviors expected are listed as follows: clear and
easy to understand, speaking at the level of the listener; fluent and
articulate, without frequent pauses; spoke at an appropriate rate (not too fast
or slow); information was in a logical order; volume of voice was appropriate,
and used appropriate inflection to emphasize main points, etc.
In any given in-basket exercise, various definitions may be combined or
modified. While the exact competencies will vary, the following competencies
will be tested in the in-basket exercise:
- Planning and Evaluating:
Organizes work, sets priorities, determines
resource requirements; determines objectives and strategies to achieve them;
monitors and evaluates progress against goals.
- Problem Solving and Decision Making (judgment): Identifies problems;
gathers, interprets, and evaluates information to determine its accuracy and
relevance; generates and evaluates alternatives; makes sound and well-informed
decisions; and commits to action to accomplish organizational goals.
- Managing and Organizing Information: Identifies need for information;
analyzes and interprets data in complex situations involving conflicting
demands, needs, and priorities; determines its importance, accuracy, and
communicates it by a variety of methods.
- Self-Management (initiative): Show initiatives; sets well-defined
and realistic personal goals; monitors progress and is motivated to achieve;
manages own time and deals with stress effectively.
Return to page top
How is an in-basket exercise developed?
Development of an in-basket exercise is a very expensive and time consuming
process. The following is a typical way an in-basket exercise is developed
- Identify critical job tasks and competencies
- Job analysis
- Training Bureau’s core
competencies
- Job code specifications
(state job descriptions identifying knowledge, skills, and abilities)
- ADC position description
questionnaires (PDQs)
- Subject matter expert (SME)
development sessions, e.g., input, discussions, data collection, item
development, review, feedback, etc.
- Collect source materials
- Variety of documents from
all levels and locations
- Related to competencies:
problem solving, planning, evaluating, delegating, etc.
- Kinds of information to
collect: memos, reports, messages, e-mails, calendars, forms, organizational
charts, policies, news articles, plans, staff schedules, training schedules,
requests, flyers, etc.
- Develop/write the scenario
- Information about
position, issues, timeframes, etc.
- Set-up organizational
charts for use in the in-basket
- Calendars for scheduling,
planning, etc.
- Develop document pool
- Reports, e-mails, memos,
incident reports, phone messages, etc.
- Information from multiple
sources (all levels and locations)
- Representative sample of
issues dealt with across locations
- Ideas about what is the
same, what is different across the job class (by level, by geography, etc.)
- Actual documents related
to real events that demonstrate the competencies
- Participation from the
field subject matter experts
- Conduct first SME session
- Introduction to the
in-basket
- Competency review
- Individual review of
scenario and documents
- Individual evaluation of
scenario and documents
- Group review and
evaluation of documents
- Solicitation of
competency-based responses to documents
- Select documents and prepare item development plan
- Revise documents based on
SME consensus and group discussion
- Revisions consistent with
the evaluation criteria (more job-related, clearer, more accurate, formatted
correctly, etc.)
- Responses now can be
focused on competency-based issues
- “Clean”
documents--formatted appropriately, technically correct, and job-related
- Competency-based reactions
to each document provides justification for writing items, response options,
and justification for each
- Write items and conduct internal technical review
- Prepare item development
plan (e.g., item writing, items linked to competencies, etc.)
- Write items (questions)
- Conduct internal technical
review (e.g., some items deal with multiple documents, equal number of items
across competencies, etc.)
- Conduct second SME session
- Review competencies and
definitions
- Review in-basket scenario
and documents
- Individual and group
evaluation of scenario and documents
- Review of questions,
response options, and justifications
- Evaluate items (questions)
- Determine or verify key(s)
for items
- Suggest revisions to
items/documents as needed
- Review assessment
materials (e.g., applicant instructions, etc.)
- Pre-test the in-basket exercise
- Plan and schedule raters
and test subjects
- Conduct an actual
in-basket simulation
- Record results and
feedback
- Refine and conduct final review
- Adjust and refine in-basket exercise
- Conduct final review
- Proofread in-basket
materials
- Print in-basket exercise
- Prepare test documentation file
- Prepare and document
in-basket development process
|
