|
Definitions
Including
below:
• The
Research Continuum (with pyramid image)
• Scientifically
Based Research
• Definitions
for Use with Review of Research
• The
Research Continuum
 The
Content Network Teams reviewed the research articles based on a continuum
that measured the research according to specific criteria. The pyramid
image and the comments that follow identify these criteria.
LEVEL
5—Gold Standard
For
the purposes of the Content Network reviews, the No Child Left Behind
criteria for quality research represents the "gold standard"
or best quality research. These criteria correspond to the top of the
pyramid. Research designs most likely to produce "Level 5"
results are research designs that randomly assign subjects to treatment
and control groups. They also provide control for most threats to internal
validity and yield findings that generate the greatest confidence in
student effect.
LEVEL
4—Strong Evidence
Research
designs most likely to produce "Level 4" results do not randomly
assign subjects to treatment and control groups, however other design
elements control many of the threats to internal validity.
LEVEL
3—Promising Studies
Research
can be classified as "promising" under certain conditions.
First, if a research design is weak but findings have been consistent
across multiple replications, the treatment under study can be said
to have promise. Second, if single strategies that have been studied
under true experimental conditions are combined with multiple strategies,
practices, and routines and the composite then studied with a weaker
design with positive results, the findings can be classified as promising.
LEVEL
2—Marginal
One-time
case studies clearly fit into the marginal category. Many of the classroom
experiments conducted by individual teachers in their classrooms and
reported in popular, but non-peer reviewed, journals fit this classification.
This is not to suggest that such reports are worthless, but rather that
they fail to control for any of the competing hypotheses that might
account for changes in the dependent variable.
LEVEL
1—No Empirical Evidence
Two
types of reports of successful innovations are common in the educational
literature on curriculum and instruction, neither of which provides
credible evidence that an innovation would consistently result in benefits
to students.
The
first is "advocacy" writing—articles that passionately
espouse specific curriculum content or an approach to teaching. These
articles often provide extensive rationales for why teachers and schools
should adopt specific practices but provide no data to document the
effects on students with whom these practices have been employed. In
some cases, claims of significant growth are made for specific practices,
again with no documentation.
A
second type of report frequently encountered in educational publications
are those that claim significant gains in student achievement for schools
of districts or states. At first glance, these reports appear to provide
empirical evidence because they present test scores that show rising
scores on a test over a period of years. Further reading of the report,
however, reveals that the treatment was a high-stakes test or a system
of rewards and sanctions for high and low performing schools. We are
left with no idea of what may have occurred differently in classrooms
with students that might account for changes in student achievement.
Finally,
testimonials and anecdotes frequently provided by publishers of educational
materials to attest to the efficacy of their products do not meet the
criteria for evidence that is commonly expected in scientifically based
research.
• Scientifically
Based Research
The
term ‘scientifically based research' —
(A) means research that involves the application
of rigorous, systematic, and objective procedures to obtain reliable and
valid knowledge relevant to education activities and programs; and
(B) includes research that —
(i) employs systematic, empirical methods
that draw on observation or experiment;(ii)
involves rigorous data analyses that are adequate to test the stated
hypotheses and justify the general conclusions drawn;
(iii) relies on measurements or observational
methods that provide reliable and valid data across evaluators and
observers, across multiple measurements and observations, and across
studies by the same or different investigators;
(iv) is evaluated using experimental
or quasi-experimental designs in which individuals, entities, programs,
or activities are assigned to different conditions and with appropriate
controls to evaluate the effects of the condition of interest, with
a preference for random-assignment experiments, or other designs to
the extent that those designs contain within-condition or across-condition
controls;
(v) ensures that experimental studies
are presented in sufficient detail and clarity to allow for replication
or, at a minimum, offer the opportunity to build systematically on
their findings; and
(vi) has been accepted by a peer-reviewed
journal or approved by a panel of independent experts through a comparably
rigorous, objective, and scientific review. No Child Left Behind
Act: Title IX — General Provisions: Part A —Definitions
Sec. 9101.
• Definitions
for Use with Review of Research
Program
– a compilation of strategies, practices and routines which are implemented
as a whole. Exact proportions of various strategies, practices and routines
are often specified in an "ideal" implementation of the program.
Examples
of Programs:
Success
for All, an elementary reading program, specifies strategies for
teaching phonics and for one-on-one tutoring, and practices such as
grouping for instruction. In addition, staff development content and
process, assessment procedures, and the monitoring of implementation
are spelled out in detail.
Cognitive
Tutors for Algebra and Geometry, secondary math programs, specify
cooperative strategies for use in teacher-directed classroom instruction,
specialized software programs for individualized instruction of students,
assessment instruments, proportions of time students are to be in
teacher directed instruction and computer-assisted instruction and
staff development for teachers.
Strategy
or Model – approaches to instruction designed around a theoretical
base of how students learn. A strategy or model of instruction combines
a series of skills and practices in a specific sequence.
Examples
of Strategies/Models:
The
Inductive Thinking model is based on a theory of human information
processing, namely that the examination of data, the classification
of data based on observed similarities and the forming of generalizations
based on multiple observations is a natural human activity and the
foundation of all higher order thinking operations. The model has
been applied to the design of curriculums in math and science and
to instructional objectives in reading, science, social studies and
math. Typically, the sequence of events in an inductive thinking model
includes: 1) The teacher presents a data set; 2) students study items
in a data set, identifying critical attributes of items; 3) students
classify items in data set by common attributes; 4) students name
categories; 5) students examine relationships between and among categories;
6) students form generalizations and apply to problem-solving situations.
The
Link-Word Mnemonic Strategy is based on theories of information
storage and retrieval developed by cognitive psychologists. Major
applications of this theory in schools is reported in the research
by Pressley and Levin. The sequence of teaching skills in this strategy
include:
Practices/Routines/Skills:
Practices, routines, and skills are discrete behaviors and procedures
employed by teachers during the course of planning, organizing for, and
conducting instruction.
Examples
of Practices include: Grouping of students by ability for instruction,
grouping students for cooperative learning tasks, and the assignment
and checking of homework.
Examples
of Routines include: Rules for student behavior and consequences
for violations of classroom rules, order of instructional activities
during a period or school day, make-up of missed work when students
are absent, etc.
Examples
of skills include: For teachers, the ability to ask questions
of varying cognitive complexity, the ability to appropriately reinforce
students for desired behavior; for students, the ability to locate
a specific word in the dictionary or textbook glossary, the ability
to identify structural components of words (e.g., prefixes, suffixes).
|
