Comments by Monica B. Mitchell

QEM/TPC Conference

December 2, 2005

Three general areas that influenced the competitiveness of TPC proposals in competitions to date.

Research Emphasis

As a research program, successful proposals fit with the goals and objectives of the TPC program. Proposals demonstrating acceptable standards for educational research and expectations outlined in the TPC solicitation were competitive.

Competitive category A proposals clearly articulated the nature and conduct of the research including research questions and/or hypotheses, and research methods that allow direct investigation of the research questions. The most serious limitation common to Full Scale Research Proposals (subcategory 2) is a focus on implementation as the primary activity or exclusive activity instead of conducting research. Oftentimes implementation may need to be an aspect of the study, but it shouldn’t be the primary activity. Implementation for TPC projects should be in the service of the research. Research must drive the activity. Subcategory 3 proposals, Research on Models, ac common limitation is the nature of the research doesn’t go beyond evaluation. Questions proposed to examine the intervention model are limited to:

Is the intervention Successful?

Did teacher understanding of content increase?

What changes occurred in classroom practice?

What was the impact on student learning?

Questions of interest to the TPC program focus on how the intervention was successful or unsuccessful; under what conditions; what is the nature of teacher learning; what contributed to change in teacher learning and classroom practice, and how.

Competitive proposals in category B demonstrated a research-basis for the needs, focus and nature of the materials. The existing research establishes a need for the materials as well as the content and nature. The pedagogy of the materials are warranted by and grounded in prevailing research. In addition, the materials development process in competitive proposals is rigorous and robust – an iterative process involving pilot- and field-testing, formative and summative evaluation, and content review. In category B, evaluation is considered critical. Unsuccessful proposals incorporated evaluation as an add-on instead of an integral project component. A project might want to develop materials that impact teacher knowledge and classroom practice, yet the evaluation concentrates on teacher beliefs and teacher confidence. Or the evaluation might simply state, “data will be collected and analyzed to assess the effectiveness of the materials.” When the evaluation is boiler-plate or completely divorced form the goals and objectives of the materials, the credibility of the project is significantly diminished. We suggest that you identify an evaluator early in the planning process so the evaluation can be aligned and respond to the purposes of the project effectively.

Finally, when a proposal is vague and lacks specificity, regardless of category, it is difficult to assess whether the proposal actually meets acceptable standards of research and expectations in the guidelines. A common mistake is that the “what” is described with limited details on the “how.”

Level of Impact

TPC intends to advance the knowledge base related to the K-12 STEM teacher continuum, preservice through inservice. As such, both categories are expected to address a recognized national need meaning it is timely, current, and established in the literature as a national need. For category A, although the phenomenon under investigation may be local in setting, the resulting finding must address a recognized national need, must be relevant to other setting and contexts. For category B, the materials can’t be limited to the needs of a local population but must have relevance to a broad audience, a national market that will have an interest in using the materials.

Level of Collaboration

The role of collaboration influences the competitiveness of TPC proposals. Competitive proposals integrate collaboration at a substantive level, beyond simply a national advisory board. TPC expects education faculty as well as STEM faculty to provide project leadership; practitioners are expected to be actively involved beyond mere subjects of the research of the pilot- and field-testing, but there should be a role for practitioners in the research. Practitioners certainly are not researchers, but they bring an important perspective to framing research questions that can inform practice; they bring an important perspective to contribute to instrument development and the materials development process. Many category B projects have teachers on the writing team, not just the pilot- and field-testing. Practitioners have a role to play in warranting interpretation of research findings and contributing to robust dissemination especially to the practitioner community. Without partnering with schools and school districts, educational research is difficult to accomplish.