|
Professional Development Workshops
August 21-26, 2009
Presenter Bios
Click on the names and photos below to learn more about our presenters.
Register for On-Site Workshops in Beautiful Claremont, CA
or
Register for the Online Versions of Our Workshops (from Anywhere in the World)
Workshop Descriptions
Friday, August 21, 2009
|
|
 
Stewart I. Donaldson & Christina A. Christie
Basics of Evaluation & Applied Research Methods
Available in person or via webcast.
This workshop will provide participants with an overview of the core concepts in evaluation and applied research methods. Key topics will include the various uses, purposes, and benefits of conducting evaluations and applied research, basics of validity and design sensitivity, strengths and weaknesses of a variety of common applied research methods, and the basics of program, policy, and personnel evaluation. In addition, participants will be introduced to a range of popular evaluation approaches including the transdisciplinary approach, program theory-driven evaluation science, experimental and quasi-experimental evaluations, empowerment evaluation, fourth generation evaluation, inclusive evaluation, utilization-focused evaluation, and realist evaluation. This workshop is intended to provide participants with a solid introduction, overview, or refresher on the latest developments in evaluation and applied research, and to prepare participants for intermediate and advanced level workshops in the series. Students are strongly encouraged to purchase and read in advance of the workshop:
Copies are available from Amazon.com by following the links above.
Questions regarding this workshop may be addressed to Stewart.Donaldson@cgu.edu.
|
|
 Allen M. Omoto
Writing Successful Grant Proposals
This workshop covers some of the essential skills and strategies needed to prepare successful grant applications for education, research, and/or program funding. It will provide participants with tools to help them conceptualize and plan research or program grants, offer ideas about where to seek funding, and provide suggestions for writing and submitting applications. Some of the topics covered in the workshop include strategies for identifying appropriate sources of funding, the components and preparation of grant proposals, and the peer review process. In addition, topics related to putting together a research or program team, constructing an appropriate budget, grants management, and the writing of an application will be discussed. The workshop is organized around key questions relating to grant support and how to become a successful grant-getter, including WHY seek grant funding or support? WHERE to look for support? WHO applies for funding and WHEN should one seek funding? WHAT is submitted in a grant application? And, HOW to structure an application and supporting materials? The workshop is intended primarily as an introduction to grant writing, and will be most useful for new or relatively inexperienced grant writers. Workshop participants are encouraged to bring their own "works in progress" for comment and sharing. At its conclusion, workshop participants should be well positioned not only to read and evaluate grant applications, but to assist with the preparation of applications and to prepare and submit their own applications to support education, research, or program planning and development activities.
Questions regarding this workshop may be addressed to allen.omoto@cgu.edu.
|
|
 Thomas Chapel
Logic Models for Program Evaluation and Planning
Many programs fail to start with a clear description of the program and its intended outcomes, undermining both program planning and evaluation efforts. The logic model, as a map of what a program is and intends to do, is a useful tool for clarifying objectives, improving the relationship between activities and those objectives, and developing and integrating evaluation plans and strategic plans. First, we will recapture the utility of program logic modeling as a simple discipline, using cases in public health and human services to explore the steps for constructing, refining and validating models. Then, we’ll examine how to use logic models effectively in some key program evaluation steps, including gaining stakeholder consensus and determining an evaluation focus. Finally, we'll present, in an overview fashion, some ways in which logic models can help in selecting performance measures and in defining program mission/vision and priority areas for strategic planning. Course presentations are supplemented by extensive small group case study work.
Questions regarding this workshop may be addressed to tchapel@cdc.gov.
|
Saturday, August 22, 2009
|
|
 William D. Crano
Quasi-Experimental Methods
Many introductory courses on research design start and end with the randomized experiment. This is a good beginning, but knowledge of "pure" experimental designs does not prepare researchers for the messier and more complicated (and often, more interesting) state of affairs commonly found in applied contexts. Just as important as knowledge of experimental designs is a thorough understanding of quasi-experimentation design and analysis. Though they often do not support causal explanations, "quasi" designs can provide evidence that may markedly reduce the range of possible explanations for an intervention or treatment. This workshop is designed to impart an understanding of quasi-experimental design and analysis.
The workshop begins with a consideration of randomized experiments, strengths and weaknesses, with particular emphasis on Campbell and Stanley’s threats to internal and external validity. Considerations of reliability validity will be an important feature of the beginning sections of our work. From here, we move to the land of the quasi-experiment. In this largest section of the workshop, we will consider designs that lack random assignment (a central feature of quasi-experiments), designs that lack a control group or pretest (or both), those that use one or another of these design features. Along the way, we will consider the dangers of matching groups to create initial equivalence, and the potential utility of the new propensity scoring approaches. After considering case control designs and analysis, we move to time series designs, with special emphasis on the use of interrupted time series studies. If time permits, we then consider regression/discontinuity designs. Analytic models useful for each of these design approaches will be discussed, but our emphasis is on design, not statistics.
Questions about this workshop should be addressed to william.crano@cgu.edu.
|
|
 Huey T. Chen & Stewart I. Donaldson
Practical Program Evaluation: A Program Theory Approach
Available in person or via webcast.
This workshop will focus on how to design practical and cost-effective program evaluations. A step-by-step approach will be used to teach participants how to conduct accurate and useful program evaluations. Key topics will include engaging stakeholders, formulating and prioritizing evaluation questions, needs assessment, expressing and assessing program theory, program implementation, managing resistance and excessive evaluation anxiety, formative evaluation and continuous improvement feedback, determining program outcomes/impact, and cost-benefit and cost-effectiveness analysis.
Students are strongly encouraged to purchase and read in advance of the workshop:
Program Theory-Driven Evaluation Science: Strategies and Applications (2007) by Stewart I. Donaldson
What Counts as Credible Evidence in Applied Research and Evaluation Practice? (2008) by Stewart I. Donaldson, Christina A, Christie, & Melvin M. Mark
Practical Program Evaluation: Assessing and Improving Planning, Implementation, and Effectiveness (2005) by Huey T. Chen
Copies are available from Amazon.com by following the links above.
Questions regarding this workshop may be addressed to Stewart.Donaldson@cgu.edu.
|
|
 Michelle C. Bligh
Introduction to Qualitative Research Methods
This workshop is designed to introduce you to different types of qualitative research methods, with a particular emphasis on how they can be used in applied research and evaluation. Although you will be introduced to several of the theoretical paradigms that underlie the specific methods that we will cover, the primary emphasis will be on how you can utilize different methods in applied research and consulting settings. We will explore the appropriate application of various techniques, and review the strengths and limitations associated with each. In addition, you will be given the opportunity to gain experience in the use of several different methods. Overall, the workshop is intended to provide you with the basic skills needed to choose an appropriate method for a given project, as well as primary considerations in conducting qualitative research. Topics covered will include field observation, content analysis, interviewing, document analysis, and focus groups.
Questions regarding this workshop may be addressed to michelle.bligh@cgu.edu.
|
Sunday, August 23, 2009
|
|
 Michael Quinn Patton
Utilization-Focused Evaluation
Available in person or via webcast.
This training is aimed at building capacity to think strategically about evaluation and increase commitment to conducting high quality and useful evaluations. The evaluation standards call for evaluations to be useful, practical, accurate and ethical. Utilization-Focused Evaluation is a process that meets these criteria and promotes use of evaluation from beginning to end. By carefully implementing evaluations for increased utility, this approach encourages situational responsiveness, adaptability and creativity.
Participants will learn:
• Key factors in doing useful evaluations, common barriers to use, and how to overcome those barriers.
• Implications of focusing an evaluation on intended use by intended users.
• Options for evaluation design and methods based on situational responsiveness, adaptability and creativity.
• Ways of building evaluation into the programming process to increase use.
This workshop will focus on learning to match an evaluation to the nature of the project and situation that are the focus of the evaluation. The field of evaluation has developed in diverse directions such that a rich variety of contrasting models, competing purposes, alternatives methods, and divergent techniques are available. Likewise, projects and organizational innovations vary along many dimensions: innovativeness, comprehensiveness, complexity, boundedness, integration, riskiness, and scope, to name but a few. The challenge, then, is to match evaluation to the nature of the initiative being evaluated. This means that evaluation is contingent on the intended users and intended users of the evaluation in the context of situational responsiveness with the goal of methodological appropriateness. An appropriate match should result in an evaluation that is useful, practical, accurate, and ethical.
For questions about this workshop, please email MQPatton@Prodigy.net.
|
|
Dale E. Berger
Introduction to Applied Quantitative Analysis
Descriptive and inferential statistics are essential for evaluating programs and for using data to guide policy decisions. This introductory/refresher course will focus on practical approaches to data analysis. Topics include first looks at data (preparing data for analysis, detecting problems, dealing with unusual data, visualizing patterns in data), selecting appropriate methods of analysis for the research questions of interest (treating your data properly, providing appropriate models, relating analyses to the motivating research questions, exploring data for unanticipated relationships, understanding limitations), and presenting your findings (interpreting results, describing results for a non-technical audience). Participants are invited to bring research questions to be used as case studies. We will discuss alternative approaches to analysis with a view toward providing a conceptual understanding of basic and advanced statistical techniques that may be used with consultants, in funding applications, and in reports. Prerequisites: None, although some familiarity with basic statistics will be helpful.
Questions regarding this workshop may be addressed to dale.berger@cgu.edu.
|
|
 Susan Kistler
Evaluation 2.0: A Hands-on Exploration of How to Measure Connected Communications and Online Identity
Are you working with programs that have started a blog, used RSS feeds, updated their website, or employed social media to spread the word about their services, increase name recognition, change behaviors, or build community? Have you been asked to evaluate connected communications or online identity but aren’t sure where to start? This workshop will examine what to measure, how to measure it, and what tools are available to assist you. In the morning, we’ll be in a traditional classroom, establishing a common understanding of the ways in which new media has changed the way we communicate and the implications for evaluation. We’ll discuss issues of feasibility, reliability, and the application and adaptation of tried and true techniques to measure change in this rapidly evolving arena. You’ll have an opportunity to explore a case study and then apply what’s learned to examples from your own work or personal experience.
During the afternoon, we’ll be in the computer lab, demonstrating in real time specific examples of how to choose and use the tools that best match your data collection, analysis, and reporting needs. We’ll provide multiple examples of how to source data, determine baselines, track change over time, and apply both qualitative and quantitative techniques to measure outcomes. Finally, we’ll examine the range of tools available and provide specific examples of how to choose those that best match your data collection, analysis, and reporting needs. You’ll have time for guided exploration of each tool and assessment of its application to your work. Throughout the workshop, you will have an opportunity to raise questions, and share your lessons learned. You will leave with multiple take-home resources including examples, checklists, worksheets, and tool comparisons.
Questions about this workshop may be emailed to susan@eval.org.
|
Monday, August 24, 2009
|
|
 Jeanne Nakamura
Introduction to the Measurement of Experience
What do people do with their time, and how do they feel about it? Within psychology, this question animates much basic, applied, and evaluation research. In recent decades psychologists have begun to study in earnest the quality of everyday life. Researchers at SBOS’s Quality of Life Research Center pioneered some of the methods that make it possible to get an accurate sense of how people actually live.
One-time, self-report measures are widely relied upon to assess people’s use of time, and their thoughts, feelings, motivations, and well-being. These measures have several advantages but depend on processes of estimation and retrospection that distort the reality they are intended to assess. In this workshop participants will learn about the strengths and limitations of various methods – surveys, diaries, and tools for real-time measurement of experience in natural settings. Primary attention will be given to the Experience Sampling Method (ESM), considered the "gold standard" of the field. Issues to be considered include how to select the appropriate tool for a researcher’s questions and resources, how to design an ESM study, and how to analyze ESM data to answer a variety of questions such as: Where during a school day are students most engaged? How do moods change in response to environmental factors? When do patients with chronic illness report the least pain?
Questions about this workshop should be addressed to loren.bryant@cgu.edu.
|
|
 Rodney K. Hopson
Considering Culture in Evaluation and Applied Research
How does culture matter in evaluation theory and practice? How does attention to cultural issues make for better evaluation practice? Does your work in an agency or organization require you to know what culturally responsive in evaluation looks like? What issues do you need to consider in building culturally competent and responsive evaluation approaches? How do agencies identify strategies for developing and disseminating culturally responsive evaluation information? This workshop aims to address these questions and others as we consider culture in evaluation and applied research for the student, professional, or academic in a plethora of non-profit, philanthropic, educational agencies and organizations.
This workshop will rely on various interdisciplinary social science theoretical frameworks to both situate and advance conversations about culture in evaluation and applied research. In particular, participants will receive information and materials that help them to develop expertise in the general topics of culture in evaluation, including understanding the value-addedness for the evaluation researcher or program specialist who needs to develop a general understanding of the topic itself. This workshop will also assist participants to assist their own agencies to become culturally responsive and competent. That is, participants will develop tips and tools for working with their own agencies who work with underserved, minoritized, or priority groups.
The instructor will encourage opportunity to raise questions for participants who address current issues in their agencies and organizations. Registered participants will be expected to communicate with the instructor prior to the workshop to best assess participant and/or agency needs for optimal learning and application upon completion of the workshop.
Level: Intermediate.
Questions regarding this workshop may be addressed to hopson@duq.edu.
|
|
 David Fetterman
Empowerment Evaluation
Available in person or via webcast.
This session provides the basic steps of empowerment evaluation, including 1) mission; 2) taking stock; and 3) planning for the future. In addition, the workshop highlights guiding concepts including: critical friend, cycles of reflection and practice, community of learners, culture of evidence, and reflective practitioner. The principles of this approach are also explored, such as: improvement, community ownership, inclusion, democratic participation, social justice, community knowledge, evidence-based strategies, capacity building, organizational learning, and accountability. Real world examples are used to demonstrate the effectiveness of empowerment evaluation, ranging tobacco prevention and education programs and academically distressed school districts in Arkansas to a $15 million Hewlett-Packard Digital Village project (to bridge the digital divide in disenfranchised communities). Digital and online tools are also presented to facilitate the empowerment evaluation process and demonstrate how to transfer the technology to users.
Questions regarding this workshop should be addressed to fettermanassociates@gmail.com.
|
Tuesday, August 25, 2009
|
|
 Michael Scriven
Meta-Evaluation: A Checklist Approach
Available in person or via webcast.
This seminar will be of general interest to evaluators interested in improving their work, and of special interest to those who have no intention of becoming professional evaluators, but realize they will probably have to use OR hire OR read and evaluate the work of professional evaluators. It will be taught so as to be accessible to both groups, as well as to evaluation students. Evaluators should have their own work meta-evaluated because doing so is crucial in showing that evaluators believe what they tell clients: that (particularly external) evaluation leads to improvement and accountability. A number of arguments for and against the use of, and particular approaches to meta-evaluation will be discussed, including the Meta-evaluation Checklist (MEC) which requires the determination of validity, credibility, utility, robustness, cost-effectiveness, and ethicality.
Questions about this workshop may be e-mailed to scriven@aol.com.
|
|
 Pauline E. Brooks
Identifying, Framing, Measuring and Interpreting Racism in Evaluation/Research Efforts
In societies where racism has been a deeply ingrained part of the history, evaluators/researchers like everyone else have been exposed to socialization to accept manifestations of racism. These manifestations are common and often taken to be normal in such societies. Evaluators and others may inadvertently bring some of these manifestations into their work. For example, evaluators/researchers may unconsciously or dysconsciously facilitate or neglect racism that may be present and influential in the contexts, programs and environments that they assess. Too, racism may similarly appear in virtually any of the other aspects of evaluation work. This workshop creates a space for exploring a variety of concepts and tools that assist evaluators/researchers, program planners and others to more clearly identify, frame, measure and interpret racism as a potential variable in diverse settings and investigative activities.
The learning format includes:
• Mini-lectures: provide historical background, diverse ways of viewing, and philosophical bases for approaching the measurement of racism in evaluation/research.
• Individual and small group work: create opportunities to apply the workshop’s content to evaluation situations, case studies and methodologies.
• Exercises: aid in surfacing denial, dysconsciousness and some of the deep-myths and sincere fictions of racism.
• Participants contribute their own examples/problems for group consideration: draws upon participants’ experiences, insights and analyses.
• Reflection on larger questions: e.g. What are some of the ethical issues, if any, when conducting evaluations/research in environments where racism or other forms of inequities are an inherent part of the context?
• Take-home materials
Basic content includes: cross-disciplinary definitions of racism; 8-10 common societal manifestations/types of racism; the BET, an equity typology useful in identifying and measuring racism and other types of societal inequities (e.g. inequities based on gender, class, religion, sexual orientation, etc.); and an introduction to the methodology of 5-Paths Analysis/Brooks. Participants learn how to identify common places and circumstances where racism and racial inequities may influence the context and content of programs and societal problems. Participants will practice applying strategies for collecting data on racism and strategies for removing/averting racism in their own evaluation/research practices.
Questions about this workshop may be addressed to pbrooks_3@hotmail.com.
|
|
 Jason Siegel
Creating Effective Surveys for Applied Research
The focus of this hands-on workshop is to instruct attendees how to create reliable and valid surveys to be used in applied research. A bad survey is very easy to create. Creating an effective survey requires a complete understanding of the impact that item wording, question ordering, and survey design can have on a research effort. Only through adequate training can a good survey be discriminated from the bad. The day long workshop will focus specifically on these three aspects of survey creation. The day will being with a discussion of Dillman’s (2007) principles of question writing. After a brieef lecture, attendees will then be asked to use their newly gained knowledge to critique the item writing of selected national surveys. Next, attendees will work in groups to create survey items of their own. Using Sudman, Bradburn, and Schwatrz’s (1996) cognitive approach, attendees will then be informed of the various ways question ordered can bias results. As practice, attendees will work in groups to critique the item ordering from selected national surveys. Next, attendees will propose an ordering scheme for the questions created during the previous exercise. Lastly, using several sources, the keys to optimal survey design will be provided. As practice, the design of national surveys will be critiqued. Attendees will then work with the survey items created, and properly ordered, in class and propose a survey design. At the end of the day, attendees will never look at a survey in the same way again.
Questions about this workshop may be e-mailed to jason.siegel@cgu.edu.
|
Wednesday, August 26, 2009
|
|
 Stewart I. Donaldson & Tarek Azzam
Using Technology to Enhance Applied Research and Evaluation
Available in person or via webcast.
Only a few seats remaining for the live workshop.
This workshop will focus on how to use a range of new technological tools to improve applied research and program evaluations. Specifically, we will explore the application of free or inexpensive software to engage clients and a range of stakeholders, collect research and evaluation data, formulate and prioritize research and evaluation questions, express and assess logic models and theories of change, track program implementation, provide continuous improvement feedback, determine program outcomes/impact, and to present data and findings. Participants will learn how to access tools such as Geographical Information Systems (GIS), data collection software, and interactive conceptual framing software to improve the quality of their applied research and evaluation projects. Participants will be provided with information sheets on each technological tool along with details about attaining free trials.
Questions about this workshop should be addressed to stewart.donaldson@cgu.edu.
|
|
 Katrina L. Bledsoe
How to Conduct Culturally Competent Community-based Program Evaluations
The dynamic cultural demographics of communities make it imperative to understand how to conduct evaluations that are culturally sensitive and competent in community-based settings. This understanding is especially important in working with communities with hard to reach populations (e.g., marginalized groups), and where evaluations, if not tailored to the community and cultural milieu, can easily overlook the mores of its members. The purpose of this workshop is multifaceted. First, it encourages participants to understand barriers that might arise in evaluative settings between evaluators, key stakeholders, and evaluation participants that can hamper the development and execution of culturally competent evaluations (e.g., power dynamics; and institutional structures that may intentionally or unintentionally promote the “isms”). Second, the workshop provides participants with basic strategies in enhancing cultural competent evaluations by addressing issues such as institutional review boards and research ethics; translation of evaluation terminology to communities; and strategies to garner community involvement and buy-in, and gain trust. Finally, participants will have the opportunity to develop a model of a culturally competent community-based program evaluation. Dr. Katrina Bledsoe will provide “real world” examples from her evaluation experiences in small urban communities, as well as a large national multi-site federally funded community-based initiative. This workshop assumes participants will engage in highly interactive discussions about program evaluations that they might have or may currently be conducting.
Questions about this workshop should be addressed to katrina.bledsoe@gmail.com.
|
|
  Tiffany Berry & Rebecca M. Eddy
Using RCTs (Randomized Control Trials) to Improve Educational Research and Evaluation
Current federal priorities for evaluation designs focus on increasing rigor in research and evaluation through the incorporation of Scientifically-based research (SBR). SBR generally involves conducting Randomized Control Trials (RCTs), which can be successfully carried out in a variety of applied settings. Despite a continuing controversy regarding the appropriateness of using these types of designs in educational settings specifically, local program and school district evaluators often find it necessary to conduct this type of research. The purpose of this workshop is to provide an overview of RCTs as well as practical strategies designed to improve educational research and evaluation. Based on our experience conducting RCTs on K-12 educational curricula, we will engage participants in class exercises and small-group discussions to examine issues related to the following: (1) recruiting districts, schools, and teachers; (2) collecting baseline and posttest data (involving multiple measures and methods) efficiently and accurately across schools; (3) monitoring and describing implementation fidelity; (4) analyzing data appropriate for designs dealing with nested data; and (5) reporting guidelines consistent with the What Works Clearinghouse (WWC). No evaluation experience required, but evaluation experience in educational settings is preferred.
Questions regarding this workshop may be addressed to rebecca.eddy@cgu.edu or tiffany.berry@cgu.edu.
|
Last updated 6/05/09.
|
