Program Evaluation for Trainers
With the proliferation of financial literacy initiatives (two-thirds of 90 financial literacy programs examined by Fannie Mae in 2002 were started in the 1990s. Of these 75% were started in the late 1990s or 2000. Financial literacy curriculums have also grown exponentially. These programs cost millions of dollars and there have been questions about the effectiveness of developing more without objective evaluation of existing programs. There has been a call to have national standards for program evaluation but until this is developed, each program must struggle with its own evaluation. An extensive program evaluation framework was proposed by Francine Jacobs in 1988 outlining the five key steps as follows. Key to effective evaluation is the setting of clear goals:
Preimplementation: Commonly known as needs assessment, this allows the program to determine its goals and plan effective execution. Studies have shown that only 22% of financial education programs conducted a formal needs assessment. Although there is much evidence to substantiate the need for financial literacy, the category and populations served are very broad. It is important to refine specific goals for specific populations. Testing the financial literacy levels of the target groups is a good method of determining the need.
Accountability: This step involves collecting information on the education and services provided (how), the cost of the program and basic program participant (who) information. Is the population of need of the program. This may be measured by collecting information during registration or an exit survey. How were the participants changed by the program?
Program Clarification: This evaluation (more formative) assesses the program's ongoing strengths and weaknesses. Review is made of mission, goals, objectives, and strategies. Is the target population being served or does the population have to be reframed? Reflections by program staff and participants. Measures include exit survey, teacher ratings, overall student satisfaction and increases in knowledge. This may involve the use of a pretest and a post-test.
Progress-toward-objectives: Objective measures are made of the effectiveness (summative) of the program on the individual in reaching desired goals. This may include long-term follow-up of program participants to see if the effects were long-lasting. The differential effects of the program may be examined as the differences in impact between males and females. An external evaluator may be contracted to provide objective evaluation. This is important to replicating the program on a broader scale.
Final Evaluation: Builds on the previous step to measure both short- and long-term impacts of the program. This stage requires formal experimental approach to analysis contrasting the educated group to a control group.
Cash Flow (Checking account, pay bills on time, record-keeping system, reconcile checkbook, have budget)
Saving (Saving account, emergency fund, save some of each paycheck, save for long-term goals, have certificate of deposits)
Investment/Retirement (Diversified investment, investment accounts, mutual fund, 401K or pension plan, IRA, calculated net worth, participate in 401K, public stock, put money in IRA, have bonds)
Credit (Have credit card, pay balance in full each month, review credit reports, compare offers for credit, refinance mortgage for home improvements)
Other (Homeowner, bought house, do own taxes, often set goals for future, read about money management)
Braunstein, S. and Welch, C. (2002) Financial Literacy: An Overview of Practice, Research and Policy
Todd, R., and Kreimer, J. (2002) Financial literacy education: A potential tool for reducing predatory lending?
This website is maintained
Last updated: 1/12/07