The Bookworm Club 2013: Evaluation Report Summary

The Bookworm Club 2013: Evaluation Report Summary
Eava n Bra dy & Kris te n Lwi n
Prac tice and Res earch Toge ther ( PAR T)
August 2014
What is the Bookworm Club?
The Bookworm Club is a program for children in out-of-home
care that aims to promote literacy and a love of reading
among children in grades one to 12. Participating children are
mailed packages
containing developmentally appropriate
books, games, and other related materials once a month over
a period of six months, from July to December. Packages are
personally addressed to the child or youth at their current
out-of-home placement.
“Dear Bookworm,
I enjoy the books
that you send every
month. They help
me with my reading
and spelling also the
stories are great
and exciting to
read. I can’t wait
for more books to
come but I am sad
that in December
will be the last
month.
Who participated in the 2013 Bookworm Club?
In 2013, the number of children participating in the
Bookworm Club grew to 532 (along with their foster carers,
n=532) representing 22 Ontario Children’s Aid Societies.
How was it evaluated?
A mixed-methods, multi-site evaluation of the 2013 Bookworm
Club was undertaken by Practice and Research Together
(PART). The following key variables were measured:
1) Child’s self-esteem.
2) Child’s motivation for learning.
3) Child-foster caregiver relationship, according to the
child.
4) Child-foster caregiver relationship, according to the
foster caregiver.
5) Literacy environment: E.g. changes in the number of
books accessible within the child or youth’s placement.
6) Foster caregiver expectations: Change in the foster
caregiver’s expectations for their foster child in terms
of education and how well they do at school.
What did we find?
Sample
·
·
467 children and foster caregivers (n=453) returned pre-tests
330 children and foster caregivers (n=330) returned posttests.
Page 2 of 3
The Bookworm Club 2013: Evaluation Report Summary
Demographics
The majority of foster caregivers were female (n=377), while the majority of
children were male (n=245). Mean age of participating children at pre-test was
10.09 years. Mean age for foster caregivers at pre-test was 49.10 years. The
highest proportion of children were in grades 5 (14.3%), 6 (17.4%), and 7
(17.1%). Forty-one percent of children were behind by one or more grade level
at pre-test while 52% were identified as being at grade level. A majority of
foster caregivers noted that they had zero biological children living in the
home and two foster children living in the home. The majority of foster
caregivers were married (79%) and a small majority had been educated to
college level (36.6%).
Key Findings: Quantitative
“My mom
reads to me at
night which I
like. Thanks
for the cool
surprises you
sent too. I
really like
them. This will
help me
become a
better reader”
Key findings from the 2013 Bookworm Club evaluation related to children’s
self-esteem and the child-foster caregiver relationship. Self-esteem
significantly decreased from pre- to post-test among children who were at
grade level while the relationship between child and foster caregiver
significantly decreased from pre- to post-test among foster caregivers who had
completed high school or less and some college/some university. All other
statistical analyses produced non-significant results.
While the 2012 evaluation indicated a significant increase in children’s
motivation for reading, the 2013 evaluation did not return any significant
results for the full data set based on the key variables identified (i.e. child
self-esteem, child’s motivation for reading, child-foster caregiver relationship
according to child and foster caregiver, and literacy environment).
Key Findings: Qualitative
Foster caregivers: Interviews were conducted with n=6 foster caregivers using
a grounded theory approach. Key themes that emerged during these interviews
related to:
·
·
·
·
Impact on Reading
Involvement of Foster Family in Supporting Program Experience
Child’s Experience of the Bookworm Club
Suitability of Books & Materials
Children: N=140 children wrote to the Bookworm sharing what they liked about
the Bookworm Club, their hobbies and interests, when and where they liked to
read and more.
The Bookworm Club 2013: Evaluation Report Summary
Page 3 of 3
Recommendations
·
·
·
·
·
·
Modify the Bookworm Club evaluation: Examine the qualitative results
from the 2012 and 2013 evaluations to guide future evaluation and what
should be explored.
Foster caregivers: Support foster caregivers to support the child’s
reading; build on the Getting Kids to Read booklet that is provided to
foster caregivers e.g. have information sessions or invite workers to
explain the steps and information outlined in this booklet more fully.
Fidelity of the program: Promote consistency in the implementation of
the Bookworm Club program and evaluation.
Bookworm Club Goals: Develop and fully understand the goals of the
Bookworm Club and ensure that the evaluation outcomes match the role
of the program.
Ensure that workers are signing up children appropriately: Children’s
reading ability matches books they are receiving, children are not
enrolled when reading level is far below actual age in order to ensure
that the child does not feel discouraged when they are receiving their
books – except in cases where is it known that the foster caregiver will
work with the child and support them through reading development.
Evaluation group: Include foster caregivers and workers in development
of the Bookworm Club program. Involving both groups may be critical to
developing the Bookworm Club program, the evaluation, and focusing on
fidelity and the child-caregiver relationship.
“I like the books
because they are
interesting. I like
the mail when it
comes to the
door. I like
getting my own
mail”
Conclusions
The 2013 Bookworm Club program evaluation yielded varied results and highlighted the need to
review and refine the current evaluation measures; a critical lesson to learn as the third year of the
program approaches the half-way point. Furthermore, an intervention and control group
methodology would offer an examination of whether or not there are any causal features of the
Bookworm Club, versus the current methodology, which is purely associational.
The 2014 Bookworm Club is almost half-way through with over 1,000 children participating. This
year’s program also includes a KOBO e-reader pilot offered to 150 youth in grades nine to 12. As the
Bookworm Club continues to grow and evolve it is vital that the program evaluation evolves at the
same rate in order to capture if the program is impacting children.
To receive a copy of the full evaluation report please email: [email protected]

Download Report