EXECUTIVE SUMMARY

The Jump with Jill-Texas Department of Agriculture partnership is creating an unprecedented swell of change in Texas schools. Employing media strategies, Jump with Jill (JWJ) is a music-based program that transforms nutrition education into a school wide rock concert. JWJ gets students dancing and singing to behaviorally-focus songs that covers:

  • increasing consumption of fruits, vegetables, low-fat milk products.
  • eating breakfast.
  • promoting local agriculture.

The JWJ program includes:

  • ASSEMBLIES (live event attendance 5,453): 16 one-hour assemblies for the entire school.
  • CRATES: 116 “Texas Farm Fresh Crates” filled with educational materials for FSD, PE, & 4th grade teachers.
  • TASTINGS: 8 taste tests to give students a hands-on experience with the featured local foods.
  • MEDIA CAMPAIGN: strategic promotion to news platforms yielding 816,361 impressions.
  • SPECIAL EVENTS: appearance at the State Fair of Texas.
  • EVALUATION: detailed analyses of 220 educators via emailed pre, post, and follow-up surveys. 

JWJ is a powerful one-hour assembly that empowers classroom teachers to implement short bursts of supplied curriculum to continue the show’s messaging.

Among all respondents:

  • program awareness increased over time
  • (1) activity books with accompanying teacher guides and (2) posters were the most popular tools
  • access to engaging nutrition education tools increased from pre- to follow-up
  • confidence for making nutrition exciting to teach was maintained in all groups
  • willingness to engage students on the subject of nutrition increased significantly from post to follow-up
  • teaching 1-15 minutes per day (the length of time requested by JWJ) of nutrition and/or movement instruction increased over time

Among groups:

Our “Champion” was always the FSD and our job is to prepare our champion. They are the people that completed the application to host the assembly and were responsible for evangelizing school staff to expose their students to the tools that support the experience. JWJ will take on more direct nudging of staff to support the FSD endorsement. They are highest in willingness and confidence for all questions and have the highest percentage doing more than 30 minutes of nutrition and/or movement education per day. We need these cheerleaders to make our exciting materials dance.

Teachers that had direct contact with JWJ in taste tests and getting their own CRATE of materials (CC) showed great increases in confidence after seeing the assembly. They reported highest levels of confidence for incorporating movement into their classroom at post. Teachers are also way more comfortable with movement-based education at baseline than with nutrition education. With JWJ, nutrition is approachable and relevant to their classroom. Both the Indirect and Diffusion group showed a decrease in the barrier of lack of expertise in nutrition. Diffusion also showed an increase in those that spent 1-15 minutes per day teaching nutrition and/or movement.

 

INTRODUCTION

In support of the Texas Department of Agriculture (TDA) Commissioner Sid Miller’s Farm Fresh Initiative, the rock & roll nutrition school assembly Jump with Jill (JWJ) was performed at 16 schools in West Texas in October of 2018. Employing media strategies, JWJ is a music-based program that transforms nutrition education into a school-wide rock concert. JWJ creates an unforgettable experience using original music, lighting, props and live characters to inspire their audiences for better nutrition.

During the 60-minute assembly, students dance and sing to behaviorally-focused songs that address increasing consumption of fruits, vegetables, low-fat milk products and eating breakfast. The Texas Farm Fresh JWJ Live Tour provides nutrition education for school-aged children while promoting local agriculture. To increase impact, schools received a “Texas Farm Fresh Crate” (CRATE) filled with educational materials for classroom teachers. In addition, taste tests were conducted with selected classrooms to give students a hands-on experience with the featured local foods.

JWJ is a powerful one-hour assembly that not only get kids motivated to eat healthy but strategically empowers classroom teachers who attend the assembly with their students to implement short bursts of supplied curriculum to continue the assembly’s messaging. The primary objective of this study is to understand how JWJ empowers teachers as nutrition educators. Specifically, we’ll look at teachers’ confidence and willingness levels at a. pre-survey (“pre”), b. post-survey (“post”)and c. follow-up survey (“follow-up”) time points (change over time). Data will be analyzed by (1) overall trends for all respondents and (2) exposure group.

Our hypothesis is that greater engagement with JWJ will lead to higher confidence. Higher  confidence leads to greater willingness to teach nutrition in the classroom. We can measure confidence by watching perceived barriers diminish over time and increase time spent teaching nutrition. This study will also provide insight into the most effective way to deliver the program.

 

METHODS

Recruitment

Texas schools participating in the National School Lunch Program are able to apply to host JWJ. Applications were completed by 227 schools for the 16 assembly stops. All applications are reviewed based on an evaluation matrix that included:

  • enthusiasm for the experience
  • adequate facilities to accommodate the performance
  • high percentage of free and reduced National School Lunch Program participation
  • participation in TDA’s Farm Fresh Challenge
  • application to TDA’s Expanding 3Es of Healthy Living Grant
  • response to TDA survey indicating participation in Farm to School
  • participation in the Fresh Fruit and Vegetable Grant Program
  • location in the geographical target of the tour (for this tour, West Texas)

Selected schools provided their current teacher roster, which were then coded by their position and then deidentified. The University of Illinois at Urbana-Champaign Institutional Review Board approved this study. No demographic information was collected.

Study Design

A purposive (non-randomized) design was created based on three delivery approaches. All exposure groups saw the assembly, with their on ramping experience to the assembly representing the exposure group to which they belong:

The Direct Group included those respondents that received their CRATE directly from JWJ Headquarters. This group represents the most exposure to JWJ:

  • Food Service Directors (FSD) who were involved in submitting their application to win the assembly, worked with other administrators to confirm their date and time, and agreed to distribute the CRATE materials to 4th grade teachers. They do not have direct contact with students in the classroom, although they have a ‘classroom’ – the cafeteria– through which the entire student body passes.
  • Physical Education Teachers (PE) who were provided a CRATE for their use. PE teachers represent important champions for health and fitness given their subject matter and for their ability to reach the entire student body throughout their instructional week.

The Indirect Group included those respondents that received their CRATE from their FSD:

  • Fourth grade teachers (CC) were all supplied a CRATE. Selected classrooms also experienced a guided taste test with JWJ characters of foods featured in the assembly.

The Diffusion Group included those respondents that only saw the assembly. The only planned exposure to JWJ was seeing the assembly. If they accessed the tools, it was by borrowing the CRATE from another teacher or accessing the tools online:

  • Teachers that did not receive CRATES (T)
  • Special Subject Teachers (SS)
  • Teaching Aides (A)

All surveys were emailed and were open for two weeks, with two email reminders. The pre-survey was given four weeks prior to the assembly. Two weeks prior to the assembly, CRATES were received and distributed (if applicable). The taste tests (if applicable) were conducted on the same day as the assembly. The post-survey was given the day after the assembly was performed. A follow-up study was given four weeks after the assembly.

Survey

MEASURED AT ONE TIME POINT:

Training & Expertise: Low to High

MEASURED AT THREE TIME POINTS:

Program Awareness, Program Engagement, Confidence & Willingness

 

Data Analysis

  • Since this study contained an unbalanced design (more participants were in the Diffusion Group) results are reported as percentages.
  • Prior to analyses, the confidence scores were combined into two groups: ‘strongly disagree/disagree/no opinion’ and ‘agree/strongly agree.’ Similarly, the willingness scores were combined into ‘I will never do this/I don’t know if I can do this/I might be able to do this’ and ‘I could do this/I’m already doing this.’
  • All statistical tests were 2-sided and performed using a 5% significance level, leading to 95% (2-sided) confidence intervals.
  • Data was analyzed for normality and adjusted as needed. Spearman correlation coefficient was used for continuous variables.
  • Categorical variables were analyzed using number of subjects and percentages. Denominators of percentage calculations are based on the number of subjects in the full sample unless otherwise specified. 
  • Comparisons of categorical variables between exposure groups were assessed using Chi-square or Fisher’s exact tests.
  • All models were analyzed using SAS version 9.4 (SAS Institute, Inc., Cary, NC).  Models were checked using standard methods including assessment of  model  fit, model  diagnostics, proportional hazards assumption, adjusted R2 and multicollinearity, where appropriate.
  • In general, missing data was not imputed unless otherwise specified for a particular parameter of interest. In that case, details are provided to describe the handling of missing data (e.g., multiple imputation, maximum likelihood).

Calculation of Scores

Groups were analyzed using longitudinal data analysis techniques. Mixed effects (multilevel model) was used to examine differences between the three study groups in the primary endpoints across the pre, post, and follow-up. To account for the non-normal distribution of the primary endpoints (Likert-like scales ranging from 1-5), PROC GLIMMIX or generalized linear models with repeated statement (PROC GENMOD) and an appropriate link function were used. Effects for random intercept, random subject (teacher), and potential nesting of teachers within a school were considered. The potential clustering of teachers within the same school was investigated using the intraclass correlation coefficient (ICC) (variability within clusters/ variability within clusters+residual variability). The cluster with an ICC of at least 0.1 was considered for inclusion in the model. Another factor considered was whether the number of teachers within each school was enough to provide sufficient variability.

Random deviation of each teacher from the overall slope for that group was considered by testing time as a random effect. Different covariance structures was tested for the repeated observations. The models were adjusted for time and exposure group. A significant group x time interaction indicated an intervention effect that varies over time. Potential confounders include length of time as a teacher and prior training and expertise. Pairwise comparisons across groups were conducted using the LSMESTIMATE statement and the P-values adjusted for multiple comparisons using the Sidak test (Stevens J et al 2017). Engagement metrics at one time point were examined by determining the mean difference among the three groups using ANOVA test (or a non-parametric analogue) by pairwise comparisons between the groups.

 

RESULTS

Description of Participants

The survey was sent to 505 persons and completed by 220 people. The sample was composed of 11.8% Direct; 16.8% Indirect; and 71.3% Diffusion. The survey itself is an important engagement metric for exposure groups. Completion indicates interest in the project. It is apparent that the greatest interest was shown by those in Direct (which contained the school contact) and the lowest interest in Diffusion (though it is the largest group).

Teaching Experience

Respondents’ overall teaching experience was somewhat evenly distributed among Novice, Experienced, and Veteran. The percent of Veterans (16+ years) was highest among Direct (42.3%) reflecting senior leadership, followed by Diffusion (34.4%) and Indirect (21.6%). The largest percent of Novice and Experienced teachers were found among Indirect (35.1%, 43.2%, respectively).

Training

Training in nutrition education for most respondents was classified as Novice (54.3%), with a higher percentage of Experts found in Direct (38.5%). Indirect had the highest percentage of Novice (64.9%). The percentage of Novice was the highest selection across all groups.

Program Awareness*

*Small sample size, interpret results with caution.

OVERALL

Program awareness increases over time and increases the most at follow-up.

Direct at pre showed notable differences for FSD (P=0.069 marginal trend), JWJ emails at pre (P=0.034), online search (P=0.069 marginal trend), and previous experience with JWJ (P=0.035). However, no differences were noted by group for post and follow-up.

Program Engagement*

*Small sample size, interpret results with caution.

OVERALL

Significant differences between pre and post were seen for student activity books and teacher guides (P=0.012) and unit posters (P=0.007). Marginal differences were seen for student activity books and teacher guides between pre and follow-up (P=0.058) and JWJ website usage rose marginally between pre and follow-up (P=0.089).

At pre, squaremeals.org (5.6%) is the most popular tool. By post, the unit posters (12.8%) and student activity books and teacher guides (10.9%) followed by danceable music videos (7.1%) and the JWJ website (7.1%) outpace squaremeals.org (which stays near pre level). By follow-up, poster use continued to increase and lead as the most popular tool (17.2%), followed closely by the JWJ website (14.9%) (suggesting the influence of Diffusion), and the student activity books and teacher guides spiked to 13.8% whereas squaremeals.org rose to 12.6% at follow-up.

GROUP

At both post and follow-up, a significant difference by group was seen for all tools. In Direct at pre, they learn about the JWJ program through squaremeals.org and get hooked with the Danceable Music Videos. By post, they heavily use the posters (47.62%, up from 9.52% at pre) and students activity books and teacher guides (42.86%, up from 4.76% at pre). By follow-up, the posters are by far the most popular item (64.71%) with students activity books and teacher guides (47.06%) remaining high. In Indirect, pre use of all items is nearly non-existent speaking to a lack of program awareness. After post (which is also after the assembly!), a spike in poster use (29.63%) is seen followed by students activity books (18.52%) and Danceable Music Videos (14.81%). Indirect increases their use of JWJ website by follow-up (25% up from 7.41%). Diffusion usage appears inconsistent and use of all tools is low compared to other groups. Their leading tool is the JWJ website and it takes until follow-up to really move. The use of squaremeals.org increases by follow-up in Direct and Indirect but decreases in Diffusion.

CONFIDENCE

OVERALL

Unlike plate waste weight loss, confidence is an unobserved characteristic that is difficult to measure. Hence several questions are used to assess the underlying construct. An overall score combining all confidence questions exhibited an acceptable internal reliability (Cronbach’s alpha = .78). The Cronbach alpha validates the internal reliability of the six questions measuring confidence to actually measure confidence.

Using the Wilcoxon signed-rank, no difference in composite scores over time were noted.

Although the confidence composite score did not change in the overall sample, significant differences were found over survey part when considering the groups. Confidence scores were highest among Direct group at all three survey parts, ranging from 4.13±0.64 to 4.29±0.96.

INDIVIDUAL QUESTIONS

Three of the six confidence questions show significant increases.

I am confident that:

  • I have access to engaging nutrition education tools.

OVERALL

Having access to engaging nutrition education tools increased from pre to follow-up.

GROUP

Having access to engaging nutrition education tools increases in all groups from baseline with the greatest increases at post for Direct and Indirect and follow-up for Diffusion.

I am confident that:

  • I can incorporate movement into my classroom.    

GROUP

Highest in indirect at post. Teachers in this group got the most one-on-one time with “Jill” and “DJ”, including the taste test.

I am confident that:

  • I can make nutrition exciting to teach.

GROUP

All groups appear to stay at their starting levels of confidence; Direct is highest at baseline and showed an increase at follow-up.

Barriers

OVERALL

At pre, the most frequently selected barrier was lack of time (48.8%), followed by lack of expertise on the subject (22.4%) and inflexibility of curriculum (19.2%). The leading factors at post are lack of time (58.33%), inflexibility of curriculum (28.21%), and lack of engaging tools (17.31%). Follow-up mirrors the four leading barriers selected at pre and post, with lack of expertise dropping from 22.4% at pre to 16.09% at follow-up, an indicator of confidence.

GROUP

At pre, significant differences were observed by group for lack of time (P=0.015) and lack of flexibility in curriculum (P=0.038). The proportion of participants who reported lack of time was highest among Indirect (72.7%), followed by Diffusion (47.57%), and Direct (28.6%). Smaller proportions yet similar trends were seen for lack of flexibility in curriculum (36.4%, 18.3%, 4.8%, respectively).

At post, a marginal association was observed for lack of flexibility in curriculum (P=0.067). The proportion of respondents who reported lack of flexibility in curriculum was highest among Indirect (40.74%), followed by Diffusion (28.7%) and Direct (9.5%). Lack of time remained an important barrier (Direct: 38.1%, Indirect, 66.7% and Diffusion 60.2%).

No differences were noted by exposure group at follow-up, with lack of time being mentioned by 47.1% Direct, 75% Indirect and 46.6% Diffusion. Inflexible curriculum was mentioned by 17.6% Direct, 33.3% Indirect and 27.6% Diffusion.

Willingness

OVERALL

An overall score combining all willingness questions exhibited an excellent internal reliability (Cronbach’s alpha = .92). Using the Wilcoxon signed-rank, no difference in composite scores over time were noted.

With the exception of willingness composite score at follow-up, mean scores significantly differed across the three groups, with highest values in Direct.

INDIVIDUAL QUESTIONS

Three of the six willingness questions significant increases.

I am willing to:

  • Prioritize nutrition in my curriculum.

Direct starts and stays high throughout the survey. At post, respondents in Direct were 2 X more willing to prioritize nutrition in their curriculum.

  • Try new nutrition education tools. 

A significant association (p= 0.039) was found between the groups in post. The proportion of participants who positively responded was highest among Direct (84.2%) compared to the Indirect (41.7%) and Diffusion (53.7%).

  • Engage students on the subject of nutrition.

OVERALL

The percentage of willing respondents increased from post (49.4%) to follow-up (51.7%) with a corresponding reduction in those who did not (P=0.099).

GROUP

High willingness was found in Direct (84.2%) at post. Furthermore, a significant association (p=0.017) was found between the groups post. The percentage of participants answering positively was highest among the Direct (84.2%) at post.

Time Spent

OVERALL

In terms of minutes per day spent on instruction in nutrition and/or movement, most respondents initially reported spending between 1-15 minutes daily and the amount of respondents in this category increased over time. This is what JWJ asks of participants and the increase was significant (from 48% pre to 55.7% post to 57.47% follow-up).

GROUP

When examined by exposure group, significant differences were observed (P=0.004) at pre. The majority (42.9%) in Direct (which contained PE teachers) reported spending > 30 minutes whereas the majority of those in Indirect (68.2%) and in Diffusion (50%) spent 1-15 minutes. In fact, Direct spent 7.6 X more minutes than Indirect and 3.9 X more minutes than Diffusion. Examining responses over time, at post, similar distributions were noted, with significant differences observed (P<.0001) (data not shown). At follow-up, these differences disappeared. When controlling for length of teaching and expertise, minutes per day spent including nutrition and/or movement in the instruction were about 2 X higher at follow-up compared to post (Odds Ratio=1.96 (95% Confidence Interval=1.08, 3.53)) (P=0.026).

 

 

DISCUSSION

From past evaluations on the assembly, we know that JWJ is successful at engaging schools. We wanted to learn more about the teachers we serve, since engagement of the teachers is different than the engagement of the school as a whole. JWJ is a powerful one-hour assembly, but relies heavily on empowering classroom teachers to implement short bursts of supplied curriculum to continue the assembly’s messaging.

Our “Champion” was always the FSD and our job is to prepare our champion. They are the people that completed the application to host the assembly and were responsible for evangelizing school staff to expose their students to the tools that support the experience. JWJ will take on more direct nudging of staff to support the FSD endorsement. They are highest in willingness and confidence for all questions and have the highest percentage doing more than 30 minutes of nutrition and/or movement education per day. We need these cheerleaders to make our exciting materials dance.

Teachers that had direct contact with JWJ in taste tests and getting their own CRATE of materials (CC) showed great increases in confidence after seeing the assembly. They reported highest levels of confidence for incorporating movement into their classroom at post. Teachers are also way more comfortable with movement-based education at baseline than with nutrition education. With JWJ, nutrition is approachable and relevant to their classroom. Both the Indirect and Diffusion group showed a decrease in the barrier of lack of expertise in nutrition. Diffusion also showed an increase in those that spent 1-15 minutes per day teaching nutrition and/or movement.

A huge surprise was how disengaged SS teachers (music, art, etc) are with our program despite its use of music and “the arts.” Our champions are those squarely in nutrition, FSD and PE. Length of teaching did not affect scores. Everyone, if they are successfully motivated, is capable of putting nutrition and movement into their classroom.

The following graphs show the patterns across time according to the major metrics of the study.

 

RECOMMENDATIONS

With the leading barrier of lack of time for all groups, the posters proved to be the most popular tool, perhaps because it doesn’t require more interaction than pinning it to the wall. Further consider that a leading barrier for FSDs, our nutrition experts and veteran teachers, was lack of engaging nutrition education tools. Because they use their kits to promote the assembly to the school, consider adding engaging tools that can be quickly activated.

The surveys themselves illustrated the usefulness of nudging to coach teachers to overcome barriers, build confidence, and encouraging at least 15 minutes per day be spent teaching nutrition and/or movement in the classroom. To improve communication to all teachers, we recommend implementing an email campaign that nudges, reminds, and coaches teachers. Similar to this survey in terms of frequency and timeframe:

  1. Getting ready for the assembly email direct from JWJ (the top videos to learn the songs) so they hear about the assembly from their FSD and directly from JWJ.
  2. Your CRATE is coming.
  3. Day after assembly, where to find the “live show segment” videos to debrief.
  4. Follow-up emails with quick suggestions for classroom learning.

 

FUNDING/SUPPORT

The Texas Department of Agriculture commissioned JWJ to conduct this study. JWJ accepted funds from the Texas Department of Agriculture to perform the assemblies and conduct this evaluation. TDA projects require USDA approval. Jill Jayne MS RD was the principal investigator and study author. Renee Zelinski and Nick Greene conducted the evaluations and taste tests on sight, providing blinded data to the PI. Salma Musaad at the University of Illinois at Urbana-Champaign obtained IRB approval IRB#: n/a and conducted statistical analyses. Madeleine Sigman-Grant PhD RD provided technical consultant.

 

REFERENCES

Stevens JR, Al Masud A, Suyundikov A (2017) A comparison of multiple testing adjustment methods with block-correlation positively-dependent tests. PLoS ONE 12(4): e0176124. https://doi.org/10.1371/journal.pone.0176124

 

CITATION INFORMATION

Jayne, Jill (2019). Teacher evaluation of the music-based nutrition education intervention Jump with Jill: Empowering educators to teach nutritionRetrieved from http://jumpwithjill.com/wp-content/uploads/2019/05/Teacher-evaluation-of-the-music-based-nutrition-education-intervention-Jump-with-Jill-Empowering-educators-to-teach-nutrition-2018.pdf

View the summary and full report.

 

COPYRIGHT PAGE

The author holds all trademarks, copyrights, and likenesses associated with Jump with Jill® including all original audio and video recordings, lyrics, and other pre-existing works.