 |
||||||||
|
 |
Learning Histories
Evaluation Design and Development Memorial wanted the dialogue between HealthWorks! and the Michiana region to be ongoing. This would help them determine if HealthWorks!, the latest addition to the work of "creating healthier communities," would have the same positive impact as other programs. The decision to document this interchange precipitated the evolution of an innovative concept in the construction of evaluation tools for museums and learning centers. Memorial discovered, through research and conversations with museum professionals, that the HealthWorks! approach to evaluation was quite unique. There was no record of any organization designing a facility, developing a curriculum and programming, and launching a new initiative with built-in evaluation at the front-end. Evaluation has always been thought of, generally speaking as a "retrofit." In this way Memorial, once again, benefitted from thinking "outside the box." For both the medical and the foundation worlds, ongoing evaluation and information tracking are the norm. Memorial now needed to determine how to best implement processes and practices they knew in an arena they were just discovering. According to Reg Wagle, Memorial Health Foundation, "When we attended conferences for health educators and museum staffs we were amazed that they were just beginning very basic discussions of the evaluation of their effectiveness in changing their own communities. Many of those museums have been in operation for decades." The Evaluation Design Team came together in Fall of 1998 to discuss how to go beyond kids (at the museum) to the people who are their primary teachers and role models, and to systems that intersect with kids in the many facets of their living. The overlapping of child, family, school, community, and museum would form a major component of the evaluation design. For their purposes, the group developed a three sphere model as a graphic illustration of the concept.
Memorial saw kids at the core, the primary reason for HealthWorks!. They are the voice at the center of Memorial's evaluation. As Memorial moved out from that core they saw the sphere of families and teachers (the two primary learning systems that engage a child) who live beside children and are their mentors. That sphere seemed essential in the teaching, integration, modeling and living experience of children. A third sphere was the subsystems that make up the wider community. Memorial's focus in this sphere was upon organizations within the community that work to create frameworks, experiences and resources for children and families. These spheres do not exist one inside the other, like the layers of an onion, but rather in overlapping fashion, like interlocking rings. At the juncture where the three connected is HealthWorks!, a place "owned" by all. Dr. Dennis W. Rudy, Ph.D., Indiana State University South Bend believes that, "This will require collaboration with a number of community agencies also interested in the long-term quality of life of the people of the region, such as local school corporations, private educational institutions, employers, social service agencies and other community resources." The process of listening to the people who make up the three spheres had begun well before Memorial had any notions of how they might go about an evaluation. They knew that HealthWorks! Kids’ Museum needed to be "measured" by what mattered most to those who would visit, network with and learn from it. This idea of measuring what matters most became a slogan for the Evaluation Design Team: "We measure what we value. We value what we measure." Particularly in a situation like HealthWorks! where a great deal of data could be generated in a short period of time, deciding what results would measure the museum's success or shortfalls would be crucial. Memorial started its process of evaluation discussions where many of the conversations regarding HealthWorks! began. They asked themselves, "What is it that this new facility has to offer the community?" Using the sphere model as their guide they developed the following: CONNECTING COMMUNITY AND EVALUATION SPHERE ONE- KIDS AT HEALTHWORKS! Evaluate:
SPHERE TWO- PARENTS Look for evidence that:
SPHERE TWO- TEACHERS An 8-year old come homes from a HealthWorks! experience, thinking he had just had lots of fun, in a group session using a 30 foot rope to learn about the digestive system. That evening at the dinner table, with a large group of extended family, midway through the meal , he launches into an extended "lecture" on the digestive system in which he names and describes the functions of each of the 6 major organs involved in processing the food they are all eating. This story is an "artifact" - evidence that cognitive learning occurred for the 8-year old; and it is evidence, as well, that a "transfer" to one of the two primary learning systems in his life, his family, has occurred. This is an example of the "mixed method" evaluation protocol vital to HealthWorks! success. When a child has been part of a school visit to HW! does he or she go home excited, and committed to getting other family members, neighbors, or groups to come back to HW! another time? Here we track evidence that the child is "turned-on" about his or her health (changed attitude) and we have evidence that the other spheres have likewise been impacted.Look for evidence of:
SPHERE THREE- WIDER COMMUNITY Seek evidence of:
Evaluating something like healthy attitudes is never easy. Opinions can change on a daily basis, particularly in a society that receives so many conflicting messages regarding nutrition, exercise and lifestyle. The evaluation of HealthWorks! success needed to be looked at in both short term and long term ways. Memorial was advised to not even consider long term evaluation until they were three or four years into daily operations. Memorial viewed short term evaluation as important early because it would allow them to gauge reaction to their start-up status. Were their exhibits functioning properly? Did their community feel welcome? Were their classroom presentations stimulating the children's interest? Short term evaluation can utilize a variety of mechanisms ranging from general observation to surveys taken after the students have returned to their schools, compilation of answers in the All About Me! kiosks to the amount of soap being used in the restrooms. In August of 1998, Dr. Dennis W. Rudy, Ph.D. at Indiana University South Bend, submitted his Evaluation Proposal for HealthWorks!. The proposal drew information and evaluation techniques from a variety of sources, including the Program Evaluation Standards put forth by the American Evaluation Association. It also outlined time lines for the Formative Evaluation and for the Summative Evaluation. The Formative Evaluation would occur in five phases, in one-year increments, during the five years following the opening of HealthWorks!. The Summative Evaluation would be done in two phases, one at the end of year two, the other at the end of year five. All of evaluation would be done by a team of professionals and reported to a variety of stakeholders and constituencies. Children have always been the central focus of HealthWorks! Kids' Museum. However, the role they would play in the evolution of the institution itself was a subject of constant analysis and one of the topics at a "National Evaluators Summit" called together by Memorial on September 22, 1999 in Indianapolis. Barry Cohen of Rainbow Research cautioned Summit participants not to forget that relationships with children are 'reciprocal.' During their visit, children would facilitate the learning of others as well as themselves. Thirteen individuals gathered at the Summit, representing Memorial Hospital HealthWorks! Kids' Museum, Volunteer Hospitals of America, Indiana Youth Institute, IUSB, Rainbow Research, Ruth Lilly Health Education Center, Search Institute, and the Children's Hospital & Health System of San Diego. They discussed and explored evaluation possibilities for HealthWorks! Kids' Museum. Dr. Rudy presented his proposal and asked the group to provide input and suggestions. The results of the evaluation would make it easier for the museum's director and staff to know what works and what doesn't work; what kids like and don't like about the museum; what parents and teachers do and don't like; and suggestions that would make better exhibits or classroom experiences. This evaluation tool would be a helpful check and balance for museum personnel. Members of the HealthWorks! "National Evaluators" will serve as expert advisors in years to come, much as the HealthWorks! "National Innovators."
Operational Elements As the development of the project began to come together, so too did the operational elements. Memorial began to think about the project from a different perspective. It was no longer a concept but was quickly becoming a reality. Operational issues and questions began to surface. What would the Director of this center do? What background and professional experience would he/she need to possess? What other staff members would need to be on board? What would a visit to the museum be like from a personnel perspective? Who will do what? Memorial realized that the vision of HealthWorks! was not simply to build and equip a floor of Memorial Leighton HealthPlex but to create a learning environment for their community's children. Vital to this mission was the staff who would actually guide and educate the center's visitors. According to Jeanne Strickland, Educational Consultant, "We began to understand HealthWorks! as more than just a really cool place. We had begun to think of it as dynamic process." The job descriptions for their Executive Director and Education Coordinator were as creatively crafted as the exhibit floor. The language capitalized on the "look and feel" of the museum, presenting positions that were as innovative and fun as they were important. Rather than a Director, they were looking for a Visionary Potentiator of Pumped-Up Kids. The Education Director was to be a High Flying Kid Motivator and HealthWorks! technical support would be provided by a Techno-Wizard. Plans also included two part-time technical staff members (Masterful Magician of Kid and Community Connectivity and a Keymaster of Incredible Knowledge), two visitor/clerical staff members (Whizbang WondersHost/ess & Sorcerer's Apprentice), a tech supper person (Techno-Wizard), and a talented corral-full of part-time performer-educators for the classrooms. Imagine the vibrations these new job descriptions caused in a hospital human resources department. The job descriptions were innovative and fun and reflected the nature of the museum: VISIONARY POTENTIATOR OF PUMPED UP KIDS (DIRECTOR)
HIGH-FLYING KID MOTIVATOR (EDUCATION DIRECTOR)
TECHNO-WIZARD
Despite the months and years of site visits and research, Memorial was mindful of the fact that, on opening day, they would be taking their maiden voyage. In a sense, they had spent their time planning for a great experiment. The individuals placed at the helm needed to be prepared for whatever unexpected surprises the journey would hold. Memorial needed a staff that would be creative, energetic and unafraid to try new things. They also needed to be able to recognize when strategies weren't working and figure out where adjustments needed to be made. More than anything, the HealthWorks' staff needed to have a passion for working with children and a respect for how very powerful a well-educated child is in a healthy community. In the best case scenario, they would need educators (whether they were actually traditional classroom teachers, nurses, physicians, medical students or community members) who understood that the most important person in the facility's classrooms would be the students. There is no formula for staffing as each community faces different challenges and has its own goals and objectives. In Memorial's case, they saw great value in empowering their educators with the ability to create unique and original programs on an ongoing basis. The Techno-Wizard needed to be able to aid the teachers by constructing new tools for their use. A decision like this is not without risk. Research offered that in-house production of audio and visual materials held a certain amount of stress, particularly as exhibit openings drew close. However, it also provided an invaluable opportunity for creating items specifically tailored to the community and their needs. Kathy Jackson believes that, "[We're starting] pretty bare bones...In talking with other people [from children's museums] they've sort of said, 'Yeah, we started with four, we started with five or six, but within a year we were at fifteen.' so we just know it's going to mushroom."
Marketing Efforts In addition to personnel and staffing, the hospital knew it was time to advance the marketing efforts and consider increasing public awareness and excitement for HealthWorks!. In order to accomplish this the hospital began to develop a Marketing Action Plan which included objectives, operational issues that impact the marketing effort and activities (including dates, costs and person who would be responsible for each activity). Marketing Action Plan - Objectives:
Operational Issues that Impact Marketing Efforts:
The Marketing Action Plan helped to set goals for the coming year and propelled Memorial into the next stage of project development. This was when all the plans and concepts become reality, when the exhibits are built and delivered, when the operational components are in place, staff is hired, and the marketing and development plans are executed.
Staffing and Operations In the summer of 1999 Memorial began putting HealthWorks! Kids' Museum' staff into place. Shortly after the 4th of July, 1999, Deborah Drendall, who had spent the previous 10 years as director of a Montessori academy, was named Visionary Potentiator of Pumped-Up Kids (Director). In addition to Montessori, Deborah's background included work in the public school system and musical performance. Her first exposure to the HealthWorks! project was as a member of the Reach and Teach committee, assisting with the creation of the center's curriculum and exhibit selection. Less than a month later Deborah would be joined by Becky Williams, HealthWorks! Kids' Museum High Flying Kid Motivator (Education Director). Williams had been working with a similar project in Michigan before joining the fledgling HealthWorks! staff and had a background in public education, nursing and had already worked in a health education center. The timing and selection of leadership and staff for a facility that is still being built, can be somewhat tricky. How do you write a job description before you've fully articulated your project? Suppose the person you bring in finds that what you envisioned for the museum does not quite measure up in reality? If you've created all your goals and curriculum, how much room have you left for the creative individuals you've hired? If your project is close to completion, how great a learning curve have you posed to your new staff? Are you creating a situation in which they can succeed? How important is it that your staff come from within the community? In Memorial's case the national search resulted in individuals who were practically under their noses. Both were already involved in some way in the process of creating HealthWorks! and shared Memorial's enthusiasm for the project. They also understood their role as an intricate part of the next phase. Memorial, and its new HealthWorks! team, recognized that there were two parts to the creation of the facility. The first was the dreaming, planning and building. Many of the individuals and committees involved in these processes would be "finished" when the front doors opened and the first school group entered. The second part was when the work toward HealthWorks! actual mission, preparing the community's children to be healthy, strong and educated, began. Memorial had been very intentional in creating exhibits and curriculum that would include all of the young people in the region. Memorial also needed to be sure that they were eliminating any obstacles that might keep them away. Early in the process they held meetings with school administrators, teachers, school nurses and counselors to let them know about the new facility and their plans. It was important that the "school community" see HealthWorks! as a valuable enrichment to their students' learning experience. Next, Memorial approached teachers and school administrators, not about idealized goals, but about the nuts and bolts of interfacing the center and their institutions. For example, Memorial expected a school bus trip to potentially occupy three to four hours of a seven hour school day. This would mean that they would need to provide a space (or other option) so that the children could have lunch. Leighton Plaza's courtyard might offer a brown bag option in the very early fall and late spring, but certainly not during an Indiana winter. The solution, since they did not have the space to provide for a cafeteria or lunchroom, was to partner with another community museum, located across the street, who had lunch facilities. Not only did it solve the problem, but enabled Memorial to reach out to another organization. Memorial also needed to be mindful of financial issues - admission charges for individuals, but also for school systems as a whole. From Memorial's perspective, they had made the early decision that no child would be kept from visiting HealthWorks! because of money. Institutionally, school budgetary issues seriously impact field trips. Some of the region's schools limited classes to, at most, one field trip a semester. Memorial, new to the attractions/museum arena, may find itself competing against other organizations for student visitors. They had to be able to assure the school systems that students would be offered a beneficial learning experience in the most fun, dynamic manner possible. Transportation impacted several decisions. The timing of HealthWorks! schedule had to be arranged so that students could be dropped off at school, meet their field trip buses, travel to HealthWorks! and be ready to be picked up when return buses arrived. All this needed to occur without disrupting regular busing schedules. Additionally, Memorial had chosen a downtown location for Memorial Leighton HealthPlex. This placement, at the busiest intersection in South Bend (a downtown with little parking for larger vehicles), resulted in the need for HealthWorks! planners to devise safe bus loading and unloading zones. With early discussion of HealthWorks! the city had made the design of the municipal parking garage accommodate the special dimensions of school buses on its ground level floor. An interesting twist to this story came when a hydrant suddenly appeared in the selected loading zone. Neither the City nor the construction firm thought to check with each other about the placement of the hydrant. Memorial contacted the City only to learn that they would have to incur an additional, substantial cost to have the hydrant moved. City and school administration are usually quite helpful when it comes to education issues, these are all subjects which must be considered and discussed. Once your staff is on board and visitation components are in place your attention should turn to development of policies and procedures. It is a critical step toward preparing for opening day. |
