Field Visit Summary: People’s Theater, Germany

Background
People’s Theater (PT) is a non-profit program based in Offenbach, Germany (staffed mainly by youth volunteers aged 18-25) that uses drama workshops to help school children to explore social responsibility and non-violent conflict resolution. Its philosophy is based on a positive image of humanity, and the view that individuals have a duty to build their own character, develop positive social values in themselves and serve the wider community. PT is a CSO partner in the ESDInds Project Consortium and has been actively involved in the project from the start.

Aims of the Field Visit
This field visit aimed to test a total of 11 indicators (seven head indicators and four sub-indicators) from four value clusters that had been selected by PT staff as relevant to the organization’s work, namely Unity in Diversity, Empowerment, Justice and Trust, and to determine which assessment tools were appropriate to measure them.

Main Findings
All of the 11 tested indicators were accepted as relevant and important in the context of PT’s program. In addition, project staff described all of the indicators as valid, i.e. useful for measuring the respective values, and measurable (without any modification) through the assessment tools that were used. These 11 indicators were also rated as independently applicable by CSOs. The exercise of values assessment was found to be valuable by both project staff and youth participants within PT. Staff members reported that the questionnaire had helped them to identify processes within PT that could be changed to make them more effective or transparent in the future, while the youth felt that participating in the evaluation had increased their own self-awareness.

Processes Utilized
Suitable assessment tools for this specific project context (small groups of literate participants working on drama-based activities) were identified through consultation between ESDInds researchers and PT staff. The criteria for structured observation (identifying specific examples of behaviour that related to each indicator) and the questions for the questionnaire were also developed through a similar consultative process.

A cycle of consultation-action-reflection was effectively employed to refine the assessment tools used. For example, the number of participants was reduced from four to three in the second round of structured observation, making it easier for observers to complete the assessment sheets.

A four-point scale (Not at all – Occasionally – Often – Very much) was tested with the second group in order to capture nuances of behaviour, but then rejected as the observers found it more difficult to use than the three-point scale. Certain criteria on the assessment sheets and questions on the questionnaire, which staff or participants found unclear or irrelevant, were reworded or reformulated during the course of the visit.

Assessment Tools
List of assessment tools used and indicators tested with each:
- Structured Non-Participative Observation: U_H2 EE_H4, I_H1
- Self Assessment with Follow-Up Dialogue: U_H2, E_H4, I_H1
- Questionnaire: J_H1, J_SH1a, J_SH1b, T_H3, E_H3, E_SH3a, E_SH3b, E_H4

(a) Structured non-participative observation: Three non-participating observers watched a small group of participants during a rehearsal. Each observer completed a structured assessment sheet for every participant, and another for the whole group, by marking statements on a three-point scale (Not at all – Sometimes - Very much). Video recordings of the rehearsal were also made and reviewed, to capture nuances missed during the direct observation. Consensus of two observers was taken as valid.
(b) Self-assessment with a follow-up dialogue: After the non-participative observation exercise, each participant was given a blank copy of the same assessment sheet (in terms of applied items) and asked to complete it themselves by reflecting on their own behaviour. Any differences between the observers’ assessment and the participant’s own self-assessment were discussed with participants in a short face-to-face dialogue.
(c) Questionnaire: Participants completed an anonymous questionnaire with closed-ended questions, and the responses to each question were counted.

Other Lessons Learned
Combining tools can generate important new information: In this case study, different assessment tools were used effectively together to give a nuanced picture of the overall situation. The participants found it interesting and useful to follow up structured observation with self-assessment and dialogue, highlighting the differences between their own understanding of their behaviour and other people’s perceptions.

Overlap of indicators: The field team noticed that there is considerable overlap between the indicators, and that two different indicators can sometimes be measured at the same time. For example, observing participants to determine whether “everyone has his/her place in the team” (U_H2) could also be relevant to a justice indicator. More research is needed into the overlap between indicators, but it may be useful in making the values assessment less time-consuming for organizations to complete.