Now, you might be asking...what about randomized control groups? Where is your observational before-and-after study with a convergent-parallel mixed method design? (Isn’t academic jargon fun?) Here are some of the reasons we DON’T use traditional “monitoring and evaluation approaches” right now. 1 Randomized Control Trials - RCT’s are an experimental approach to measuring the social change in a policy or intervention. (Think randomly giving houses to one group of beneficiaries and not to another and then surveying both groups to assess the change in quality of life over time.) 2 Why we don’t do an RCT (right now) . They’re expensive - think hundreds of thousands of dollars and waiting years to see data and results. As a growing organization our approach is centered on the end-user. Right now, we’re focused on building the best houses possible through a community process. It’s not a feasible priority to go back and survey families who haven’t received housing yet still live in slum communities. In this case, our end-users are some of the most vulnerable and impoverished women, men and children in this hemisphere. When optimizing data to understand value creation and areas for operational improvement, we don’t think it’s a smart use of time and money to go back and survey non-selected families. However, we do believe that there are ways to optimize diligence in our methods to make sure our data collection process and data is as scientifically rigorous as possible. Output Only (Laundry List) Approach This is a laundry list of program outputs such as number of homes funded, number of families living in homes, number of women and children helped. This list is often required by funders, but doesn’t go any deeper to help us understand the behavioral and environmental changes created by giving someone a home? Did it help someone create a business or feel trust their neighbors? Did it actually lift a family out of poverty or create a thriving community? Working with a local partnership model, this approach can also create a top-down compliance culture rather than a bottom up optimization of impact from all stakeholders. (Note: we do track some of these numbers, you can see many of them here. We just don’t think it should be the only or most important metric to track impact.) 1 New Story does not believe in “monitoring” and “evaluating” our beneficiaries. If our goal is to create thriving communities by learning, iterating and improving our impact along the way, there are many methods to optimize for value and impact. 2 Randomized control trials are the gold standard because they are most accurate scientific method to evaluate causal effect. They are not the only indicator of optimizing value and impact. They are often more feasible for established organizations operating at scale. They do not however take into account the dynamic nature of humans conditions in slums and growing organizations where products and processes are changing to optimize efficiency and value. We are excited, however, about the possibility for a quasi-experimental study in the future through a pipeline approach. This approach would allow us to optimize construction timelines to utilize families who have yet received housing as a control group. More on this soon.