Making Social Science Actionable for the NWS: The Brief Vulnerability Overview Tool (BVOT) (2024)

Keywords: Social science; Operational forecasting; Communication/decision-making; Decision support; Vulnerability

1. BVOT: An inductive approach to actionable vulnerability decision support

The mission of the U.S. NWS is to “provide weather, water, and climate data, forecasts, warnings, and impact-based decision support services for the protection of life and property, and enhancement of the national economy.” In many cases, this provision is in the form of impact-based decision support services (IDSS), which was codified into law with the Weather Research and Forecasting Innovation Act of 2017 as part of the Weather-Ready Nation initiative (Public Law 115-25). This paper describes the Brief Vulnerability Overview Tool (BVOT), a social science–derived tool being developed that can help the NWS improve IDSS by integrating local vulnerability data into the decision-making and messaging of NWS Weather Forecasting Office (WFO) meteorologists.

For IDSS to be most effective throughout the forecast to warning continuum, NWS meteorologists need insight and understanding of local communities and their vulnerabilities (NWS 2019) (see the “Vulnerability” sidebar for more information). While many vulnerability-based tools already exist, it is clear when considering the specific needs of the NWS that these vulnerability data do not fit well into operational practices. Rather than considering the universe of ways of subdividing, categorizing, and analyzing vulnerability data and then searching for a “use for” some dashboard or database, a different approach would be to consider—inductively—what NWS meteorologists do with vulnerability data, and then consider the structure of decision support needs based on actual practices and mission requirements. Our in situ, ethnographic observations began in 2015 (Friedman 2016, 2017; Friedman and Wagner 2017) and have revealed that real-world NWS WFO vulnerability data needs generally fall into two temporal categories: 1) undefined, long-term, strategic needs and 2) discrete, hazard-specific, short-term tactical needs.

Long-term (strategic) needs for vulnerability data can address mission requirements including establishing outreach and engagement with core partners; determining and implementing established protocols regarding warnings; and/or planning for and implementing focused interventions regarding “structural” factors associated with vulnerability to hazardous weather (e.g., working with coastal communities to understand the extent of storm surge, sea level rise, and/or tsunami exposure so that communities and planners/decision-makers might better prepare). Many of these longer-term, strategic NWS goals fall under the umbrella of the NWS’s Weather-Ready Nation mission and combine both outreach and engagement to assess and address the broad needs of core partners and publics at the national, tribal, regional, state, and local levels. These general outreach and engagement needs can be improved by drawing on many of the existing vulnerability-data-informed decision support tools that have been built by Federal Emergency Management Agency (FEMA) (discussed below)—including the National Risk Index (NRI; Zuzak et al. 2022, 2023) and the Resilience Analysis and Planning Tool (RAPT; FEMA 2023c). Those FEMA tools can meet these outreach and engagement needs because they allow the NWS WFO to identify broad patterns of vulnerability across their county warning area (CWA) that can act as the building blocks for identifying partners for longer-term strategic engagement.

However, while there is an explicit relationship between vulnerability data and many of the long-term, strategic needs of the NWS, the relationship between vulnerability data and short-term, everyday needs (e.g., the strategic vulnerability data one needs to prepare a town for the risk of tornadoes vs the tactical vulnerability data one needs to operationalize that plan with 15-min notice) is less clearly distinguished in the NWS. Short-term (tactical) vulnerability data needs can address NWS mission requirements that focus on the (relatively) immediate threat to life and property posed by discrete, existing weather incidents: a forecasted hurricane, ice storm, blizzard, dust storm, riverine flood, flash flood, tornado, etc. Short-term warnings are, ideally, purely focused on the scientific understanding of atmospheric or hydrological conditions over and above discrete impacts. But this ideal does not do justice to the real-world decision-making of most NWS meteorologists who tacitly account for vulnerabilities throughout their decision-making.

While there have been a limited number of studies that have been conducted to quantitatively assess the potential impact of awareness of vulnerabilities on NWS product issuance (Davis and LaDue 2004; Dobur 2005; Barrett 2008, 2012; White and Stallins 2017; Naylor and Sexton 2018), this has generally remained an understudied topic. Most of these studies have confirmed what most operational meteorologists know—if there is a known concentration of people (i.e., a city or other municipality), it is more likely that the awareness of the assumed vulnerability of humans to severe weather will result in the higher likelihood of warning being issued on “marginal” events. The influence of known vulnerability awareness on warning issuance should hardly be a surprise, despite the fact that there have been few (if any) efforts to systematically study the pathways and nature of this influence on warning issuance. While this remains “anecdotally accepted” as just the way that meteorological warnings happen, this almost everyday occurrence is understudied and underdocumented. This lack of research and/or documentation on the influence of vulnerability awareness on NWS decision-making and product issuance can be seen as a gap in our empirical understanding of how to improve operations and meet core partner needs.

The BVOT was a response to this gap. Specifically, we observed that, on the one hand, NWS meteorologists are aware of the influence of vulnerability knowledge on their product issuance; while, on the other hand, they struggle with justifying and communicating these decisions. This means that there tends to be a very uneven distribution of vulnerability awareness in any given NWS WFO: some meteorologists have “learned” where vulnerabilities exist, others have not. In addition, this knowledge can ebb and flow at any NWS WFO as meteorologists retire or leave for other offices. To add more complication to this institutional knowledge of vulnerabilities, though, emergency managers (EMs) themselves can regularly rotate through jobs, meaning that it becomes increasingly difficult to catalog and share a definitive understanding of weather-hazard-relevant vulnerabilities at any WFO. The BVOT is created by working with local and county EMs who identify vulnerabilities of greatest concern, then mapping, and annotating those spatially discrete vulnerabilities in a shapefile (see below). The BVOT is meant to provide an experimentally grounded, well-tested, shared, and durable understanding of vulnerabilities across an NWS WFO so that the decision to factor in vulnerability information to any IDSS is both defensible and is based on core partners’ forecast needs.

This paper provides an introduction to the BVOT, an experimental (not operational) decision support tool—and an accompanying set of methods—designed to provide hazard-specific, operationally useful vulnerability information to NWS meteorologists in WFOs around the United States. As a tool, the BVOT is, at its most basic level, a shapefile that displays locally known, spatially specific, and weather-hazard-related vulnerabilities in a format that can be easily integrated into the existing operational forecasting, warning responsibilities, and tasks of NWS meteorologists. The method for mapping those vulnerabilities in a WFO’s BVOT builds, strengthens, and maintains the relationships that NWS WFOs already have with their local EMs and core partners through the process of working together to collect, map, and sort those local vulnerability data that will be included in a BVOT. Each CWA has unique sociodemographic and political histories, cultural contexts, economic realities, built and natural environments, and jurisdictional infrastructures that engender what vulnerabilities exist. Therefore, the BVOT is populated with discrete, locally known vulnerability data in order to ensure that NWS meteorologists are spatially situationally aware of those people, places, and things that are of greatest concern to their core partners.

The BVOT improves spatial situational awareness of NWS meteorologists and improves messaging between them and their core partners. The BVOT has been designed to be integrated into any GIS-based software platform, including the NWS’s Advanced Weather Interactive Processing System (AWIPS), used by the NWS meteorologists to view a variety of weather models, aid in the production of official forecasts, and issue weather alerts. Each BVOT shapefile identifies areas of people, places, and things that are most vulnerable to a particular meteorological hazard. Meteorologists can, then, overlay each BVOT layer with meteorological data [e.g., radar, satellite, convection-allowing models (CAMs), and surface observations]. Each BVOT shapefile can also be toggled on or off like any data in AWIPS. Having the BVOT shapefiles separated by weather hazard allows forecasters to focus on only those vulnerabilities susceptible to a particular relevant hazard, helping to eliminate unnecessary information (Fig. 1). BVOT polygons have a built-in text layer that provides specific information about what each vulnerability is and why it is vulnerable (Fig. 2). This text information can be used by the meteorologists to know when and how to relay information to core partners. Central to the BVOT design is the update-able nature of an office’s BVOT. Vulnerabilities may change over time; likewise, knowledge about vulnerabilities can and should develop over time. As such, BVOT can be thought of as a living document for vulnerability knowledge within a WFO that we recommend is updated/reviewed at least once a year.

The BVOT was created with both NWS meteorologists and a particular core partner, EMs, in mind. The term “emergency manager” is multifaceted in nature, depending on the scale of responsible jurisdiction. EMs work on all levels of government operations ranging from federal, state, local, tribal, or territorial. Their positions include part-time volunteers, working with or under local government, or specialized to a single entity such as a hospital or university. EMs have a number of responsibilities primarily focusing on prevention, protection, mitigation, response, and recovery (FEMA 2020). At the local level, EMs are uniquely positioned to know how weather hazards can—and do—impact rural and urban areas across the United States. While they are responsible, among other things, for identifying risks and mapping vulnerabilities (discussed below), they have no particular existing mechanism for identifying the instances of vulnerabilities that are particularly acute (e.g., the school in a flood plain). However, they are often bearers of significant knowledge and subtle, tacit understandings of who or what is most vulnerable to different hazards. The creation of a BVOT provides EMs with the opportunity to share these concerns to the NWS.

The BVOT is designed to prompt NWS meteorologists to more effectively communicate that specific, identified locations may be at risk from weather hazards. The BVOT may also help to close the “hedge gap” (see the “Closing the hedge gap” sidebar for more information) in situations of uncertainty. While the goal of the BVOT is to display critical, known vulnerabilities, the BVOT is always a subsample of all potential vulnerabilities (i.e., it is not designed to capture all vulnerabilities) and is used to make meteorologists aware of those weather-hazard-specific vulnerabilities that “keep their partners awake at night.”

2. Rationale behind the BVOT

The BVOT concept emerged from the real-time, in situ observation of and interviews with NWS meteorologists and their partner EMs before, during, and after severe weather. As we collaborated across these two groups, the authors increasingly recognized that understanding these actors individually meant understanding how these actors interacted with each other. For instance, NWS meteorologists worried about briefing their core partners at regular times before forecasted severe weather. However, these briefings had, as reported to the authors, increasingly been shaped by meteorologists’ growing understanding of the needs of those core partners, such as the need to make decisions at specific times of the day regarding school closures, opening Emergency Operations Centers (EOCs), mobilizing and preparing resources, and staffing. At the same time, EMs and other core partners had gradually learned, over years of interaction with NWS meteorologists, various limitations faced by meteorologists in their day-to-day work, and those EMs had learned to adapt to what they perceived as the cautious “hedging” (e.g., limiting or qualifying forecast communication, often due to uncertainties) of the NWS (cf. Roeder et al. 2021).

To study the feedbacks between these multiple elements and how they influence the work of NWS meteorologists, we developed an approach that sought to both study and assist these meteorologists to improve their awareness of vulnerabilities across their warning area. What follows, then, is a discussion of 1) some of the existing efforts to turn vulnerability data into decision support tools; 2) what makes BVOT different; 3) what observed impact BVOT has had on NWS meteorologists’ practices and how this has been received and understood by EMs; 4) the co-development of methods that have been developed to implement the BVOT across the NWS; and 5) some of the outcomes, implications, and continuing challenges that emerged from this work.

3. Existing efforts to integrate vulnerability into decision support tools

Many efforts to make sense of vulnerabilities have drawn on pre-existing data—usually at the census-tract level—to identify spatial units that can be tied to various social, economic, demographic, and/or built environment data as proxies for social vulnerability. For instance, vulnerabilities in the Centers for Disease Control’s social vulnerability index (CDC SVI; CDC 2020) can provide a spatially aggregated measure of social vulnerability (based on census-tract geographical units) and can also be displayed based on four major vulnerability themes that aggregate categories of census data in order to shed light on types of vulnerabilities (Table 1).

Something like the CDC’s SVI can be used to get a bird’s eye view of specific census tracts that might be of concern to the NWS and others who are trying to draw on vulnerability data to decide on longer-term strategies. But, in reality, we have seen few meteorologists in the NWS actually use the SVI data in operations. One of the reasons for this is that the SVI, by itself, requires significant social science background and skill in order to interpret these data. A more practical and important reason is that the SVI is hazard-agnostic and focused on the census tract, which is rarely the actionable and operationalizable geographic unit of concern for most NWS meteorologists.

Other recent efforts have focused on bridging these gaps between census-derived vulnerability indicators and the need for actionable units of analysis. Notably, recently, the U.S. FEMA created the NRI in an effort to produce a broadly accessible dashboard that brings together SVI data on both vulnerability and resilience along with hazard exposure and historical loss data for 18 natural hazards. Similar to other efforts (cf., Edgemon et al. 2023), the NRI balances vulnerabilities with a region’s capacity to absorb and/or recover from loss through the inclusion of measures of resilience and provides more granular data on specific natural hazards so that a decision-maker can understand the historical impacts of some hazards (e.g., tornadoes in the southeast). The NRI provides useful decision-support-focused risk assessments for both county and census-tract-level regions. A second, but related, effort by FEMA—the RAPT—focuses more narrowly, but in greater detail, on mapping the intersection of critical vulnerabilities (e.g., schools, hospitals, and mobile home parks) with a range of natural hazards.

While FEMA’s NRI and RAPT are improvements over naive uses of SVI data, we would argue, they still fail to meet the day-to-day operational needs of most NWS meteorologists. This is, of course, not a fault in these FEMA tools or their developers. They, in fact, were never designed for the kind of operational needs of an NWS meteorologist since they were mostly designed to meet requirements for longer-term, strategic decision-making regarding mitigation or recovery planning. Thus, while the NRI or RAPT allows an operational meteorologist to visualize the level of risk posed by certain weather hazards to a census tract or county, this level of awareness, in general, does not translate into short-term, storm-specific, actionable information that can inform communicating risks or hazards to core partners or refining product issuance. What operational meteorologists need to identify to help their core partners fulfill some of their most critical roles is whether objects of concern—people, places, or things—are currently or imminently at risk from a specific weather hazard. Ultimately, by helping NWS operational meteorologists better communicate risks or hazards to core partners through more specific messaging about potential impacts on vulnerabilities, the BVOT aids in the larger NWS mission to provide IDSS.

4. Methods for creating a BVOT

An IDSS approach requires fostering and maintaining relationships between NWS WFOs and their core partners. Creating a BVOT involves both an internal training and mapping process that happens within the WFO, as well as the external mapping that happens between WFOs and their core partners. The methods for building a BVOT should encourage and leverage collaboration between the local WFO and core partners, which may reduce uncertainty (regarding communication and a shared understanding of the vulnerabilities of greatest concern to core partners) and increase trust between partners (see the “Reducing interpersonal uncertainty through collaborative mapping” sidebar for more information). The building of a BVOT, then, can act as a doorway for outreach, helping to improve and maintain relationships between the WFO and core partners. Unlike other existing vulnerability maps—such as SVI, NRI, and RAPT—the BVOT is created both for and by individual NWS WFO. Crucially, the methods for collecting vulnerability data for use in the BVOT rely on eliciting specific and localized knowledge from locals: the NWS meteorologists and their core partners.

The initial BVOT interview protocol for mapping vulnerabilities, Interactive Mapping of Vulnerabilities Exercise (IMoVE; Friedman and Wagner 2018), involved one-on-one sessions with participants (n = 84, ∼60–90 min each). Researchers created the maps later, based on recorded interviews and screen recordings of the informant identifying and describing discrete, weather-related vulnerabilities (Fig. 3). This was a time-intensive process that could not, realistically, be scaled up. The next iteration retained a high level of researcher involvement but was interactive, with participants telling researchers where the vulnerabilities were, and the participant mapping those vulnerabilities. In both initial iterations, the research team combined and created maps using open-source software and created the GIS shapefiles for the participating WFOs to use in operations.

Successful implementation of the BVOT across all WFOs will require less technical skill to be feasible. Ongoing research is piloting ways to provide adequate training resources for the BVOT, along with streamlined mapping and configuration processes, so that WFOs can complete all necessary steps entirely independently, without the help of researchers. At present, WFOs spend approximately 30–60 min completing the mapping process individually or in small groups within the WFO, as well as 30–60-min sessions with contributing partners. Following this process, additional time is required to assemble the maps, which has varied across partnering WFOs. Not only does this updated set of methods allow the BVOT to be scaled across the NWS without the need for a team of social scientists to build it with them, but it also provides opportunities to better engage with their EM partners so that the process engenders a sense of ownership and trust in the vulnerability data.

Again, the BVOT is made for and by the WFO. Thus, in the next iteration of the development of methods for collecting vulnerability knowledge and information to populate a BVOT, training modules were created for the WFO to enable researchers to take a supportive, rather than primary role. In this iteration, the research team met with the NWS meteorologists and a subset of partnering EMs within the CWA and coached the partnering WFOs in their efforts to collect, map, and prepare vulnerability data for use in operations. The researchers guided the WFO along the way and worked on ways to make the mapping and preparation of files for operations more user-friendly. As the methods for building a BVOT are further tested, less focus is placed on researcher expertise; instead, the individuals who are doing the mapping are empowered as the experts. In other words, NWS meteorologists and core partners involved in collaboratively mapping BVOT vulnerabilities become stewards of local knowledge.

The continued streamlining of the methods for collecting BVOT data is being tested in WFOs across the country. We have conducted in-person fieldwork, online interviews, and surveys with the participating WFOs to learn about how to improve these methods as well as learn about benefits from building and using the BVOT. Before beginning outreach with offices for participation, all 122 CWAs were compared across socioeconomic variables derived from census data, geography, and climate. We have deliberately selected each study WFO, and the CWA that they cover, due to its unique concerns. They vary in terms of climate, infrastructure, culture, history, geography, and more. Successful BVOT implementation across the country will ultimately require a clear set of methods for mapping, creating, and updating a BVOT that will meet the needs and provide guidance for all CWAs.

During in-person setup and piloting of BVOT methods, the research team engaged in conversations with WFO management regarding ways in which BVOT methods capture institutional knowledge, both externally, on the part of EM knowledge, and internally, regarding WFO knowledge. Many regions experience high EM turnover. Likewise, many WFOs experience turnover among meteorologists. The methods for building a BVOT allow for the capturing of individual local knowledge about the people, places, and things most vulnerable to hazardous weather; that knowledge is not lost when an EM or meteorologist moves. Discussions with participating EMs and meteorologists have helped researchers think about the communication implications resulting from the BVOT methods. EMs have reflected on linear communication transactions: the NWS WFO provides IDSS to the EMs before hazardous weather, and after hazardous weather, EMs send observation information back to the WFO. The methods we have tested to create a local BVOT allow for more transactional flows of communication around potential impacts, and ongoing relationships can be formed between the WFO and their core partners. During this research, warning coordination meteorologists (WCMs) have also noted how the process of working with partners to build a BVOT provides a script for outreach with their partners. It opens the door to increased communication and information exchange (Hurst et al. 2022).

5. Is a BVOT effective?

6. Conclusions

This paper has described the origin of the BVOT, how we have created and tested methods for putting it into practice, and how we have validated and tested its usefulness and usability. Throughout, our goal has been to demonstrate how social science–informed research can be operationalized within the institutional and organizational context of the NWS and across to its relationships with core partners like EMs. From its inception, collaboration has been at the heart of the BVOT. The methods used for collecting vulnerability data are used to build a local BVOT that can provide actionable decision support for NWS meteorologists. The process for collecting this information requires collaboration across meteorologists within a WFO, capturing institutional knowledge, as well as interorganizational collaboration between the WFO and core partners, such as EMs. In addition to developing and assessing these collaborative methods, this research has also experimentally evaluated the impact of the BVOT on NWS product issuance and communication with EMs, as well as examined how EMs value the “enhanced” messaging that access to a BVOT can afford NWS meteorologists. This work has demonstrated the potential value of moving the BVOT from research to operations.

But, still, the BVOT, as with many social science–derived tools, faces challenges in its transition to operations. Studies have shown that NWS meteorologists are enthusiastic in their support for social science (Sherman-Morris et al. 2018). NOAA, itself, prioritizes matching “understanding and prediction of high-impact weather” with “the urgent need imposed by climate trends, population and infrastructure increase, and disproportionate impacts on vulnerable communities” (NOAA Science Advisory Board 2021). Regardless, social science–derived tools like the BVOT face an uphill battle to find their way into everyday WFO operations. Operational innovations have traditionally involved new technologies, software, or models that, while at times facing resistance from operational meteorologists (e.g., Fine 2007; Daipha 2015), fit the expected paradigm of the meteorological work: highly technical, focused on atmospheric observations, and focused on complex models. The BVOT, as with other innovations, could find itself lost in the “valley of death” (or the “valley of lost opportunities”; National Research Council 2003, 12–21) in the transition from research to operations (Friedman et al. 2023). We conclude, then, with a call for action on the part of the NWS and researchers across the academic, industry, and government spectrum to come together to support and encourage efforts to transition social science–derived, operationally relevant research into meteorological operations.

Data availability statement.

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to confidentiality restrictions (e.g., they contain information that could compromise the privacy of research participants) established through agreement with the University of Oklahoma’s Institutional Review Board to protect human subjects involved in this study.