Start Submission Become a Reviewer

Reading: Developing and Evaluating Survey Items About Women Veterans’ Care-seeking for Cardiovascular...

Download

A- A+
Alt. Display

Research

Developing and Evaluating Survey Items About Women Veterans’ Care-seeking for Cardiovascular Disease Prevention

Authors:

Jennifer Jill Orshak ,

William S. Middleton Memorial Veterans Hospital; University of Wisconsin – Madison, School of Nursing, US
X close

Diane Ruth Lauver

University of Wisconsin – Madison, School of Nursing, US
X close

Abstract

The purposes of this study were to develop and evaluate survey items designed to describe younger women veterans’ reasons for, and barriers to, care-seeking for cardiovascular disease (CVD) prevention. Although these women are likely to be at higher risk for CVD than comparable men, we found little research on younger women veterans and CVD prevention. With a descriptive design, eight experts rated items on clarity, relevance, comprehensiveness, and gender-sensitivity. We computed content validity indices by item and sets of items. Using cognitive interviews, we assessed women veterans’ understandings of directions and items, abilities to perform mental operations to produce responses to items, abilities to produce acceptable responses to items. The clarity and relevance of all items were 0.89 and 0.98, respectively. The average comprehensiveness and gender-sensitivity of the sets of items were 0.96 and 0.96, respectively. We revised directions and items that were difficult to understand, revised items for that were difficult to answer, and developed a new items and response options based on participants’ feedback. Spontaneous feedback during the cognitive interviews affirmed the importance of describing care-seeking for CVD prevention in this population. Researchers can use the valid survey in future research.

How to Cite: Orshak, J. J., & Lauver, D. R. (2023). Developing and Evaluating Survey Items About Women Veterans’ Care-seeking for Cardiovascular Disease Prevention. Journal of Veterans Studies, 9(1), 1–14. DOI: http://doi.org/10.21061/jvs.v9i1.319
45
Views
7
Downloads
  Published on 06 Jan 2023
 Accepted on 06 Jul 2022            Submitted on 10 Nov 2021

Background

More than a third of women in the United States (US) have cardiovascular disease (CVD), and CVD is the primary cause of women’s deaths (Benjamin et al., 2018). As compared to civilian women, women veterans have a higher prevalence of traditional risks for CVD such as being overweight and “obese” (Breland et al., 2017; Lehavot et al., 2012), using tobacco (Lehavot et al., 2012; Odani et al., 2018), and developing gestational diabetes and hypertensive disorders during pregnancy (Katon et al., 2014). Women veterans have a higher prevalence of nontraditional risks for CVD such as posttraumatic stress disorder (PTSD; Lehavot et al., 2018), depression (Whitehead et al., 2013) and military sexual trauma (Lutwak & Dill, 2013; Wilson, 2018; Wolff & Mills, 2016), as compared to civilian women. Also, women veterans have low awareness of their risks for CVD, as compared to the civilian population (Biswas et al., 2002; Canter et al., 2009). More women than men veterans seek care in the community than in the US Department of Veterans Affairs (VA; Lehavot et al., 2012; Murdoch et al., 2006). Few community-based clinicians know of their patients’ military history or appreciate how military service can impact health (Murdoch et al., 2006).

In the VA the percent of younger women veterans, ages 18–44, with at least one major risk for CVD is 57%. This risk rises with age across all age groups, 68% at age 45–64, and 84% at age 65 or older. Among the ages 18–44, more women than men veterans who receive care in the VA already have a CVD diagnosis (Whitehead et al., 2013). Early detection and prevention efforts are crucial to reduce this population’s risk for CVD.

The American College of Cardiology and the American Heart Association recommend clinicians assess all women for their risks for CVD (Arnett et al., 2019; Mosca et al., 2011). Scholars (e.g., Wenger, 2019; Wilbur et al., 2018) advise that all women be assessed for pregnancy-associated complications, such a pre-eclampsia, gestational diabetes, and pregnancy-induced hypertension. Because these conditions are early indicators of CVD, knowing the history of these conditions can help clinicians identify women at risk for developing CVD. Then, clinicians could tailor CVD prevention (e.g., smoking cessation, blood pressure monitoring, stress management) among those at risk.

Despite expert recommendations to assess all women for their risks for CVD, many women veterans forgo or delay health care in general (Delcher et al., 2013; Lehavot et al., 2012); and little is known about younger women veterans’ use of health care (i.e., care-seeking) for CVD prevention (Whitehead et al., 2013). Experts have called to focus on CVD prevention among younger women veterans (Whitehead et al., 2013). One way to do this is to assess younger women veterans’ care-seeking for CVD prevention. If researchers understood younger women’s reasons for, and barriers to, care-seeking then they could collaborate with clinicians to design interventions to target barriers, and support particular reasons, in this population.

Survey research is a useful and valued approach to better understand a phenomena of interest (Ponto, 2015). Surveys are an important means of collecting health information from a large sample of people (National Library of Medicine, n.d.). We proposed that conducting a descriptive study using a survey to understand women veterans’ care-seeking for CVD prevention would be useful to guide future research and practice. Before doing so, we sought a survey broad enough to reflect concepts relevant to care-seeking in general and for CVD prevention in particular; yet specific enough to reflect concepts relevant to women veterans’ care-seeking behavior as it relates to their military service. At the time of the first phase of this research, we had not found a survey that addressed care-seeking, CVD prevention, and women veterans. Thus, we reviewed the literature to: (a) identify tools used to assess care-seeking in different contexts and populations (e.g., Backonja et al., 2014; US Department of Veterans Affairs, 2015; Mochari-Greenberger et al., 2010); (b) identify what concepts other scholars had used to describe individuals’ care-seeking in general, related to women veterans, and CVD prevention in particular; and (c) explore whether a theory could guide our research.

Review of the Literature

Prior to developing survey items, we became familiar with the literature on barriers to seeking health care in general (e.g., Byrne, 2008) and barriers to seeking health care among women veterans in particular (e.g., US Department of Veterans Affairs, 2015; Orshak et al., 2022; Washington et al., 2011, 2015). Next, we reviewed the literature on CVD and prevention (e.g., Arnett et al., 2019; Benjamin et al., 2019; Wenger, 2019), CVD research among women (e.g., Cushman et al., 2021; Mochari-Greenberger et al., 2010, 2012; Mosca et al., 2011; Mosca et al., 2013), and CVD prevention and women veterans (e.g., Goldstein et al., 2014, 2017). In reviewing these bodies of literature, we noted that concepts described by other scholars were consistent with concepts in the theory of care-seeking behavior.

Theory of Care-Seeking Behavior (TCSB)

TCSB is a middle-range theory, and as such, is specific enough to guide research, and abstract enough to guide researchers and clinicians (Im & Chang, 2012; Lor et al., 2017). TCSB was developed to explain health-related behaviors which involved contact with the health care system, such as for secondary prevention (Lauver, 1992). Many health behaviors involve care-seeking for evaluation or treatment and are undertaken in the health care system (Lauver, 1992) such as blood pressure screening for CVD prevention.

Lauver (1992) originally proposed that the probability of engaging in health behavior is a function of psychosocial variables (i.e., affect, beliefs, norms, habits), external conditions, clinical and socio-demographics. Figure 1 shows the concepts and propositions in TCSB. According to TCSB, affect refers to feelings associated with care-seeking. In these bodies of literature, some examples of affect included fear of a disease diagnosis (Backonja et al., 2014; Honein-AbouHaidar et al., 2016), fear of mental health stigma (Orshak et al., 2022), fear of behavior change (Mosca et al., 2006), and embarrassment (Tsai et al., 2015). Beliefs refers to the perception of risks and/or benefits, and relevant outcomes of care-seeking. Some examples of beliefs included beliefs of being at risk for CVD (Leifheit-Limson et al., 2015), of having symptoms of CVD (Mosca Lori et al., 2013), and confidence in making lifestyle changes (Cavanagh et al., 2020). Normative influences, norms, refer to social and personal norms, such as professional’s recommendations. Social support had been documented a reason for care-seeking for CVD prevention (McSweeney et al., 2004; Mosca Lori et al., 2013); inadequate social support had been documented as a barrier to women veterans’ care-seeking (Cavanagh et al., 2020). Habit reflects one’s usual behavior, such as one’s past experience with care-seeking. Habits have influenced individuals’ care-seeking in general (Backonja et al., 2014; Melnyk, 1988, 1990) and women’s care-seeking for CVD in particular (Mosca et al., 2006).

Concepts and Propositions in Theory of Care Seeking Behavior
Figure 1 

The concepts and propositions in Theory of Care Seeking Behavior (Lauver, 1992).

External conditions are specific, objective conditions relevant to care-seeking, often referred to as the affordability, accessibility, and acceptability of care (Lor et al., 2017). The affordability of care has influenced women veterans’ care-seeking in general (Washington et al., 2011), women’s care-seeking for CVD prevention (McSweeney et al., 2004; Mosca et al., 2006), and would likely influence women veteran’s care-seeking for CVD prevention in particular (Goldstein et al., 2017). The accessibility of care, such as distance to a health care facility, has influenced individuals’ reasons to seek care in general (Wang et al., 2019), women veterans’ care-seeking (US Department of Veterans Affairs, 2015; Hamilton et al., 2012; Washington et al., 2015), and likely would influence women veteran’s care-seeking for CVD prevention in particular (Goldstein et al., 2017). The acceptability of care, such as the perceived quality of services, had influenced women veterans’ reasons to seek care (Washington et al., 2006, 2015) and women’s care-seeking for CVD prevention (Mosca et al., 2006).

According to the TCSB, clinical and socio-demographic variables may influence care-seeking behavior directly or indirectly through the psychosocial variables. Moreover, the influence of psychosocial variables on care-seeking behaviors could be modified by external conditions (Lauver, 1992; Lauver et al., 2003). We propose that TCSB has a breadth of concepts to describe younger women veterans’ reasons for, and barriers to, care-seeking in general and for CVD prevention in particular. For brevity, the terms women veterans are used herein to refer to younger women veterans between the ages of 18–44 and care-seeking is used to refer to care-seeking for CVD prevention unless otherwise noted.

Thus, we proposed to develop and evaluate new survey items to be used in future research to describe women veterans’ reasons for, and barriers to, care-seeking. This study (a) builds upon others’ works to describe women veterans’ barriers to care-seeking in general, (b) addresses a gap in the literature to describe women veterans’ reasons for engaging in care-seeking for CVD prevention in particular, and (c) includes a breadth of variables that could influence women veterans’ care-seeking behaviors as guided by theory. By doing so, this study adds to the literature on reasons for care-seeking in general.

The specific aims were to: (a) develop survey items based on relevant CVD literature, women veteran literature, and prior TCSB research (Version 1.0); (b) assess content validity of proposed survey items, based on ratings from content experts (Version 2.0); and (c) pretest and assess women veterans’ understanding of proposed items with cognitive interviewing, and refine the survey for administration with women veterans, using findings from the cognitive interviews (Version 3.0).

Methods

This descriptive study was conducted in three phases. Briefly, first we developed survey items. Second, we assessed the content validity of the items with ratings from content experts. Third, we assessed women veterans’ understanding of proposed items and refined the survey based on findings from cognitive interviews.

Aim 1: Development of Survey Items

Item Generation

The first author (J. O.) generated survey items based on prior literature and theory. Next, she invited nurse researchers to evaluate the clarity and conciseness of the newly developed items. This team of nurse researchers included one PhD-prepared nurse (D.R.L.) and four PhD nursing students (J.O., B.L.O., L.A., C.O.) from a midwestern university in the US. The team had a combined expertise in health promotion, disease prevention, CVD research, instrument development, and working with women veterans. In an iterative process, the team developed 10 open-ended items and 12 closed-ended items to reflect women veterans’ reasons for, and barriers to, care-seeking. The survey also included three screening and as well as clinical and sociodemographic items that were relevant to this research.

We developed open-ended items to invite women veterans to serve as experts of their own health regarding care-seeking for CVD prevention, consistent with feminist and qualitative research methods in nursing (e.g., Chinn, 1995; Speziale et al., 2011). Open-ended items can (a) provide a flexible mode of communication, (b) empower marginalized women so their voices can be heard, and (c) provide a more comprehensive understanding of women veterans’ experiences than closed-ended items alone (Im & Chee, 2001). Briefly, we wanted to understand, in women veterans’ own words their understanding of CVD, and why they may or may not seek care for CVD prevention. Open-ended items were not designed to reflect concepts from TCSB; rather, they were written broadly to invite a breadth of responses in women veterans’ own words.

We developed closed-ended items, often in the form of checklists, to reflect concepts from TCSB relevant to care-seeking for CVD prevention. Items were designed to be inclusive of all types of prevention (primary, secondary, and tertiary) to describe these phenomena. The response options were designed to include several reasons for, and barriers to, care-seeking. We assumed that barriers to care-seeking, in general, are not necessarily conceptually the opposite of reasons to seek care, as documented in previous research (Backonja et al., 2014; Lor et al., 2013). Two items were designed to ask participants to rank their reasons for, and barriers to, care-seeking in order of importance from “1” most important to “7” least important.

Survey Organization

We designed the survey to be administered online via a web-based platform (Qualtrics, 2019). We programmed the online survey so that three screening questions would appear first to ask potential participants if they were a woman, between the ages 18–44, and a US military veteran. Next, we programmed the survey to display open-ended items prior to closed-ended items so the response options of the closed-ended items would not bias participants’ responses to open-ended items. One of the first open-ended items read, “Describe what ‘cardiovascular disease’ means to you, using your own words. We value your honest thoughts. Please write your thoughts below.” Afterwards, we provided a definition of CVD to increase the accuracy of responses to the remaining items.

In one open-ended item, we asked participants to describe their reasons for care-seeking in their own words. Then we asked participants to answer closed-ended items about their reasons for care-seeking from checklists of response options, reflecting concepts from TCSB. For instance, “I have a risk for cardiovascular disease (for example, a family history or high blood pressure),” reflecting the concept beliefs. After participants endorsed reasons for care-seeking from the checklists, we asked them to rank their most important reasons.

In another open-ended item, we asked participants to describe their barriers to care-seeking. Then we asked participants to answer closed-ended items about their barriers to care-seeking from checklists of response options reflecting concepts from TCSB. For example, “the healthcare services available are unaffordable,” reflected the concept affordability. After participants endorsed barriers to care-seeking from the checklists, we asked them to rank their most important barriers to care-seeking. We anticipated participants would endorse multiple reasons and barriers to care-seeking, thus, we asked participants to rank their most important reasons and barriers to inform future research.

We concluded the survey with clinical and sociodemographic items. For example, one clinical item included on the survey was, “Are you a current smoker or tobacco user?” Response options included “yes” and “no” (Arnett et al., 2019). One sociodemographic item included on the survey was, “What is your Marital Status?” Response options included “single (never married),” “married, or in a domestic partnership,” “separated,” “divorced,” and “widowed” (US Department of Veterans Affairs, 2015).

Aim 2: Assessing Content Validity of Survey Items

Design

With a cross-sectional descriptive design, we assessed the content validity of proposed survey items, using Qualtrics (Qualtrics, 2019). Content validity is the degree to which items in a survey or questionnaire represent the concept being measured (Polit & Beck, 2021). This phase of our research was deemed exempt from further review by our university Health Sciences Institutional Review Board (IRB).

Sample

To assess content validity, experts (Polit & Beck 2021) advise recruiting a sample of five to 12 content experts. For this study, content experts were health care professionals (i.e., researchers, clinicians, faculty) from relevant disciplines (e.g., nursing, medicine) with expertise in gender-sensitivity or care of women veterans, women veterans’ health research, women’s risks for CVD, and applying TCSB in research. We chose this sample because they had knowledge of the target population as well as the theory and concepts applied (Polit & Beck, 2021).

Recruitment and Data Collection

One member of our team (B.L.O.) sent 27 emails to invite experts to participate in the study. Experts were asked to rate each item for clarity of wording and relevance to women veterans and CVD prevention (Polit & Beck, 2021). Experts also rated sets of items for comprehensiveness, that is, breadth of content, and gender-sensitivity, that is, how suitable the content would be for women veterans (deKleijn et al., 2015). For clarity and relevance, experts rated items on a 4-point scale from “1” = “Not at all” to “4” = “Very.” For comprehensiveness and gender-sensitivity, experts rated sets of items from “1” = “Not at all” to “4” = “Very.” If content experts rated an item, or a set of items, less than “3,” then we asked them to offer comments on how to revise items or set of items.

Data Analysis

We computed content validity indices by item (I-CVIs). To evaluate agreement among content experts by item, we collapsed ratings of “3” or “4” on each scale, based on expert recommendation (Polit & Beck, 2021). Next, we summed the number of content experts who gave a rating of either “3” or “4” and divided it by the number of content experts. Typically, experts (Polit & Beck, 2021) consider values close to .80 as good content validity. Thus, we considered the content experts’ assessments and revised items for clarity and comprehensiveness if a corresponding I-CVI was lower than 0.78.

Aim 3: Assessing Women Veterans Understanding of Survey Items

Design

With a descriptive design, we conducted semi-structured, in-depth cognitive interviews (Beatty & Willis, 2007; Conrad & Blair, 1996) by telephone among women veterans. Cognitive interviewing was chosen because it (a) is a method to study how targeted audiences comprehend, mentally process, and respond to materials (Beatty & Willis, 2007); (b) has been used in health care research to pretest and validate survey items and to improve completion and response rates (Drennan, 2003; Yu et al., 2019); and (c) can be particularly useful to determine whether or not participants understand the wording of the items and how participants may answer survey items (Drennan, 2003). This phase of our research was deemed exempt from further review by the university IRB.

Setting and Sample

We conducted this study in a midwestern city via telephone over the course of two months during the COVID-19 pandemic. We recruited a convenience sample of participants (N = 7). All participants self-identified as a woman, military veteran, and 18–44 years of age. Experts who have previously conducted cognitive interviews recommend researchers identify potential participants most relevant to the study, conduct between 5 and 15 interviews, and interview participants until relatively few new insights are identified (Beatty & Willis, 2007).

Sampling Method and Recruitment Plan

We recruited the convenience sample of participants using mass email services, word of mouth, and snowball sampling. Using the university’s mass email services, we randomly selected potential participants to receive recruitment material in one email at one point in time. Using word of mouth, we reached out via email to one informal leader of a local veteran’s group. We asked email recipients to either consider participating in this study or forward the email to one who was is eligible.

Study Procedures

Due to the COVID-19 pandemic, and to maintain safe social distancing, this study was conducted over the phone. First, we replied via email to potential participants to set a time for contact by phone and screening for eligibility. Second, we provided each participant information about the study including: the purpose of the study, procedures that will be involved, duration, potential risks, and benefits, and their right to opt out of the study at any time without penalty. Third, we sought verbal consent and obtained it from all participants. Fourth, we encouraged participants to ask questions or express concerns during the study. Based on prior research among veterans (e.g., Cavanagh et al., 2020; Littman et al., 2018), we offered one $20 digital cash card to participants who completed the cognitive interviews; this amount of money has been documented as adequate compensation for military veterans to participate in non-invasive, health-related research.

Data Collection

The 52-item survey was programmed into Qualtrics. Prior to data collection, items were previewed for accessibility using Qualtrics’ mobile preview technology (Qualtrics, 2021). Eligible participants were sent an email with directions and a link for accessing the online version of our survey. With verbal consent, we recorded the interviews using a digital, audio-recorder. Next, we assessed whether participants were able to access the online survey by their smart phone, tablet, or computer.

We used the Three-Step Test Interview (TSTI) method to guide the interview process (Jansen & Hak, 2005). See Appendix A for details about the guide used. The TSTI has been a useful guide for researchers to evaluate surveys by observing participants and their responses to items on a particular survey. The TSTI stages include: (a) observing for response behaviors (e.g., skipping questions), (b) follow-up probing to identify gaps in understanding, and (c) debriefing to elicit participants’ experiences and opinions of the survey (Jansen & Hak, 2005).

We adapted the first step of the TSTI from observing for response behavior to listening for response behavior (e.g., changes in tone, hesitation, silence), because data collection occurred over the telephone. We encouraged participants to “Think Aloud,” that is, we encouraged participants to read the stem of each item out loud, read each response option out loud, and answer each item out loud (Beatty & Willis, 2007). We utilized cognitive probes as needed (Beatty & Willis, 2007). Examples of cognitive probes used in this study included: “You paused for a moment. What were you thinking then?” and “Are you having difficulty answering that question?” Debriefing occurred after participants answered all of the survey items to elicit any last opinions of the survey. Examples of debriefing questions included: “How many response options do you think are appropriate for each question and why?” and “Is there anything you would like to add or share with the researchers who designed the survey before we finish up today?”

Data Analysis

To analyze responses, we transcribed audio recordings from the cognitive interviews and compiled the narrative data across interviews (Beatty & Willis, 2007). We followed Conrad and Blair’s (1996) three stage approach for analyzing Think Aloud data, that is, understanding, performing the primary task, and response formatting. Understanding involves understanding the directions, survey items, and the implied task. Performing the primary task involves the mental operation to produce a response. Response formatting involves providing an acceptable response from among the available response options. Conrad & Blair’s (1996) approach is useful for analyzing participants’ verbal responses to survey items, because participants may not complete items that are complex, lengthy, or sensitive (e.g., about health history or income). This approach is systematic and thorough and as such can assist in identifying potential issues with a survey.

Results

Aim 1: Development of Survey Items

Through an iterative process, a 45-item survey was developed to be used in future research to describe women veterans’ reasons for, and barriers to, care-seeking (Version 1.0).

Aim 2: Assessment of Content Validity of Survey Items

Eight of 27 content experts (30%) assessed the content validity of the 45-item survey. I-CVIs ranged from 0.55 to 1.00. Content experts rated the clarity and relevance of all items 0.89 and 0.98, respectively. Content experts rated the comprehensiveness and gender-sensitivity of the sets of items as 0.96 and 0.96, respectively.

Based on feedback, we retained 19 items, lightly refined 26 items, and added seven items. For example, for clarity, we refined one response option from “my partner/spouse says I should go” to “my partner/spouse says I should not go.” We did not omit any items; none were scored less than 0.78 on relevance or gender-sensitivity. Experts suggest 0.90 is excellent content validity (Polit & Beck, 2021). The average content validity score of our newly developed items was 0.99, revealing excellent content validity. After survey revisions, we had a 52-item survey (Version 2.0).

Aim 3: Assessment of Women Veterans Understanding of Proposed Survey Items

We conducted cognitive interviews once with each of seven participants. On average, cognitive interviews lasted approximately 52 minutes. All participants were able to access the online survey. Forty-three percent of participants reported using a smart phone, 57% a computer, none reported using a tablet. Most participants self-identified as White. All participants had some form of health insurance, and all had at least some college education. The characteristics of participants are presented in Table 1.

Table 1

Sociodemographic Characteristics of Women Veterans Who Participated in Cognitive Interviews.


PARTICIPANT AGE RACE ETHNICITY MARITAL STATUS HIGHEST LEVEL OF EDUCATION EMPLOYMENT STATUS HOUSEHOLD INCOME TYPE OF HEALTH INSURANCE BRANCH OF SERVICE LENGTH OF MILITARY SERVICE

1 36 White White Married Doctorate Part-time >$100,000 Private, VA health coverage Army 6–10 years

2 42 White White Single Bachelor’s degree Full-time $50,000–$74,999 Employer-based,
VA health coverage
Army 11–15 years

3 35 White White Married Associate degree Full-time $75,000–$99,999 Private Army 16–20 years

4 36 White White Married Master’s degree Full-time $75,000–$99,999 Private, VA health coverage Army 6–10 years

5 32 Black or African American Non-Hispanic Single Professional degree Part-time, student <$10,000 Private Air Force 6–10 years

6 42 White White Married Bachelor’s degree Student $50,000–$74,999 Tricare Navy >20 years

7 23 White White Separated Some college Part-time, student $20,000–$34,999 VA health coverage Marines 5 years or less

Note: VA = US Department of Veterans Affairs.

Understanding

Overall, most participants were able to understand the directions, survey items, and the implied task. However, a few participants had difficulty understanding a few directions and a few survey items. Of the participants who accessed the survey using phones, most suggested reducing the length of directions. Participants identified that repeating directions were redundant and unnecessary, and lengthy questions were difficult to comprehend. For example, Participant 3 stated, “I’ve seen other surveys with the directions on one slide, and in the Army, they keep firing at ya; they put directions on one slide and then the next slide will have the very simple instructions.” Also, a few participants suggested shortening the stem of a few items to make them easier to comprehend. For example, Participant 1 commented on one open-ended item. She stated, “this is way too wordy.” She went on to state, “You don’t have to say ‘describe’ and then ‘write your thoughts below.’ Maybe just have a statement in the beginning and say write your thoughts.”

Performing the Primary Task

Overall, most participants were able to conduct the mental operations to produce responses to most survey items. However, we identified that some participants had difficulty developing responses to some items. For instance, some participants suggested there were too many response options to mentally process. Participants who accessed the survey using smart phones reported that most of the response options did not fit on their screens. This caused some participants to have to scroll to finish reading the item and the corresponding response options. For example, Participant 1 stated, “My brain is trying to figure out what you are trying to ask me … [pause] there are too many options.” Similarly, Participant 5 stated, “I just had to go back up to the top to figure out which way I am answering this … There are a lot of choices to pick from. It would be helpful to scale it down a bit.”

During the cognitive interviews, we asked participants how many response options would be reasonable for each item. Participants’ answers varied from three to seven. When asked to rank their reasons for, and barriers to, care-seeking, some participants had a difficult time answering those items. Participants deemed it difficult to rank seven options from “1” “most important” to “7” “least important.” For example, Participant 3 stated, “[It] is asking me to put a number in here and I can do that … I am only going to rank three … and then move on to the next one.”

Response Formatting

Overall, most participants were able to provide acceptable responses to items from among the available response options in the survey. However, we identified that a few of participants’ verbal responses did not coincide with the available response options. For example, one participant suggested that feeling harassed by other veterans in the healthcare setting was her reason to avoid health care. Her desired response did not coincide with any response options on the survey. She stated,

One thing that I have noticed as a woman vet is that when I walk into the VA … There are a lot of stares, like staring, like looking at me … There is a little less awareness about ‘keeping your eyes to yourself.’ I don’t know if there is a way to describe that.

Additional Topics

During the cognitive interviews, we identified two additional topics relevant to the survey. First, from participants’ responses to items, we identified that the concept awareness, that is, the “knowledge or perception of a situation or fact,” (Lexico Oxford Dictionary, 2021) is relevant to care-seeking for CVD prevention. For instance, some participants described their lack of knowledge of CVD, some were unaware of their risks for developing CVD, some were not aware that they had been tested for some risks for CVD (e.g., blood pressure, weight), and some were not aware of where they could receive care for CVD prevention. For example, Participant 1 revealed, “I am counting on my primary care provider to tell me if I have a problem [with CVD] or not.” Moreover, Participant 6 stated, “some [women veterans] don’t realize that they can go to the VA for health care.” These examples reflect the concept awareness.

A second topic identified was that some women veterans have mental health concerns that may interfere with care-seeking in general and for CVD in particular. Participants mentioned posttraumatic stress disorder (PTSD) and military sexual trauma. For example, Participant 1 stated, “Being a veteran, I might be more at risk [for CVD] because of past trauma.” She revealed that she believes some women veterans may have a higher chance of developing CVD in their lifetime because of “trauma, military sexual trauma, and the chronic effects of trauma.” Participant 2 stated, “I don’t know if there is a place to add this, but as you know, people who have PTSD have a hard time in some public settings, so I don’t know if that is completely described here.”

Acceptability of the survey. During the debriefing, most participants voluntarily expressed their appreciation with this study. Regarding the survey, most participants recognized the importance of studying women veterans and CVD prevention. Participant 2 stated, “I believe in research, so that is why this [study] is important. Thank you for doing it [conducting this study].” Participant 5 exclaimed, “This is important. I didn’t know about this whole cardiovascular thing, so it is making me check my health different, right after I get off the phone with you!” See Appendix B for examples of some revised directions and items based on participants’ feedback during cognitive interviews.

Discussion

We have described the development, evaluation, and pretesting of a new survey designed for women veterans. This research builds upon others’ work to describe women veterans’ barriers to care-seeking in general (e.g., US Department of Veterans Affairs, 2015), has added to the literature on reasons for care-seeking in general, and has addressed a gap in the literature to describe women veterans’ reasons for care-seeking for CVD prevention in particular. Also, this research is novel because it was guided by theory and included a breadth of variables that could influence women veterans’ care-seeking for CVD prevention.

By addressing Aim 1, we generated survey items based on relevant literature and prior TCSB research to be evaluated by content experts. By addressing Aim 2, we found support for the clarity, relevance, comprehensiveness, and gender-sensitivity of most proposed items. Based on experts’ ratings and feedback, we revised our survey for pretesting with women veterans. The survey had near excellent content validity. By addressing Aim 3, we evaluated the target population’s understanding of proposed items with cognitive interviews. Overall, women veterans understood most survey directions and items and answered the items accurately and reliably. The implications of these findings lend support for the validity of the survey.

With findings from these interviews, we identified five areas to improve our survey. One, regarding understanding, we identified that a few directions and items were difficult to comprehend because some were too lengthy. Therefore, we removed a few redundant and unnecessary instructions, and made a few items clearer and more concise, to make the survey more understandable.

Two, regarding performing the primary task, we identified that participants, especially those taking the survey on their smart phones, were unable to answer items if there were too many response options. Thus, we reduced the number of response options for each item, so participants were able to mentally process each option and produce responses to items. We added the option “other” so participants could elaborate, if needed. Also, participants endorsed that they would have a difficult time ranking their top seven responses. Thus, we revised the two ranking items, so participants only ranked their top five responses, reducing the burden on participants. This is important because we desired accurate responses from participants.

Three, regarding response formatting, we identified that some participants’ desired responses did not correspond with available response options. Thus, we developed a few new response options to address these omissions, if needed. For example, we included the response option “harassed in the healthcare setting” as a barrier to care-seeking, designed to reflect the concept affect. Our finding is supported by prior research (e.g., Klap et al., 2019). Klap and colleagues (2019) found that women veterans who identified as being harassed were more likely to delay health care.

Four, regarding response formatting, we identified that some participants did not know what CVD is, that it is serious, and is the leading cause of death of women (Virani et al., 2021). Also, we identified that women veterans may not be aware that health care services are available to help reduce that risk. We propose that awareness is relevant to women veterans’ care-seeking, as evidenced in prior research (Biswas et al., 2002; Canter et al., 2009). Thus, we added two items to reflect the concept awareness to the survey.

Five, we identified that some participants have mental health concerns such as PTSD that may influence their care-seeking behavior for CVD prevention. Because women veterans’ mental health has been associated with their cardiovascular health, we added some valid and reliable measures to the survey, such as the Primary Care PTSD screen (Ouimette et al., 2008; Prins et al., 2004) and the military sexual trauma classification (Kimerling et al., 2007). The way in which we have asked some of these items would make these findings comparable to research by others interested in women veterans’ cardiovascular health (e.g., Cavanagh et al., 2020). After incorporating all refinements, the Women Veterans Health Study (Survey Version 3.0) was developed. See Appendix C.

Findings from interviews indicated that some women veterans were familiar with the term CVD, others were not. Surprisingly, most participants in this study were not aware that they themselves may be at risk for developing CVD in their lifetime. This finding is consistent with other research which suggests young women, as well as health care providers, often underestimate women’s risk for CVD (Leifheit-Limson et al., 2015; Wilbur et al., 2018).

Because some women veterans in this study were unaware of their risk for CVD and lacked knowledge of available health care services for women veterans for CVD prevention, adding the concept awareness to the survey may prove to be useful in a study to describe women veterans’ care-seeking. By adding the concept awareness to survey, the most recent version would be consistent with some previous healthcare research on heart disease (e.g., Cushman et al., 2021), women veterans’ knowledge of available health services (e.g., US Department of Veterans Affairs, 2015), and CVD prevention (e.g., Cavanagh et al., 2020). Future research could build upon previous TCSB research (e.g., Backonja et al., 2014; Lauver et al., 2003; Lor et al., 2013) by incorporating and testing the refined survey, including the concept awareness.

We demonstrated that assessing content validity as well as incorporating participants’ feedback through cognitive interviewing are complementary methods that can be useful to evaluate a newly developed survey. Pretesting the survey among experts as well as the population of interest has offered generous data from which we revised items for future use with women veterans.

Limitations

This study has its limitations. Due to the pandemic, the primary author conducted cognitive interviews over the telephone rather than in person and, as a result, was unable to view participants’ behaviors. However, by utilizing Willis’ (2007) Think Aloud method participants were able to answer survey items and tell the researcher what they were thinking synchronously, and the researcher was able to identify verbal behaviors such as hesitation, pauses, and skipping of questions that led to changes in the survey (Drennan, 2003). Willis’ method proved to be a useful approach to guide cognitive interviews when researchers were unable to meet interviewees in person. Additionally, only one interviewer conducted one round of cognitive interviews with participants. Beatty and Willis (2007) have suggested that additional interviewers could yield new insights about or problems with the survey. Yet, there have been well developed, published, peer-reviewed studies that have only employed one interviewer and one round of interviews (e.g., Yu, et al., 2019).

Conclusion

In an iterative manner, we developed and evaluated a survey about women veterans’ care-seeking for CVD prevention. This study has several strengths. The survey items were based on bodies of literature regarding care-seeking, CVD prevention, and women veterans’ health research. Also, we generated items based on concepts from TCSB and included a breadth of variables that could influence women veterans’ care-seeking. Findings from content experts informed our revisions resulting in a more clear, relevant, comprehensive and gender-sensitive survey than previously found in the literature. These findings lend support for the content validity of the survey. Cognitive interviews yielded rich, meaningful feedback from participants about how they processed, that is, understood and answered the items accurately and reliably. The implications of these findings lend support for the validity of the survey. Spontaneous feedback during the cognitive interviews affirmed the importance of describing younger women veterans’ reasons for, and barriers to, care-seeking for CVD prevention. Future research could use the refined survey to collect survey data from this population. Also, researchers could utilize TCSB to guide future research on care-seeking in different contexts and populations.

Additional Files

The additional files for this article can be found as follows:

Appendixes

Appendixes A, B and C. DOI: https://doi.org/10.21061/jvs.v9i1.319.s1

Supplemental Files

Supplemental File 2 and 3. DOI: https://doi.org/10.21061/jvs.v9i1.319.s2

Acknowledgements

The authors would like to thank Brenda Owen, Lacey Alexander, and Cissy Ondoma for their assistance with the content validity phase of this study, as well as the content experts who provided feedback. Also, the authors would like to thank Zhiyuan Yu and Maichou Lor for their guidance on the cognitive interview process, as well as the women veterans who participated in the cognitive interviews.

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States Government.

Funding information

Conger Research and Scholarship Award, School of Nursing, University of Wisconsin–Madison.

Competing Interests

The authors have no competing interests to declare.

References

  1. Arnett, D. K., Blumenthal, R. S., Albert, M. A., Buroker, A. B., Goldberger, Z. D., Hahn, E. J., Himmelfarb, C. D., Khera, A., Lloyd-Jones, D., McEvoy, J. W., Michos, E. D., Miedema, M. D., Muñoz, D., Smith, S. C., Virani, S. S., Williams, K. A., Yeboah, J., & Ziaeian, B. (2019). 2019 ACC/AHA guideline on the primary prevention of cardiovascular disease: A report of the American College of Cardiology/American Heart Association task force on clinical practice guidelines. Journal of the American College of Cardiology, 74(10), e177–e232. DOI: https://doi.org/10.1016/j.jacc.2019.03.010 

  2. Backonja, U., Royer, H. R., & Lauver, D. R. (2014). Young women’s reasons to seek sexually transmitted infection screening. Public Health Nursing, 31(5), 395–404. DOI: https://doi.org/10.1111/phn.12125 

  3. Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287–311. DOI: https://doi.org/10.1093/poq/nfm006 

  4. Benjamin, E. J., Muntner, P., Alonso, A., Bittencourt, M. S., Callaway, C. W., Carson, A. P., Chamberlain, A. M., Chang, A. R., Cheng, S., Das, S. R., Delling, F. N., Djousse, L., Elkind, M. S. V., Ferguson, J. F., Fornage, M., Jordan, L. C., Khan, S. S., Kissela, B. M., Knutson, K. L., Kwan, T. W., … American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. (2019). Heart disease and stroke statistics—2019 update: A report from the American Heart Association. Circulation, 139(10), e56–e528. DOI: https://doi.org/10.1161/CIR.0000000000000659 

  5. Benjamin, E. J., Virani, S. S., Callaway, C. W., Chamberlain, A. M., Chang, A. R., Cheng, S., Chiuve, S. E., Cushman, M., Delling, F. N., Deo, R., de Ferranti, S. D., Ferguson, J. F., Fornage, M., Gillespie, C., Isasi, C. R., Jiménez, M. C., Jordan, L. C., Judd, S. E., Lackland, D., Lichtman, J. H., … American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. (2018). Heart disease and stroke statistics—2018 update: A report from the American Heart Association. Circulation, 137(12), e67–e492. DOI: https://doi.org/10.1161/CIR.0000000000000558 

  6. Biswas, M. S., Calhoun, P. S., Bosworth, H. B., & Bastian, L. A. (2002). Are women worrying about heart disease? Women’s Health Issues, 12(4), 204–211. DOI: https://doi.org/10.1016/S1049-3867(02)00136-6 

  7. Breland, J. Y., Phibbs, C. S., Hoggatt, K. J., Washington, D. L., Lee, J., Haskell, S., Uchendu, U. S., Saechao, F. S., Zephyrin, L. C., & Frayne, S. M. (2017). The obesity epidemic in the Veterans Health Administration: Prevalence among key populations of women and men veterans. Journal of General Internal Medicine, 32(1), 11–17. DOI: https://doi.org/10.1007/s11606-016-3962-1 

  8. Byrne, S. K. (2008). Healthcare avoidance: A critical review. Holistic Nursing Practice, 22(5), 280–292. DOI: https://doi.org/10.1097/01.HNP.0000334921.31433.c6 

  9. Canter, D. L., Atkins, M. D., McNeal, C. J., & Bush, R. L. (2009). Risk factor treatment in veteran women at risk for cardiovascular disease. Journal of Surgical Research, 157(2), 175–180. DOI: https://doi.org/10.1016/j.jss.2008.07.014 

  10. Cavanagh, C., Rosman, L., Bastian, L., Brandt, C., Haskell, S., & Burg, M. (2020). Barriers to cardiovascular disease preventive behaviors among OEF/OIF/OND women and men veterans. Health Psychology. DOI: https://doi.org/10.1037/hea0000844 

  11. Chinn, P. L. (1995). Feminism and nursing. Annual Review of Nursing Research, 13(1), 267–289. DOI: https://doi.org/10.1891/0739-6686.13.1.267 

  12. Conrad, F., & Blair, J. (1996, August). From impressions to data: Increasing the objectivity of cognitive interviews. In Proceedings of the section on survey research methods, annual meetings of the American Statistical Association (Vol. 1, No. 10). Alexandria: American Statistical Association. https://www.bls.gov/osmr/research-papers/1996/pdf/st960080.pdf 

  13. Cushman, M., Shay, C. M., Howard, V. J., Jiménez, M. C., Lewey, J., McSweeney, J. C., Newby, L. K., Poudel, R., Reynolds, H. R., Rexrode, K. M., Sims, M., Mosca, L. J., & null null. (2021). Ten-year differences in women’s awareness related to coronary heart disease: Results of the 2019 American Heart Association National survey: A Special report from the American Heart Association. Circulation, 143(7), e239–e248. DOI: https://doi.org/10.1161/CIR.0000000000000907 

  14. deKleijn, M., Lagro-Janssen, A. L. M., Canelo, I., & Yano, E. M. (2015). Creating a roadmap for delivering gender-sensitive comprehensive care for women veterans: Results of a national expert panel. Medical Care, 53(4 Suppl 1), S156–164. DOI: https://doi.org/10.1097/MLR.0000000000000307 

  15. Delcher, C., Wang, Y., & Maldonado-Molina, M. (2013). Trends in financial barriers to medical care for women veterans, 2003–2004 and 2009–2010. Preventing Chronic Disease, 10. DOI: https://doi.org/10.5888/pcd10.130071 

  16. Drennan, J. (2003). Cognitive interviewing: Verbal data in the design and pretesting of questionnaires. Journal of Advanced Nursing, 42(1), 57–63. DOI: https://doi.org/10.1046/j.1365-2648.2003.02579.x 

  17. Goldstein, K. M., Melnyk, S. D., Zullig, L. L., Stechuchak, K. M., Oddone, E., Bastian, L. A., Rakley, S., Olsen, M. K., & Bosworth, H. B. (2014). Heart matters: Gender and racial differences cardiovascular disease risk factor control among veterans. Women’s Health Issues, 24(5), 477–483. DOI: https://doi.org/10.1016/j.whi.2014.05.005 

  18. Goldstein, K. M., Oddone, E. Z., Bastian, L. A., Olsen, M. K., Batch, B. C., & Washington, D. L. (2017). Characteristics and health care preferences associated with cardiovascular disease risk among women veterans. Women’s Health Issues, 27(6), 700–706. DOI: https://doi.org/10.1016/j.whi.2017.08.002 

  19. Hamilton, A. B., Poza, I., Hines, V., & Washington, D. L. (2012). Barriers to psychosocial services among homeless women veterans. Journal of Social Work Practice in the Addictions, 12(1), 52–68. DOI: https://doi.org/10.1080/1533256X.2012.647584 

  20. Honein-AbouHaidar, G. N., Kastner, M., Vuong, V., Perrier, L., Daly, C., Rabeneck, L., Straus, S., & Baxter, N. N. (2016). Systematic review and meta-study synthesis of qualitative studies evaluating facilitators and barriers to participation in colorectal cancer screening. Cancer Epidemiology and Prevention Biomarkers, 25(6), 907–917. DOI: https://doi.org/10.1158/1055-9965.EPI-15-0990 

  21. Im, E.-O., & Chang, S. J. (2012). Current trends in nursing theories. Journal of Nursing Scholarship, 44(2), 156–164. DOI: https://doi.org/10.1111/j.1547-5069.2012.01440.x 

  22. Im, E.-O., & Chee, W. (2001). A feminist critique on the use of the internet in nursing research. Advances in Nursing Science, 23(4), 67. DOI: https://doi.org/10.1097/00012272-200106000-00006 

  23. Jansen, H., & Hak, T. (2005). The productivity of the Three-Step Test-Interview (TSTI) compared to an expert review of a self-administered questionnaire on alcohol consumption. Journal of Official Statistics: An International Quarterly, 21(1), 103–120. 

  24. Katon, J., Mattocks, K., Zephyrin, L., Reiber, G., Yano, E. M., Callegari, L., Schwarz, E. B., Goulet, J., Shaw, J., Brandt, C., & Haskell, S. (2014). Gestational diabetes and hypertensive disorders of pregnancy among women veterans deployed in service of operations in Afghanistan and Iraq. Journal of Women’s Health, 23(10), 792–800. DOI: https://doi.org/10.1089/jwh.2013.4681 

  25. Kimerling, R., Gima, K., Smith, M. W., Street, A., & Frayne, S. (2007). The Veterans Health Administration and military sexual trauma. American Journal of Public Health, 97(12), 2160–2166. DOI: https://doi.org/10.2105/AJPH.2006.092999 

  26. Klap, R., Darling, J. E., Hamilton, A. B., Rose, D. E., Dyer, K., Canelo, I., Haskell, S., & Yano, E. M. (2019). Prevalence of stranger harassment of women veterans at Veterans Affairs Medical Centers and impacts on delayed and missed care. Women’s Health Issues, 29(2), 107–115. DOI: https://doi.org/10.1016/j.whi.2018.12.002 

  27. Lauver, D. (1992). A theory of care-seeking behavior. The Journal of Nursing Scholarship, 24(4), 281–288. DOI: https://doi.org/10.1111/j.1547-5069.1992.tb00734.x 

  28. Lehavot, K., Hoerster, K. D., Nelson, K. M., Jakupcak, M., & Simpson, T. L. (2012). Health indicators for military, veteran, and civilian women. American Journal of Preventive Medicine, 42(5), 473–480. DOI: https://doi.org/10.1016/j.amepre.2012.01.006 

  29. Lehavot, K., Katon, J. G., Chen, J. A., Fortney, J. C., & Simpson, T. L. (2018). Post-traumatic stress disorder by gender and veteran status. American Journal of Preventive Medicine, 54(1), e1–e9. DOI: https://doi.org/10.1016/j.amepre.2017.09.008 

  30. Leifheit-Limson, E. C., D’Onofrio, G., Daneshvar, M., Geda, M., Bueno, H., Spertus, J. A., Krumholz, H. M., & Lichtman, J. H. (2015). Sex differences in cardiac risk factors, perceived risk, and health care provider discussion of risk and risk modification among young patients with acute myocardial infarction: The VIRGO study. Journal of the American College of Cardiology, 66(18), 1949–1957. DOI: https://doi.org/10.1016/j.jacc.2015.08.859 

  31. Lexico Oxford Dictionary. (2021). Aware. Retrieved November 1, 2021, from https://www.lexico.com/en/definition/aware 

  32. Littman, A. J., True, G., Ashmore, E., Wellens, T., & Smith, N. L. (2018). How can we get Iraq- and Afghanistan-deployed US Veterans to participate in health-related research? Findings from a national focus group study. BMC medical research methodology, 18(1), 88. DOI: https://doi.org/10.1186/s12874-018-0546-2 

  33. Lor, M., Backonja, U., & Lauver, D. R. (2017). How could nurse researchers apply theory to generate knowledge more efficiently? Journal of Nursing Scholarship, 49(5), 580–589. DOI: https://doi.org/10.1111/jnu.12316 

  34. Lor, M., Khang, P. Y., Xiong, P., Moua, K. F., & Lauver, D. (2013). Understanding Hmong women’s beliefs, feelings, norms, and external conditions about breast and cervical cancer screening. Public Health Nursing, 30(5), 420–428. DOI: https://doi.org/10.1111/phn.12043 

  35. Lutwak, N., & Dill, C. (2013). Military sexual trauma increases risk of post-traumatic stress disorder and depression thereby amplifying the possibility of suicidal ideation and cardiovascular disease. Military Medicine, 178(4), 359–361. DOI: https://doi.org/10.7205/MILMED-D-12-00427 

  36. McSweeney, J. C., O’Sullivan, P., Cody, M., & Crane, P. B. (2004). Development of the McSweeney Acute and Prodromal Myocardial Infarction Symptom Survey. Journal of Cardiovascular Nursing, 19(1), 58–67. https://journals.lww.com/jcnjournal/Fulltext/2004/01000/Development_of_the_McSweeney_Acute_and_Prodromal.10.aspx?casa_token=Kv-RnSr-ogIAAAAA:VztMCgMGMoq6CF9nKcq3Ryc97VNh7dqwkVIoVu122qCmdoug8ZqTS4ORgNGAE1p-zmNYsA4_VF-n1K_WS5Gly7H-cjE 

  37. Melnyk, K. M. (1988). Barriers: A critical review of recent literature. Nursing Research, 37(4), 196–201. DOI: https://doi.org/10.1097/00006199-198807000-00002 

  38. Melnyk, K. M. (1990). Barriers to care: Operationalizing the variable. Nursing Research, 39(2), 108–112. DOI: https://doi.org/10.1097/00006199-199003000-00013 

  39. Mochari-Greenberger, H., Miller, K. L., & Mosca, L. (2012). Racial/ethnic and age differences in women’s awareness of heart disease. Journal of Women’s Health, 21(5), 476–480. DOI: https://doi.org/10.1089/jwh.2011.3428 

  40. Mochari-Greenberger, H., Mills, T., Simpson, S. L., & Mosca, L. (2010). Knowledge, preventive action, and barriers to cardiovascular disease prevention by race and ethnicity in women: An American Heart Association national survey. Journal of Women’s Health, 19(7), 1243–1249. DOI: https://doi.org/10.1089/jwh.2009.1749 

  41. Mosca, L., Benjamin, E. J., Berra, K., Bezanson, J. L., Dolor, R. J., Lloyd-Jones, D. M., Newby, L. K., Piña, I. L., Roger, V. L., Shaw, L. J., Zhao, D., Beckie, T. M., Bushnell, C., D’Armiento, J., Kris-Etherton, P. M., Fang, J., Ganiats, T. G., Gomes, A. S., Gracia, C. R., … Wenger, N. K. (2011). Effectiveness-based guidelines for the prevention of cardiovascular disease in women—2011 update: A guideline from the American Heart Association. Circulation, 123(11), 1243–1262. DOI: https://doi.org/10.1161/CIR.0b013e31820faaf8 

  42. Mosca, L., Hammond, G., Mochari-Greenberger, H., Towfighi, A., & Albert, M. A. (2013). Fifteen-year trends in awareness of heart disease in women. Circulation, 127(11), 1254–1263. DOI: https://doi.org/10.1161/CIR.0b013e318287cf2f 

  43. Murdoch, M., Bradley, A., Mather, S. H., Klein, R. E., Turner, C. L., & Yano, E. M. (2006). Women and war: What physicians should know. Journal of General Internal Medicine, 21(S3), S5–S10. DOI: https://doi.org/10.1111/j.1525-1497.2006.00368.x 

  44. National Library of Medicine. (n.d.). Surveys [Training Material and Manuals]. Retrieved February 17, 2021, from https://www.nlm.nih.gov/nichsr/stats_tutorial/section3/mod1_surveys.html 

  45. Odani, S., Agaku, I. T., Graffunder, C. M., Tynan, M. A., & Armour, B. S. (2018). Tobacco product use among military veterans—United States, 2010–2015. Morbidity and Mortality Weekly Report, 67(1), 7–12. DOI: https://doi.org/10.15585/mmwr.mm6701a2 

  46. Orshak, J., Alexander, L., Gilmore-Bykovskyi, A., & Lauver, D. (2022). Interventions for women veterans with mental health care needs: Findings from a scoping review. Issues in Mental Health Nursing, 43(6), 516–527. DOI: https://doi.org/10.1080/01612840.2021.2011506 

  47. Ouimette, P., Wade, M., Prins, A., & Schohn, M. (2008). Identifying PTSD in primary care: Comparison of the Primary Care-PTSD Screen (PC-PTSD) and the General Health Questionnaire-12 (GHQ). Journal of Anxiety Disorders, 22(2), 337–343. DOI: https://doi.org/10.1016/j.janxdis.2007.02.010 

  48. Polit, D. F., & Beck, C. T. (2021). Nursing research: Generating and assessing evidence for nursing practice (11th ed.). Wolters Kluwer Health. 

  49. Ponto, J. (2015). Understanding and evaluating survey research. Journal of the Advanced Practitioner in Oncology, 6(2), 168–171. DOI: https://doi.org/10.6004/jadpro.2015.6.2.9 

  50. Prins, A., Ouimette, P., Kimerling, R., Camerond, R. P., Hugelshofer, D. S., Shaw-Hegwer, J., Thrailkill, A., Gusman, F. D., & Sheikh, J. I. (2004). The primary care PTSD screen (PC–PTSD): Development and operating characteristics. Primary Care Psychiatry, 9(1), 9–14. DOI: https://doi.org/10.1185/135525703125002360 

  51. Qualtrics. (2019). Qualtrix. https://www.qualtrics.com 

  52. Qualtrics. (2021). Preview survey. https://www.qualtrics.com/support/survey-platform/survey-module/preview-survey 

  53. Speziale, H. S., Streubert, H. J., & Carpenter, D. R. (2011). Qualitative research in nursing: Advancing the humanistic imperative. Lippincott Williams & Wilkins. 

  54. Tsai, J., Mota, N. P., & Pietrzak, R. H. (2015). U.S. female veterans who do and do not rely on VA health care: Needs and barriers to mental health treatment. Psychiatric Services, 66(11), 1200–1206. DOI: https://doi.org/10.1176/appi.ps.201400550 

  55. US Department of Veterans Affairs. (2015). Study of barriers for women veterans to V.A. health care. https://www.womenshealth.va.gov/docs/Womens%20Health%20Services_Barriers%20to%20Care%20Final%20Report_April2015.pdf?source=post_page 

  56. Virani, S. S., Alonso, A., Aparicio, H. J., Benjamin, E. J., Bittencourt, M. S., Callaway, C. W., Carson, A. P., Chamberlain, A. M., Cheng, S., Delling, F. N., Elkind, M. S. V., Evenson, K. R., Ferguson, J. F., Gupta, D. K., Khan, S. S., Kissela, B. M., Knutson, K. L., Lee, C. D., Lewis, T. T., Liu, J., … American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. (2021). Heart disease and stroke statistics—2021 update: A report from the American Heart Association. Circulation, 143(8), e254–e743. DOI: https://doi.org/10.1161/CIR.0000000000000950 

  57. Wang, H., Roy, S., Kim, J., Farazi, P. A., Siahpush, M., & Su, D. (2019). Barriers of colorectal cancer screening in rural USA: A systematic review. Rural and Remote Health, 19(3). DOI: https://doi.org/10.22605/RRH5181 

  58. Washington, D. L., Bean-Mayberry, B., Riopelle, D., & Yano, E. M. (2011). Access to care for women veterans: Delayed healthcare and unmet need. Journal of General Internal Medicine, 26(2), 655. DOI: https://doi.org/10.1007/s11606-011-1772-z 

  59. Washington, D. L., Farmer, M. M., Mor, S. S., Canning, M., & Yano, E. M. (2015). Assessment of the healthcare needs and barriers to VA use experienced by women veterans: Findings from the National Survey of Women Veterans. Medical Care, 53, S23. DOI: https://doi.org/10.1097/MLR.0000000000000312 

  60. Washington, D. L., Yano, E. M., Simon, B., & Sun, S. (2006). To use or not to use. Journal of General Internal Medicine, 21(S3), S11–S18. DOI: https://doi.org/10.1111/j.1525-1497.2006.00369.x 

  61. Wenger, N. K. (2019). Female-friendly focus: 2019 ACC/AHA guideline on the primary prevention of cardiovascular disease. Clinical Cardiology, 42(8), 706–709. DOI: https://doi.org/10.1002/clc.23218 

  62. Whitehead, A. M., Davis, M. B., Duvernoy, C., Safdar, B., Nkonde-Price, C., Iqbal, S., Balasubramanian, V., Frayne, S. M., Hayes, P. M., & Haskell, S. G. (2013). The state of cardiovascular health in women veterans volume 1: VA Outpatient Diagnoses and Procedures in Fiscal Year (FY) 2010. Women’s Health Services, US Department of Veteran Affairs. https://www.womenshealth.va.gov/docs/08272014_Cardio_Report_FINAL.pdf 

  63. Wilbur, J., Braun, L. T., Arslanian-Engoren, C., Lauver, D. R., & Halloway, S. (2018). Assessing and addressing cardiovascular risk in young women. Nursing Outlook, 66(3), 325–328. DOI: https://doi.org/10.1016/j.outlook.2018.04.005 

  64. Wilson, L. C. (2018). The prevalence of military sexual trauma: A meta-analysis. Trauma, Violence, & Abuse, 19(5), 584–597. DOI: https://doi.org/10.1177/1524838016683459 

  65. Wolff, K. B., & Mills, P. D. (2016). Reporting military sexual trauma: A mixed-methods study of women veterans’ experiences who served from World War II to the war in Afghanistan. Military Medicine, 181(8), 840–848. DOI: https://doi.org/10.7205/MILMED-D-15-00404 

  66. Yu, Z., Lauver, D. R., Wang, L., & Li, H. (2019). Developing and pretesting a survey about immigrant women’s experiences with postpartum depression screening. Journal of Obstetric, Gynecologic & Neonatal Nursing, 48(2), 216–226. DOI: https://doi.org/10.1016/j.jogn.2019.01.008 

comments powered by Disqus