Toward Veteran-Centered Research: A Veteran-Focused Community Engagement Project

Although it is widely acknowledged that consumer engagement in healthcare research is a promising pathway to actionable patient-centered findings, there remains some ambiguity regarding effective patient-centered engagement strategies. Therefore, the purpose of this US-based multi-state project was to create a platform from which US veterans could dialogue and articulate (1) their research priorities; (2) perceived barriers to research partnerships and participation; (3) recommendations for engaging other veterans in the research enterprise; and (4) preferences about how they would like to receive research findings. A total of 283 veterans and 101 community stakeholders participated across 54 dialogue sessions or Think Tank Meetings (TTMs) held in six states with the largest veteran populations: California, Texas, Florida, Pennsylvania, New York, and Ohio. Each TTM was led by a local veteran with strong ties to the veteran community. To address veteran concerns about the research process, a local researcher from the project team recorded field notes rather than audiotaping, which were analyzed by the lead research team using open and axial coding. Based on veterans’ recommendations, a list of veteran-centered research priorities is presented, as are two checklists designed to guide research teams seeking to engage veterans in research and disseminate findings to the veteran community.


Introduction
The engagement of consumers in health-related research is a widely accepted goal within the scientific community. According to the Patient-Centered Outcomes Research Institute (PCORI), this patient-centered approach allows patients' voices to be heard in the selection of research questions, study methods, recruitment of participants, interpretation of findings, and dissemination of results (Sheridan, Schrandt, Forsythe, Hillard, & Paez, 2017). This crucial patient-researcher collaboration is dependent, however, upon the success of the research team in engaging members of the target patient population in these processes.
Unfortunately, effective patient-centered engagement strategies remain elusive (Fagerlin, 2018;McGavin, 2015). This is particularly evident among the nation's 22 million US veterans. Despite a critical need for evidence-based approaches to service-related health issues, veterans are frequently reluctant to partner with or participate in the research enterprise (Bush, Sheppard, Fantell, & Bell, 2013;Braun, Kennedy, Sadler, & Dixon, 2015;Littman, True, Ashmore, Wellens, & Smith, 2018;Williams, Gatien, & Hagerty, 2012;Funderburk, Spinola, & Maisto, 2015). The purpose of this multi-state project was to invite veterans to dialogue about (1) their research priorities; (2) perceived barriers to research participation; (3) recommended strategies for engaging veterans in research; and (4) their preferences for receiving research findings. Two checklists outlining veteran-centered strategies for veteran engagement were derived from these extensive dialogue sessions. These strategies offer researchers the opportunity to engage veterans as full partners in the research enterprise.
Background The healthcare needs of the nation's military veterans differ strikingly from those of the general US population. Unlike civilians, many veterans have been exposed to the potential hazards of open burning pits, chemical agents such as Agent Orange, or are suffering from traumatic injuries resulting from improvised explosive devices (Krause-Parello & Morales, 2018). Moreover, veterans are significantly more likely than civilians to suffer from post-traumatic stress disorder (PTSD), bipolar disorder, depression, and anxiety (Fortney, et al., 2016;Lehavot et al., 2018;Liu, Collins, Wang, & Bie, 2019;National Center for PTSD, 2018;Shepardson, Kosiba, Bernstein, & Funderburk, 2019). Not surprisingly, veterans are also 40% more likely to suffer from chronic pain when compared to the general population (Akhtar, Ballew, Orr, Mayorga, & Khan, 2018;Cichowski, et al., 2017: Nahin, 2017. Obviously, many of these health conditions are the direct result of combat conditions, and a sufficient body of evidence pertaining to the acceptability, practicality, and efficacy of various treatment options has yet to be developed. Clearly, there is a critical need for comparative effectiveness research (CER) and veteran-centered outcomes research (V-COR) to inform high value, efficacious healthcare for our nation's veterans (Eckardt, et al., 2017).
Despite ethnic, racial, and gender diversity, veterans frequently share in a distinctive military culture characterized by duty, honor, self-sacrifice, commitment, and a loyal comradery with fellow service members and veterans. Similarly, veterans also share the challenges of reintegration into civilian life (Olenick, Flowers, & Diaz, 2015). According to Military Service Members and Veterans (MSMV) Reintegration Theory (Elnitsky, Blevins, Fisher, & Magruder, 2017), many veterans feel closer to their fellow veteran comrades than to their families and civilian friends following military separation. Moreover, they may feel disconnected from civilians and institutions, and may sense a distinct loss of purpose. Importantly, this theory explains that successful reintegration needs to occur at four nested ecological levels: the individual, the interpersonal, the community, and the societal.
Applying these principles to patient-centered engagement, participation in V-COR can give veterans the opportunity to: (1) create a sense of connectedness and purpose by participating as full partners in the research process; (2) develop expanded interpersonal ties through the research enterprise; (3) give back to their veteran community through facilitating actionable discovery; and (4) drive veteran-focused research initiatives. Thus, informed by MSMV Reintegration Theory and the principles of a patient-centered approach to healthcare research, this project aimed to create a multi-state platform for veteran dialogue. Through dialogue, the team sought to elicit veteranrecommended strategies for engaging veteran communities as a full partner in the research process.
Methods A qualitative descriptive design (Kim, Sefcik, & Bradway, 2017;Lambert & Lambert, 2012) was used to accomplish the project aims. This type of design, informed by a naturalistic philosophy, seeks to describe an event or community perspective using data derived from focus groups or field notes (Bradshaw, Atkinson, & Doody, 2017). All project procedures were approved by the appropriate university institutional review boards for the protection of human subjects.
Field teams: Composition and training To achieve project goals and create a national platform for dialogue, the project team created a Veteran Action League (VAL) in each of the six US states with the largest veteran populations: California, Texas, Florida, Pennsylvania, New York, and Ohio. Consulting with veteran advocacy groups, the project team identified and selected a veteran with strong ties to the local veteran community in each of these states. This veteran was then invited to join the research team by serving as the VAL Unit Leader for his state. In each of the six states, the project team also identified and invited a local academic researcher, or collaborative academic researcher member (CARM), whose program of research or clinical practice included a veteran-focus. Thus, six Veteran Unit Leaders and six CARMS comprised the field team.
Veteran Unit Leaders and CARMS were trained by the project team prior to the initiation of think tank meeting (TTM) sessions. They each attended two synchronous, online training sessions during which they were oriented to (1) the purpose of the project, (2) the approved IRB human subjects protection protocol; (3) the design and use of the field note template developed by the project team to address veteran concerns about the research process; (4) project inclusion criteria; (5) participant recruiting strategies; (6) dialogue facilitation methods; and (7) field note creation guidelines and submission.
During the early phases of the project, project leaders also conducted site visits in each state to meet with the Unit Leader and the CARM and observe a TTM session. Afterwards, project leaders held a debriefing session with both the Unit Leader and the CARM to provide feedback, make suggestions, and answer questions. Throughout the project, the leadership team continued to meet with each of the six VAL units every two months using video conferencing technology. These meetings provided opportunities to monitor the research process by reviewing recruitment strategies, answering facilitation questions, discussing field notes, stakeholder engagement at TTMs, and maintaining project operations. Moreover, these meetings provided a mechanism for ensuring the dependability of the data by allowing project leaders to ensure that TTMs were conducted in the same manner across all field teams (Noble & Smith, 2015). Near the end of the project period the project team and VAL Unit Leaders and CARMS teams attended a two-day project workshop hosted by the project team. To ensure the credibility of the findings, recurrent themes, interpretations, and conclusions derived through analysis of field notes were presented and validated by the Veteran Unit Leaders and CARMS in attendance.

Participants: Exclusion criteria and recruitment
Over the course of the community engagement project, the Veteran Unit Leader and CARM in each state collaborated to plan and hold nine TTMs over the two-year project period. Open invitations were extended to veterans and key veteran community stakeholders, including family members of veterans, friends of veterans, healthcare providers working with the veteran community, and representatives from veteran advocacy organizations. These were circulated by each Veteran Unit Leader and CARM through a variety of mechanisms, including veteran, healthcare, and university organizations.
Project participation was only open to English-speaking, self-identified veterans and veteran stakeholders aged 18 years and older who were willing and able to attend a 60-minute dialogue session exploring their thoughts and concerns about research conducted within the veteran community. In total, 54 TTMs were held across six states. These meetings were attended by 384 participants, including 283 self-identified veterans and 101 self-identified community stakeholders. Although some participants attended more than one meeting, there were a total of 257 unduplicated attendees. Think tank meetings were held on a university campus in four of the six states; two states held their meetings at veteran advocacy organizational offices. Each TTM was led by the Veteran Unit Leader.

Data collection and analysis
The project design incorporated the findings of prior research indicating that military personnel might be reluctant to participate in research due to the sensitivity of the topics, lack of trust, and concerns regarding lack of anonymity and perceived risks to privacy (Littman, True, Ashmore, Wellens, and Smith, 2018). To reduce veteran concerns about privacy and address this community's reluctance to participate in recorded dialogue sessions, a field note template was developed to guide discussions and enable the CARM to take careful notes about what was said including direct quotes. Each CARM provided an extensive written summary of the dialogue generated at each meeting that included anonymous, representative direct quotes. This method has been used in studies targeting vulnerable populations for whom audio taping was either not feasible or not desired by study participants (Collica, 2012;Easterling & Johnson, 2015;Pelletier, Rowe, Francois, Bordeleau, & Lupien, 2013;Tessier, 2012;Thomas, 2012). The field note template (Appendix) included topics such as veteran research priorities, perceived barriers and facilitators to research participation, recommendations for engagement strategies, and veteran-recommended dissemination strategies for research findings.
At each session in all six states, the CARM ended the meeting by reading the field notes, direct quotes, and recommendations made by participants and then obtained feedback from the participants to ensure accuracy. This form of member checking was yet another technique to support the credibility of the findings (Korstjens & Moser, 2018).
Field notes were then sent to the project's lead research team for analysis via open and axial coding (Summer, Guendelman, Kestler, & Walker, 2017). Identified themes were then circulated by the lead research team back to the Veteran Unit Leaders and CARMs for further validation. Lastly, final themes, exemplars, veteran-generated research priorities, and elements of the engagement checklists were presented by the lead research team at the two-day project workshop and validated by the Veteran Unit Leaders and CARMs.
In summary, several techniques were used to ensure the credibility and dependability of the findings. The lead project team provided field team training, observed TTMs in each state, and held ongoing virtual meetings with each field team to ensure that TTMs were conducted in the same manner across all field teams. Field notes were read and validated with the participants at the close of each meeting as a form of member checking. Field notes were consistently submitted to the lead research team following each TTM and themes identified through analysis were validated with field teams.

Engagement
While many veteran participants across all six states expressed enthusiasm and interest in participating in PCOR and CER, profound mistrust and skepticism of the research process emerged as a strong reoccurring theme. This mistrust was voiced by veteran participants from multiple states, as was a lack of exposure to current federal regulations regarding the protection of human subjects. Veterans frequently commented that the military does not have a good reputation for protecting human research subjects and referred to the testing of LSD on active military personnel without their explicit consent. One veteran commented that they were even skeptical at first about attending the TTM.
Veterans across the six states also expressed unfamiliarity with research processes, explaining this unfamiliarity created a barrier to engaging in research activities. As one veteran noted, "We don't know what comparative research means and even when explained, it's still hard to get." Moreover, veterans who were open to participating in research explained that they were never made aware of opportunities to do so and did not know how to find information on research participation opportunities. Thus, lack of familiarity with the research process, human subject protection, and opportunities for research participation were identified as the biggest barriers to veteran research participation.
In addition to identifying barriers to research participation, veterans in the TTMs made several recommendations about how to engage veterans in research activities. The most commonly recommended community engagement strategy was to extend "…veteran comradery" into research processes. Veterans in dialogue sessions across all six states emphasized that veterans are much more willing to extend trust to other veterans than they are to non-veterans. Therefore, they strongly recommended that research studies be informed by veterans and include veterans as full partners on the research team. Veterans across all six states emphasized the importance of an authentic, collaborative relationship between veterans and research scientists. Such partnerships, they advised, could be operationalized through the appointment of veterans to research teams or to the research advisory boards. Importantly, veterans stressed that when attempting to recruit veterans as research participants, outreach and invitations should be conducted primarily by veterans. Stakeholders in at least one of the states reiterated this recommendation, stating that they work with veterans who conduct the intake process for their organization. These stakeholders emphasized that it makes a big difference to veterans' willingness to participate when they are invited to join the study by other veterans. Additionally, stakeholders attending the dialogue sessions who conducted their own studies emphasized the importance of research designs that place veterans in group settings with other veterans. They reported that in their experience, having other veterans in the room helped veterans to engage. These stakeholders also emphasized that veteran participants must be clearly appraised of the benefits of the proposed research to other veterans. They warned that asking veterans probing questions without fully and clearly explaining the potential benefits of their answers would simply serve to alienate potential veteran participants.
These veteran-recommended strategies have been incorporated into a Veteran-Centered Research Engagement Strategies Checklist (see Table 1  SUBJECT RECRUITMENT Veteran organizations agree to collaborate with recruitment and outreach efforts. Recruitment of veteran subjects is conducted by veterans. Strategies used to protect human subjects are clearly described in lay terms during recruitment. The potential benefits of the research for the veteran community at large are clearly described during recruitment. The opportunity for veterans to "give back" to other veterans through research participation is discussed.
RESEARCH ACTIVITIES All researcher-subject meetings / interfaces are held in a "safe space" such as space occupied by a veteran organization. Travel time for research participation is minimal. Mechanisms to provide participants with a summary of findings are ready for implementation.
Additional research engagement strategies included (1) designing time commitments to accommodate veterans' employment hours, (2) issuing invitations through social media and veteran organizations, and (3) encouraging "word of mouth" recruitment by inviting trusted members of the veteran community to inform them of research participation and partnership opportunities.

Dissemination of findings
Veterans across all six states repeatedly acknowledged that the Veterans Administration (VA) conducts active research aimed at testing treatment options for veterans. However, they also acknowledged that many veterans are not involved in the VA, so other sources of dissemination should be used. Veterans across states consistently emphasized that veteran advocacy organizations should provide the primary mechanism by which to disseminate CER and V-COR findings.
Veterans across all six states also emphasized the importance of a multi-media approach to the distribution and dissemination of research findings that includes the use of social media, email blasts, websites, podcasts, YouTube videos, and magazine articles in journals favored by veterans such as Military Officers Journal and Disabled American Veterans Journal. Participants underlined the fact that research findings need to be presented or published using language that the average person can understand.
Finally, veterans recommended that researchers develop a project-specific website on which to post research findings that are communicated in lay language. They suggested that researchers create videos featuring veteran research participants who summarize study findings and explain their importance to the veteran community. They further recommended that the link to this website or online video be distributed through veteran advocacy organizations and other veteran-friendly outlets. Reiterating the theme of "comradery among veterans," veterans across multiple states expressed the importance of receiving information from another veteran-someone they trust. They concluded that a vet can become "…an ambassador of sorts," linking the research community to the veteran community. Participants emphasized that research teams should recruit veterans to assist in the translation and dissemination of research findings.
Overall, veteran participants in the TTMs expressed both interest and willingness to assist in the translation and dissemination of research findings to the broader veteran community. They also said that they would be interested in volunteering to disseminate and translate research findings via their involvement with veteran organizations and advocacy groups. Veteran organizations and advocacy groups were described as very effective dissemination partners, particularly organizations such as Veterans of Foreign Wars (VFW), the Military Order of the Purple Heart, The American Legion, Rotary International, and other service leagues.

Veteran-centered research priorities
Lastly, veterans who participated in the TTMs were very vocal in identifying their research priorities. The following list compiles veteran research priorities as reported in TTMs held across all six states. The list does not represent any order of priority: • testing models to improve access to care Outcomes from this veteran-centered community engagement project provide veteranrecommended strategies for engaging veterans in research and disseminating research findings to the veteran community. The large sample size coupled with very consistent reports across all six states support the transferability of the recommendations to the larger veteran community (Noble & Smith, 2015). Thus, these recommendations have been compiled into checklists for use by research teams seeking to partner with veterans for the purpose of conducting research. They have been developed to maximize veteran research engagement (see Table 1 pp. 269-270) and veteran-focused dissemination of findings (see Table 2 below). Use of these checklists will facilitate the efforts of research teams working to ensure that they are employing a veteran-centered approach consistent with the aims of PCORI. Such a patient-centered approach will not only allow patients' voices to be heard but will also incorporate veterans' recommendations regarding research question selection, research methods, participant recruitment, and dissemination strategies (Sheridan, Schrandt, Forsythe, Hillard, & Paez, 2017). Table 2. Checklist for Veteran-Centered Research Dissemination Strategy Yes No Required Action Veterans are recruited to assist in developing dissemination strategies. Veterans assist in dissemination within veteran groups, advocacy organizations, and social groups. A research study website has been developed, findings are posted in lay language, and the link has been distributed by veteran organizations and through social media. A video has been created featuring veteran research participants who describe study findings in lay language. The video has been posted on YouTube or other online video platform and the link has been distributed by veteran organizations and through social media. A podcast has been created that describes findings in lay language. The link has been distributed through veteran organizations and social media. A manuscript, presenting findings in lay language, has been submitted to a magazine or journal preferred by veterans such as Military Officers Journal, Disabled American Veterans Journal, American Legion Magazine, or similar publication including veterans as authors if

Discussion
The many veterans and stakeholders who participated in the TTMs were vocal in identifying barriers to veterans' participation in research. Fortunately, they were also willing to provide recommendations to reduce these barriers and further engage the veteran community in the research enterprise. These recommendations have been incorporated in a checklist for research teams (see Table 1 pp. 269-270). One reoccurring barrier was an underlying mistrust of research, as well as an unfamiliarity with research processes and the measures required to protect human subjects. Based on these findings, this project team has been awarded additional grant funding to develop and test a research participation and partnership training program for veterans. This training program is being operationalized as a series of publicly accessible, web-based educational modules designed specifically to prepare veterans to become full partners in research activities. The training modules, which will be narrated by veterans, are being designed in collaboration with a team of veteran advisors (for more information visit our website).
Veterans and stakeholders also provided specific recommendations for the effective dissemination of research findings to the veteran community. A multimedia approach, including veteran spokespersons, a research website, YouTube videos, and dissemination through veteran organizations and veteran-focused journals are included in the recommended dissemination plan and veteran-centered checklist for research teams (see Table 2 above).
Interestingly, veteran and stakeholder recommendations mirror the postulates of Military Service Members and Veterans (MSMV) Reintegration Theory (Elnitsky, Blevins, Fisher, & Magruder, 2017), reflecting the trust that veterans place in other veterans. Consistent with this theory, recommendations elicited in this project strongly emphasize the important role of veterans as members of the research team throughout all phases of study design, participant recruitment, and dissemination of findings. Clearly, veterans extend trust to other veterans and they are more likely to engage in research participation and the uptake of findings, when veterans are heavily represented on the team and serve as respected advisors, chief consultants, and even the public face of the study and its findings.

Limitations
This engagement project is not without limitations. Many factors including personal biases, age, or racial/gender characteristics could influence participating veterans' perceptions, responses, and their willingness to engage in research. Potential nuances in the perceptions of different groups of veterans, however, were not captured in TTMs. It is also possible that despite efforts of field and research team members to identify personal biases related to discussion topics, subjectivity may have influenced the recording or analysis of field notes. For example, the lead analyst on this engagement project embraces the six domains of health quality as conceptualized and published by the US government's Agency for Healthcare Quality. Included among the six domains is the concept of appropriate or acknowledged in the acknowledgement section with permission. A veteran-focused conference has been organized where findings will be presented. The conference has been marketed to the veteran community, providers, and other veteran stakeholders. Strategies have been developed to disseminate findings to providers in ways that inform evidence-based practice.
equitable access to quality care. Despite these limitations, there was notable consistency in the perceptions and recommendations from 257 unduplicated participants as recorded in field notes from the 54 TTMs held in six geographically diverse US states.
Conclusion Although patient and consumer engagement in healthcare research is a worthy goal, little is known regarding the best ways to include these important "end-users" into research decisions and processes. Clearly, consumer engagement must be meaningful, moving from token representation to a true partnership between consumers and the research team. But in order to develop meaningful partnerships, researchers must hear, understand, and operationalize the recommendations of their community stakeholders.
The outcomes revealed from this veteran-centered community engagement project demonstrate that veterans want their voices to be heard regarding the aims, designs, recruitment activities, and dissemination plans related to veteran-focused research. The checklists developed as a result of this community engagement project represent a concrete step in this direction. With the help of these veteran-informed tools, research teams can operationalize this community's recommendations and move towards the goal of engaging veterans as true partners in the research enterprise.
Many thanks to the veterans who attended TTM sessions. Their willingness to share their expertise and engage in this project is invaluable. The authors also gratefully acknowledge the following Veteran Unit Leaders for their service to their country and for their noteworthy assistance on this project: California Mr. Aaron Siebert -U.S. Navy