Skip to main content
Download PDF

Developing an mHealth program to improve HIV care continuum outcomes among young Black gay and bisexual men

BMC Public Health volume 24, Article number: 1247 (2024) Cite this article

Abstract

Background

Young Black gay and bisexual men (YBGBM) in the United States face significant disparities in HIV care outcomes. Mobile health (mHealth) interventions have shown promise with improving outcomes for YBGBM across the HIV care continuum.

Methods

We developed an mHealth application using human-centered design (HCD) from 2019–2021 in collaboration with YBGBM living with HIV and with HIV service providers. Our HCD process began with six focus groups with 50 YBGBM and interviews with 12 providers. These insights were used to inform rapid prototyping, which involved iterative testing and refining of program features and content, with 31 YBGBM and 12 providers. We then collected user feedback via an online survey with 200 YBGBM nationwide and usability testing of a functional prototype with 21 YBGBM.

Results

Focus groups and interviews illuminated challenges faced by YBGBM living with HIV, including coping with an HIV diagnosis, stigma, need for social support, and a dearth of suitable information sources. YBGBM desired a holistic approach that could meet the needs of those newly diagnosed as well as those who have been living with HIV for many years. Program preferences included video-based content where users could learn from peers and experts, a range of topics, a community of people living with HIV, and tools to support their health and well-being. Providers expressed enthusiasm for an mHealth program to improve HIV care outcomes and help them serve clients. Rapid prototyping resulted in a list of content topics, resources, video characteristics, community features, and mHealth tools to support adherence, retention, goal setting, and laboratory results tracking, as well as tools to help organization staff to support clients. Online survey and usability testing confirmed the feasibility, acceptability, and usability of the content, tools, and features.

Conclusions

This study demonstrates the potential of a video-based mHealth program to address the unique needs of YBGBM living with HIV, offering support and comprehensive information through a user-friendly interface and videos of peers living with HIV and of experts. The HCD approach allowed for continuous improvements to the concept to maximize cultural appropriateness, utility, and potential effectiveness for both YBGBM and HIV service organizations.

Peer Review reports

Background

Young Black gay and bisexual men (YBGBM) face substantial disparities in HIV diagnosis and care outcomes. While gay, bisexual, and other men who have sex with men (MSM) constitute approximately 3.9% of the total U.S. population [1], they accounted for a disproportionate 67% of new HIV diagnoses in 2021 [2]. Among MSM, Black MSM experience the highest rate of HIV diagnoses by race/ethnicity, with 37% of new HIV diagnoses in 2021 attributed to this group [2]. Among 13–24-year-olds, Black youth experienced a rate of HIV diagnosis 14.3 times that of their White counterparts [2]. Furthermore, youth between the ages of 13–24 made up 24% of HIV diagnoses among MSM, but 53% of new diagnoses among Black MSM [2].

Enhancing engagement in the HIV care continuum, from diagnosis to viral suppression [3], is crucial to reducing HIV morbidity, mortality, and transmission [4,5,6,7]. However, disparities persist, as Black MSM have poorer linkage to care, retention in care, and viral suppression outcomes compared to their White counterparts, even after controlling for access to care [8, 9]. Of particular concern is that, among Black MSM living with HIV, only 65% of those ages 13–24 and only 61% of those 25–34 have achieved viral suppression [10].

Multiple factors negatively impact HIV care continuum outcomes among YBGBM, including stigma related to HIV and sexual orientation [11,12,13,14,15,16]; low self-efficacy related to health management and provider communication [17, 18]; mental health issues, such as depression, anxiety, and post-traumatic stress disorder [18, 19]; substance use [17, 18, 20,21,22]; and insufficient social support [23, 24]. Moreover, many of these factors intersect, creating syndemics that magnify negative outcomes for YBGBM [24,25,26,27,28,29]. In light of this spectrum of challenges, it is essential to support YBGBM using holistic approaches [30, 31] that engage them in the HIV care continuum through acceptable and accessible solutions [32, 33].

Mobile health (mHealth) interventions can positively impact HIV prevention, testing, and treatment outcomes [33], including those related to HIV self-management, and have been found to be feasible and acceptable for YBGBM [32,33,34,35,36,37]. These interventions can also be very convenient and accessible, as 96% of Americans ages 18–29 own a smartphone [38], and they can be used in rural areas where in-person HIV interventions might otherwise be unavailable [39,40,41]. Though evidence on the effects of mHealth strategies for stigma reduction among youth is nascent [42], emerging research suggests that these approaches can help build social support and resilience among youth who face discrimination due to their race/ethnicity or sexual orientation [43, 44].

In addition to the wide array of mHealth tools and text-based information that can be provided via smartphones, these devices enable the delivery of video-based media [32]. Prior research has found that targeted HIV prevention videos for Black MSM increased understanding and recall of messages [45]. Furthermore, online personal testimonial videos of people with chronic conditions, including people living with HIV, may improve education and social support, and reduce stigma [46]. Several studies have demonstrated the effectiveness of video-based interventions, including some specifically created for young MSM living with HIV [47,48,49,50,51]. These interventions have demonstrated impacts on HIV testing [47], risky sexual behaviors [47, 49], HIV disclosure [47, 49], and HIV treatment initiation [51].

Studies have identified preferences for mHealth HIV prevention and care programs in general, as well as among youth and YBGBM specifically. Desired characteristics include confidentiality protections, motivational content, both video and text content, adherence and lab results tracking, an online community, resource lists, and an overall focus on holistic health rather than only on HIV [31, 52, 53]. However, few if any mHealth programs include most or all of these features [52, 53]. In this article, we detail the human-centered design (HCD) approach used to create an mHealth app, called Amp, intended to enhance HIV care continuum outcomes and to holistically support the health and well-being of YBGBM. Amp was developed in close collaboration with YBGBM living with HIV and with HIV service providers. It includes comprehensive, video-based information, a range of mHealth tools to support HIV care outcomes, resources, and a community of people living with HIV. In the subsequent sections, we discuss the impetus for this app, the methods used to generate initial insights, the transformation of these insights into testable prototypes, and the iterative refinement of the intervention concept.

Methods

Inspiration for AMP

The concept for Amp emerged from a series of informal conversations between our team and HIV linkage and retention staff from a large HIV service organization in Los Angeles, CA, beginning in 2015. These staff expressed a need for programs to better engage gay and bisexual men living with HIV in care, to keep them retained in care, and to help them achieve viral suppression. We had additional conversations with staff at HIV service organizations in San Francisco, CA, Los Angeles, CA, and Huntsville, AL, in 2018. Organization staff suggested a program that could function as both a stand-alone intervention for people living with HIV and something that could be integrated into their organization’s HIV testing, linkage, and retention workflow to improve HIV care continuum outcomes.

From 2016–2018, we held individual and group discussions with 16 young gay and bisexual men who were diverse in age and race/ethnicity to assess their needs with regard to HIV care and to investigate potential solutions. These conversations revealed that despite the availability of highly effective treatment, an HIV diagnosis remained devastating. Many expressed feeling isolated after their diagnosis and overwhelmed with the information they were provided to manage their condition. When discussing possible solutions with these men, we developed the initial concept of an mHealth program (e.g., mobile website or app) that could include content (e.g., personal narrative videos and articles), tools, and a community of peers to holistically support people living with HIV. In 2018, we received funding to use HCD to fully develop the concept. We decided to focus the program on YBGBM due to the HIV care continuum disparities experienced by this group.

HCD is an approach to creating programs or products that requires a deep understanding and intensive involvement of stakeholders to ensure that programs meet the needs of the intended users [54, 55]. This process has been used to create innovative and culturally appropriate programs [56] that are high in satisfaction, usability, and effectiveness [57] among intended audiences. The mixed methods approach described below follows the stages of HCD developed by the design firm IDEO [55]. This approach is highly iterative, enabling continuous improvement throughout the development process, and refinement even after the program is launched.

Stage 1: ideation

The first stage of the HCD process was intended to build upon the initial concept and to generate a large number of ideas for the functions, features, and design characteristics of the program. We conducted focus groups with YBGBM and key informant interviews with HIV service providers in several U.S. cities to gain a better understanding of the needs of intended users and organizations that would implement the program. Interview and focus group data were analyzed for themes, which were used to develop prototype features in the subsequent stages.

Focus groups

From March–April 2019, we conducted six in-person focus groups with a total of 50 YBGBM, ranging from 18–30 years old (see Table 1). Focus group participants were recruited through our team’s network of contacts. Groups were held at HIV service organizations or community spaces in Alabama (Huntsville) and California (San Francisco, Los Angeles, and Oakland). Participants gave written consent and received a $100 incentive for a 90-min group. We developed a semi-structured discussion guide to investigate the experiences of YBGBM living with HIV, specifically barriers and facilitators to HIV care continuum outcomes. During the discussion, we asked for feedback on The Body, an HIV website to which patients are often referred for HIV information. We showed participants two sections, the home page and the newly diagnosed section, and asked what they thought of this website, if it seemed helpful, and how it could be improved, if at all. We then showed four existing videos available online (two of young men living with HIV and two of HIV providers) to assess preferences for video content. Finally, we read a description of our initial program concept and solicited feedback both generally and on specific content topics, tools, and features. Focus groups were audio recorded and professionally transcribed.

Table 1 Sample characteristics for focus groups, key informant interviews, rapid prototyping, and usability testing (% n)
Full size table

Key informant interviews

In July 2019, we conducted 12 key informant interviews in person or by telephone with HIV service providers at five different organizations in Alabama (Huntsville), Louisiana (New Orleans), Texas (Dallas), and California (Los Angeles and Riverside; see Table 1). Providers gave either written or verbal consent and received a $75 incentive for a 60-min interview. All worked at HIV service organizations that provide case management and linkage to care to YBGBM, and 10 worked at organizations that also offer HIV treatment. We interviewed providers from different levels within the organization, including executive, senior, and frontline staff. We developed a semi-structured interview guide to solicit both general and specific input on program content, tools, and features that could be useful to YBGBM living with HIV and that would help staff to better support clients along the HIV care continuum. Provider interviews were audio recorded and professionally transcribed.

Stage 2: rapid prototyping

Rapid prototyping is a methodology used in HCD that involves the iterative testing and refining of low-fidelity prototypes to efficiently optimize program elements. YBGBM and providers were recruited for rapid prototyping through our team’s HIV service organization contacts. Rapid prototyping sessions were carried out over two days each in California (Los Angeles), Alabama (Huntsville), and Mississippi (Jackson), with a total of 31 YBGBM ages 20–30 and 12 HIV service providers (see Table 1). Sessions took place from August–September 2019. Each session was conducted in-person with individual participants by one of three study staff. Participants provided written informed consent and received a $100 incentive for a 60-min session.

During these sessions, low-fidelity hand-drawn and digital prototypes were used to gather participants’ input on: the user interface; content, tools, and resources to be included in the program; how YBGBM might want to interact in a community of peers; ways to maximize program engagement; and whether the program should include an app, mobile website, or both. At the end of each day of prototyping, responses were integrated into revised prototypes and tested again the next day.

Prior to the rapid prototyping, we produced 13 videos on eight topics based on needs expressed by YBGBM participants and providers in the ideation stage. We varied the settings (a person’s home or a film studio), speakers (YBGBM or HIV clinician), and video characteristics (e.g., subject looking directly into the camera or off camera, and heavy or light editing) in order to give participants the opportunity to provide feedback on both video content and style. Each participant was shown at least two videos, and videos were evenly divided between participants.

Stage 3: user feedback

Online survey

An online survey was conducted from September–October 2019 to gather detailed feedback on the range of program features that had been developed and refined in stages 1 and 2 and to assess the generalizability of those findings. The survey instrument was designed to capture preferences for content topics, tools, features, and resources, as well as to assess the acceptability, feasibility, and usefulness of the concept among a national sample of YBGBM living with HIV, ages 18–29 (n = 200; see Table 2). This sample size was sufficient given the intention was for descriptive purposes rather than statistical analysis. The survey began with questions designed to screen out participants who did not meet these inclusion criteria. Survey participants were recruited through our team’s network of HIV service organization contacts and through paid advertisements on geo-spatial networking apps and social media platforms. The survey took an average of 20 min to complete. Duplicate responses (e.g., consecutive surveys from the same IP address or similar email addresses) and responses completed in under six minutes (deemed too fast given the survey length) were excluded from analysis. Participants provided consent online before taking the survey and received a $20 incentive.

Table 2 User feedback survey sample characteristics (n = 200)
Full size table

Usability testing of the functional prototype

Using data from stages 1 and 2 and the online survey, we worked with a website developer to create a working digital prototype of the app. The objective of the online usability testing was to identify potential functionality issues and use the findings to maximize program acceptability and effectiveness. Participants were sent a link to the prototype and asked to view it on their smartphone. The prototype included the proposed topics, tools, and videos developed in stage 2, but was designed in grayscale without any branding or images. This approach allowed us to test usability of the interface without distracting users with design elements.

Usability testing participants (n = 21; see Table 1) were recruited through our team’s network of HIV service providers and from the online survey, with participants contacted in the order in which they completed the survey. Usability testing sessions were conducted in October 2021, with study team members observing participants interacting with the prototype over Zoom. During the session, participants were presented with several tasks related to finding information, tools, or resources on the prototype (e.g., “find and use the medication reminder” or “find a video about disclosing HIV status to sex partners”). Participants were also asked open-ended questions about topics, community features, tools, and resources; ease and predicted frequency with which they would use the app; and whether most of the content should be video- or text-based. Finally, each participant orally completed the System Usability Scale, or SUS [58], a reliable tool for measuring usability comprised of 10 questions with five response options, from “strongly agree” to “strongly disagree.” Each testing session lasted approximately one hour. Participants provided verbal consent and received a $100 incentive.

Analyses

Stage 1 focus group and interview transcripts were coded using deductive and inductive approaches by two of the team members experienced with qualitative data analysis. They developed codes based on the interview and discussion guides and added codes as they emerged from the data. Transcripts were coded to identify themes relevant to developing the app and its components (e.g., content, tools, community features, and resources). In stage 2, written notes of participants’ feedback during rapid prototyping sessions were entered into a digital spreadsheet after each session and were categorized into prototype elements that received positive feedback and those needing improvement. In stage 3, survey data and the SUS results were assessed using descriptive statistics (e.g., frequencies) in SPSS v.26. During usability testing of the functional prototype, task completion times were captured by the interviewer using a timer and were used to calculate mean and range values. Qualitative responses were analyzed for common themes with regard to satisfaction with content, tools, usability, and likely uptake of the program.

Results

Stage 1: ideation

Seven main themes emerged from the focus groups and interviews. Sample quotes for each theme can be found in Table 3.

Table 3 Sample quotes from YBGBM participants and HIV service providers
Full size table

Learning about HIV after a new diagnosis is overwhelming, and available online resources do not meet the needs of YBGBM

Many focus group participants described being devastated after receiving their HIV diagnosis (e.g., feeling isolated, depressed, and afraid of dying and social rejection). Participants also reported having many questions about living with HIV (e.g., effects of HIV and medications on long-term health, and transmission to partners). Many participants felt overwhelmed with receiving so much information about HIV from their providers right after their diagnosis.

Participants reported seeking HIV information online, often even years after their diagnosis. Several participants stated that it was difficult to find trustworthy and accurate information. When participants were shown The Body website, many felt it was too text-heavy and visually busy to easily find needed information.

Most interviewed HIV service providers said that they wanted a medically accurate, easy-to-navigate informational website to offer clients, but that they were not aware of such a resource. Few providers reported referring clients to websites or apps, other than several who recommend apps specifically for medication adherence. Providers said that clients receive HIV information, including written materials, during the first clinic visit after their diagnosis. However, many providers said clients struggle with important concepts related to living with HIV (e.g., CD4 count and viral load) and would benefit from ongoing education.

Participants expressed a desire for video-based content and programs

YBGBM participants and HIV service providers felt that the new program should include videos of both people living with HIV and experts. Participants and providers said that program videos (and the content overall) should be informational and motivational, should feel authentic as opposed to overly polished, and should include a diversity of stories (e.g., people at different stages of living with HIV, and stories of challenges and successes). Some YBGBM and providers felt that the content should be delivered through both text and video, as people learn in different ways.

When participants were shown sample video testimonials of other young men living with HIV they responded very favorably; many said that seeing peers speak openly about their HIV status and sexual orientation was motivating and empowering. Some providers felt that including videos of peers navigating challenges related to living with HIV could increase YBGBM’s self-efficacy related to care continuum outcomes and help them build skills to stay healthy (e.g., videos could show YBGBM discussing their strategies for taking medication and staying in care). They also thought that the program could reduce fears and offer hope by showing people thriving while living with HIV.

YBGBM need more support related to disclosure and stigma, which can impact their HIV care outcomes

Numerous YBGBM participants across groups described the impact that disclosure issues and HIV stigma have had on their lives. Some YBGBM reported positive experiences with disclosure, describing telling others as cathartic and facilitating social support. However, many shared fears or negative stories of disclosing their HIV status, with some experiencing stigma from friends, partners, and family members, which added to feelings of isolation. YBGBM participants also said HIV stigma within the gay community (e.g., on dating apps) exacerbated challenges with finding partners and establishing romantic relationships. Both YBGBM and providers talked about stigma being related to fear and misinformation about HIV transmission (e.g., family members believing that HIV can be transmitted from hugging or sharing dishes). Several providers suggested that the program provide accurate information to family members to reduce this stigma and demonstrate how to support a loved one living with HIV.

YBGBM participants and providers mentioned disclosure issues and stigma as barriers to accessing HIV care, to medication adherence, and to staying in care. For instance, some YBGBM reported missing medical appointments due to fears of being seen at a clinic or missing medication doses because they did not want to take the pills in front of others. Both providers and YBGBM participants suggested that the program include videos with examples for when and how to disclose their status. Further, participants and providers emphasized that the program should address confidentiality concerns (e.g., HIV should not be in the program name).

YBGBM would benefit from opportunities to learn and get support from others living with HIV

Most YBGBM participants said that they rarely discuss HIV with anyone, even with others whom they know are living with HIV. However, participants in all groups discussed the value of getting support and information from other people living with HIV. Providers also emphasized the importance of spaces where people living with HIV can support one another in a safe environment. Both YBGBM and providers responded enthusiastically to the idea of building a community of people living with HIV where they can share their experiences (e.g., through creating a profile, direct messaging, or posting on a forum) as part of the program. Some providers also suggested that the program could include trained peer mentors living with HIV who can answer questions and provide support and resources.

A holistic approach would be most useful for people living with HIV and for service organizations

Participants across focus groups wanted a wide range of content about living with HIV, including information about medical treatment, disclosure, sex and relationships, nutrition and exercise, mental health, and resources. Participants explained that having holistic content would be most beneficial, as information needs vary from person to person and also change over time (e.g., right after a diagnosis versus several years later). All HIV providers felt the program should include content, tools, and resources on a wide range of issues that affect well-being and HIV care outcomes of YBGBM (e.g., unaddressed mental health issues, disclosure, stigma, substance use, HIV misinformation, medication adherence, awareness of available resources, and medical distrust). Providers felt that a “one stop shop” approach would make the program more acceptable, engaging, and useful to both YBGBM and to HIV organizations.

YBGBM want an HIV program that is not exclusively focused on them

The majority of YBGBM participants felt that the program’s videos and community should represent people living with HIV of diverse races/ethnicities, gender identities, sexual orientations, and experiences. Many participants said that they would not relate to a program that was only for Black men living with HIV, or they felt that this would be stigmatizing (as a program for only YBGBM would suggest that they are the primary group affected by HIV). Others felt that a diverse community of people living with HIV should be included yet the experiences of young people of color should be emphasized. Very few participants felt that the program should only be for YBGBM.

Digital programs can help HIV providers and their organizations reach HIV care continuum outcome goals

All HIV service providers interviewed felt that an mHealth program could help them to support their clients through linkage, treatment initiation, adherence, retention, and viral suppression. Providers mentioned several features that could be useful, including a messaging platform that could be used by case managers, clinic appointment and medication reminders, and a way to track CD4/viral load test results. Providers at organizations that already offer some of these features (e.g., a messaging system) thought that an mHealth program could be very useful for other components (e.g., videos addressing HIV stigma, HIV education, and a community for people living with HIV). Providers suggested that the program could be offered to clients as soon as they receive a diagnosis in order to provide social support and encourage linkage during that critical time. All providers said that their staff are open to using technology and that the proposed mHealth program could be feasibly integrated into their workflow.

Stage 2: rapid prototyping

Insights from the focus groups and interviews were incorporated into the development of a series of prototypes covering potential content topics, sample videos, tools, and resources (See Table 4 for ways that findings from each stage of research were used to improve the program).

Content

When presented with a list of program topics and sub-topics, participants expressed that the proposed content was comprehensive and that it would appeal to people at different stages of living with HIV. When asked which topics would be most helpful to them personally, participants offered a range of responses, which supported findings from stage 1 that having holistic content would engage a broader audience. All participants emphasized the value of information specifically for individuals newly diagnosed with HIV and said that content should emphasize that people can lead a long, healthy life with HIV. Specific topics that participants found helpful included: U = U (Undetectable = Untransmittable); that people living with HIV can have a fulfilling love life and sex life; mental health (e.g., addressing depression and anxiety, and dealing with trauma and stress); maintaining overall health (e.g., diet and exercise, preventing sexually transmitted infections, and managing substance use); understanding medications and their side effects; and examples of how to disclose their HIV status to partners, family, and friends.

Nearly all participants supported the idea of delivering most of the content through videos featuring individuals living with HIV and experts. The majority preferred video over text-based content or desired a combination of video and text. Similar to focus group participants, prototyping participants expressed the desire for videos featuring diverse people living with HIV. Participants also liked the idea of featuring videos of doctors, mental health specialists, and case managers from the organization where they receive their HIV care. Participants suggested several ways to keep them engaged with the program, such as incorporating motivational content (e.g., daily affirmations and tips), regularly updating the program with new videos, sending notifications about new content (via app notifications or text messages), the ability to post their own content, and having a way to save and share content.

Participants responded favorably to the 13 sample videos we created and found the people in the videos to be relatable and authentic. Most considered the video lengths (ranging from 1.5 min to 4.5 min) to be appropriate. Notably, some participants who did not favor a particular video indicated they would simply switch to another video rather than discontinue using the program. We tested versions of these videos where the subject did or did not speak directly into the camera, with participants overwhelmingly preferring the former.

Tools

Proposed tools, including adherence and appointment reminders, viral load and CD4 trackers, a tool to meet health and personal goals, and a notepad to write questions for providers received highly favorable feedback. Some participants expressed the desire to interact with mentors or case managers through the program. Others suggested customizability or the inclusion of games. While some participants said they might not use every tool, they felt that including a range of tools was needed to meet the diverse needs of people living with HIV.

Community

Participants strongly advocated for having various ways to interact with other people living with HIV, including creating a profile, direct messaging, and responding to questions on a forum. They expressed a desire to share information and offer help to others, and they emphasized the potential for such a community to reduce stigma and isolation, and facilitate the exchange of useful information, strategies, and support. Some participants stressed the importance of mechanisms to address online bullying and proposed rules for community engagement, such as prohibiting trolling and having methods for removing users who breach community guidelines. One participant recommended creating online support groups centered around specific topics.

Resources

Most participants felt the resources proposed, which included HIV care and treatment, mental health, housing, food assistance, career, legal, and education were comprehensive and useful. Participants emphasized the need to understand the full range of available resources and to provide instructions and support for accessing them. Several participants suggested including hotlines as additional resources.

Rapid prototyping with providers

Providers reacted enthusiastically to the prototypes and felt that the mHealth program would help improve HIV care continuum outcomes. They felt it would provide their clients with an efficient means of engagement and support, as well as enhance the provision of medically accurate information. Some providers proposed introducing the app to newly diagnosed clients on their first visit to support linkage into care and treatment initiation. One provider noted that newly diagnosed people are often overwhelmed with information, which makes it difficult to retain. This provider felt the program would allow clients to access important information when they are ready to receive it. Providers also suggested potential improvements, such as renaming specific topics and including a feedback mechanism so clients could suggest service improvements for their organization. One provider early in stage 2 suggested developing a dashboard to enable staff to support clients as they use the app (e.g., a tool to message clients and suggest specific content or tools to assist them with specific challenges). This idea was tested with all subsequent providers and they were all very supportive of this feature. All providers considered the mHealth program to be feasible for implementation within their organization Table 4.

Table 4 Improvements to the program concept after each stage of research
Full size table

Stage 3: user feedback

Online survey

Results from the online survey among YBGBM nationwide echoed the findings from stages 1 and 2 (See Table 5). For instance, a high percentage of respondents indicated that: including topics such as mental health (92%), recent diagnosis (91%), and sex and relationships (86.5%) is “very important”; having a community of young men living with HIV to communicate and share experiences with through the program would be “very helpful” (77%); and the program’s community should not be exclusive to YBGBM living with HIV (90%).

Table 5 User feedback survey results (n = 200)
Full size table

A large majority of respondents agreed that the program concept was appealing (91.5%) and that it would be “very helpful” for both people recently diagnosed with HIV (90.5%) and those who have had HIV for two or more years (76%). Most respondents said the program should include both a mobile app and a website (53.5%), followed by an app only (41.5%) or mobile website only/other (5%). All participants said that they would use the program. When asked how often they thought they would use the program, 35.5% of respondents said they would use it at least once a day, and 82.5% said they would use it at least once a week.

Respondents reported the proposed tools and resources would be useful and helpful, in particular the CD4/viral load tracker and adherence tool (81.5% and 75%, respectively, would “definitely” use these). Over 60% said they would “definitely” use a series of proposed community features (e.g., message forum, virtual support groups, and direct messages). Respondents indicated that the following would motivate them to continue using the app: interacting with other people living with HIV in the app community (83.5%); regularly adding new videos of people living with HIV telling their stories (73.5%) and of experts (72.5%); and the resources (73%) and tools (65.5%) on the app.

Usability testing of the functional prototype

Most usability testing participants completed their tasks on the working web-based prototype quickly, with many completing them within a few seconds (see Table 6 for average task completion times), indicating that the program prototype was intuitive and easy to navigate. The prototype scored a 95.7 out of 100 on the SUS, indicating very high usability [58]. When presented with lists of tools (e.g., medication reminder, CD4/viral load tracker, and provider messaging), resources (e.g., mental health and support, housing, substance use treatment, and job and career), and community features (e.g., virtual support groups, ability to message mentors and other users, and ability to post videos or photos) that the program would include, participants shared that the content and features included the most important things that a person living with HIV would need. All participants said that they had never seen a website or app with our proposed program’s comprehensive content and tools. Nearly all favored having program content that is mostly video-based versus text-based. All participants said they would use the program, though with varying frequency—some would use it daily and others weekly or monthly.

Table 6 Mean and range of time for each task, in seconds
Full size table

Discussion

This study details the use of HCD to create Amp (Fig. 1), a holistic mobile app created to improve the health and well-being of YBGBM living with HIV. Insights gathered across three stages of research—ideation, rapid prototyping, and user feedback—were iteratively incorporated to develop a program prototype that received high acceptability and usability ratings. HCD is a comprehensive approach that has been increasingly utilized in the development of mHealth programs [54, 59]. As demonstrated in this study, HCD is an approach that can yield high usability, feasibility, acceptability, and potential uptake from end users. The use of HCD in this study, which included engaging YBGBM living with HIV and HIV service providers as co-creators, resulted in a desirable program with a combination of content, tools, and resources that participants had not previously seen all in the same place. Further, the Amp concept was described by providers as promising for engaging YBGBM living with HIV across the care continuum, which is imperative to reducing HIV-related transmission, morbidity, and mortality [4, 5, 7].

Fig. 1
figure 1

Final Amp user interface, content topics, tools, and features

Full size image

Findings from our HCD research reinforced those identified in prior research regarding the needs of people living with HIV, including: social support [60], adherence support [53], HIV knowledge [61], programs to address stigma and discrimination [6, 11, 25, 62], and the general demand for more holistic approaches to HIV care [30, 31, 53]. Amp includes content, tools, and features to address all of these needs. Amp offers several alternatives to the ways that YBGBM living with HIV usually receive support—including community features that allow users to create a profile, communicate with peers via forums and direct messaging, professionally produced video narratives, trained mentors who can be messaged on-demand, and a range of tools to support medication adherence, retention in care, and viral suppression. This wide variety of tools and resources is designed to meet the needs of YBGBM where they are, and as their needs change over time. Further, HIV service providers were very enthusiastic about using Amp within their organization, to improve their services and help clients reach their HIV care continuum goals. While several existing mHealth HIV care continuum interventions take a comprehensive approach [63], Amp is unique with regard to the holistic range of tools and content, and in its extensive use of custom-created personal narrative video content.

Our study has strengths stemming from the mixed methods approach it utilized. Intensive involvement of program end users and stakeholders is fundamental to designing interventions using HCD. During every stage of our research, we collected data from geographically diverse groups of YBGBM and providers, validating and triangulating findings at each stage. This is likely the reason that Amp was so well received in the final user feedback stage.

While this study outlines a design process that was effective, there are several limitations to our findings that should be noted. Recruiting a sample of 200 HIV positive YBGBM for participation in a survey is logistically challenging, and it was not possible to ensure that this sample is representative of the broader population. Thus, our survey findings may not be generalizable to all YBGBM living with HIV in the U.S. Similarly, relying on our professional network of HIV service providers for much of the recruitment may have introduced some bias, especially as this meant that most YBGBM participants were linked to care. We attempted to mitigate this potential bias by working with multiple HIV service providers from diverse organizations who had direct connections to local populations, as well as by recruiting from a geographically diverse sample of YBGBM in stage 3, mostly via geospatial apps. Survey findings mirrored findings from other stages, which suggests that responses from stages 1 and 2 were minimally biased by our recruitment methods. Further, we administered the SUS verbally with usability testing participants over Zoom, which is different from the typical questionnaire-based approach. While we requested that participants give honest feedback in order to facilitate improvements to the app, it is possible that this approach resulted in response bias.

The results of these activities have led to the production of Amp, a mobile app that has the content, tools, community, and resources identified as important among YBGBM and HIV service providers in our HCD process. At the time of writing, the Amp mobile app is being developed with Phase II funding through a Small Business Innovation Research Grant provided by the National Institutes on Minority Health and Health Disparities. Amp will be evaluated in a nationwide randomized controlled trial in 2024, which will test the effectiveness of the application on knowledge, social support, linkage to care, medication adherence, retention in care, viral suppression, mental health, and HIV related stigma among a relatively large sample of YBGBM living with HIV. Our team plans to disseminate Amp to organizations nationwide, including HIV service organizations, public health departments, and community organizations. Ideally, Amp will be offered to people immediately after an HIV diagnosis as well as to existing clients who are living with HIV.

This study adds to the limited research on using HCD to develop HIV mHealth programs. The evaluation study will be among the first randomized controlled studies of an HIV intervention developed with HCD. Although we attribute the early signs that Amp will be feasible, acceptable, usable, and effective to the HCD process through which it was developed, further research is needed to understand the impact of HCD relative to other approaches. Future studies could also investigate using HCD to create mHealth self-management programs for other chronic illnesses.

Conclusions

We used HCD in a highly collaborative, multi-stage process to develop an innovative mobile app that holistically addresses multiple health and wellness needs identified by YBGBM and HIV service providers. Each stage of the process identified barriers and facilitators to program uptake, with findings used to make the app more useful and engaging. These results support prior studies finding that intensive involvement of end users and stakeholders using HCD can yield innovative, acceptable, and culturally-appropriate programs.

Availability of data and materials

Fully de-identified data can be made available from the corresponding author on reasonable request (EMAIL ADDRESS BLINDED FOR REVIEW).

Abbreviations

YBGBM:

Young Black gay and bisexual men

MSM:

Men who have sex with men

HCD:

Human-centered design

CD4:

CD4 T lymphocytes; CD4 cells or T-cells

SUS:

System usability scale

References

  1. Grey JA, Bernstein KT, Sullivan PS, Purcell DW, Chesson HW, Gift TL, Rosenberg ES. Estimating the population sizes of men who have sex with men in US states and counties using data from the American Community Survey. JMIR Public Health Surveill. 2016;2(1):e14. https://doi.org/10.2196/publichealth.5365.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Centers for Disease Control and Prevention. HIV surveillance report, 2021. 2023. Available from: https://www.cdc.gov/hiv/library/reports/hiv-surveillance/vol-34/index.html. Accessed 21 Jul 2023.

  3. Gardner EM, McLees MP, Steiner JF, Del Rio C, Burman WJ. The spectrum of engagement in HIV care and its relevance to test-and-treat strategies for prevention of HIV infection. Clin Infect Dis. 2011;52(6):793–800. https://doi.org/10.1093/cid/ciq243.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Hogg RS, Yip B, Chan KJ, Wood E, Craib KJ, O’Shaughnessy MV, et al. Rates of disease progression by baseline CD4 cell count and viral load after initiating triple-drug therapy. JAMA. 2001;286(20):2568–77. https://doi.org/10.1001/jama.286.20.2568.

    Article  CAS  PubMed  Google Scholar 

  5. Paterson DL, Swindells S, Mohr J, Brester M, Vergis EN, Squier C, et al. Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann Intern Med. 2000;133(1):21–30. https://doi.org/10.7326/0003-4819-133-1-200007040-00004.

    Article  CAS  PubMed  Google Scholar 

  6. Centers for Disease Control and Prevention. HIV prevention in the UNITED States: new opportunities, new expectations. 2015. Available from: https://stacks.cdc.gov/view/cdc/39420/cdc_39420_DS1.pdf. Accessed 30 Nov 2023.

  7. Lima VD, Harrigan R, Bangsberg DR, Hogg RS, Gross R, Yip B, et al. The combined effect of modern highly active antiretroviral therapy regimens and adherence on mortality over time. J Acquir Immune Defic Syndr. 2009;50(5):529–36. https://doi.org/10.1097/QAI.0b013e31819675e9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Singh S, Mitsch A, Wu B. HIV care outcomes among men who have sex with men with diagnosed HIV infection—United States, 2015. MMWR Morb Mortal Wkly Rep. 2017;66(37):969–74. https://doi.org/10.15585/mmwr.mm6637a2.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Vaughan Sarrazin MS, Ohl ME, Richardson KK, Asch SM, Gifford AL, Bokhour BG. Patient and facility correlates of racial differences in viral control for Black and White veterans with HIV infection in the Veterans Administration. AIDS Patient Care STDS. 2018;32(3):84–91. https://doi.org/10.1089/apc.2017.0213.

    Article  PubMed  Google Scholar 

  10. Centers for Disease Control and Prevention. Monitoring selected national HIV prevention and care objectives by using HIV surveillance data—United States and 6 dependent areas, 2021. HIV Surveill Rep. 2023;28(4). Available from: https://www.cdc.gov/hiv/library/reports/hiv-surveillance/vol-28-no-4/index.html. Accessed 29 Nov 2023.

  11. Arnold EA, Rebchook GM, Kegeles SM. “Triply cursed”: racism, homophobia and HIV-related stigma are barriers to regular HIV testing, treatment adherence and disclosure among young Black gay men. Cult Health Sex. 2014;16(6):710–22. https://doi.org/10.1080/13691058.2014.905706.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Hussen SA, Harper GW, Bauermeister JA, Hightow-Weidman LB, Adolescent Medicine Trials Network for HIV/AIDS Interventions. Psychosocial influences on engagement in care among HIV-positive young black gay/bisexual and other men who have sex with men. AIDS Patient Care STDS. 2015;29(2):77–85. https://doi.org/10.1089/apc.2014.0117.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Quinn K, Dickson-Gomez J. Homonegativity, religiosity, and the intersecting identities of young Black men who have sex with men. AIDS Behav. 2016;20(1):51–64. https://doi.org/10.1007/s10461-015-1200-1.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Radcliffe J, Doty N, Hawkins LA, Gaskins CS, Beidas R, Rudy BJ. Stigma and sexual health risk in HIV-positive African American young men who have sex with men. AIDS Patient Care STDS. 2010;24(8):493–9. https://doi.org/10.1089/apc.2010.0020.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Quinn KG. Applying an intersectional framework to understand syndemic conditions among young Black gay, bisexual, and other men who have sex with men. Soc Sci Med. 2022;295:112779. https://doi.org/10.1016/j.socscimed.2019.112779.

    Article  PubMed  Google Scholar 

  16. Quinn K, Voisin DR, Bouris A, Jaffe K, Kuhns L, Eavou R, et al. Multiple dimensions of stigma and health related factors among young Black men who have sex with men. AIDS Behav. 2017;21(1):207–16. https://doi.org/10.1007/s10461-016-1439-1.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Voisin DR, Quinn K, Kim DH, Schneider J. A longitudinal analysis of antiretroviral adherence among young Black men who have sex with men. J Adolesc Health. 2017;60(4):411–6. https://doi.org/10.1016/j.jadohealth.2016.10.428.

    Article  PubMed  Google Scholar 

  18. Hightow-Weidman L, LeGrand S, Choi SK, Egger J, Hurt CB, Muessig KE. Exploring the HIV continuum of care among young Black MSM. PLoS One. 2017;12(6):e0179688. https://doi.org/10.1371/journal.pone.0179688.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Leserman J. Role of depression, stress, and trauma in HIV disease progression. Psychosom Med. 2008;70(5):539–45. https://doi.org/10.1097/PSY.0b013e3181777a5f.

    Article  PubMed  Google Scholar 

  20. Aralis HJ, Shoptaw S, Brookmeyer R, Ragsdale A, Bolan R, Gorbach PM. Psychiatric illness, substance use, and viral suppression among HIV-positive men of color who have sex with men in Los Angeles. AIDS Behav. 2018;22(10):3117–29. https://doi.org/10.1007/s10461-018-2055-z.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Gwadz M, de Guzman R, Freeman R, Kutnick A, Silverman E, Leonard NR, et al. Exploring how substance use impedes engagement along the HIV care continuum: a qualitative study. Front Public Health. 2016;4:62. https://doi.org/10.3389/fpubh.2016.00062.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Rajasingham R, Mimiaga MJ, White JM, Pinkston MM, Baden RP, Mitty JA. A systematic review of behavioral and treatment outcome studies among HIV-infected men who have sex with men who abuse crystal methamphetamine. AIDS Patient Care STDS. 2012;26(1):36–52. https://doi.org/10.1089/apc.2011.0153.

    Article  PubMed  PubMed Central  Google Scholar 

  23. George S, Garth B, Wohl AR, Galvan FH, Garland W, Myers HF. Sources and types of social support that influence engagement in HIV care among Latinos and African Americans. J Health Care Poor Underserved. 2009;20(4):1012–35. https://doi.org/10.1353/hpu.0.0210.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Friedman MR, Coulter RWS, Silvestre AJ, Stall R, Teplin L, Shoptaw S, et al. Someone to count on: social support as an effect modifier of viral load suppression in a prospective cohort study. AIDS Care. 2017;29(4):469–80. https://doi.org/10.1080/09540121.2016.1211614.

    Article  PubMed  Google Scholar 

  25. Earnshaw VA, Bogart LM, Dovidio JF, Williams DR. Stigma and racial/ethnic HIV disparities: moving toward resilience. Am Psychol. 2013;68(4):225–36. https://doi.org/10.1037/a0032705.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Glynn TR, Safren SA, Carrico AW, Mendez NA, Duthely LM, Dale SK, et al. High levels of syndemics and their association with adherence, viral non-suppression, and biobehavioral transmission risk in Miami, a U.S. city with an HIV/AIDS epidemic. AIDS Behav. 2019;23(11):2956–65. https://doi.org/10.1007/s10461-019-02619-0.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Quinn KG, Reed SJ, Dickson-Gomez J, Kelly JA. An exploration of syndemic factors that influence engagement in HIV care among Black men. Qual Health Res. 2018;28(7):1077–87. https://doi.org/10.1177/1049732318759529.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Yigit I, Turan B, Gülşah KU, Weiser SD, Johnson MO, Mugavero MJ, Turan JM. Longitudinal associations of experienced and perceived community stigma with ART adherence and viral suppression in new-to-care people with HIV: mediating roles of internalized stigma and depression symptoms. J Acquir Immune Defic Syndr. 2022:10–97. https://doi.org/10.1097/QAI.0000000000003360.

  29. Houang ST, Kafka JM, Choi SK, Meanley SP, Muessig KE, Bauermeister JA, Hightow- Weidman LB. Co-occurring epidemic conditions among southern US Black men who have sex with men in an online eHealth intervention. AIDS Behav. 2023;27(2):641–50. https://doi.org/10.1007/s10461-022-03799-y.

    Article  PubMed  Google Scholar 

  30. Areri HA, Marshall A, Harvey G. Interventions to improve self-management of adults living with HIV on antiretroviral therapy: a systematic review. PLoS One. 2020;15(5):e0232709. https://doi.org/10.1371/journal.pone.0232709.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Giovenco D, Muessig KE, Horvitz C, Biello KB, Liu AY, Horvath KJ, Golinkoff JM, Reback CJ, Hightow-Weidman L. Adapting technology-based HIV prevention and care interventions for youth: lessons learned across five US adolescent trials network studies. Mhealth. 2021;7. https://doi.org/10.21037/mhealth-20-43.

  32. Sullivan PS, Hightow-Weidman L. Mobile apps for HIV prevention: how do they contribute to our epidemic response for adolescents and young adults? Mhealth. 2021;7:36. https://doi.org/10.21037/mhealth-20-71.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Nelson KM, Perry NS, Horvath KJ, Smith LR. A systematic review of mHealth interventions for HIV prevention and treatment among gay, bisexual, and other men who have sex with men. Transl Behav Med. 2020;10(5):1211–20. https://doi.org/10.1093/tbm/ibaa007.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, Internet and Web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep. 2015;12(1):173–90. https://doi.org/10.1007/s11904-014-0239-3.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Evans KN, Hassan R, Townes A, Buchacz K, Smith DK. The potential of telecommunication technology to address racial/ethnic disparities in HIV PrEP awareness, uptake, adherence, and persistence in care: a review. AIDS Behav. 2022;26(12):3878–88. https://doi.org/10.1007/s10461-022-03715-4.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Dworkin MS, Lee S, Chakraborty A, Monahan C, Hightow-Weidman L, Garofalo R, et al. Acceptability, feasibility, and preliminary efficacy of a theory-based relational embodied conversational agent mobile phone intervention to promote HIV medication adherence in young HIV-positive African American MSM. AIDS Educ Prev. 2019;31(1):17–37. https://doi.org/10.1521/aeap.2019.31.1.17.

    Article  PubMed  Google Scholar 

  37. Erguera XA, Johnson MO, Neilands TB, Ruel T, Berrean B, Thomas S, et al. WYZ: a pilot study protocol for designing and developing a mobile health application for engagement in HIV care and medication adherence in youth and young adults living with HIV. BMJ Open. 2019;9: e030473. https://doi.org/10.1136/bmjopen-2019-030473.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Pew Research Center. Demographics of mobile device ownership and adoption in the United States. Fact Sheet. 2021. Available from: https://www.pewresearch.org/internet/fact-sheet/mobile/. Accessed 29 Nov 2023.

  39. Bowen AM, Williams ML, Daniel CM, Clayton S. Internet based HIV prevention research targeting rural MSM: feasibility, acceptability, and preliminary efficacy. J Behav Med. 2008;31:463–77. https://doi.org/10.1007/s10865-008-9171-6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Jones J, Edwards OW, Merrill L, Sullivan PS, Stephenson R. Interest in HIV prevention mobile phone apps: focus group study with sexual and gender minority persons living in the rural Southern United States. JMIR Form Res. 2022;6(6):e38075. https://doi.org/10.2196/38075.

    Article  PubMed  PubMed Central  Google Scholar 

  41. McKenney J, Sullivan PS, Bowles KE, Oraka E, Sanchez TH, Dinenno E. HIV risk behaviors and utilization of prevention services, urban and rural men who have sex with men in the united states: results from a national online survey. AIDS Behav. 2018;22(7):2:2127–2136. https://doi.org/10.1007/s10461-017-1912-5.

    Article  Google Scholar 

  42. Mulawa MI, Rosengren AL, Amico KR, Hightow-Weidman LB, Muessig KE. mHealth to reduce HIV-related stigma among youth in the United States: a scoping review. Mhealth. 2021;7:35. https://doi.org/10.21037/mhealth-20-68.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Barry MC, Threats M, Blackburn NA, LeGrand S, Dong W, Pulley DV, et al. “Stay strong! keep ya head up! move on! it gets better!!!!”: resilience processes in the healthMpowerment online intervention of young black gay, bisexual and other men who have sex with men. AIDS Care. 2018;30(sup5):S27–38. https://doi.org/10.1080/09540121.2018.1510106.

    Article  PubMed  Google Scholar 

  44. Bauermeister JA, Muessig KE, LeGrand S, Flores DD, Choi SK, Dong W, Sallabank G, Hightow-Weidman LB. HIV and sexuality stigma reduction through engagement in online forums: results from the HealthMPowerment intervention. AIDS Behav. 2019;23:742–52. https://doi.org/10.1007/s10461-018-2256-5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Wang AL, Lowen SB, Shi Z, Bissey B, Metzger DS, Langleben DD. Targeting modulates audiences’ brain and behavioral responses to safe sex video ads. Soc Cogn Affect Neurosci. 2016;11(10):1650–7. https://doi.org/10.1093/scan/nsw070.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Huh J, Liu LS, Neogi T, Inkpen K, Pratt W. Health vlogs as social support for chronic illness management. ACM Trans Comput Hum Interact. 2014;21(4). https://doi.org/10.1145/2630067.

  47. Chiasson MA, Shaw FS, Humberstone M, Hirshfield S, Hartel D. Increased HIV disclosure three months after an online video intervention for men who have sex with men (MSM). AIDS Care. 2009;21(9):1081–9. https://doi.org/10.1080/09540120902730013.

    Article  PubMed  Google Scholar 

  48. Harshbarger CL, O’Donnell LN, Warner L, Margolis AD, Richardson DB, Novey SR, et al. Safe in the city. Am J Prev Med. 2012;42(5):468–72. https://doi.org/10.1016/j.amepre.2012.01.029.

    Article  PubMed  Google Scholar 

  49. Hirshfield S, Chiasson MA, Joseph H, Scheinmann R, Johnson WD, Remien RH, et al. An online randomized controlled trial evaluating HIV prevention digital media interventions for men who have sex with men. PLoS One. 2012;7(10):e46252. https://doi.org/10.1371/journal.pone.0046252.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Hirshfield S, Downing MJ, Chiasson MA, Yoon IS, Houang ST, Teran RA, et al. Evaluation of sex positive! a video eHealth intervention for men living with HIV. AIDS Behav. 2019;23(11):3103–18. https://doi.org/10.1007/s10461-019-02498-5.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Neumann MS, Plant A, Margolis AD, Borkowf CB, Malotte CK, Rietmeijer CA, et al. Effects of a brief video intervention on treatment initiation and adherence among patients attending human immunodeficiency virus treatment clinics. PLoS One. 2018;13(10):e0204599. https://doi.org/10.1371/journal.pone.0204599.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Schnall R, Mosley JP, Iribarren SJ, Bakken S, Carballo-Diéguez A, Brown W. Comparison of a user-centered design, self-management app to existing mHealth apps for persons living with HIV. JMIR Mhealth Uhealth. 2015;3(3):e91. https://doi.org/10.2196/mhealth.4882.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Cooper V, Clatworthy J, Whetham J, Consortium E. mHealth interventions to support self-management in HIV: a systematic review. Open AIDS J. 2017;11(1):119–32. https://doi.org/10.2174/1874613601711010119.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Beres LK, Simbeza S, Holmes CB, Mwamba C, Mukamba N, Sharma A, et al. Human-centered design lessons for implementation science: improving the implementation of a patient-centered care intervention. J Acquir Immune Defic Syndr. 2019;82(3):S230–43. https://doi.org/10.1097/QAI.0000000000002216.

    Article  PubMed  PubMed Central  Google Scholar 

  55. IDEO.org. The field guide to human-centered design 2015. Available from: https://www.designkit.org/resources/1.html. Accessed 29 Nov 2023.

  56. Bairaktarova D, Bernstein WZ, Reid T, Ramani K. Beyond surface knowledge: an exploration of how empathic design techniques enhances engineers understanding of users’ needs. Int J Eng Educ. 2016;32(1):111–22.

    Google Scholar 

  57. Altman M, Huang TTK, Breland JY. Design thinking in health care. Prev Chronic Dis. 2018;15(9):E117. https://doi.org/10.5888/pcd15.180128.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Sauro J. Measuring usability with the System Usability Scale (SUS). 2011. Available from: https://measuringu.com/sus/. Accessed 29 Nov 2023.

  59. Macharia P, Pérez-Navarro A, Inwani I, Nduati R, Carrion C. Developing an unstructured supplementary service data-based mobile phone app to provide adolescents with sexual reproductive health information: a human-centered design approach. BMC Med Res Methodol. 2022;22(1):213. https://doi.org/10.1186/s12874-022-01689-4.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Mi T, Li X, Zhou G, Qiao S, Shen Z, Zhou Y. HIV disclosure to family members and medication adherence: role of social support and self-efficacy. AIDS Behav. 2020;24:45–54. https://doi.org/10.1007/s10461-019-02456-1.

    Article  PubMed  Google Scholar 

  61. Leblanc NM, Flores DD, Barroso J. Facilitators and barriers to HIV screening. Qual Health Res. 2016;26(3):294–306. https://doi.org/10.1177/1049732315616624.

    Article  PubMed  Google Scholar 

  62. Golub SA, Gamarel KE. The impact of anticipated HIV stigma on delays in HIV testing behaviors: findings from a community-based sample of men who have sex with men and transgender women in New York City. AIDS Patient Care STDS. 2013;27(11):621–7. https://doi.org/10.1089/apc.2013.0245.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Budhwani H, Kiszla BM, Hightow-Weidman LB. Adapting digital health interventions for the evolving HIV landscape: examples to support prevention and treatment research. Curr Opin HIV AIDS. 2022;17(2):112–8. https://doi.org/10.1097/COH.0000000000000721.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to express their gratitude to the young Black gay and bisexual men who provided their time and input throughout the HCD research process and to the HIV service organizations and their staff who participated in developing Amp: AIDS Healthcare Foundation, APLA Health, San Francisco AIDS Foundation, Thrive Alabama, and the University of Mississippi Medical Center. We also thank N-tonic for producing the functional prototype and the Amp mobile app.

Funding

Research reported in this publication was supported by the National Institute On Minority Health And Health Disparities of the National Institutes of Health under Award Number R44MD013644. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and Affiliations

  1. Sentient Research, 231 North Walnuthaven Drive, West Covina, CA, 91790, USA

    Aaron Plant, Paul Sparks, Deborah Neffa Creech, Ta’Jalik Morgan & Jorge A. Montoya

  2. Keck School of Medicine, University of Southern California, 1975 Zonal Avenue, Los Angeles, CA, 90033, USA

    Jeffrey D. Klausner

  3. Rietmeijer Consulting, LLC, 533 Marion Street, Denver, CO, 82018, USA

    Cornelis Rietmeijer

Authors
  1. Aaron Plant
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Sparks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deborah Neffa Creech
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ta’Jalik Morgan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeffrey D. Klausner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cornelis Rietmeijer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jorge A. Montoya
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AP and JM both contributed to program development, study design, data analysis, and original draft manuscript preparation. PS contributed to original draft manuscript preparation. DNC contributed to data analysis and original draft manuscript preparation. TM contributed to program development and manuscript preparation. JK and CR contributed to program development and study design. All authors reviewed and edited the draft manuscript and approved the version to be published.

Corresponding author

Correspondence to Aaron Plant.

Ethics declarations

Ethics approval and consent to participate

All methods were performed in accordance with the relevant ethical guidelines and regulations. This study was reviewed and approved by Sterling Institutional Review Board on Oct. 16, 2018. Informed consent was obtained from all participants. All eligible participants were read a brief introduction and informed consent information. Participants provided verbal or written consent. Participation was voluntary for all participants and confidentiality was assured.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Plant, A., Sparks, P., Creech, D.N. et al. Developing an mHealth program to improve HIV care continuum outcomes among young Black gay and bisexual men. BMC Public Health 24, 1247 (2024). https://doi.org/10.1186/s12889-024-18652-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-024-18652-1

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us