Gray

health outcomes and greater barriers to using digital technologies as compared to their non-Indigenous counterparts. 32-35


Introduction
Mental health has been recognized by health system leaders around the globe as an important priority for policy and practice during the ongoing COVID-19 pandemic, as reflected by an increase in research, service development, and public interest worldwide. 1,2 Necessary public health protection measures, coupled with a loss in social and economic opportunities, have been stressors on communities worldwide. [3][4][5][6] The mental health impacts of COVID-19 are profound and have led to a significant impact on population mental health. 7,8 The COVID-19 pandemic presented an accelerated opportunity for the implementation of asynchronous virtual mental health (AVMH) technologies. 9 In response to COVID-related public health policies and stay-at-home-orders, health care providers quickly shifted to virtual and digital health care resulting in a rapid deployment and uptake of virtual mental health services or platforms since 2021. 10,11 Beyond the pandemic's near-ubiquitous effects on access to in-person mental health care, some population sub-groups face disproportionate disadvantages in accessing mental health services, including individuals who identify as 2SLGBTQIA, those with severe mental health concerns, or concurrent mental health and substance use, persons with disabilities, as well as Black and other communities marginalized by race, including Indigenous Communities. 12,13 Virtual mental health services or platforms may have potential to overcome traditional barriers to care among these equity-deserving communities through reductions in cost, transportation, and perceived stigma associated with accessing mental health care. 14,15 Virtual mental health services include both synchronous and asynchronous modalities: • Synchronous care (occurring in real time) provided virtually via consultations with providers through phone or video modalities. • Asynchronous care (not occurring in real time), such as mental health literacy resources, selfmanagement programs, 16 mood/symptom trackers, and treatment such as internet-based cognitive behavioural therapy (iCBT).
This report focuses only on asynchronous services and technologies for adults (i.e. AVMH), which refers to any app, website, online tool, or other online support application that does not engage in direct contact or synchronous consultation with a mental health care provider. For the purposes of this report, the term 'mental health care providers' refers to any trained health care professional who works to provide mental health services. 17 Our definition is inclusive of those working within or across various sectors (e.g., public and private health care settings).
The integration of virtual mental health technologies within medical practice can help health professionals in the following ways: 1. improve care coordination between primary and specialists. 2. Improve health care coverage in areas where there is a relative lack of service providers 3. Enhance service offerings by extending service opening times (e.g., not limited to weekdays 9 AM to 5 PM), lessening wait times, and being cost effective. 18 Equity and appropriateness of care are two major considerations for virtual mental health care. [19][20][21][22] Equity refers to removing systematic disparities that exist between groups who live with different levels of social advantage and social (in)justice. 23,24 Appropriateness refers to the most advantageous clinical modality for delivering care to achieve the best possible patient outcomes. 25 People living in rural and remote communities have poorer access to high-speed and unlimited broadband internet access than the rest of the country (i.e., 62% of the population having access in rural and remote communities as compared to 91.4% in more urban areas 26 ), which may affect their ability to access digital health information and virtual mental health care. [27][28][29][30] In Canada and beyond, equitydeserving groups including Indigenous peoples make up a large proportion of communities living in remote areas. 31 In addition, Indigenous-identifying people are at a higher risk of reporting poorer mental health outcomes and greater barriers to using digital technologies as compared to their non-Indigenous counterparts. [32][33][34][35] To better understand these experiences, this report synthesizes some of the literature on AVMH services and platforms to better understand what technologies are being used by individuals living in remote or rural communities in Canada and abroad. We also summarize effectiveness and uptake of AVMH services among people living in rural and remote areas, with particular considerations for Indigenous peoples, to offer a more nuanced understanding of these two important outcomes.
Our review of the existing literature includes studies that have been conducted around the globe, however we are particularly interested in distilling considerations and lessons for Canada. Because the delivery of health care services in Canada is organized by provincial and territorial governments, 36,37 mental health services varies across the country.

Questions
We describe AVMH health services, platforms, and technologies used by individuals living in remote or rural communities, including Indigenous peoples, with examples from Canada and internationally. We address two broad questions: • Which AVMH services or platforms are used by adults living in remote or rural communities in Canada and abroad, and what are the reported effectiveness and uptake outcomes of these platforms?
• What are the patient reported outcome measures (PROMs) of AVMH care delivered to adults living in in rural, remote, and Indigenous Communities?

Methods
We retrieved data for this report through three academic (i.e., MedLine, Scopus, Cochrane) and three grey literature databases (i.e., WHO COVID-19 Global Literature, Google Scholar [limited to the first 50 pages] and Open Grey). Search terms were loosely defined around geography, virtual, and mental health. We conducted hand searches in various journals focused on virtual care, mental health, and rural and remote health. We included empirical research and literature reviews of digital health platforms. The search was limited to English language for which full-text articles were available from 2021-2022, although a hand search of reference lists included some widely cited articles from 2019-2020. This was a rapid review, not a full systematic review (see Appendix 1).
Eligibility criteria included articles that reported on Indigenous peoples or rural and remote populations, aged 18 or older, using at least one AVMH service or modality (e.g., self-guided resources, blogs, AI technology, bots).
Citations were managed using Zotero. 38 We screened titles and abstracts of potentially eligible studies, and then full texts. All screening was performed by one reviewer, and uncertainties resolved through team discussion. Data abstraction was performed independently by three members of the team using Microsoft Excel. All authors reviewed the data abstraction form to identify themes and gaps. 39 To augment the findings from the literature review, between September 2022-December 2022 two PhDtrained qualitative researchers (KMK and SM) undertook 60 minutes semi-structured Zoom interviews with three key informants in Canada, each with expertise on the state of the evidence and current practices (see Appendix 2 for a Sample Interview Guide). We selected participants based on their scholarship and/or professional practice. Our Interview Guide included questions about asynchronous technologies/modalities being used in rural and remote communities, the roll-out strategies for using these technologies, and barriers and facilitators to their use. We asked key informants to provide any insights that were not addressed by these questions, which helped to inform the recommendations of this report. We systematically analyzed these interview data to identify key themes/patterns, tracking themes in a Word file. We compared these findings to our emerging synthesis of the literature. All three key informants offered to provide feedback on our report and their feedback was incorporated into the final version. The key informants have asked to remain anonymous but include: • A practicing family physician and clinical researcher with experience in providing mental health care using digital technology in rural and remote communities and evaluating AVMH platforms; • A practicing nurse practitioner and health service researcher with experience providing mental health care in rural and remote communities and researching Indigenous mental health needs, including in the realm of digital health technologies; and • An Indigenous researcher conducting research in the area of digital technology for rural, remote, and Indigenous people and communities.
KMK conducted the literature search; data extraction; key informant interviews; acquired, analyzed, and interpreted interview data; synthesized literature; and wrote report. SM conducted the literature search; data extraction; assisted with key informant interviews and thematic analysis; and edited the report. EBH assisted with the literature search; data extraction; and reviewed the report. KP and CC reviewed the report. SL critically reviewed the report and provided supervisory support.

Limitations
• To meet rapid deadlines, relevant publications and data may have been missed or excluded due to our rapid review methodology (e.g., we did not screen in duplicate which may have led to missing relevant articles), limited search strategy, and limited databases. Articles may have been missed given variations in terminology and definitions -asynchronous interventions are not defined or described consistently in the literature. The intent of the report was not to capture all articles but to provide an overview of the key literature with a focus on review papers. 40 Our search strategy (e.g., imposing language and date restrictions) might have excluded relevant studies. • The language used to describe AVMH services is inconsistent and still evolving. There is still no consensus in the literature on the terms used to classify various types of technologies and services. As such, the terms/definitions we used for this report may vary from other familiar terms. • Studies in this review were largely from high-income, western cultures (i.e., Canada, United States, Europe, and Australia). • There was limited evidence and reporting in the primary literature on long-term effectiveness, uptake, and cost-effectiveness of AVMH care. • Across jurisdictions, both asynchronous and synchronous mental health services or platforms were often used simultaneously, therefore the impacts of asynchronous interventions are difficult to parse out. • Articles often reported on the feasibility and acceptability of AVMH interventions, rather than other clinical outcomes and PROMs. • Articles often reported on rural and remote populations as homogenous populations, without considering the unique impact of particular contexts, geographies, social settings, and histories. Indigenous peoples were often reported as part of remote communities and, thus, we could not comment on any differences in outcomes between non-Indigenous and Indigenous peoples in these communities. • In our recommendations and lessons learned we did not consider the implications of jurisdictional differences in health care delivery across Canada in the implementation and delivery of AVMH services.

Results
Our search produced 53 articles. The published literature consisted mainly of review studies and quantitative articles. Most of the literature reported on Australia, Canada, and the United States. A summary of the literature included in this report can be found in Appendix 3 and our commentary on the overall state of the literature can be found in Appendix 4. Insights from the key informants further informed our lessons learned and key considerations for policy and practice in Canada.

AVMH Technologies Used by Adults Living in Rural/Remote Communities
A wide variety of AVMH services and virtual platforms have been studied in a range of settings and across several groups and individuals. These technologies have been reported as helpful to provide surveillance, tracking, and the monitoring of symptoms to help individuals manage their illness and receive appropriate support. The range of mental health services provided through AVMH includes: i) mobile apps and wearable devices (e.g., apps for smart phone, tablet; wearables such as smart watches, clothing) ii) text-based counselling (e.g., text reflections from a mental health worker sent in between face-to-face or virtual real-time sessions with a mental health worker) iii) web-based platforms that support self-care (e.g., guided exercises to improve resilience) and provide educational materials (e.g., skills training, mental health literacy).

AVMH Technologies Used by Indigenous Peoples
AVMH was identified as a key strategy to deliver interventions in Indigenous communities. 83 AVMH uses different types of devices and technologies to enable geographically separated individuals to exchange health information among themselves, 63,83-87 including those in Canada. 83 The types and uses of these AVMH technologies by Indigenous peoples were similar to those reported for rural and remote communities ( Table 1).
AVMH interventions include mobile applications and web-based platforms that promote healthy lifestyles involving culturally relevant information 84 (i.e., promotion of wellbeing through spiritual concepts, and culturally tailored motivational messages [e.g., proverbs]). 83 AVMH services were used to address suicidal thoughts, psychosis, borderline personality disorder, and emotion dysregulation in Indigenous peoples. 53,54,56,63,66,86 To be effective, AVMH interventions have to reflect the cultural values, social contexts, and views of health held by Indigenous peoples. 64,83 Below we indicate some of the ways AVMH services were tailored to Indigenous populations.

Mobile Apps and Wearable Devices for Indigenous peoples
Mobile phone technologies, particularly in the forms of mobile applications and social media applications, 64 have aimed to support the self-management of Indigenous peoples. 88 Self-management includes applications that support information sharing (i.e., provide free, asynchronous peer support) and applications that host questionnaires for individuals to better understand their symptoms (i.e., identifying their risk of suicide, or coping techniques for borderline personality disorder). 63,64 Other applications sent reminders about when to attend in-person appointments. 65 A systematic review noted that Indigenous peoples' expertise was strongly sought in the development and refinement of mobile phone technologies to ensure cultural responsiveness and clinical effectiveness. 65

Text-Based Counselling for Indigenous Peoples
Text-messaging was used to facilitate the asynchronous exchange of information, 54,59,64,65,89 particularly by individuals who do not prefer face-to-face interactions 64 and mental health care professionals who are able to provide support without formal referral. 89 Text-messaging has also been used for asynchronous suicide detection, with positive evaluations of cultural appropriateness by Indigenous peoples. 65,85

Web-Based Platforms for Indigenous Peoples
Web-based platforms were used to deliver educational and resource support through Health Information Technology (HIT). HIT is any web-based platform that provides individuals with mental health support and helps people to locate resources efficiently and effectively to help them self-manage their mental health symptoms 53 and find coping-strategies. 18,86,89 HIT also supported information-giving and sharing amongst Indigenous peoples, which improved mental health awareness and wellbeing among Indigenous people. 86 Some forms of HIT helped to facilitate digital storytelling, as a culturally responsive way for Indigenous people to share information. 90

Approaches to Evaluating Uptake and Effectiveness of AVMH Care
A variety of PROMs and other outcome measures and methodologies have been used to evaluate the uptake and effectiveness of AVMH. Effectiveness of AVMH has been measured by outcomes such as patient-reported satisfaction, self-management, medication adherence, symptoms, rehospitalization, client empowerment and engagement, and feasibility of use ( Table 2).

Effects of AVMH on patient-reported satisfaction:
As a proxy measure for satisfaction, clients' word counts have been evaluated to determine if, and how often, individuals have been able to reach out and ask questions whenever they wanted. 60 Word counts were obtained through asynchronous two-way messages between clients and health care teams pre-and post-treatment. 60,62 An analysis of the word counts determined that AVMH increased communication with providers to address health care issues about symptom self-assessment and tests and results. 60 Effects of AVMH on self-management, medication adherence, symptoms, and/or rehospitalization: Another study used the Generalized Anxiety Disorder Scale (GAD-7) and Patient Health Questionnaire (PHQ-9) to assess improvements in mental health care in rural and urban communities. 62 The study concluded those with the most distressing mental health symptoms from rural communities experienced the most significant improvement of symptoms as measured by a significant decrease in reported symptomology over time. 62 This study also compared Therapeutic Alliance Scores to measure patient satisfaction by the degree to which a client and provider are aligned in their views of the goals of treatment. 62 Text-messages that facilitated reminders of treatment plans and education messages for individuals with schizophrenia was reported to improve medication adherence, symptoms, and reduce rehospitalization using a generalized estimating equation model in comparison to using just attendance at an existing government community-mental health program. 73,74 Smartphone-delivered applications for self-management (e.g., medication tracking and symptom monitoring) were successfully installed by adults, but with relatively low usage rates. 88 Psychopathology was assessed through the Positive and Negative Syndrome Scale (PANSS) and low psychotic and general symptoms (measured by the PANSS) were generally low while using the applications, 88 suggesting a stable sample. Thus, smartphone applications can safely promote self-management with no related serious adverse events reported. 88 For other applications that targeted borderline personality disorder, symptoms such as anger, suicidality, impulsivity, and general psychopathology were measured, although no significant improvements in mental health symptoms were noted. 63 Rural male and female veterans with PTSD and depression who were enrolled in an asynchronous webbased skills training program reported significant improvements in PTSD and depression symptoms, as measured by PTSD Checklist for DSM-5 and Patient Health Questionnaire (PHQ-9). 42 Similarly, improvements in social functioning and emotion regulation, as measured by the Difficulties in Emotion Regulation Scale post-treatment and 3-month follow-up. 42

Effects of AVMH on empowerment and engagement among Indigenous peoples:
Some applications specifically designed for Indigenous peoples, such as AIMhi Stay Strong 65,91 and iBobbly, 65,66 have been evaluated to determine if they helped improved client empowerment, cultural appropriateness, and enhanced engagement. 65,91 These apps have aimed to provide visual representations, voiceovers, action-based content, and goal setting with the goal of building mental health coping strategies to reduce risk of suicide. 65 This research found an acceptability of culturally-appropriate applications by Indigenous clients and mental health practitioners, who reported that the applications are a culturally safe AVMH and well-being tool, 70,91 particularly in the area of suicide prevention. 65,66 By making the applications culturally-appropriate, the applications were effective in supporting client goal setting, increasing client self-insight, improving client empowerment, and promoting engagement. 45,91

Feasibility of AVMH in rural communities:
Web-based platforms, such as webSTAIR, 42 were reported to be feasible in rural communities, 42,50,55,78 for individuals with depression, anxiety and suicidal ideation. 52 However, individuals with psychosis living in rural areas expressed concerns about their ability to use the platform because of poor internet speed in rural locations. 76

Common Barriers and Enablers to Using AVMH Services
The use of asynchronous technologies has increased during the pandemic when physical distancing prompted the need to leverage the internet and mobile technologies for delivering mental health services. 76 These advances have contributed to improvements in ease of use and convenience for providers and clients, including lower health care costs and reduced requirements for travel, as well as enhanced access. 84 As such, these technologies may be a suitable mode of service delivery in rural and remote communities where there is a higher burden of unmet mental health needs and human resource shortages. 61 Especially for Indigenous peoples, these technologies require careful consideration and engagement of community partners. Consultations and collaboration are required to define priorities for reducing inequalities and developing context-appropriate interventions. There is increasing recognition in the scientific literature that health technologies often fail to reach their optimal potential if they are not adapted to the cultural, social, and economic contexts in which they are used. 61 These products should be designed and adapted to the cultural context with empowerment and engagement of the client in mind. 91 Barriers to the use of technologies can be grouped into two perspectives: clients/users and health care providers. Barriers that health care providers in AVMH uptake include a lack of technological support, demanding workloads, difficult organizational procurement and practice policies, and negative staff perceptions. 51,89 Client/user-level barriers and facilitators can be categorized as technological, socioeconomic, or socio-cultural.

Technological barriers and enablers to uptake of AVMH services
The most commonly reported barrier was related to the technology itself, reflecting the challenges in designing technologies for complex problems that overlap with medical, regulatory and social domains, such as mental health care. 18 One of the major barriers was connectivity or the bandwidth in rural/remote and Indigenous populations; rural communities may lack consistent internet and/or cellular coverage or people may not own smart devices, 67,77 limiting the ability of clients to download apps or access webbased materials.
Studies have described special considerations for the use of digital technologies, especially in severe mental health diseases like schizophrenia 70 or when mental health concerns coexist with chronic conditions like diabetes. 48 Individuals with comorbidities may require high quality emotional support. 48 Therefore, virtual mechanisms for treatment may not reach the threshold of care required for these individuals.
Digital apps require attention to issues of privacy and confidentiality, particularly as they may involve repositories of sensitive patient information. 50 Issues of regulation and licencing of apps also arises, particularly in validating apps where the therapeutic effects may be questionable and where there is potential for harm or a worsening of a patient's condition. 92 Catering to clinical and institutional needs is also a consideration for adopting technology; integration of these technologies into clinical practices will require robust infrastructure and a supportive culture to encourage technology adaptation. 79 Some health care providers value access to digital technologies as they believe it can provide tools that overcome traditional barriers like travel, time, resistance to talk, and save costs. Evidence on provider views about feasibility and acceptability suggested that implementing digital cognitive behavioural interventions would require building greater awareness around apps and integrating platforms into clinical workflows, for example, integrating data collected through digital platforms into medical records. 51 Physicians would be more comfortable with the technology if there were standards in place for data collection and storage to help overcome concerns over privacy. 51,93 The importance of endorsement from other providers as facilitating providers' comfort with technology suggests early adopters may be able to influence integration by the broader community. 51

Socio-economic barriers and enablers to uptake of AVMH services
Digital literacy was commonly reported as a barrier to the use of AVMH technologies. Higher technological competency or confidence is associated with the use of technology, interest in using virtual mental health options such as iCBT, and adherence to iCBT treatment. 66,74,75 Increasing population age and rural location have been associated with challenges in adoption of digital health tools. 70 In families with a lower household income, devices are often shared between family members which raises concerns for privacy and make AVMH less preferable compared to in-person visits. 72,80 Black, Indigenous peoples, rural communities, refugees, older adults and other equity-deserving individuals have been particularly challenged by a lack of housing and technology infrastructure and/or costs (e.g., high-speed internet), making it difficult to use asynchronous technologies. 45, 61 Racialized communities tend to have lower digital literacy, posing as an additional challenge to engaging in virtual technologies. 45,61

Socio-cultural or culturally appropriateness as barriers and enablers to uptake of AVMH services
Socio-cultural sensitivity and appropriateness were often cited as determining factors for acceptance and usability within Indigenous populations. 45, 65 Indigenous peoples may face various individual barriers, such as language and cultural barriers. 53 Culturally-adapted asynchronous technologies are more likely to be used by Indigenous, Black, and other equity-deserving individuals than non-culturally adapted products and can help to overcome barriers to digital mental health services. 45 Identity empowerment is important in the well-being of Indigenous peoples, which is deeply rooted in the appreciation and conservation of cultural practices and knowledge. 86 Cultural expertise can be used to ensure appropriate language, imagery, and design, which can increase the uptake of technology. 65 Education on how to use new AVMH applications should target key respected champions from diverse backgrounds to promote the use of products in their communities. 64 Co-design with users of technology can make the design aspects culturally appropriate and user-friendly and affirms users' identity and preferences. Participants from a series of workshops in designing mental health information technologies in a non-urban area in Australia recommended that technology should provide meaningful guidance and consider the fact that different cultures treat mental health differently and, thus, destigmatize culturally-appropriate approaches to mental illness. 18 To help make asynchronous technologies more socio-cultural or culturally appropriate, language translation, modification of audio/visual content, and inclusion of culturally salient messaging have been employed. 45 Design features need to be attractive and digitally secure platforms also enable safe use. 89 An example of visual modification includes producing digital avatars on mobile applications that are consistent with the racial/ethnic group for which the AVMH intervention was developed. 45 Translation of an AVMH intervention into participants' native language was the most common strategy to enable adaptation identified in a scoping review. 45 The inclusion of culturally relevant messages 45 aligned with worldviews and perceptions health 83 further enabled adoption.

AVMH Policies, Practices, and Lessons for Canada
What lessons can Canada learn about implementing and evaluating asynchronous mental health technologies in rural, remote, and Indigenous communities?
Lesson 1: AVMH services/platforms may not be effective for some adults.
This lesson is targeted at health care providers. While there is evidence that AVMH can be widely used, including within rural and remote communities, this modality does not solve broader issues related to equitable access to mental health care. One way to change that would be to bridge the digital divide so that reliable high-speed internet is available to everyone, no matter where they live in Canada. Until that happens, AVMH care will continue to be inaccessible to some. Other factors limit uptake of AVMH interventions across individuals and settings, including as cultural factors, lack of social support, behaviours and beliefs, and the environment. Although the benefits of asynchronous virtual care may not be equally distributed across all Canadians, research into its limitations will elucidate how, why, and for whom there is merit in the application of these technologies and services.

Lesson 2: A micro-focus on implementation of AVMH in specific settings/populations may reveal factors affecting client/patient engagement, which might improve uptake and effectiveness.
This lesson is targeted at researchers. Key informant interviews highlighted that while virtual care was previously used more often in rural and remote settings due to a lack of mental health providers, the COVID-19 pandemic resulted in a shift where these technologies are now often used and evaluated in urban settings. Consequently, we have a better understanding of the implementation process, particularly how individual communities incorporate new AVMH services which may inform organizations looking to adopt new technologies. 94 Rigorous studies of AVMH products, such as randomized control trials that explore socioeconomic and cultural covariates may be useful. As much literature has highlighted barriers that prevent rural and remote populations and Indigenous peoples from using asynchronous technologies, a deeper understanding of all the factors that influence engagement and enrollment is needed. As such, qualitative and co-designed research methodologies can help supplement quantitative research and offer a more holistic understanding of adoption and use of AVMH technologies. This is particularly important as much of the evidence on this topic has been gathered via quantitative research, whereas the nuances of acceptability, context, and preferences would benefit from qualitative approaches for a deeper understanding. For instance, rigorous enrollment processes with strict eligibility criteria may have influenced uptake and the extent of engagement, but these were not described in enough detail to understand which strategies would increase engagement.
While rural and remote communities as compared to Indigenous communities may each be unique in their engagement with asynchronous technologies, the nuances in experiences and factors that influence engagement can differ across communities and groups. Yet, studies in the existing literature have tended to approach and study these communities as a homogenous group, ignoring the diversity of the peoples and their communities. As such, there is a need for further study to better understand how AVMH can support inclusive engagement.

Lesson 3: Uptake and effectiveness of AVMH services is affected by cultural appropriateness.
This lesson is targeted at content developers and vendors. Indigenous peoples understand mental health within traditional knowledge and culture. 95 This complexity related to setting, culture, and local knowledge contributes to challenges in designing and implementing health services that are culturally-appropriate -both methodologically and conceptually, 96 necessitating consideration of local environments and social relationships. 100 As such, those involved in the development of mental health services are encouraged to consider the role of community, land, and spirit in the design of asynchronous technologies and in the delivery of mental health service. 101 While this is not an exhaustive list, at a minimum, some consider that it would be beneficial to offer methods for amplifying and expressing Indigeneity, and address colonialism through trauma-informed care. 102 Language revitalization in the development of technologies can also be used to support this and health literacy, 103 as can the use of peer support. 104 In considering the cultural relevance of existing and future asynchronous mental health interventions, we suggest that researchers and content developers examine outcomes of interest at multiple levels, including patients, providers, organizations, and systems. Critical investigations of how AVMH interventions may have contributed to the exacerbation of cultural inequities may provide guidance for culturally appropriate interventions to eliminate such disparities. We recommend that future AVMH interventions report comprehensive details of their methods to enable future reviews to critically appraise the quality of evidence.

Lesson 4: Focus on AVMH services/care, rather than products, might improve effectiveness and uptake.
This lesson is targeted at funders, regulators, and vendors. Although key informants reported that there are thousands of technologies and applications worldwide that offer virtual mental health services, most of the existing research has focused on individual applications as a product, rather than as part of a wider service or program of services within a complex system of care. By exploring and evaluating the efficacy of various technologies themselves, less attention has been placed on the broader portfolio of local and regional services and resources required to ensure delivery of AVMH care suited to the local context and needs of rural, remote, and Indigenous communities. Consequently, existing evaluations of health applications and technologies may have adopted a narrow approach and are less considerate of the real-life complexities around people's experiences of health and illness that influence their needs from services. 105 Evaluations tend to focus on use for a population with a single illness or diagnosis, rather than considering complexities in treating of mental health, such as multimorbidity and polypharmacy. 105 Government-vendor contracts can help develop products that meet the service needs of health care providers, as existing vendors may not have access to clinical staff who influence how technologies are introduced and offered to patients. 106 For example, existing trials and evaluations may specially exclude those with poor literacy skills and may not therefore reflect the needs of those in that demographic who require services or use services. Better product evaluations can inform developers so that their products become more impactful and beneficial to the public and health professionals.

Conclusion
Overall, we found positive experiences and outcomes related to usage and acceptability of AVMH interventions by users in rural, remote, and Indigenous communities.

Appendix 1: More information on Included Papers and Search Strategy
Detailed information on search for academic and grey literature.
In searching academic and grey literature, the following key search terms and definitions were used and inclusion criteria applied. Data on review approach, study design, setting, jurisdiction, population studied (e.g., rural/remote, Indigenous, illness), and type of impacts were extracted when reported. We summarized the results narratively due to variation in methodology in the included papers.
In total, 3,625 documents were identified, 122 were screened for relevance, and 53 included. These papers spanned various populations, as illustrated on the following pages.

Zaslavsky et al, 2022 United States Depression Rural
The 53 papers comprise the following: • 11 were quantitative studies • 20 were review articles • 6 were qualitative studies • 5 used mixed-methods • Other study designs included protocols, quality improvement studies, reports, commentaries, and a thesis.
The list of included papers with key findings is attached as Appendix 3. -Web-based resource (farewell) built based on acceptance and commitment therapy, with the ultimate goal of reducing barriers to engagement with traditional mental health and well-being strategies -Resource considered an easily accessible and confidential source of self-help techniques for farmers -5 interactive modules (written, drawn, and audio-and video-based psychoeducation, exercises, and farmingrelated jokes, metaphors, examples, and imagery) used to deliver techniques -SMS text messages provided personalized reminders and support -Although results showed high acceptability, some improvements such as additional reminders, higher quality audio recordings, and shorter modules were recommended.

Australia
To determine the characteristics of mHealth interventions in relation to First Nations populations, while outlining the intervention outcomes and user perspectives (ex. cultural responsiveness and clinical effectiveness)

Scoping review
Indigenous N/A -13 studies (5 randomized controlled trials and 8 quasiexperimental designs) conducted in the various countries; Australia (n=9), the United States (n=2), and New Zealand (n=2) were analyzed -Mental health and suicide were frequently discussed among the studies (n=5) -Intervention modalities included text messaging (n=5), apps (n=4), multimedia messaging (n=1), tablet software (n=1), or a combination of short messaging service (SMS) and apps (n=1) -Findings suggested mixed engagement with the intervention (n=3); favorable user perspectives, including acceptability and cultural appropriateness (n=6); and mixed outcomes for clinical effectiveness (n=10) Hollan et al.

United States
To examine the pre-post treatment outcomes of a two-way message-based asynchronous therapy service (Talk space).

Quantitative
Rural 460 -Statistically significant pre-post improvements on both outcomes' measures for both rural and urban clients were demonstrated -Results supported in favour of the two-way messagebased asynchronous therapy as a treatment method -Rural clients had significantly better outcomes for depression compared to urban participants -Further investigation on the interactions between telemental health care and sociocultural identities required Multi-country To examine the digital interventions that could be used to support Canadians' mental health during the COVID-19 pandemic, by identifying the target population, the effect and the barriers/facilitators of the interventions Rapid review Indigenous N/A -31 mobile apps and 114 web-based resources (e.g., telemedicine, virtual peer support groups, discussion forums) to support the mental health of the Canadian population during the pandemic were identified -Further evaluation on equity-related issues is required -All of the interventions identified in this synthesis were not reported to have an effect, but among the ones reported, it was established that they were efficient in the setting in which they were applied -Access, cost, and connectivity were recognized as barriers/facilitators Thenral & Annamalai (2020)

India
To examine the state of telepsychiatry in India and the function of artificial intelligence (AI) in mental health and potential applications

Scoping Review
Rural N/A -Although there are a number of potential opportunities, the time is not yet right for telepsychiatry and AI to be widely used in the field of mental health care -Psychiatrists must select the best tool based on their needs, the resources that are available, and the practicality of deployment -Harmony between conventional care and technologybased care must be attained gradually Tighe et al.

Australia
To outline the pilot usage and acceptability of the iBobbly, suicide prevention app.
Mixed methods (Randomized controlled trial)

Indigenous 13 10F, 3M
-Regression analysis showed that app use improved psychological outcomes, although only minimally (insignificant effect) -Thematic analysis results indicated that the iBobbly app was deemed effective, acceptable, and culturally appropriate -iBobbly app considered to be culturally safe and of therapeutic value -All 13 participants stated they would recommend the app to others -92% (12/13) stated that they would take part in a similar trial again Toombs et al.
Multi-country To assess the use of eHealth interventions for Indigenous youth to communicate empirically supported practices and to provide suggestions on developing future digital interventions.
Systematic review Indigenous (youth) N/A -Preliminary findings demonstrate the usefulness and affordability of eHealth interventions -Qualitative outcomes generally indicated positive community or individual response to eHealth interventions from service providers or service users -Quantitative findings described outcomes for alcohol screening, brief intervention, smoking cessation and suicide prevention

Study Results
Turmaine et al.

France
To determine whether the local use of the eHealth tool StopBlues (SB), which aims to prevent psychological distress and suicide, varied depending on local contexts and if implementation was related to the use of the tool.

Cluster Randomized Controlled Trial
Rural & urban -3 distinct promotion patterns were identified -From highest to lowest utilization rates, they are listed as follows: the privileged urban localities, investing in health that implemented a high-intensity and digital promotion, demonstrating a greater capacity to take ownership of the project; the urban, but less privileged localities that, in spite of having relatively little experience in health policy implementation, managed to implement a traditional and high-intensity promotion; and the rural localities, with little experience in addressing health issues, that implemented low-intensity promotion but could not overcome the challenges associated with their local context Yu et al. (2022)