An exploration of inter-organisational partnership assessment tools in the context of Australian Aboriginal-mainstream partnerships: a scoping review of the literature
BMC Public Health volume 15, Article number: 416 (2015)
The need for better partnerships between Aboriginal organisations and mainstream agencies demands attention on process and relational elements of these partnerships, and improving partnership functioning through transformative or iterative evaluation procedures. This paper presents the findings of a literature review which examines the usefulness of existing partnership tools to the Australian Aboriginal-mainstream partnership (AMP) context.
Three sets of best practice principles for successful AMP were selected based on authors’ knowledge and experience. Items in each set of principles were separated into process and relational elements and used to guide the analysis of partnership assessment tools. The review and analysis of partnership assessment tools were conducted in three distinct but related parts. Part 1- identify and select reviews of partnership tools; part 2 – identify and select partnership self-assessment tool; part 3 – analysis of selected tools using AMP principles.
The focus on relational and process elements in the partnership tools reviewed is consistent with the focus of Australian AMP principles by reconciliation advocates; however, historical context, lived experience, cultural context and approaches of Australian Aboriginal people represent key deficiencies in the tools reviewed. The overall assessment indicated that the New York Partnership Self-Assessment Tool and the VicHealth Partnership Analysis Tools reflect the greatest number of AMP principles followed by the Nuffield Partnership Assessment Tool. The New York PSAT has the strongest alignment with the relational elements while VicHealth and Nuffield tools showed greatest alignment with the process elements in the chosen AMP principles.
Partnership tools offer opportunities for providing evidence based support to partnership development. The multiplicity of tools in existence and the reported uniqueness of each partnership, mean the development of a generic partnership analysis for AMP may not be a viable option for future effort.
Australia, along with other high income countries with colonial pasts, has struggled to minimise the disparities between its Aboriginal and non-Aboriginal populations, where the gap in life expectancy is variously reported as being between 11 and 18 years . Partnerships between Aboriginal and mainstream organisations are seen as fundamental for improving Aboriginal health outcomes [2-7].
Aboriginal community controlled organisations, led by Aboriginal boards, provide culturally appropriate services as well as reflecting the aspirations Aboriginal people have for self-determination . Many of these corporations and services are now significant enterprises [9,10] and have important roles in supporting mainstream delivery of effective services. For example, many Aboriginal Community Controlled Health Services (ACCHS) in addition to providing programs and services directly to their local Aboriginal community often support mainstream organisations to access community members by allowing them utilise their infrastructure [11-13], and participate in partnerships to deliver services or conduct research [14-21]. Baeza and Lewis also point out the role that ACCHSs have played in educating mainstream services on providing appropriate care for and working with Aboriginal people .
Successful Aboriginal-mainstream partnerships (AMP) offer multiple opportunities for improving health outcomes [2,3,5] and community development, particularly in rural and remote Australia . However, with poor Indigenous health outcomes/indices widely attributed to colonisation and its ongoing expressions [24-26], building such partnerships can be challenging and complex particularly as they continue to be affected by Australia’s historical and current context of race and political relations .
Partnerships are also affected by disparate ways of working; power dynamics and funding timeline pressures to deliver outputs that often overwhelm the necessary development of trust and relationships [17,27,28]. ACCHS face particular governance challenges balancing community obligations with financing and reporting accountabilities that impact on their ability to maintain partnerships .
Partnerships are particularly undermined and collaborations weakened through lack of clarity about respective roles resulting in confusion about the partnership’s purpose, its objectives and how to measure its success [5,30]. The ability to assess the success of Aboriginal-mainstream partnerships is therefore critical in both communicating the value of the partnership internally and externally and to ensure continued growth and strengthening of the partnership .
Objective of the review
Approaches to assessing and improving Aboriginal-mainstream partnerships, and the value of using appropriate tools, remain open to debate. This scoping review  explores existing partnership assessment tools for their potential value in measuring the success of AMP and supporting partnership development. As partnership assessment is intrinsically linked to developing and strengthening partnerships, the review focuses on self-assessment tools, not external evaluation processes.
While an independent facilitator may usefully be involved in supporting partnership self-assessment, the tools reviewed here demand a degree of participation by the partners themselves. As suggested by Taylor and Thompson, the focus of partnerships needs to be on developing genuine, trusting relationships that are tangibly linked to the Aboriginal community. ‘Failure to invest in this relational process and push forward with ‘business as usual’ can ultimately have negative ramifications on client outcomes’ . This review therefore focuses on partnership assessment tools that emphasize the importance of understanding successful partnerships, recognising that both process and relational factors are important in successful partnerships .
Review and analysis of partnership assessment tools
The review was conducted in three distinct but related parts, described in detail below.
Part 1: identifying and selecting reviews of partnership tools
A search was conducted in October 2012 and updated in October 2014 using Science Direct, Proquest, Sage, PubMed, Informit, Wiley Online (Journal), and Google Scholar and the key phrases: partnership assessment tools review, partnership tool applicability, partnership tool application, and Aboriginal-mainstream partnership. Additional searching was undertaken using the Google search engine to identify relevant additional literature not captured in the previous search.
Articles were excluded from further consideration if they did not have specific reference to partnership evaluation procedures or were reviews of partnership tool analysis without a clear documentation of methodology. The remaining articles were reviewed to identify potential partnership self-assessment tools (see Part 2 below), and learnings regarding implementing tools.
Part 2: identifying and selecting partnership self-assessment tools
Partnership assessment tools identified from the articles identified in Part 1 were downloaded for detailed review. The inclusion criteria for partnership tools were set as follows:
elements included in tools were based on a literature review or systematic examination of multiple practitioner experiences of a successful partnership, or were included in at least two of the tool review articles found in the first part of this analysis;
tools were designed to facilitate self-assessment and contain checklist or Likert scale-like components as opposed to a conceptual framework, open-ended questions or activity instructions to support partnership design or the partnering process;
tools were openly accessible from the internet.
Partnership tools designed to foster partnership within a single organisation were excluded as our interest was in AMP as inter-organisational partnerships.
Part 3: analysis of selected tools using AMP principles
Process and relational elements within the items in the identified AMP principles were identified. Similarly, process and relational elements contained in the partnership self-assessment tools were also identified. Process and relational AMP principles were then used as the basis to assess the usefulness of partnership assessment tools to AMP. This process identified elements in the partnership assessment tools appropriate to the AMP context as well as gaps or deficiencies in assessing adherence of partnerships to identified AMP principles. Figure 1 summarises this analysis, including references to the tables in which results are documented.
Findings from literature search
The output from database searches and processes of eliminating articles for detailed review and analysis are summarised in Figure 2. Three broad types of data sources were identified: AMP principles, review of partnership tools or documentation of experience in applying partnership tools in inter-organisational partnerships and partnership self-assessment tools.
Table 1 summarises articles included for abstract review and initial list of partnership tools reviewed against inclusion criteria.
Identification and selection of Aboriginal and mainstream partnership principles
To determine the suitability of existing partnership assessment tools required identifying best practice principles for satisfactory AMP functioning in Australian context. We selected from three key sources [5,33,83] identified through the data base searches and guided by authors’ knowledge and experience. The use of comprehensive original data sources, in-depth analysis, or their association with respected national bodies with extensive experience in Aboriginal and mainstream health service partnerships (Reconciliation Australia, Secretariat of National Aboriginal and Islander Child Care) were important considerations.
Taylor and Thompson identified 16 key learnings from 24 published reports of AMP on improving AMP in health services . Reconciliation Australia asserted ten ingredients for successful Aboriginal and Torres Strait Islander policies and programs outlined in past reports, studies and research papers . Burton outlined eight core principles underpinning genuine and successful partnerships between Aboriginal controlled community organisations and mainstream service providers from a case study review . Elements in all three sources had some similarities and were useful in providing the lens through which elements in the partnership tools are assessed.
Partnership tool reviews
Two partnership assessment tool summaries [38,59]; three reviews of partnership assessment tools [61-63] and four papers documenting experiences of implementing partnership analysis tools [57,60,64,84] were downloaded to identify tools relevant to consider for AMP assessment. One comparative analysis of four selected partnership analysis tools was excluded due to a substantial gap in methodology documented . At the time this paper was revised, Markwell’s Partnership Working: A Consumer Guide to Resources  was no longer accessible through the internet and therefore is not included in Table 2 which summarises the author(s), year of publication, aim of the review, methods of the review, study location and intended audience of the seven partnership tool reviews. None of the identified reviews or discussions made any specific reference to AMP or working in cross-cultural partnerships. The following sections outline relevant learnings from these papers regarding using tools, which inform the subsequent analysis of the identified tools using the AMP principles.
Using partnership tools
Typically partnership evaluation frameworks and tools are selected or compiled by the agency, partner or evaluation funder  in order to measure the effectiveness of collaborative endeavours and to demonstrate to funding bodies that the partnership has been worthwhile . In keeping with partnership development aims, self-assessment tools are designed to generate discussion among partners although the literature is divided regarding the value of partnership tools. The strengths and weaknesses/limitations in partnership tools identified in the reviewed articles are summarised in Table 3.
Suggestions for using tools
The experience of applying a structured tool in partnership evaluation has highlighted conditions to be met in order for a useful application of assessment tools.
The first and most important condition to be met prior to applying a partnership tool is having the understanding of organizational settings and the operational environment alongside any agreed measurement of partnership effectiveness .
Secondly, the funding or governing authorities requesting a tool supported partnership evaluation process need to ensure that all partners understand the value of ongoing self-evaluation or reflection for partnership development, and that this should ideally take place in the early formation stage .
Thirdly, appropriate time and support should be given to master the technical aspects of using self-assessment tools . It is important to consider whether the chosen method offers a means of analysis for the partnership as an entity as well as meeting the needs of individual partners .
Application of partnership tools can be time consuming and require substantial commitment; otherwise it is unlikely to foster learning and development . Partners need to be prepared to invest the necessary resources in broad-based evaluation activities, including predetermining the components of partnership to be measured, the time and effort a partnership is willing to invest in the evaluation process, and the most appropriate way to evaluate the partnership .
Finally, as suggested by Granner and Sharpe, valid and reliable tools could be applied across multiple partnerships in the same community context in order to achieve a better understanding of the associated factors which influence partnership functioning and community health impacts and outcomes .
Adapting partnership tools
A cross-cutting theme from discourses on the role of an assessment tool in partnership evaluation has been the importance of reflecting the context in which the partnership operates [59,60,62,64,85]. Similarly, there is considerable skepticism around the ability of existing tools to generate valid and reliable data to reflects changes in program quality, shifts in success factors and the impact of working in partnership on desired outcomes [62,85]. The question of whether or not, and how, to adapt existing tools is examined in many papers.
Some papers document experiences of using adapted partnership tools to meet the needs of the partnerships in question [57,60,84]. These authors generally warn against over-reliance on a structured tool without contextualizing the assessment findings to the circumstances in which the partnerships are operating [62,84,85]. Other practitioners have chosen from available tools as starting points for adapting or creating an evaluation instrument to suit the circumstances [57,60,84].
Joss and Keleher point out that different tools represent variations in focus rather than intrinsic superiority, so evaluators need to be guided by the requirements of the partnership . They suggest that it is better to design a bespoke tool that reflects the organic and context-specific nature of partnerships and to capture the composite and complex partnership characteristics in a valid and reliable manner .
The approach in adapting partnership tools is not well documented. Discussion predominantly focuses on whether the difficulty of adaptation is associated with the design of the instrument in the data collection phase, or if more emphasis should be placed on contextualizing findings in the analysis stage. We return to the discussion of contextualizing findings in the Discussion section of this paper.
Elements of partnership self-assessment tools
Seven tools were identified that met the inclusion criteria. They were subjected to detailed analysis and the aims, audience and basis of tool elements summarised (Table 4), including reporting whether the basis of the elements came from a review of literature on partnership success factors and/or on practitioner experiences.
In total, 190 process, relational and outcome elements were identified from the partnership tools, 100 of which were relational elements, 80 were process elements and only 10 were associated with partnership outcomes (data not shown). The outcome elements were not included for the purpose of this analysis.
The process and relational elements of the reviewed tools were mapped against the process and relational elements of partnership functioning, identified through the thematic analysis of the three sets of AMP principles. The identified process elements (themes) are presented together with their respective sources and principle states in the first three columns of Table 5, while those associated with relational elements are presented in the first three columns of Table 6.
Process elements in partnership assessment tools mapped to AMP principles
Nine of the 15 AMP principles associated with partnership processes were reflected in elements of the reviewed partnership assessment tools. Table 7 provides a summary of the process elements found in the partnership self-assessment tools corresponding with relevant AMP principles.
In terms of process assessments in AMP, the elements in the reviewed tools concerning common aims, partnership and membership structure, communication, valuing process, outcome and innovation, flexibility and adaptability to ensure involvement of all partners and the reflective elements of monitoring and evaluation are all applicable in the AMP context.
The key deficiency in process elements found in the tools reviewed with reference to the AMP principles related to the timeframes and level of resourcing required to achieve determined outcomes. The three sources of the AMP principles emphasise realistic investment [5,83] for long-term achievements  and long-term wellbeing outcomes . Whilst the tools reviewed also highlighted the importance of realistic and adequate resourcing [74,77,78,82], the focus has been on the agreed level of resourcing, outcome and achievements by all partners. Thus, the timeframe in which resources should be allocated appears to be a point of distinction in Australian AMP circumstances.
Relational elements in partnership assessment tools mapped to AMP principles
Sixteen of the 22 AMP principles associated with relational elements were reflected in the partnership assessment tools included in this review (Table 8).
A good alignment was found between elements of mutual respect/trust, equal participation/democracy and equity, innovation/creativity in power sharing and problem solving, and sharing of accountability and responsibility. The cross-cutting theme in the leadership elements involves a need to nurture motivated individuals not only to work in partnerships but also to have the skills and preparedness to cross traditional boundaries, a role referred to as the ‘boundary spanner’ in a number of partnership discourses [58,86,87]. Elements associated with equal participation found in the Nuffield, VicHealth, New York PSAT and Verona benchmark tools are all potentially appropriate to assess the AMP principle of equal participation in planning and power sharing found in Taylor and Thompson . Elements of shared responsibility found in the Wilder Collaborative Inventory  are also potentially appropriate in this context.
Mapping exercises contained in two of the reviewed tools appear helpful in unpacking the complex relationships in which some partnerships operate. The VicHealth Tool is a simple but useful activity for mapping partnership, assessing the nature of relationships between participating partners, and differentiating between networking, coordinating, cooperating and collaborating relationships . Tennyson’s partnering tool book proposes mapping stakeholders according to the level of interest and degree of influence of each partner .
There are also items extending the analysis beyond internal partnership relationships to include relational aspects of the partnership with the community and broader society. Community understanding, partners’ perceptions that the partnership’s operation and achievements are meeting the needs of the community and stakeholders, and that there is adequate understanding of these benefits among community members are included. The impact of the historical, social and political environment in which the partnership operates is also subject to assessment, including issues such as the distribution of resources, class and community, gender and race. Connections between the partnership and political decision makers and external organisations are also viewed as conducive to successful collaboration [74,78].
Elements associated with self-determination found in the Verona benchmark  and New York PSAT  are comprehensive and potentially useful in the Australian AMP context. Similarly, elements associated with community engagement found in the New York PSAT and VicHealth tools have potential to support and analyse aspects of sustainable relationships (Table 8).
Elements associated with the AMP principles of respecting the needs of the Australian Aboriginal and Torres Strait Islander communities were less neatly aligned. Elements associated with responsiveness to community needs (Table 8) should reflect being responsive to the needs articulated by the Aboriginal community , working with Aboriginal people to knit together local knowledge within a national framework , and openness to approaches that are different from the mainstream conventions [33,83]. While these principles found their counterparts in elements of the New York PSAT and the Verona benchmarking tools, they are generally poorly captured.
One of the recommendations made to improve AMP is related to visibility of partnership projects in order to ensure engagement by the members of the local communities . Whilst the New York PSAT and the VicHealth tools have corresponding elements associated with raising public profile and prestige of the partnership, they are referring to ‘public profile’ in different contexts. Whether or not the purpose of such an element is to encourage participation by potential partners is not apparent from the literature reviewed.
Cultural components in the AMP principles include elements of respecting Aboriginal cultural knowledge, lived experience and honouring the ‘Aboriginal ways’ and assessment of these was not available in any of the tools reviewed. Furthermore, openly negotiating what Taylor and Thompson refer to as “historical baggage”  is another element specific to the Australian AMP context not captured in the reviewed tools.
Alignment between partnership tools and AMP principles in the Australian context
Table 9 summarises the number of corresponding AMP principles the process and relational elements contained in each of the tools reviewed.
Five out of seven tools have potential to assess the process aspects of Aboriginal-mainstream partnerships. Hardy et al’s Nuffield Partnership Assessment Tool  and the VicHealth Partnership Analysis Tool  contain elements corresponding to seven of the principles. Markwell et al’s Working Partnership Book , previously known as the Verona Benchmark, contain elements corresponding to six principles, while the Wilder Collaborative Inventory developed by Mattessich et al.  and the New York Partnership Self-Assessment Tool  contain elements corresponding to five of the AMP ‘process’ principles.
Two of the seven tools have greater potential to assess the relational aspects of AMP principles. The New York Partnership Self-Assessment Tool  contains elements corresponding to six AMP ‘relational’ principles, while the VicHealth Partnership Analysis Tool  contains elements corresponding to five principles. In addition, the Wilder Collaborative Inventory contains elements corresponding to mutual respect and accountability which distinguished it from the New York PSAT and VicHealth tools which did not assess these elements.
Overall assessment indicated that the New York PSAT and the VicHealth Partnership Analysis tools reflect the greatest numbers of AMP principles followed by the Nuffield Partnership Assessment tool. The New York PSAT has the strongest alignment with the relational elements of the AMP principles adopted in this paper while VicHealth and Nuffield tools showed greatest alignment with the process elements in the chosen principles.
The focus on relational and process elements in the partnership tools reviewed is consistent with the focus of Australian AMP Principles by reconciliation advocates. The potential of a large number of elements contained in available partnership self-assessment tools has been affirmed. However, the historical context, lived experience, cultural context and approaches of Australian Aboriginal people represent key deficiencies in the tools reviewed.
A history of oppression is not a distant memory to the Australian Aboriginal population remaining part of their lived experience. It is important to recognise and respect the world views of Aboriginal and Torres Strait Islander people  when interpreting the process, relational and outcome data collected. Relevant to the current Aboriginal and mainstream partnership (AMP) discourse is Johnstone’s argument that advocates for health researchers to engage in the distinctive political process of ‘recognition’ and ‘reconciliation’,
‘If the future of Indigenous health research is to be strengthened, researchers must confront rather than deny the past………researchers need to engage in the political process of reconciliation….As a matter of human decency, researchers (even though they may have had nothing to do with the past abuses), should express profound sorrow for those ‘dark bits’ 
Further, within some AMP it has been agreed that the partnership not be “equal” in the sense that the Aboriginal worldview and agency will be privileged, with mainstream evidence and energy used in a complementary way, for what it can add . This may be a necessary approach in a period where power and privilege are being redressed and balanced, but is likely to require a substantially different assessment tool to those reviewed. In this context, relational elements with particular reference to the sociocultural and political context of partnership working will be particularly important  in addressing data collection and analysis issues. This in turn supports the argument for the development of an appropriate assessment instrument.
Bespoke tool or adapting a tool
Rather than constructing a tool by compiling elements from various tools, the analysis in this review has demonstrated the potential to use either the VicHealth Partnership Analysis Tool or the New York PSAT as starting points. Where elements of mutually respectful relationship and sharing of responsibilities are weaker in these tools, associated elements can be borrowed from other tools as summarised in Table 8 to ensure comprehensive coverage of elements of working partnerships. The relevant parts of the New York PSAT together with Verona Benchmark tool also offer a good inventory for community self-determination which is emphasised in the Burton, Reconciliation Australia and Taylor and Thompson principles [5,33,83].
Developing and maintaining a successful AMP requires mutual learning processes and the comparative analysis in this review identified opportunities for mainstream partnership assessment to incorporate learnings from Aboriginal-mainstream partnerships. Whilst the New York PSAT and the Verona benchmark contain corresponding elements associated with responsiveness to community needs, elements in the AMP principles suggest there are opportunities to enrich the community centred culture elements in these tools in order to support community-based programs (Table 8).
Given the global reality of the inadequate life-span of funded partnership programs, existing partnership tools have placed emphasis on ‘agreed’ levels of resourcing and realistic outcomes for the partnerships to achieve. This, however, does not reflect the AMP principles. In the Australian context, policy makers and practitioners have recognised the significance of sustainable trusting relationships and the need to set long term goals, aim for long term achievements and long term investments in partnerships initiatives [33,83]. This presents an opportunity to integrate structured assessment to monitor changes in partnership process and relations to support longer term changes in the desired outcomes to be achieved.
Contextualising partnership assessment tools findings
The key challenge faced in applying structured tools to assess partnership working has been on contextualisation of findings. An unresolved question is whether it is possible to capture the complex dynamics using a structured partnership analysis tool even if using a bespoke tool designed for the specific purpose of capturing aspects of AMP. The answer is uncertain; however, elucidating the historical, sociocultural and political background of Australian AMP when interpreting data collected using structured tools is essential to ensure that findings are as close to reality as possible. If the underpinning rationale for the drive to work in partnerships in this special context is to redress and balance power and privileges between the Aboriginal and mainstream partners, then discussions of equity and equality must be included when analysing findings from partnership assessments. This also has implications for the dissemination of findings.
The impact of current policy and sociocultural environments, and the power dynamics which operate in Aboriginal and mainstream settings are of crucial importance when building and maintaining AMP . That is, the political, economic and social disadvantage of Aboriginal and Torres Strait Islander people in Australia is especially important to consider when assessing the health of the partnerships. Therefore, tools to evaluate partnerships need to include culturally appropriate and community relevant outcomes. If the criteria related to partnership success are not culturally appropriate for an AMP or do not reflect the social and political context, then the quality and appropriateness of the data collected must be questioned.
Regardless of the type of tool used, the importance of complementing the data collected with a broader examination of relevant public policy, service delivery, and community outcomes must be acknowledged. All assessment should be placed in a broader social ecological context which recognises that various levels of the political environment can impact on the effectiveness of self-determination and genuine AMP. Instead of focusing only on equality in participation and accountability, similarities as well as differences in basic conditions affecting access to information, knowledge, resources and services must be made transparent in order to achieve equity.
An emerging example from the above analysis (Table 8) may allow us to infer on the elements of nurturing leadership quality in this special context. One of the AMP principles involves investment in and support for local Aboriginal and Torres Strait Islander leadership  when its mainstream counterpart highlights the importance of ‘initiatives to inspire, motivate and empower people to be involved’ as the key to nurturing leadership . This could mean that when mainstream organisations partner with Aboriginal communities, greater emphasis should be placed on understanding the culture of Aboriginal and Torres Strait Islander leadership while inspiring, motivating and empowering mainstream counterparts to work collaboratively in the AMP context.
Data collection considerations
The role of the facilitator in implementing a partnership evaluation tool should not be underestimated. Any partnership evaluation is reliant on the sensitivity and awareness of the facilitator and the data analysis process to surface the true underlying issues. For example, people who are reluctant to speak in a public forum may need to be encouraged to participate, or have a more low key opportunity to contribute created for them.
There are barriers to using a partnership tool and participants require preparation to use them in a way that can contribute to high quality sustainable local partnerships . Flexibility must be allowed in terms of the stages in which the tool is applied and how to ensure maximum benefits to strengthen the partnership.
In the initial phases of introducing a partnership tool to an AMP, effort to promote the idea of testing a structured tool to help guide partnership analysis is needed. This means allowing time to assess whether the right questions have been asked to explore partners’ experiences in the context of a wider partnership networks that include local advocacy groups and state, and even federal level government agencies. In the context of Aboriginal-mainstream partnership, it is imperative that the entire evaluation process, starting from data collection, is not separated from the historical, social and political context in which the partnership operates .
Long term relationships and trust are especially important in the development and evolution of Aboriginal-mainstream partnerships [5,92]. Even if there is an arbitrary starting point assigned, a genuine partnership rarely has a neat starting point. Partnerships often start with a small number of individuals, the boundary spanners, and in some instances, a desirable partnership outcome may be the increased number of partners willing to participate in the assessment process. This questions the feasibility of a ‘before’ and ‘after’ comparison when that has been deemed necessary. It is more than likely that a tool is most useful to help structure informal conversations to achieve a balance between systematic understanding of partnership dynamics and conversational nature of data collection in this context.
In conclusion, partnership tools are instruments to help work out where and why the partnership is working or not working. The willingness of partners to engage in any formal self-assessment process in itself suggests the partnership is robust. Tools offer opportunities for providing evidence based support to partnership development. Evaluation of any partnership is really only a means of strengthening the collaboration, and the assessment or analysis process relies upon honesty and openness from the partners and a preparedness to change ways of working if the needs of partners are not being met adequately. It is likely that where these prerequisites are met, the partnership is already on a secure footing, but further improvements may still be possible.
The unique nature of partnerships means an established tool that has been shown efficiency in other contexts (AMP or non-AMP) may not realise the same or may be difficult to implement in some AMP. The multiplicity of tools in existence and the reported uniqueness of each partnership, means the development of a generic partnership analysis tool for AMP may not be a viable option for future effort.
No documented evidence was found in the use of partnership tools in an AMP setting. However, the use of a structured tool, particularly when adapted or used in combination with other data collection techniques to explore the context of program or community development evaluations may add value to partnership assessment. Future research should focus on documenting experience in the application of partnership tools to support Aboriginal-mainstream partnership operations using an appropriate interpretive framework that is cognisant of the factors involved in the process of recognition and reconciliation, including cultural context, self-determination, mutuality and equity.
Haynes and colleagues  have argued that reflexivity and dialogical theory are essential theoretically informed ways to work in practice that ensure attention is paid to the nature of partnerships in terms of power, strangeness, borders and intercultural relations . Working in the space of AMP building and improvement, partners, evaluators and researchers face the challenges of getting the balance right between contrasting cultures and customs and the efficacy and efficiency of the process of partnership analysis. The inherent challenges of operating in the intercultural space contribute to the hurdles of applying either established protocols or adapted tools in this highly unfamiliar area in evaluation. As the reason for assessing AMP should be to support long term sustainable relationships based on trust, this strive for balance is an on-going process.
Aboriginal Community Controlled Health Service
Partnership Self-Assessment Tool
The Victorian Health Promotion Foundation
Australian Bureau of Statistics: Experimental life tables for Aboriginal and Torres Strait Islander Australians, 2005–2007. Cat. no. 3302.0.55.003. In. Canberra Australian Bureau of Statistics; 2009.
Bailey S, Hunt J. Successful partnerships are the key to improving Aboriginal health. New South Wales Public Health Bulletin. 2012;23(4):48–51.
Taylor KP, Bessarab D, Hunter L, Thompson SC. Aboriginal-mainstream partnerships: exploring the challenges and enhancers of a collaborative service arrangement for Aboriginal clients with substance use issues. BMC Health Serv Res. 2013;13(1):12.
Selsky JW, Parker B. Cross-sector partnerships to address social issues: challenges to theory and practice. J Manag. 2005;31(6):849–73.
Taylor KP, Thompson SC. Closing the (service) gap: exploring partnerships between Aboriginal and mainstream health services. Aust Health Rev. 2011;35:297–308.
Vurro C, Dacin MT, Perrini F. Institutional antecedents of partnering for social change: How institutional logics shape cross-sector social partnerships. J Bus Ethics. 2010;94:39–53.
Anderson IP. Recent developments in national Aboriginal and Torres Strait Islander health strategy. Aust New Zealand Health Policy. 2004;1(1):3.
Marles E, Frame C, Royce M. The Aboriginal medical service redfern–improving access to primary care for over 40 years. Aust Fam Physician. 2012;41(6):433–6.
Dwyer J, O’Donnell K, Lavoie J, Marlina U, Sullivan P. The Overburden Report: Contracting for Indigenous Health Services. Darwin: Cooperative Research Centre for Aboriginal Health; 2009.
Haynes E, Holloway MT, Thompson SC. Survey findings of reporting requirements 2013. Report back to Aboriginal community controlled health services. Perth: Western Australian Centre for Rural Health, University of Western Australia; 2014.
Black AP, Vally H, Morris P, Daniel M, Esterman A, Karschimkus CS, et al. Nutritional impacts of a fruit and vegetable subsidy programme for disadvantaged Australian Aboriginal children. Br J Nutr. 2013;110(12):2309–17.
Brady M, Sibthorpe B, Bailie R, Ball S, Sumnerdodd P. The feasibility and acceptability of introducing brief intervention for alcohol misuse in an urban Aboriginal medical service. Drug Alcohol Rev. 2002;21(4):375–80.
DiGiacomo M, Davidson PM, Davison J, Moore L, Abbott P. Stressful life events, resources, and access: key considerations in quitting smoking at an Aboriginal Medical Service. Aust N Z J Public Health. 2007;31(2):174–6.
Dimer L, Dowling T, Jones J, Cheetham C, Thomas T, Smith J, et al. Build it and they will come: outcomes from a successful cardiac rehabilitation program at an Aboriginal medical service. Aust Health Rev. 2013;37(1):79–82.
DiGiacomo ML, Thompson SC, Smith JS, Taylor KP, Dimer LA, Ali MA, et al. ‘I don’t know why they don’t come’: barriers to participation in cardiac rehabilitation. Aust Health Rev. 2010;34(4):452–7.
Digiacomo M, Abbott P, Davison J, Moore L, Davidson PM. Facilitating uptake of Aboriginal adult health checks through community engagement and health promotion. Qual Prim Care. 2010;18(1):57–64.
Fuller JD, Martinez L, Muyambi K, Verran K, Ryan B, Klee R. Sustaining an Aboriginal mental health service partnership. Med J Aust. 2005;183(10):S69–72.
Allan J, Campbell M. Improving access to hard-to-reach services: a soft entry approach to drug and alcohol services for rural Australian Aboriginal communities. Soc Work Health Care. 2011;50(6):443–65.
Liaw ST, Lau P, Pyett P, Furler J, Burchill M, Rowley K, et al. Successful chronic disease care for Aboriginal Australians requires cultural competence. Aust N Z J Public Health. 2011;35(3):238–48.
Nel P, Pashen D. Shared antenatal care for indigenous patients in a rural and remote community. Aust Fam Physician. 2003;32(3):127–31.
Kildea S, van Wagner V. Birthing on Country’ maternity service delivery models: an Evidence Check rapid review brokered by the Sax Institute on behalf of the Maternity Services Inter-Jurisdictional Committee for the Australian Health Ministers. Advisory Council. Sydney: Sax Institute; 2013.
Baeza JI, Lewis JM. Indigenous health organizations in Australia: connections and capacity. Int J Health Serv. 2010;40(4):719–42.
Packer J, Spence R, Beare E. Building community partnerships: an Australian case study of sustainable community-based rural programmes. Community Dev J. 2002;37(4):316–26.
Paul D. Casting shadows: persisting colonial influences and Aboriginal health. Balayi: Culture, Law and Colonialism. 2000;1:207–26.
Dudgeon P, Fielder J. Third spaces within tertiary places: indigenous Australian studies. J Community Appl Soc Psychol. 2006;16:396–409.
Kowal E, Paradies Y. Ambivalent helpers and unhealthy choices: public health practitioners’ narratives of Indigenous ill-health. Soc Sci Med. 2005;60(6):1347–57.
Voyle JA, Simmons D. Community development through partnership: promoting health in an urban indigenous community in New Zealand. Soc Sci Med. 1999;49(8):1035–50.
Hooper K, Thomas Y, Clarke M. Health professional partnerships and their impact on Aboriginal health: an occupational therapist’s and Aboriginal health worker’s perspective. Aust J Rural Health. 2007;15(1):46–51.
Waples-Crowe P, Pyett P. Learning from a successful partnership between mainstream and Indigenous organisations. Aboriginal and Islander Health Worker J. 2006;30:4–5. 33.
Taylor KP, Thompson SC, Smith JS, Dimer L, Ali M, Wood MM. Exploring the impact of an Aboriginal Health Worker on hospitalised Aboriginal experiences: lessons from cardiology. Aust Health Rev. 2009;33(4):549–57.
Armstrong R, Hall BJ, Doyle J, Waters E. ‘Scoping the scope’ of a Cochrane review. J Public Health. 2011;33(1):147–50.
Health and Community Development Branch ATSIC. Aboriginal and Torres strait islander health: current status and recent initiatives. Aust Health Rev. 1993;16(4):332–9.
Burton J. Opening doors through partnerships: practical approaches to developing genuine partnerships that address Aboriginal and Torres strait islander community needs. North Fitzroy, Victoria: Secretariat of National Aboriginal and Islander Child Care; 2012.
Hoffmann BD, Roeger S, Wise P, Dermer J, Yunupingu B, Lacey D, et al. Achieving highly successful multiple agency collaborations in a cross-cultural environment: experiences and lessons from Dhimurru Aboriginal Corporation and partners. Ecol Manag Restor. 2012;13(1):42–50.
ACT Governmen. ACT Closing the Gap Report 2013, Progress Outcome for Aboriginal and Torres Strait Islander Peoples. Canberra: ACT Government; 2013.
Jamieson LM, Paradies YC, Eades S, Chong A, Maple-Brown L, Morris P, et al. Ten principles relevant to health research among Indigenous Australian populations. Med J Aust. 2012;197(1):16–8.
Larkin S. Evidence-based policy making in Aboriginal and Torres Strait Islander Health. Aust Aborig Stud. 2006;2:17–26.
Partnership Working: A consumer guide to resources [www.hda.nhs.uk]
World Bank. Sourcebook for evaluating global and regional partnership programs: Indicative principles and standards. Washington, DC: World Bank, Independent Evaluation Group; 2007.
Southerland J, Behringer B, Slawson DL. Using the give–Get grid to understand potential expectations of engagement in a community–academic partnership. Health Promot Pract. 2013;14(6):909–17.
Sweet SN, Ginis KA, Estabrooks PA, Latimer-Cheung AE. Operationalizing the RE-AIM framework to evaluate the impact of multi-sector partnerships. Implement Sci. 2014;9:74.
Atkinson M. The development of an evaluation framework for partnership working. Electron J Bus Methodol. 2005;3(1):1–10.
Appleton-Dyer S, Clinton J, Carswell P, McNeill R. Understanding evaluation influence within public sector partnerships: a conceptual model. Am J Eval. 2012;33(4):532–46.
Caplan K, Jones D. Partnership indicators–Measuring effectiveness of multi-sector approaches to service provision. Practitioner note series. London: BPD Water and Sanitation Cluster; 2002.
de Crespigny C, Emden C, Kowanko I, Murray H. A ‘partnership model’ for ethical Indigenous research. Collegian. 2004;11(4):7–13.
Fletcher J, King K, Christo J, Machlin A, Bassilios B, Blashki G, et al. An evaluation of an Australian initiative designed to improve interdisciplinary collaboration in primary mental health care. Eval Program Plann. 2014;45:29–41.
Giunta N, Thomas ML: Integrating Assessment and Evaluation Into Partnership Initiatives: Lessons From the Community Partnerships for Older Adults. J Appl Gerontol 2013, [Epub ahead of print].
Glendinning C. Partnerships between health and social services: developing a framework for evaluation. Pol Polit. 2002;30(1):115–27.
Glasbergen P. Understanding partnerships for sustainable development analytically: the ladder of partnership activity as a methodological tool. Environ Policy Governance. 2011;21(1):1–13.
Javed AA. A model of output specifications for public-private partnership projects [PhD Thesis]. Hong Kong: Hong Kong Polytechnic University; 2013.
Rieker PP, Jernigan J. Partnership Evaluation. Atlanta, Georgia: Centers for Disease Control and Prevention; 2010.
de Ramírez LM, Lamancusa JS, Zayas-Castro JL, Jorgensen JE. Making a partnership work: outcomes assessment of the manufacturing engineering education partnership. J Eng Educ. 1998;87(S5):519–27.
Pope J, Lewis JM. Improving partnership governance: using a network approach to evaluate partnership in Victoria. Aust J Publ Admin. 2008;67(4):443–56.
Lasker RD, Weiss ES, Miller R. Partnership synergy: a practical framework for studying and strengthening the collaborative advantage. Milbank Q. 2001;79(2):179–205.
Lamie J, Ball R. Evaluation of partnership working within a community planning context. Local Govern Stud. 2010;36(1):109–27.
Vance A, Soloe C, Stone-Wiggins B, Holden DJ. Partnership Literature Review and Framework. Research Triangle Park, NC: Centers for Disease Control and Prevention; 2004.
Ball R, Forbes T, Parris M, Forsyth L. The evaluation of partnership working in the delivery of health and social care. Publ Pol Admin. 2010;25(4):387–407.
Fuller JD, Hermeston W, Passey M, Fallon T, Muyambi K. Acceptability of participatory social network analysis for problem-solving in Australian Aboriginal health service partnerships. BMC Health Serv Res. 2012;12(1):152.
Granner ML, Sharpe PA. Evaluating community coalition characteristics and functioning: a summary of measurement tools. Health Educ Res. 2004;19(5):514–32.
Halliday J, Asthana SNM, Richardson S. Evaluating partnership: the role of formal assessment tools. Evaluation. 2004;10(3):285–303.
Horton D, Prain G, Thiele G. Perspectives on partnership: A literature review. Lima, Peru: International Potato centre (CIP); 2009.
Joss N, Keleher H. Partnership tools for health promotion: are they worth the effort? Glob Health Promot. 2011;18(3):8–14.
Pietsch J. A study tour of the United Kingdom, Canada and New Zealand on partnership management in primary health care focusing on Governance, Leadership, Partnership Evaluation and Clinical Governance across network partners. Acton, ACT: Winston Churchill Memorial Trust of Australia; 2006.
Serafin R, Stibbe D, Bustamante C, Schramm C. Current practice in the evaluation of cross-sector partnerships for sustainable development. London UK: International Business Leaders Forum; 2008.
Wildridge V, Childs S, Cawthra L, Madge B. How to create successful partnerships—a review of the literature. Health Inf Libr J. 2004;21:3–19.
Hausman AJ, Becker J, Brawer R. Identifying value indicators and social capital in community health partnerships. J Community Psychol. 2005;33(6):691–703.
Andrews JO, Cox MJ, Newman SD, Meadows O. Development and evaluation of a toolkit to assess partnership readiness for community-based participatory research. Prog Community Health Partnersh. 2011;5(2):183–8.
Spink L, Merrill-Sands D. Successful Collaborative Partnerships: Key Elements and a Self–Assessment Inventory. Organizational Change Briefing Paper no. 5. Boston: Simmons College; 1999.
Caplan K, Gomme J, Mugabi J, Stott L. Assessing partnership performance: Understanding the drivers for success. London: Building Partnerships for Development in Water and Sanitation (BPD); 2007.
Hartwich F, A. E, J. EJ, Ghezan G, Gonzalez C, Silva JA, J. V: Guidelines for building public–private partnerships for agricultural and agroindustrial innovation. Food Policy in Practice. In. Washington DC: International Food Policy Research Institute; 2007
Whitty B. Accountability principles for research organizations: Toolkit. London: One World Trust; 2008.
Frearson A. Partnership Self-Assessment Toolkit: A Practical Guide to Creating and Maintaining Successful Partnerships. Leeds, UK: East Leeds Primary Care Trust; 2002.
Audit Commission. A fruitful partnership – Effective partnership working. London: Audit Commission for Local Authorities and the National Health Service in England and Wales; 1998.
Center for the Advancement of Collaborative Strategies in Health: Partnership Self-Assessment Tool. In. New York: CACSH 2002.
European Commission. Stott L: EQUAL Guide for Development Partnerships: Learning from the experience of EQUAL partnerships. In: Employment & Social Affiars. Luxembourg: Office for Official Publications of the European Communities; 2005.
Gormley W, Guyer-Miller L. Training Resource Group Inc: Partnership Building: Practical tools to Help You Create, Strengthen, Assess and Manage Your Partnership or Alliance More Productively. Chapel Hill, NC: The Capacity Project, IntraHealth International Inc; 2007.
Hardy B, Hudson B, Waddington E. Assessing Strategic Partnership: The Partnership Assessment Tool. London: Nuffield Institute for Health; 2003.
Markwell S, Watson J, Speller V, Platt S, Younger T. The Working Partnership. London, UK: Health Development Agency; 2003.
Mattessich PW, Murray-Close M, Monsey BR. Wilder Research Centre: Collaboration: What Makes It Work. 2nd ed. Saint Paul, Minnesota: Fieldstone Alliance; 2001.
McCabe A, Lowndes V, Skelcher C. Partnerships and networks: an evaluation and development manual. York, UK: Joseph Rowntree Foundation; 1997.
Tennyson R. The Partnering Toolbook. 4th ed. London, UK: International Business Leaders Forum (IBLF); 2011.
VicHealth. The partnerships analysis tool. Melbourne: Victorian Health Promotion Foundation; 2011.
Respectful Relationships: Ten Top Tips [http://shareourpride.reconciliation.org.au/sections/respectful-relationships/]
Sunderland N, Domalewski D, Kendall E, Armstrong K. Which comes first: the partnership or the tool? Reflections on the effective use of partnership tools in local health partnerships. Aust J Prim Health. 2009;15(4):303–11.
Huxham C, Vangen S. Managing to collaborate: The theory and practice of collaborative advanage. London: Routlege; 2005.
Williams P. The competent boundary spanner. Public Adm. 2002;80(1):103–24.
Cross R, Prusak L. The people who make organizations go-or stop. Harv Bus Rev. 2002;80(6):104–12.
Kowal E. The politics of the Gap: indigenous Australians, liberal multiculturalism, and the End of the self-determination Era. Am Anthropol. 2008;110(3):338–48.
Johnstone MJ. Research ethics, reconciliation, and strengthening the research relationship in Indigenous health domains: An Australian perspective. Int J Intercul Rel. 2007;31(3):391–406.
Altman JC. Beyond closing the gap: Valuing diversity in Indigenous Australia. Canberra: Centre for Aboriginal Economic Policy Research, ANU; 2009.
Singleton GE, Linton C. Courageous conversations about race. Thousand Oaks, California: Corwin Press; 2006.
Durey A, Thompson S. Reducing the health disparities of Indigenous Australians: time to change focus. BMC Health Serv Res. 2012;12(1):151.
Haynes E, Taylor KP, Durey A, Bessarab D, Thompson SC: Examining the potential contribution of social theory to developing and supporting Australian Indigenous-mainstream health service partnerships. International Journal for Equity in Health 2014, 13(75).
The research for this review was supported by the Western Australian Health Promotion Foundation (Healthway) through the research project More Than Talk (grant #20340). WACRH acknowledges funding support from the Commonwealth Department of Health. We thank the Midwest Aboriginal Organisations Alliance (MAOA) for contributing the Australian Aboriginal perspectives. We also thank Dr. John Woods for his support in revising the literature sourced and referencing of the manuscript during the review process.
The authors declare that they have no competing interests.
CT contributed to the conception and design of the review paper including literature search, collection, analysis and interpretation. EH critically reviewed, provided suggestions and contributed to the structuring of the initial and final drafts. WDW critically reviewed and contributed an Australian Aboriginal perspective. GG contributed the Australian Aboriginal perspectives with extensive lived experience in Western Australia, mentoring the research team in the community engagement process which has influenced the interpretation and synthesis of literature reviewed. SCT assisted with the conception and design of the review paper and critically revised methodology, intellectual content and structure of the final draft. All authors read and approved the final manuscript.
CT, Western Australian Centre for Rural Health (WACRH), University of Western Australia (research officer); Central and North West Queensland Medicare Local (Population Health Planner); Inner East Primary Care Partnership (IEPCP) (Health Promotion Project Coordinator). EH, Western Australian Centre for Rural Health (WACRH), University of Western Australia (Assistant Professor, Primary, Aboriginal and Rural Health Care). WDW, Western Australian Centre for Rural Health (WACRH), University of Western Australia, (Assistant Professor, Aboriginal Health). GG, Midwest Aboriginal Organisations Alliance (MAOA) (Chairperson - MAOA, Midwest Aboriginal Health Environmental Health Program Coordinator). SCT, Western Australian Centre for Rural Health (WACRH), University of Western Australia (Director of WACRH, Professor, Rural and Remote Health).
About this article
Cite this article
Tsou, C., Haynes, E., Warner, W.D. et al. An exploration of inter-organisational partnership assessment tools in the context of Australian Aboriginal-mainstream partnerships: a scoping review of the literature. BMC Public Health 15, 416 (2015). https://doi.org/10.1186/s12889-015-1537-4
- Aboriginal-mainstream partnership