Evaluation SEC remote assignments January 2021 - June 2022

Overall satisfaction – rating «good» or «excellent»

Evaluation SEC remote assignments, Icon Smiley
  • 90% of the SEC Clients
  • 66% of the SEC Experts

Advantages – in comparison to on-site assignments

Evaluation SEC remote assignments, Icon Smiley
  • No costs for SEC clients
  • Facilitation of access to expert advice to SMEs with limited financial means
  • Flexibility in organising meetings
  • Higher attendance rate due to scheduled meetings
  • More time available to gather and prepare necessary information
  • Offer consultancy on topics where it is not worth travelling to the client

Disadvantages – in comparison to on-site assignments

Evaluation SEC remote assignments, Icon Disadvantages Minus
  • Difficulties in efficiently identifying and understanding client’s need and context
  • Limited number of participants
  • Lack of feeling/verification for the client’s capacity to understand the content
  • Limited efficiency due to lack of observation, feeling and witnessing products/services/processes
  • Limited number of topics can be addressed

Positive effects of remote assignments, observed by clients

Evaluation SEC remote assignments, Icon Positive effects
  • Improved HR management
  • More efficient processes
  • Improved product quality
  • Product diversification or diversified distribution channels

EVALUATION BRIEF OVERVIEW

Purpose of the evaluation

The ongoing global crisis caused by the pandemic of the COVID-19 virus, and the corresponding restrictions in the countries in which Senior Expert Contact (SEC) operates as well the international travel restrictions, resulted in the suspension of SEC’s main project activity of on-site consultancy assignments in 2020 and 2021. To continue to support clients, SEC introduced remote assignments. The exchange and collaboration between SEC experts and clients took place online.

The objective of the present evaluation is: to assess the new modality of remote assignments; to draw learnings compared to traditional assignments; to inform the future design and management of remote assignments; and to appraise the overall potential of remote assignments to become a regular part of SEC’s offering even after the COVID-19 pandemic.

In particular, the objective of the evaluation is to analyse in detail how well remote assignments met clients’ needs, how efficiently they have been executed, how effectively the planned objectives have been met and if there has been any impact observed already.

Methodology

The evaluation applied a mixed method approach, using desk research to review the available assignment request forms and final report forms submitted by SEC clients, experts and coordinators, semi-structured interviews with selected SEC experts and clients, as well as an online survey with the SEC coordinators. To ensure comparability with the 2020 evaluation, the same methods were used.

For 2020 clients, every client from the 2020 evaluation (except those from Tunisia and Guatemala) were included (29 clients). The sample for 2021 consisted of 34 clients.

In total, 59 interviews with SEC clients from nine selected SEC countries (Bolivia, Benin, Cambodia, Kosovo, Mongolia, Nepal, Nicaragua, Peru, and Uganda), as well as seven interviews with SEC experts have been conducted. The online survey was filled in by the SEC coordinators of the nine SEC countries.

Key findings of the evaluation

Relevance of remote assignments: meeting the needs and requirements of the SEC Clients

Also in 2021, the remote assignments seem to have largely met clients’ expectations and needs in terms of content and output. However, the relevance of remote assignments depends on the client and expert discussing expectations and goals for the remote assignment either before or at the very start of the assignment. Often, topics submitted by clients need to be adjusted to be suitable for a remote assignment.

Most clients stated an interest and need for a follow-up assignment. While this indicates their satisfaction with the received remote consultancy, half the SEC clients stated a preference for practical on-site support for their follow-up assignment.

Efficiency of remote assignment: use of resources and achievement of planned outputs

Satisfaction with the implementation of the remote assignments:

SEC clients who had remote assignments in 2021 showed a high overall satisfaction, with 90% of the interviewed SEC clients rating the implementation of the remote assignments as “good” or “excellent”.

The main reasons for their satisfaction were professionalism, expertise, availability, and openness of the experts to share their experience and knowledge with them.

SEC experts who carried out remote assignments in 2021 were less satisfied with the implementation, showing an overall satisfaction of 66%. The lower satisfaction is attributable to the difficulty in connecting with the client (personally and technically), and the reluctance of some clients to implement SEC experts’ recommendations.

Communication, contact frequency, assignment duration:

For the remote communication with each other, the Clients and the SEC Expert used different tools and formats for specific purposes:

Purpose Communication tool
Virtaul meetings and Ad-hoc exchange and phone calls Zoom, Skype and Messenger services such as WhatsApp
Sharing formal information and documents Email
Documenting work environment Video

The main technical challenge that SEC clients and experts encountered during their exchanges was unstable internet connectivity. On occasion, the language barrier, or the expert’s lack of the understanding of the client’s context rendered the exchange difficult.

Most SEC clients and experts met virtually once or twice a week. Both SEC clients and experts appreciated the flexibility to find a rhythm of exchange that suited them best.

Even though more than half of the remote assignments had to be extended, 72% of the clients perceived the duration of the remote assignment as just right. There was general agreement among experts that the duration of an assignment should be kept flexible, depending on the topics, as well as on the pace of implementation.

Comparison to on-site assignments:

SEC clients and experts noted the advantages and disadvantages of remote assignments:

Advantages Disadvantages
  • No costs for SEC clients
  • Facilitation of access to expert advice to SMEs with limited financial means
  • Flexibility in organizing meetings
  • Higher attendance rate due to scheduled meetings
  • More time available to gather and prepare necessary information
  • Difficulties in efficiently identifying and understanding client’s need and context
  • Limited number of participants
  • Lack of feeling/ verification for the client’s capacity to understand the content
  • Limited efficiency due to lack of observation, feeling and witnessing products/services/processes

Coordination of remote assignments:

Most SEC coordinators invest more resources in preparing and accompanying a remote assignment than an on-site assignment. Despite some process improvements and the omission of accommodation, the workload increased for the SEC coordinators as the verification of the information given clients and an assessment of their commitment levels, as well as the training in remote working and online tools took more time.

This also held true for the accompanying of the remote assignments, which according to the SEC coordinators needed more or even many more resources compared to on-site assignments due to a higher frequency of meetings, the longer duration of remote assignments and the more intense follow-ups. On the other hand, this also led to closer monitoring and time management. Record keeping also improved.

Effectiveness of remote assignment: contribution to increased capacities and behaviour change

Most SEC clients were either satisfied or even highly satisfied with the achievements of their remote assignments. In some cases the clients’ expectations were even exceeded. Flexibility in adjusting assignment objectives in case of unexpected problems was key to this.

Some clients observed early signs of tangible changes in their business performance such as:

  • improved HR management,
  • more efficient processes,
  • improved product quality,
  • product diversification or diversified distribution channels.

This is an early indication that remote assignments lead to a successful adoption of the SEC expert’s advice by the clients. Still, the re-interviews with the 2020 clients indicate that the effects of remote assignments only emerge over time as most clients are still implementing the plans worked out with the SEC expert.

In contrast to 2020, only 55% of the SEC coordinators consider remote assignments as effective as on-site assignments. The other 45% point out that the experts’ lack understanding of the local context and of the clients’ situation, which is seen as crucial for formulating recommendations. In general, the SEC coordinators argue that the effectiveness of remote assignments depends strongly on good communication and the topic of the assignment.

Impact of remote assignment: contribution to increased resilience and retention of jobs in the COVID-19 crisis

Of the clients who reported that their businesses were still negatively affected by the Covid-19-crisis, 23% of 2021 clients and 60% of 2020 clients explicitly stated that the support of the SEC expert helped to mitigate the effect of the COVID-19 crisis. The findings indicate that the remote assignments have contributed to their resilience.

Two thirds of 2021 clients and almost half of 2020 clients recorded a stabilization or even growth of their profit and/or turnover. Furthermore, most of the clients reported job retention (including rehiring) or creation of jobs. For the other clients, it was either too early to provide information on turnover and jobs or no changes have been observed.

Yet, directly linking this rather positive performance of SEC clients to the SEC remote assignments is not possible as several other external factors influence this development, above all the general economic recovery that varies from country to country. However, following the theory of change of the modality of the remote assignments at least a contribution to the impacts in terms of jobs and income can be assumed, given that SEC clients improved their business performance thanks to the adoption of the experts’ advice.

Conclusion

That SEC continued its expert consultancy support, despite the restrictions of the COVID-19 pandemic, was highly appreciated by the SEC clients.

As we saw in the previous section, the remote assignments contributed to the clients’ economic resilience to mitigate the impact of the pandemic on their businesses. Furthermore, the remote support was also of great importance for the clients, as several 2020 and 2021 clients reported

that the SEC expert gave them moral support and motivated them to continue and not give up.

As remote assignments appear to be effective, SEC should continue offering them also after the COVID-19 pandemic.

On the one hand, they can broaden the scope of SEC assignments; as a cost-efficient alternative to on-site assignments, they facilitate access for clients with little to no budget for an expert.

On the other hand, with travelling becoming possible again in 2022, the development of a “hybrid model” combining remote support with a shorter on-site assignment should be further explored/developed. To ensure the continuous implementation of the results, SEC should explore the integration of low-threshold follow-up exchanges between the client and expert for some period after the remote assignment officially concludes.

Whether remote or on-site, the expert assignments of the Senior Expert Contact are important to their clients and remain relevant also in the light of the recovery from the COVID-19 crisis (“build back better”) but also related to any future economic, social or environmental shock that might affect the clients’ businesses.

Impact Evaluation of SEC in Nepal 2018

Scribble video of a study conducted in August 2018 by a Swisscontact internal team. Interviews with proprietors and managers of 45 of the 138 SEC clients from 2013 – 2017 formed the largest part of the evaluation.