Evaluation SEC remote assignments June - December 2020

Overall satisfaction – rating “good” or “excellent”

Evaluation SEC remote assignments, Icon Smiley
  • 93% of the SEC Clients
  • 83.3% of the SEC Experts

Advantages – in comparison to on-site assignments

Evaluation SEC remote assignments, Icon Smiley
  • No costs for SEC Clients
  • Increased accessibility to Expert advice for SMEs with limited financial means
  • Better processing of information between meetings
  • Flexibility in organizing meetings

Disadvantages – in comparison to on-site assignments

Evaluation SEC remote assignments, Icon Disadvantages Minus
  • Difficulties in efficiently identifying and understanding Client’s needs and context
  • Limited number of participants
  • Lack of feeling for/verification of the Client’s capacity to understand the content
  • Lack of commitment from the Client

Positive effects of remote assignments, observed by clients

Evaluation SEC remote assignments, Icon Positive effects
  • Improved HR management
  • More efficient processes
  • Improved product quality
  • Product diversification or diversified distribution channels

EVALUATION BRIEF OVERVIEW

Purpose of the evaluation

Purpose of the evaluation

The global COVID-19 pandemic in 2020 and the corresponding restrictions in the Senior Expert Contact (SEC) countries in response to the health crisis resulted in the halt of SEC’s main project activity of on-site assignments. Due to inter­national travel restrictions, the SEC Experts were no longer able to travel to the Clients and provide expert consultancy on site. To continue supporting their Clients with expert consultancy from a distance, SEC thus established a new assignment model: the remote assignment, in which the exchange and collaboration between the SEC Experts and Clients was exclusively virtual.

The present evaluation was carried out six months after the start of the first remote assignment. Its purpose was to determine whether the planned objectives of the remote assignments had been achieved, to provide a comparison to traditional assignments, to provide information for the future design and management of remote assignments and to appraise the overall potential of remote assignments to become an institutionalized part of the SEC as a model that could also be offered beyond the COVID-19 pandemic.

In particular, the objective of the evaluation was to analyse in detail how well the remote assignments met Clients’ needs, how efficiently they had been put into practice, how effectively the planned objectives had been met and if there has already been any impact observed.

Methodology

Methodology

The evaluation applied a mixed-method approach, using desk research to review the available assignment request and final report forms submitted by SEC Clients, Experts and Coordinators, semi-structured interviews with selected SEC Experts and Clients as well as an online survey with the SEC Coordinators.

In total, 32 interviews with SEC Clients from ten selected SEC countries (Bolivia, Benin, Cambodia, Guatemala, Mongolia, Nepal, Nicaragua, Peru, Tunisia and Uganda) as well as with six SEC Experts have been conducted. The online survey was filled out by all SEC Coordinators of the ten SEC countries.

Key findings of the evaluation

Key findings of the evaluation

Relevance of remote assignments: meeting the needs and requirements of the SEC Clients

The remote assignments seem to have largely met Clients’ expectations and needs in terms of content and outputs. However, in some cases, the time frame was not sufficient to carry out all the activities requested by the SEC Client. Also, the objectives often only became more specific after the first remote meetings with the SEC Expert had taken place.

In addition, not all activities are suitable for remote assignments: SEC Clients and SEC Experts alike stated that it was not possible to address activities related to practical implementation, demon­strations or dealing with operational problems in the same way as in on-site assignments.

Most of the SEC Clients stated an interest in and a need for a follow-up assignment. While this indicates their satisfaction with the remote consultancy they received, also as a solution in times of travel restrictions, the SEC Clients re-emphasized the need for practical on-site support in their follow-up assignment.

Efficiency of remote assignment: use of resources and achievement of planned outputs

Satisfaction with the implementation of the remote assignments:

SEC Clients as well as SEC Experts showed a high overall satisfaction, with 93% of the interviewed SEC Clients and 83.3% of the interviewed SEC Experts rating the implementation of the remote assignments as “good” or “excellent”.

The main reasons for their satisfaction being:

SEC Clients SEC Experts
  • Professionality, expertise, motivation, and flexibility of the SEC Expert
  • Clients’ openness and transparency regarding needs and problems
  • Clients’ responsiveness to the Experts’ questions and inquiries

The statements of the SEC Clients as well as the SEC Experts also showed that building a trustful relationship between the Client and the Expert is key for a successful implementation of the remote assignment.

Communication, contact frequency, assignment duration:

For the remote communication with each other, the Clients and the SEC Expert used different tools and formats for specific purposes:

Purpose Communication tool
Ad-hoc exchange and phone calls Messenger services such as WhatsApp
Sharing formal information and documents Email
Documenting work environment Video

During their remote assignments, the SEC Clients and Experts also encountered certain challenges such as:

  • Language barriers [1]
  • Time differences
  • Finding the appropriate frequency of meetings or trainings

In some cases, these challenges prevented the participation of other staff members outside the management team.

Even though one third of the remote assignments had to be prolonged, 87% of the Clients perceived the duration of the remote assignment as just right. Both Clients and SEC Experts mentioned the need for continued support during the process of implementing the Experts’ advice.

Comparison to on-site assignments:

In direct comparison to on-site assignments, SEC Clients and Experts alike noted both advantages as well as disadvantages of remote assignments:

[1] Mainly mentioned by SEC Clients in Latin America

Advantages Disadvantages
  • No costs for SEC Clients
  • Increased accessibility to Expert advice for SMEs with limited financial means
  • Better processing of information between meetings
  • Flexibility in organizing meetings
  • Difficulties in efficiently identifying and understanding Client’s needs and context
  • Limited number of participants
  • Lack of feeling for/ verification of the Client’s capacity to understand the content
  • Lack of commitment from the Client

Coordination of remote assignments:

Citing efficiency gains due mainly to a lack of organizational logistics, 40% of the SEC Coordinators reported investing the same amount of resources in preparing a remote assignment and 30% even reported fewer.

At the same time, the workload increased for the SEC Coordinators, as the verification of the Clients’ information and their commitment, as well as the training in remote modalities and tools took more time due to movement restrictions.

This also held true for the support of the remote assignments, which, according to the SEC Coordinators, needed more or even many more resources compared to on-site assignments. This was due to more frequent progress meetings, the longer duration of remote assignments and the more intense follow-ups. However, this did also lead to closer monitoring and time management. Record keeping also improved.

Effectiveness of remote assignment: contribution to increased capacities and behaviour change

While all the SEC Clients reported being either satisfied or even highly satisfied with the achievements of their remote assignments, flexibility in adjusting objectives of the assignment in the event of unexpected problems was mentioned as being a key factor.

Although for most interviewed Clients it was still too early (or they were still under COVID-19 restrictions) to observe tangible changes in their business performance, some Clients had already observed positive effects such as:

  • improved HR management,
  • more efficient processes,
  • improved product quality,
  • product diversification or diversified distribution channels.

This is a promising indication that remote assignments can also lead to the successful implementation of the SEC Expert’s advice by the Clients.

As for the SEC Coordinators, 80% evaluated the remote assignments as being just as effective as on-site assignments. They stated that the effectiveness of remote assignments depends strongly on a mutual understanding between SEC Expert and Client as well as on the topic and the type of the assignments.

Impact of remote assignment: contribution to increased resilience and retention of jobs in the COVID-19 crisis

The majority of the SEC Clients interviewed stated that the support of the SEC Expert helped to mitigate the effect of the COVID-19 crisis, which indicates that the remote assignments have contributed to their resilience.

Thanks to performance improvements, Clients stated that they felt better prepared for a future crisis and almost half of the interviewed SEC Clients recorded increased turnover and profit combined with job retention (incl. rehiring) and even cases of job creation. For the other half of the Clients, it was either too early to provide information on turnover and jobs or no changes had been observed.

It is, however, not possible to directly link this rather positive performance of SEC Clients to the SEC remote assignments, as several other external factors influence this development, above all the general economic recovery that varies from country to country. However, following the Theory of Change of the assignment model to remote assignments, at least a contribution to the impact in terms of jobs and income can be assumed, given that SEC Clients improved their business performance thanks to their implementation of the Experts’ advice.

Conclusion

Conclusion

The fact that SEC continued its expert consultancy support despite the restrictions of the COVID-19 pandemic was highly appreciated both by the SEC Clients as well as the Experts.

As we saw in the previous section, the remote assignment, on the one hand, contributed to the Clients’ economic resilience, which mitigated the impact of the pandemic on their businesses. On the other hand, the remote support was also of great mental importance for the Clients, as quite a few reported that the SEC Expert gave them moral support and motivated them to continue and not give up.

Looking beyond the COVID-19 pandemic, the findings of the present evaluation also indicate a strong potential for SEC remote assignments to be included in the future as an assignment model of SEC expert consultancy.

As a cost-efficient alternative to on-site assignments, remote assignments could be enhanced to include a broader range of Micro, Small and Medium Enterprises (MSMEs) for specific types and topics.

This could also be explored in the form of a “hybrid model”, combining remote support with a shorter on-site assignment. For instance, this form could be interesting for assignments where the preparation or the first theoretical input could take place remotely, followed by on-site support during the implementation. The third part – the follow-up of the implementation of the Expert’s advice – could again be done remotely in the form of backstopping support.

Whether remote or on-site, the expert assignments of the Senior Expert Contact are important to their Clients and remain relevant also in light of the recovery from the COVID-19 crisis (“built back better”) but also regarding any future economic, social or environmental shock that might affect the Clients’ businesses.

Impact Evaluation of SEC in Nepal 2018

Scribble video of a study conducted in August 2018 by a Swisscontact internal team. Interviews with proprietors and managers of 45 of the 138 SEC clients from 2013 – 2017 formed the largest part of the evaluation.