face to face session

How to evaluate an executive search shortlist: a practical guide for employers

Receiving a shortlist of candidates is an important milestone in an executive search process. After weeks of research, candidate approaches and interviews, your headhunter has identified a group of people who could potentially be a strong fit for the role.

But how do you decide who to hire?

In some cases, the decision may seem relatively straightforward. One candidate might stand out because of their relevant experience, track record or particular skills. In other cases, you might find yourself choosing between several strong candidates who bring very different qualities to the table.

This is where evaluating an executive search shortlist becomes more complicated than simply comparing CVs or interviewing candidates against a list of requirements.

In this guide, we explore how employers can evaluate executive candidates and make a well-informed hiring decision.

1. Three CTO candidates, three different strengths

Let’s imagine a technology company is recruiting a Chief Technology Officer to lead an engineering team of around 40 people.

Following an executive search process, three candidates have been shortlisted. All three are intelligent, credible candidates with relevant experience, but their backgrounds and personalities are quite different.

Candidate A: the up-and-coming technologist

Candidate A has around 10 years of professional experience and is relatively early in their leadership career.

They have managed engineering teams of up to 15 people and remain very hands-on technically, still writing code on a daily basis. Their technical knowledge is current, and they have relevant experience that makes them a credible candidate for the position.

In interviews, they come across as bright, enthusiastic and highly motivated. They have a genuine passion for technology and seem excited by the opportunity to take on greater responsibility.

However, moving from managing 15 people to leading an organisation of 40 would be a significant step up.

Would they be able to develop the management structures and leadership skills required? Could they transition from being heavily involved in technical decisions to spending more time managing people and setting strategy?

These are reasonable questions, but they don’t necessarily make Candidate A a weaker option. They may have considerable potential to grow into the role.

Candidate B: the experienced CTO

Candidate B has approximately 25 years of experience and has held the CTO position several times before.

They have managed engineering organisations of up to 100 people and have considerable experience dealing with the challenges associated with technology leadership.

Although their recent positions have been more managerial than hands-on, they retain a willingness to roll up their sleeves when necessary.

Their interview style is perhaps less polished than that of the other candidates. They can come across as slightly rough around the edges, but their answers demonstrate considerable substance, depth of knowledge and experience.

There is little doubt that Candidate B understands what it means to lead an engineering organisation.

However, would they enjoy managing a relatively small team again? Would they be sufficiently hands-on for the company’s needs? And how would their communication and management style work within the existing leadership team?

Their experience offers reassurance, but that doesn’t automatically make them the best appointment.

Candidate C: the product-minded technology leader

Candidate C began their career as an engineer but has also spent time working in product leadership.

As a result, they bring a combination of technical understanding, product experience and commercial awareness.

They are intelligent, approachable and particularly effective at communicating with people outside engineering. They appear capable of bridging the gap between technology, product and the wider business.

Of the three candidates, they might be the most polished and well-rounded in interviews.

For a company where engineering and product teams struggle to work together effectively, this combination of skills could be especially valuable.

However, the employer would still need to explore their depth of technical expertise and experience managing engineering organisations.

Would they be sufficiently strong technically to provide the leadership the company needs? Or would their broader product and commercial experience be a particularly valuable asset?

So, who should the company hire?

There isn’t necessarily a correct answer.

Candidate A brings technical depth, enthusiasm and potential. Candidate B offers extensive leadership experience and a proven track record. Candidate C brings a broader combination of technology, product and communication skills.

All three could potentially succeed in the role, depending on the business’s circumstances.

The real question is which combination of strengths, experience and personal qualities is most appropriate for your organisation.

2. Decide what matters most to your business

It’s rare to find a candidate who ticks every box, particularly when hiring for a senior executive role. Even the strongest candidates will have their own strengths and weaknesses, so it’s important to decide which qualities matter most to your business.

Consider which skills and experiences are essential from day one, and which could be developed as the candidate grows into the role. It’s also worth thinking about whether other people in the organisation could complement the candidate’s strengths and compensate for any gaps in their experience.

Ultimately, evaluating a shortlist is about understanding these trade-offs and deciding which compromises you’re comfortable making, rather than searching for a candidate who meets every requirement perfectly.

3. The limitations of numerical candidate assessments

Some companies use numerical scoring sheets or competency matrices to compare candidates.

Interviewers might be asked to rate each candidate out of five for technical skills, leadership, communication, commercial awareness and other requirements. The scores may then be combined to produce an overall ranking.

On paper, this looks like a structured and objective way to make a hiring decision. In practice, however, numerical assessments can be highly subjective.

What exactly distinguishes a candidate who scores four out of five for leadership from somebody who scores five? How do you accurately measure commercial awareness or the ability to work with a CEO using a numerical scale?

Different interviewers may interpret the same criteria differently. One person might give Candidate B a high leadership score because they have managed 100 engineers, while another might favour Candidate C because of their communication skills.

Assigning numbers to these opinions doesn’t necessarily make them more objective.

4. See how candidates actually work

Senior executives are often experienced interviewees. They may be highly skilled at presenting their achievements, answering difficult questions and communicating a compelling vision.

But being impressive in an interview doesn’t necessarily tell you what working with someone will be like.

For very senior appointments, particularly C-level positions, one of the most revealing ways to assess a candidate can be to spend time working through a realistic business challenge together.

Arrange a face-to-face working session

For example, a company recruiting a CTO could arrange a two-hour, face-to-face session between the candidate and the CEO or founder.

Rather than asking the candidate to prepare a formal presentation about their first 100 days, the CEO could introduce a genuine business challenge.

Perhaps the company is struggling with product delivery, considering a significant technology investment or deciding how to organise its engineering teams for the next stage of growth.

The discussion could explore how the candidate would approach the situation.

The objective isn’t necessarily to solve the problem during the session. Instead, it is to understand how the candidate thinks and interacts with the people they would be working alongside.

Do they ask thoughtful questions before suggesting solutions? Are they comfortable challenging the CEO’s assumptions? Can they explain their reasoning clearly? Are they willing to acknowledge uncertainty or consider alternative approaches?

This kind of session can reveal aspects of a candidate’s personality and working style that might not emerge during a conventional interview.

It also gives the candidate an opportunity to understand the CEO and the business, making the assessment valuable for both sides.

Involve future colleagues

Another useful approach is to introduce candidates to people they would work closely with, including employees outside the senior leadership team.

For a CTO appointment, this could involve engineering managers, developers, product leaders or other relevant colleagues.

These discussions don’t necessarily need to be formal interviews.

A conversation about an existing technical challenge, product development or how teams currently collaborate may be more revealing.

An executive who is impressive when speaking to investors or board members may behave differently when interacting with junior employees. Understanding how candidates engage with their future colleagues can therefore provide valuable information about their leadership style.

Chemistry matters, especially at senior level

Regardless of the size of the organisation, working chemistry is an important consideration when hiring an executive.

What matters is whether they can establish mutual trust, respect each other’s judgement and work together productively.

This is why the most approachable or polished candidate isn’t necessarily the best choice.

Candidate B’s communication style, for example, might initially seem less appealing than Candidate C’s. But a working session could reveal that the CEO values Candidate B’s directness, practical judgement and willingness to challenge assumptions.

Alternatively, the session might expose genuine difficulties in communication that would create problems in the role.

Neither outcome can be assumed in advance.

Reserve these sessions for the final candidates

A collaborative working session requires a meaningful commitment of time and effort, both from candidates and from the employer’s team.

It therefore makes sense to reserve this type of assessment for the final two or three candidates.

It is also important to respect candidates’ time. The session should assess their judgement and working style, not become an opportunity for the company to obtain substantial unpaid consulting work.

Where genuine business challenges involve commercially sensitive information, appropriate confidentiality agreements, such as non-disclosure agreements (NDAs), should be put in place before sharing that information.

Even with an NDA, employers should only disclose information that is necessary for the discussion.

5. How practical is it to hire each candidate?

Even if a candidate is considered a strong fit for the role, how realistic it is to hire them depends on several practical factors, particularly the company’s financial resources, the candidate’s compensation expectations and their willingness to leave their current employer.

As the recruitment process approaches its final stages, it becomes increasingly important to understand what each candidate would expect in order to accept an offer. This is where experienced headhunters can be particularly valuable. Having managed similar negotiations many times before, they are often well placed to understand candidate psychology, interpret subtle reactions and assess how much flexibility there may be around compensation and other terms.

It’s also important to understand whether candidates are involved in other recruitment processes and how far those processes have progressed. If a candidate is already approaching the offer stage with another company, you may need to accelerate your own process and demonstrate why your opportunity is more attractive. Otherwise, you risk losing a strong candidate before you’ve had the chance to make an offer.

Another consideration is the possibility of a counteroffer from the candidate’s current employer. Even after accepting an offer and handing in their notice, candidates may be persuaded to stay through a significant salary increase, a promotion or an expanded role.

This is why understanding the candidate’s relationship with their current employer is so important. How closely do they work with the CEO or senior management team? Have there already been discussions about a promotion or greater responsibilities? Equally, are there underlying reasons why they want to leave that a salary increase or new title would be unlikely to resolve?

An experienced headhunter can help explore these factors throughout the process, identify potential risks and advise on how best to secure the preferred candidate.

Conclusion

Evaluating an executive search shortlist is about understanding each candidate’s strengths and weaknesses, deciding what matters most to your business and considering the practicalities of hiring them.

For senior appointments, real-life case studies and face-to-face discussions with the leadership team can be particularly valuable. They offer a better sense of how candidates think, communicate and collaborate than traditional interviews alone.

Above all, working chemistry matters. You’re not simply hiring someone with the right experience, but choosing a person you and your team can trust, work with and build a successful relationship with over the long term.