5 ways to combat the data engineering skills shortage in 2023

Much like potholes and graffiti, the data skills shortage is very hard to miss. It’s a challenge most organisations will have to overcome.

Only 20 percent of companies employ a data engineer, according to a recent government report. This is likely less for smaller organisations, many of which don’t employ any data experts.

Of course, without these skills, you’ll struggle to:

• Locate your data and identified siloed information
• Understand what your data means
• React to issues and opportunities in real-time
• Meet your business intelligence needs

All of this can lead to stunted growth, inefficiencies, and a lack of competitiveness. But let’s not focus on bleak scenarios.

It’s time to roll up your sleeves and learn how to combat this issue. Here are five ways you can bridge the skills gap.

1. Focus on staff retention

Don’t take your talented engineers for granted. Work hard to retain them. (And, no, this doesn’t just mean increasing pay.)

To keep your engineers happy and engaged:

• Challenge them. Data engineers and analysts love to sink their teeth into complex problems. You could follow Google’s lead and adopt the ‘20 percent rule’. This will encourage creativity, innovation and autonomy.
• Work on your culture. Over half of adults believe company culture is more important than their salary in regards to job satisfaction. So, work on improving your workplace relationships, providing compelling benefits, and finding a collective purpose.
• Use a hybrid model. We’ll let the data talk for us here. Eighty-five percent of working adults would like to adopt a hybrid work approach.
• Stay up to date. Data engineers don’t want to work on a clunky Oracle database from 1995. The more modern your systems, the better.
• Value your employees. At Tech City Teams, we’re big validators and gift-givers. If your data experts are doing a great job, show them! Employees who feel valued at work are more likely to be happier and more productive.

2. Upskill existing tech workers and newcomers

You may not have a team of data-specific experts at your disposal. But that doesn’t mean your current tech workers can’t learn new skills.

In fact, upskilling your current employees may be a welcome career development opportunity. Which ties nicely into our first point.

So, identify employees that fit the bill, then provide them with the right training. You could start with certifications that align with your current IT estate. For instance, AWS offer their own digital training courses.

From there, create an ambitious career development plan for these employees.

3. Venture outside of your IT department

Do you have any Excel whizzes in your wider team? Perhaps someone in sales or marketing is a pro at making sense of complex spreadsheets.

There’s no reason why you can’t nurture these skills. You may choose to fund beginner-friendly training courses. Or, you could let these non-techies shadow the data engineers and analysts you already employ.

4. Assess your hiring pipeline

If you’re reading this blog, we’re almost certain you’ve tried hiring... with little success.

That’s okay. The good news is, you may just need to finetune a few things.

We’d recommend revisiting your job postings. Have you fully sold your company’s benefits and culture? Have you made the data engineer job role sound enticing? With clear progression?

Beyond this, we’d suggest adjusting the way you review CVs. Instead of focusing on certifications and degrees, look for core competencies and indicators of success. When reading through a resume, pinpoint:

• Achievements. What positive outcomes have they delivered for their previous workplaces?
• Projects they’ve worked on. Have they got proven experience building data pipelines and analysing data sets?
• Contextual evidence. Does your candidate really understand data? Can they see the context behind the numbers?

5. Know when to outsource data skills

If you’ve exhausted your options and the skills gap is still frustratingly evident... It may be time to find reputable outsourced experts.

From hard-learned experience, we’d recommend avoiding recruitment agencies. Not every recruiter is a data expert, which means the CVs they send you won’t always hit the mark.

Instead, find a provider that not only has data in their DNA, but your business’s best interests at heart.

Every business is a data business

Data skills aren’t a ‘nice to have’. They’re crucial for all businesses.

This is what makes the current data engineering skills shortage daunting. Without these skills, you’ll fall behind those who can harness their data.

To combat the shortage, you can improve your hiring practices and employee retention. This will help you find and keep great talent. But if you’re still struggling to find the right fit, it’s okay to look elsewhere.

At Tech City Teams, we build bridges over your data knowledge gaps. Working back from your business problems, our data engineers and analysts will always suggest end-to-end solutions that make the most sense for you.

If you’re ready to lean on our trusted expertise, get in touch.