Blog.

Assembling the dream team to deliver your analytics platform is vital. Really, your data team structure is just as important as the design and technology supporting it.

To begin with, let’s all pretend headcount and budget aren’t a thing (if only…!). No doubt this is a challenge that everyone faces, but I want to lay out an effective data team structure that can deliver.

Data Team Structure Outline

I’ll quickly summarise the points made in this post, but read on for the full guide to building a data team. You need to put together the following roles in your data team structure:

• a Head of Data, to define the vision and sell the team internally
• a Delivery Manager, to keep everyone on task and communicating
• a Solution Lead, to come up with customer-focused solutions
• a Framework Architect, to design solutions that are flexible
• Software engineers, who build out the frameworks
• Data engineers, who to deliver the solutions to customers

There are more skill sets that go into a bigger team, but they’ll come later. You need to adapt the team to your circumstances and focus on the solution lead and framework architect to start with.

It goes without saying that defining leadership in this space is important. What I don’t want to talk about here is the kind of data leadership a Chief Data Officer represents. I’m much more interested in the on-the-ground men and women doing the delivery.

Head of Data

First up, we have the head of data. From my experience, this person needs to focus on three things:

1. Setting the vision for the data team and the platform
2. Selling that vision internally
3. Unblocking their team members

Having a great vision for your ambitions is crucial; it’s your tool to both motivate and inspire your team, and it’s how you get buy-in from the rest of your organisation. The business is going to have plenty of project ideas.

If you can use sophisticated technology and smart design to achieve these, all the better. Take a common example of targeted offers. What if you need to run a batch job allocating targeted offers?

Turn that into a real-time next best offer powered by machine learning. Sometimes, a huge improvement like that isn’t feasible and that’s quite an extreme example. But it works on a smaller scale too.

What if someone has asked for you to load and transform some data? Provide them with automated alerts in the event of an error, and a dashboard to track the data quality with. It’s not about the head of data coming up with these kinds of ideas themselves, it’s about inspiring the rest of the team to do it.

Selling the team internally is often overlooked. You think because you’ve got the money and people in place you’re done. Think again. There needs to be a constant push from the head of data in all directions about what the team is doing and capable of.

Make sure when people think data, they think of your team. Finally, the head of data must be unblocking the developers in the team. So much time gets wasted in arguments with other departments. You need to make sure that the team is working efficiently, so make sure they’re not stuck on their priorities.

Delivery Manager

The delivery manager is a hybrid project manager/scrum master/product owner for the team. It’s their job to make sure everyone is on task and that communications are flowing in and out of the team properly. Their key tasks are as follows:

1. Manage the backlog of work for the team and allocate it out
2. Manage customer relationships and communicate out to them anything that they need to know
3. Lead stand-ups and sprint retrospectives

In contrast to a lot of the other roles on here, the delivery manager has the tough job of making people commit to a plan, and get work out on time. That makes it more of a governance role than a creative one.

Recruiting previous project managers or scrum masters should help you pin down the right candidate. Managing the incoming requests and priorities of items in the backlog is vital, as well as organising the various scrums and sprints – it’s a rhythm that the delivery manager needs to establish early, and not let drop.

If they go on holiday for a week, it should be muscle memory for the rest of the team to scrum in their absence. Beyond the day-to-day management side of things, they need to be a strong leader that can rally people to their cause and bring customers on side.

They are going to be one of the most visible people in the team so need to be comfortable managing customers. A strong delivery lead can make all the difference between being a well-oiled machine that’s hitting deadlines, and a group of well-intentioned talent that gets nothing done.

Solution Lead

The solution leads job is very simple (to define at least!). Own the overarching design of the platform, from data ingestion right through to data provision. Their key focus should be to:

1. Understand what the customers of the platform need.
2. Maintain a coherent approach for everything going on in the platform.
3. Be an emissary to customers, and a cheerleader for the platform.
4. Make quick decisions on how things will be done.

You may think that this person needs to be very technical, but they don’t. While a deep understanding of data is important, being able to model and transform data, they also need to understand their customers. They need to be able to take the questions that they are being asked, and understand what is actually required.

They should be able to come up with simple and elegant solutions to these questions, and communicate the solution across the data team structure. The Solution Lead will also be the go-to person for questions about the platform. They’re one of the primary representatives out to the community.

They need to have good people skills to build a rapport with everyone. Likewise, they are going to need to be able to make decisions fast so that the pace of delivery is not slowed down. The development team will have a lot of demands on this role, and they all need to be turned around quickly.

Framework Architect

The framework architect is your technical lead. They should have a deep level of understanding in the tool, technologies and languages you are using (more on that coming up in the series). Their key responsibilities are:

1. Design the technical frameworks for ingestion, provision, logging, etc.
2. Validate work done by other engineers conforms to the spec.
3. Keep an eye on the future.

This is probably going to be the role you have the most difficulty hiring – finding a good framework architect is hard. Hiring a great one can be like climbing Everest.

They need to be an experienced software engineer, someone who has extensive experience doing the delivery work. Someone who is always looking at what the industry is doing. This includes new and emerging technologies, so it should be someone with a keen interest in trying new things.

An eye for detail is also crucial. Not only in how they implement their designs, but also in reviewing the rest of the work in the data team structure. This is because they are one of the primary touch-points before code goes into production.

The frameworks form the basis for a lot of the delivery you do and getting them right will mean you have a flexible, easily supportable and quickly deployable base from which to work. We’ll be going into a lot more detail on these in some of the later posts.

Platform Architect

The platform architect is someone you’re going to need to get the project started as they will put together the design of the infrastructure and technology you’re going to deliver on. They’ll probably pick up a hands-on role implementing and configuring the platform too. Their key responsibilities are:

1. Design the high-level architecture for your system.
2. Install and configure the platform.
3. Plan for the integration of new technologies.

The platform architect needs to be thinking about how the system will look in year 1, year 2, and so on. Do you need to set-up encryption from the start? How can you scale the platform over time? What do the development and test environments look like?

These are all questions that the platform architect needs to answer. In a larger data team structure, you’re likely to want to introduce an admin who will take care of a lot of the configuration and upgrade tasks. However, this is often a luxury you can’t afford, so it’s import your platform architect can pick up both roles to start with.

Finally, they need to be looking ahead to the future and how the tech your using is developing, so as not to back you into a technical cul-de-sac which requires a complete rebuild at some point.

Software Engineer

These are the developers who are writing code, and you’ll scale out the number of them as the team’s work grows. Finding people experienced in the tools and technologies you are working with can be helpful. Yet, I’ve worked with some great people who’ve learnt on the job. They’ve ended up being some of the most productive members in the data team structure.

If you’ve fostered a culture of inclusivity amongst your team and hired the right sort of people, then this is a totally valid approach for some of your hires. But not all of them. Day-to-day, the software engineers are going to be implementing the frameworks designed by our architects.

They’re doing the initial builds, and then applying it to many different use cases. Hire people who feel responsible for the work they do. The people who have an innate sense of ownership about their work,  dedicated to making it better than passable. If you can find these people (and they do exist), you’ll always be on top of your delivery.

Data Engineer

Much of the above remains true to the data engineer, certainly when it comes to attitude. Instead of working on the frameworks they’re going to be spending their time getting hands-on with the data; modelling it, testing it, transforming it, everything you’d expect.

As such, it’s important that they have a strong background in working with data. Similar to the software engineer, you can train people up in SQL and other techniques if you’ve got the right people and culture in your team, but you might want a couple of experts to begin with.

Much like the solution lead, they need to have a strong focus on what the customer wants, and be able to work closely with them to define solutions.

Gaps in the data team structure

There are quite a few roles I haven’t mentioned; operations, administrators, testers, and business analysts. As your team grows, these are the kind of people that you are going to want to bring in. Testers, in particular, are a role you should try and prioritise as soon as you can.

In many organisations, the administration is done by other infra teams, so it’s not something you need to worry about. That being said, my preference has always been to own as much as possible of your environment, development, and direction within the team. This helps remove blockers, and keeps you agile.

Rely on external support as little as possible to get things done – you don’t want to get bogged down in someone else poor process or planning. Your solution lead and data engineers should be able to double as business analysts for what you need here.

Likewise, you’ll find that projects come to you for part of delivery already with their own BA’s, so there are more important people that you can focus this headcount on, in my opinion.

Adapting the data team structure

Depending on where you are in your journey, how much budget and headcount you have, and how much work there is, you’re going to have to customise and adapt what’s outlined here to suit your ambitions.

While working with M&S, I started with a four-man team. That data team structure had grown to thirty strong by the time I left. If you’re a small team, you need to focus on the solution lead and framework architect roles.

Your data teams are going to double up as developers; one focused on software and one focused on data. The head of the team can fill in for the delivery lead to start.

As you expand, you need to balance software engineering and data engineers. In the long term, you’ll likely want more data engineers overall, though this will be driven somewhat by the types of use cases that you’ve decided to tackle.

When work hits a critical mass, you must bring in your delivery lead. By the time you’ve got 4 or 5 development resources, you should be aiming to have this role in place. From there, you just scale with demand.

Contractor or consultancy resources are a good way to deal with short-term demand or tackle a particular technical challenge, but you want to focus on owning as much of the design and knowledge in-house as you can.

If this is a topic you would like to find out more about, take a look at our whitepaper:

Assembling your Dream Data Analytics Team