Extract. Transform. Read.

A newsletter from Pipeline​

Hi past, present or future data professional!

When I worked as a resume consultant, the toughest mental block for clients was identifying and expressing material contributions at work; avoiding this communication is why so many job hunters revert to regurgitating their job duties rather than clarifying the outcomes of their work.

In addition to overcoming the hurdle of distilling a complex technical role for non-technical recruiters to understand, data engineers face an additional challenge, which led me to ask: How do I measure my impact? I initially struggled with self-reviews and self-advocacy because the work data engineers do isn’t as quantifiable as say, that of a data scientist. A data scientist’s output is literally judged by performance, i.e. what is the accuracy/precision of a given model? Data analysts, if they’re supporting a particular division, can typically tie their contributions to a portion of a group’s revenue.

The most obvious metrics data engineering teams can use to measure utility and impact include:

• Gigabytes ingested/stored
• Load times
• Slack/teams’ threads or JIRA tickets with stakeholder requests embedded

If there had to be one universal metric that defined a data engineering team’s success it would be uptime/downtime percentage, i.e. when a data customer tries to access a dashboard or data product how often is there an interruption/”downtime” instance?

Being a driving force on a team that tracks downtime has value in the job market, so a resume bullet like the following would catch a hiring manager’s attention:

• Maintained [x] number of daily data ingestion pipelines, hitting [uptime percentage] goal quarter-over-quarter

Out of those three areas, you’d be surprised to know how telling customer interest and feedback is when it comes to measuring a data engineering team’s efficacy. At its core, data engineering is a customer-oriented profession. If no one is requesting data, there isn’t a need for DE.

For us, the challenge is to reframe our value. It’s not about the final insight or the model’s accuracy. It's in the reliability of the data, the speed of the pipelines, the robustness of the infrastructure. Our work reduces risk, improves efficiency, and empowers every other data role in the company.

So, how do you start to articulate your impact in interviews and performance reviews? Think about these potential wins:

• Time Savings: Did your work automate a task that used to take an analyst a full day each week? Highlight the time and effort saved for others
• Risk Reduction: Did your new process reduce data quality issues or "dirty data" incidents? This builds business trust and reduces risk, especially if you're working on projects that touch aspects of data privacy
• Scalability: Consider how your work, such as a robust, well-documented pipeline, can be leveraged by other teams in the future with minimal effort. This demonstrates a long-term investment in the company's data capabilities

Intuiting and confidently communicating the scale of your individual and team impact is a soft skill that makes you marketable, promotable and (nearly) un-fireable.

To dive into more impact metrics, read the piece on Medium that inspired this edition.

Thanks for ingesting,

-Zach Quinn