Hi fellow data professional!

Ken Jee, who you heard from last week, drops some sobering career advice in one of the earliest editions of AI Survival Guide: Making a senior-level tech role is no longer about advancement; it’s about survival. The post talks about the growing moat or "wall" between those breaking into the industry, those in entry-level roles and those in a mid-career phase.

In the spirit of AI Survival Guide’s advice to bridge the gap separating the early and mid-career engineer, I’d like to offer a glimpse “behind the wall” to demonstrate how finally succumbing to internal encouragement to use AI actually made me better at programming; this in turn allowed me more time to become a true subject matter expert (SME) in my domain, digital subscriptions (which, incidentally, includes the backend for my org’s portfolio of newsletters).

To give you an idea of how transformative AI is, I’ll recap a typical day, highlighting AI usage.

At the beginning of the week, I have my weekly check in with my boss. We’re discussing the capabilities of a new vendor’s API; I plug the URL into Gemini and get a high-level recap ready to reference. After the meeting I review notes on my GitHub pull request (PR); my boss hasn’t yet seen the code because Copilot just completed its review.

Copilot reminds me I forgot to define a schema for a metadata table. In my IDE, I access the data frame’s columns and use the field names and types as an input in the LLM to yield a schema; it is 98% correct. After adding the schema, my code passes my boss’ review and is deployed to production where it causes an error in the Kubernetes pod I’ve spun up in Airflow.

The error message is long so I copy and paste it into my chat with the LLM. The suggestion is to increase resources, which I do in my VS Code environment. But with a meeting approaching, I don’t have time to remember the exact config, so I hit tab and CoPilot autofills my DAG task based on previous work.

After the call I need to answer a stakeholder’s question about missing data; I go way down the rabbit hole with a technical explanation so before I hit send in Slack I let the LLM rewrite my message for a non-technical audience.

You’ll notice none of the tasks I’m asking the LLM to conduct is revolutionary.

I’m not seeking complete pipeline builds or model deployments. The value I get is in compounding time saved.

Schemas take 10-15 minutes to write and review, documentation review and synthesis can take 1-2 hours and troubleshooting infrastructure failures can take 2-4 hours for a pesky bug. And this is why it feels difficult for entry-level devs right now.

If a senior engineer can free up hours of work time, how can a junior be expected to work as quickly and efficiently? I sympathize with that frustration.

So as I peer over the wall, my advice to you isn’t to sink time into shiny tools and abstract projects; instead, find and implement small-scale automations that “win back” time and 10x your bandwidth.

Because the only thing more valuable than a busy dev is one with more time to build.

​Read the full story in PipelineToDE.

Thanks for ingesting,

-Zach

​Medium | LinkedIn | Ebooks