[ETR #35] AI Can Train You In DE


Extract. Transform. Read.

A newsletter from Pipeline

Hi past, present or future data professional!

Software engineers can package anything— including buzzwords. “Learn new, industry-relevant skills” was compressed to “upskilling.”

And while I’m a proponent of continuous learning, especially when it helps you avoid stagnation, at the end of the day, upskilling is a lot of work. Without proper structure and no mandate from a school or employer, it’s difficult to remain engaged, no matter how interesting the content.

My Udemy cart with 5 unfinished courses can attest to that.

So when I wanted to brush up on the ever-relevant PySpark, I booked some time with Professor LLM; LLMs like Google’s Gemini (my choice to be compatible with my GCP tech stack) make incredible teachers because, like a real tutor, they can engage in a dialogue and adjust to your learning style on the fly.

As a former tutor, I appreciate that it explains concepts as it provides results. This is the educational equivalent of “showing your work.”

To get the most out of your chat bot study sesh, I recommend:

  • Providing experience level context and desired trajectory: “I have 2 years’ experience with SQL but I’d like to learn more about CTEs as they relate to query optimization
  • Prompting the LLM to explain concepts as they relate to a specific role: “Show me examples of how a data engineer might use this skill/tool to build a data pipeline”
  • Repeating subject matter for confirmation: “Let me make sure I have this right, certain Python versions are no longer compatible with Pandas?”
  • Code correction/optimization: “How might I make this code more concise?”
  • Demonstrating errors: “What are the common errors associated with this method? How might I troubleshoot? How might I handle the errors?“

Lost in the AI hype is the power of being able to streamline simple tasks. I did this recently by using function calls to automate the conversion of Google Docs to markdown files I can render as blog posts.

Answering a string of questions as a “teacher” isn’t as revolutionary as an AI creating “original” movies or podcasts.

But by optimizing your learning, you’re creating an abundance of something far more valuable: Time.

Thanks for ingesting,

-Zach Quinn

Extract. Transform. Read.

Reaching 20k+ readers on Medium and nearly 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.

Read more from Extract. Transform. Read.

Extract. Transform. Read. A newsletter from PipelineToDE Hi past, present or future data professional! After choosing a dataset, one of the most significant decisions you must make when creating displayable work is: How am I going to build this thing? For some, you may try to “vibe code” along with an LLM doing the grunt technical work. If you choose this approach, be warned: Nearly half of all “vibe code” generated contains security vulnerabilities and that’s before you even consider its...

Extract. Transform. Read. A newsletter from PipelineToDE Amid layoff announcements from Meta, Amazon and even UPS, it's job aggregator Indeed that signals a different concern for entry-level data job seekers. This week a post on Blind revealed Indeed’s plan to quietly reduce junior roles. They’re not necessarily going to stop hiring or layoff juniors (though they are losing 1300 employees by end of year)—they’re just going to stop paying attention to them. Specifically, Indeed will no longer...

Extract. Transform. Read. A newsletter from PipelineToDE Hi past, present or future data professional! I want to share the single most important realization I had back in the summer of 2021. I was burned out, juggling two part-time jobs, trying to plan a wedding, and drowning in full-time job applications. I felt overwhelmed and underprepared as I plunged into a sea of candidates I perceived to be more intelligent and better "fits" than me. My portfolio was full of the usual Titanic, Iris,...