[Pipeline] Provision A Python VM In 5 Minutes


The Latest From Pipeline: Your Data Engineering Resource

Hi past, present or future data professional!

One of the worst habits you can adapt in data engineering is to accept a tool or technology as a black box you’ll simply never understand. For me, for the longest time, this was virtual machines. I would commit and troubleshoot code that would run within a VM but never quite knew how to interact with it.

Recently, I provisioned a Google Cloud Compute Engine instance for a junior teammate and learned how to simply and logically create a virtual environment that can boost compute power and accurately mimic production.

The best part is, I’ve distilled this into a process that shouldn’t take you more than 5 minutes.

Read the latest here.

And if you want to learn why automation isn’t always the end goal of software engineering, read last week’s article.

Thanks for ingesting,

-Zach

Extract. Transform. Read.

Reaching 20k+ readers on Medium and over 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.

Read more from Extract. Transform. Read.

Hi fellow data professional! Remember when the world ended? This month, 6 years ago, the world shut down and entered “unprecedented times.” Shortly after COVID-19 was designated a pandemic, I was unceremoniously furloughed from my day job at Disney World for 3-ish months. During COVID while others quarantined, I was on the move. After quickly feeling isolated in our third floor Central Florida apartment, my now-wife and I joined millions of other American 20-somethings who took a pandemic as...

Hi fellow data professional! I’ve broken my own data project rule. I’ve used the same data over and over again. For 3 years. It sounds boring but that depth exposure may actually be one of the few moats that slows encroaching AI. A little context: I support subscriptions, newsletters and growth for my employer. Spoiler alert: These areas are all basically the same thing. And they use basically the same three data sets. While I have opportunities to jump to other projects, this has been my...

Hi fellow data professional! I had a very adult weekend after baby bedtime. I uncorked a bottle of wine, cracked open my laptop and… stayed up late making a dashboard tracking my assets and debts. While I learned some important high-level insights that will help me make financial decisions as I renovate and prepare to move into my home, I realized a basic development truth that could help anyone knee-deep in their own project. If your goal is commit or ship then the simplest implementation...