Extract. Transform. Read.A newsletter from Pipeline: Your Data Engineering ResourceHi past, present or future data professional! Few aspects of data engineering are as shame-inducing as saying, after a failed deployment, “But it ran in my environment!” In my first year as a data engineer I was that guy who made excuses like this and grew frustrated that I would complete a build and then struggle to push it over the finish line. Here’s what helped me:
I don’t track my deployment success rate (probably for the best given my initial failures), but I estimate that following the above advice has reduced my failure rate from 20% to between 1-5%. None of these bullets, however, is a substitute for hands-on experience. To step through your own deployment, enroll in my free 5-day Deploy Your First Cloud Function course. As always, please send me any questions: zach@pipelinetode.com. Thanks for ingesting, -Zach |
Reaching 20k+ readers on Medium and over 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.
Hi fellow data professional! Remember when the world ended? This month, 6 years ago, the world shut down and entered “unprecedented times.” Shortly after COVID-19 was designated a pandemic, I was unceremoniously furloughed from my day job at Disney World for 3-ish months. During COVID while others quarantined, I was on the move. After quickly feeling isolated in our third floor Central Florida apartment, my now-wife and I joined millions of other American 20-somethings who took a pandemic as...
Hi fellow data professional! I’ve broken my own data project rule. I’ve used the same data over and over again. For 3 years. It sounds boring but that depth exposure may actually be one of the few moats that slows encroaching AI. A little context: I support subscriptions, newsletters and growth for my employer. Spoiler alert: These areas are all basically the same thing. And they use basically the same three data sets. While I have opportunities to jump to other projects, this has been my...
Hi fellow data professional! I had a very adult weekend after baby bedtime. I uncorked a bottle of wine, cracked open my laptop and… stayed up late making a dashboard tracking my assets and debts. While I learned some important high-level insights that will help me make financial decisions as I renovate and prepare to move into my home, I realized a basic development truth that could help anyone knee-deep in their own project. If your goal is commit or ship then the simplest implementation...