7 Tips To Achieve A 99% Cloud Deployment Success Rate


Extract. Transform. Read.

A newsletter from Pipeline: Your Data Engineering Resource

Hi past, present or future data professional!

Few aspects of data engineering are as shame-inducing as saying, after a failed deployment, “But it ran in my environment!”

In my first year as a data engineer I was that guy who made excuses like this and grew frustrated that I would complete a build and then struggle to push it over the finish line.

Here’s what helped me:

  • Learning the subtle but important difference between a dependency-related error and a code-oriented issue
  • Taking time to actually read documentation rather than skimming it
  • Understanding my chosen cloud platform (Google Cloud Platform)
  • Distinguishing the important bits of an error string to properly Google a mistake (both in local and cloud dev contexts)
  • Not running to my seniors for answers; StackOverflow, Medium, Reddit and platform-specific communities (like Google Community) are hive minds for solving specific errors
  • Logging status codes and outputs; you can’t fix what you can’t see
  • Creating “clean” dev environments that contain only the dependencies I need

I don’t track my deployment success rate (probably for the best given my initial failures), but I estimate that following the above advice has reduced my failure rate from 20% to between 1-5%.

None of these bullets, however, is a substitute for hands-on experience.

To step through your own deployment, enroll in my free 5-day Deploy Your First Cloud Function course.

Enroll here: https://pipe_line.ck.page/33a3ad0f36

As always, please send me any questions: zach@pipelinetode.com.

Thanks for ingesting,

-Zach

Extract. Transform. Read.

Reaching 20k+ readers on Medium and over 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.

Read more from Extract. Transform. Read.

Hi fellow data professional! It finally happened. I fell for a job scam. Luckily I realized my naivety after responding to the initial email. But let’s back up. We’ll examine Why this particular attempt was so “real” What made me skeptical How to prevent this from happening to you Established professionals in any field have the privileged problem of receiving unsolicited recruiter inquiries. If it’s from a random firm I typically move it to junk; if it’s a big name company, I give a look...

Hi fellow data professional! The best data skills to develop right now might just be cutting and measuring. While that statement might be a bit facetious, the hot media narrative is to push the idea of blue collar work as a viable fallback if you’re having trouble breaking into a conventional tech role. Outlets like CNN have touted the fact that data center engineer is the hottest role in tech. Executives, specifically Nvidia’s Jensen Huang, speculate that data center construction (despite...

Hi fellow data professional! This is the 100th week I’m reaching out into the void of the Internet to connect with you in order to democratize data engineering career knowledge. In the golden age of cable TV, shows would celebrate the 100th episode milestone by airing extended content like a 1-hour special for a sitcom that would typically consume 30 minutes. But I know your time is valuable so I’m going to do the opposite and make this a shorter newsletter than normal. Since I live what I...