7 Tips To Achieve A 99% Cloud Deployment Success Rate


Extract. Transform. Read.

A newsletter from Pipeline: Your Data Engineering Resource

Hi past, present or future data professional!

Few aspects of data engineering are as shame-inducing as saying, after a failed deployment, “But it ran in my environment!”

In my first year as a data engineer I was that guy who made excuses like this and grew frustrated that I would complete a build and then struggle to push it over the finish line.

Here’s what helped me:

  • Learning the subtle but important difference between a dependency-related error and a code-oriented issue
  • Taking time to actually read documentation rather than skimming it
  • Understanding my chosen cloud platform (Google Cloud Platform)
  • Distinguishing the important bits of an error string to properly Google a mistake (both in local and cloud dev contexts)
  • Not running to my seniors for answers; StackOverflow, Medium, Reddit and platform-specific communities (like Google Community) are hive minds for solving specific errors
  • Logging status codes and outputs; you can’t fix what you can’t see
  • Creating “clean” dev environments that contain only the dependencies I need

I don’t track my deployment success rate (probably for the best given my initial failures), but I estimate that following the above advice has reduced my failure rate from 20% to between 1-5%.

None of these bullets, however, is a substitute for hands-on experience.

To step through your own deployment, enroll in my free 5-day Deploy Your First Cloud Function course.

Enroll here: https://pipe_line.ck.page/33a3ad0f36

As always, please send me any questions: zach@pipelinetode.com.

Thanks for ingesting,

-Zach

Extract. Transform. Read.

Reaching 20k+ readers on Medium and nearly 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.

Read more from Extract. Transform. Read.

Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! While many tech-oriented companies have (in one way or another) reneged on remote working arrangements, my employer made an extreme gesture to demonstrate its commitment to the ongoing office-less lifestyle: It removed an entire floor of our two-floor New Jersey office space. Other companies, like Spotify, have unveiled slogans like “Our employees aren’t children. Spotify will continue working...

Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! The only thing worse than summer temperatures (if you’re in the western hemisphere, that is) is a summer job search. Conventionally, summer isn’t the best time to apply for work; you could probably tell this if you’re currently working and find yourself accepting an overwhelming amount of OOO cal invites. If you are braving the heat of the job market, I want to share a more targeted and...

Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! Well, it finally happened; AI has replaced a build I created and I’ve been made redundant. Thankfully, the person that created the AI integration was also me. And I did this on personal time so this isn’t an apocalyptic scenario. I’ve previously written about a handful of tools I created to optimize the “busy work” of blogging. One of the ways is by adding links to past relevant articles and...