Top data engineering writer on Medium & Senior Data Engineer in media; I use my skills as a former journalist to demystify data science/programming concepts so beginners to professionals can target, land and excel in data-driven roles.
[ETR #17] Warning: Your Google Cloud Function Might Fail Next Week
Extract. Transform. Read.A newsletter from PipelineTechnically, this title is misleading. Not because your Google Cloud Function won’t fail. It may. And we’ll get to that. I promise. But because Google Cloud Functions are now called Google Cloud Run functions, selecting a name that reflects a fusion between Cloud Run and Cloud Functions, which were previously two distinct Google Cloud Platform products. While both products leverage serverless architecture to run code, Cloud Run was geared more toward those developing apps while Cloud Functions was more of a “quick and dirty” way to get simpler scripts, like ETL pipelines, into production. No matter what GCP calls this product, you’ll still be able to run scripts using a serverless configuration. As a bonus, you’ll now be able to use NVIDIA-based CPUs to boost runtime compute power. To leverage this though, you’ll need to upgrade to a gen 2 cloud function. With many new releases comes obsolescence. This case is no different. Effective October 14th, Google Cloud Functions (excuse me, Google Cloud Run functions), will no longer support Python 3.8, as Python itself is ending support for 3.8 in the same time frame.
If you need to go into more depth with updating runtimes or other aspects of deployment, you can learn to deploy a cloud function in 5 days. It’s 100% free and comes with access to a dedicated GitHub repository. Enroll here. I want to make sure I keep you sufficiently updated, so here are this week’s links:
Until next time—thanks for ingesting, -Zach Quinn |
Pipeline To DE
Top data engineering writer on Medium & Senior Data Engineer in media; I use my skills as a former journalist to demystify data science/programming concepts so beginners to professionals can target, land and excel in data-driven roles.
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! It’s the most wonderful time of the year… performance reviews. Depending on your outlook on your job/org, that “wonderful” could be sarcastic. However, if you’ve landed on your manager’s nice list, this can be the time to recap your achievements to maintain credibility and work toward the next rung on the corporate ladder. If you’re on a small team serving demanding stakeholders it’s possible...
Extract. Transform. Read. A newsletter from Pipeline. Hi past, present or future data professional! This holiday season will be a little less bright thanks to my lack of personal GitHub commits. Like you, I began 2024 full of ideas and motivation that, let’s be honest, was depleted by the end of Q1 when I was cranking out enough code at work that would please even notorious code volume stickler Elon Musk. Despite my lacking output, I managed to hunker down to create useful “bespoke” (a.k.a....
Extract. Transform. Read. A newsletter from Pipeline. Hi past, present or future data professional! When I worked at Disney there was one line (aside from “Have a Magical Day”) that was borderline beaten into us: “We are all custodial employees.” The line meant, of course, to keep areas under your purview neat and presentable (“show ready” in Disney-speak). Using the same logic, I’d like to emphasize that while the various data roles (data analyst, data scientist, data engineer, etc.) have...