Reaching 20k+ readers on Medium and over 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.
[ETR #14] Build Your Staging Tables Faster/Safer
Extract. Transform. Read.A newsletter from Pipeline: Your Data Engineering ResourcePresented by Basejump AI Use natural language prompts to chat with your database in Basejump’s intuitive interface, or embed it directly in your application. Book your demo here.
Hi past, present or future data professional! One thing that makes my work day easier is when I’m Google-ing (as all software developers do) a problem and I come across the holy grail of solutions: A one-line implementation. Like anything, however, a one-liner that is too complex can become a bad thing. Think: Chained Pandas expressions that become unreadable. Or cramming a multi-line query inside of a BigQuery client method. My favorite one line (at least in recent memory) is a clause used with SQL’s ALTER TABLE statement: RENAME TO. You may find renaming a table as compelling as schema creation. But this simple clause can be especially useful in lieu of a more dangerous phrase: CREATE OR REPLACE. The RENAME command allows you to rename a table without having to completely recreate its contents–and risk a SQL statement failing and losing some or all of your data. Specifically, I use RENAME TO when I want to convert a copy table with some change, like an updated schema, to a production table. I do so using these steps:
The best part is that this is a true one-liner. No chains–or headaches–involved. To save you a headache, here are this week’s links:
If you want to read more about this method, I cover the process in more detail here. Questions? You know where to find me: zach@pipelinetode.com. Until next time–thanks for ingesting, -Zach Quinn |
Extract. Transform. Read.
Reaching 20k+ readers on Medium and over 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.
Hi fellow data professional! The best data skills to develop right now might just be cutting and measuring. While that statement might be a bit facetious, the hot media narrative is to push the idea of blue collar work as a viable fallback if you’re having trouble breaking into a conventional tech role. Outlets like CNN have touted the fact that data center engineer is the hottest role in tech. Executives, specifically Nvidia’s Jensen Huang, speculate that data center construction (despite...
Hi fellow data professional! This is the 100th week I’m reaching out into the void of the Internet to connect with you in order to democratize data engineering career knowledge. In the golden age of cable TV, shows would celebrate the 100th episode milestone by airing extended content like a 1-hour special for a sitcom that would typically consume 30 minutes. But I know your time is valuable so I’m going to do the opposite and make this a shorter newsletter than normal. Since I live what I...
Hi fellow data professional! For my health I try not to argue with my wife; but when she told me her networking plan I had to push back. Some context. She’s exploring career paths within the multinational corp she works for and wanted to meet with a friend of a family member. The catch? She felt weird about leveraging a personal connection and wanted to reach out cold. This is the wrong approach. Western culture demonizes nepotism but, truthfully, sometimes a connection is so painfully...