Reaching 20k+ readers on Medium and nearly 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer and time as a former journalist to demystify data science/programming concepts while sharing career strategies so you can target, land and excel in data-driven roles.
Why Put Emojis In A Data Pipeline? 😎
Extract. Transform. Read.A newsletter from Pipeline: Your Data Engineering ResourceHi past, present or future data professional! I bet when you think of data engineering tools, you don’t think of smiley faces. But emojis, when used properly (and sparingly), can be a powerful way to emphasize logging messages and highlight infrastructure failures. Failing in data engineering can make you :sad_face:. Professionally, I’ve made every code mistake imaginable, with the highlight being omitting a column and needing to process a 45 TB table—twice! Even as I gain experience, I still make (but recover from) plenty of mistakes, which I document on a yearly basis. Since we’re a little more than halfway through the year, I’d encourage you to use this framework to honestly assess both your technical and interpersonal shortcomings as an aspiring data engineer. Proactively identifying areas of improvement is growth—and areas for improvement will pay more dividends than cramming in upskilling sessions. It might even make you :smiling_face: Here are this weeks un-embedded links:
P.S. – there’s a fun announcement coming in next week’s email. Thanks for ingesting, -Zach |
Extract. Transform. Read.
Reaching 20k+ readers on Medium and nearly 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer and time as a former journalist to demystify data science/programming concepts while sharing career strategies so you can target, land and excel in data-driven roles.
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! For years, a start-up cliche was being the “Uber” of (product, service, etc.). Now, it seems like any content platform wants to be the “Tik Tok” of a given subject area. Case in point for the latter: A fun app I came across called, fittingly, “Gittok.”* Like Tik Tok, Gittok feeds users an endless stream of distraction but instead of dance challenges it serves up a random GitHub repository, like...
Extract. Transform. Read. A newsletter from Pipeline For a STEM discipline, there is a lot of abstraction in data engineering, evident in everything from temporary SQL views to complex, multi-task AirFlow DAGs. Though perhaps most abstract of all is the concept of containerization, which is the process of running an application in a clean, standalone environment–which is the simplest definition I can provide. Since neither of us has all day, I won’t get too into the weeds on containerization,...
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! From 2014-2017 I lived in Phoenix, Arizona and enjoyed the state’s best resident privilege: No daylight saving time. If you’re unaware (and if you're in the other 49 US states, you’re really unaware), March 9th was daylight saving, when we spring forward an hour. If you think this messes up your microwave and oven clocks, just wait until you check on your data pipelines. Even though data teams...