Extract. Transform. Read.A newsletter from Pipeline: Your Data Engineering ResourceHi past, present or future data professional! I bet when you think of data engineering tools, you don’t think of smiley faces. But emojis, when used properly (and sparingly), can be a powerful way to emphasize logging messages and highlight infrastructure failures. Failing in data engineering can make you :sad_face:. Professionally, I’ve made every code mistake imaginable, with the highlight being omitting a column and needing to process a 45 TB table—twice! Even as I gain experience, I still make (but recover from) plenty of mistakes, which I document on a yearly basis. Since we’re a little more than halfway through the year, I’d encourage you to use this framework to honestly assess both your technical and interpersonal shortcomings as an aspiring data engineer. Proactively identifying areas of improvement is growth—and areas for improvement will pay more dividends than cramming in upskilling sessions. It might even make you :smiling_face: Here are this weeks un-embedded links:
P.S. – there’s a fun announcement coming in next week’s email. Thanks for ingesting, -Zach |
Top data engineering writer on Medium & Senior Data Engineer in media; I use my skills as a former journalist to demystify data science/programming concepts so beginners to professionals can target, land and excel in data-driven roles.
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! Despite falling into the realm of engineering, data infrastructure construction is a bit like basic art. At times building a data pipeline is as simple as filling in one of those color-by-numbers books. Other times, the process of extracting and ingesting data can be as abstract and disconnected as paint flicked onto a canvas, Jackson Pollack style. No matter the complexity of your build, there...
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! To those in the U.S.: Happy Halloween! In the spirit of the spooky season, I’d like to scare—I mean warn—you about 3 truly creepy trends that might give you goosebumps during a job search. A Shady Recruiter “Ghost” Writing Your Resume When in the job market, one of the first things you learn, after how to write a resume, is how to format one. I’m sure you know about headings, bullet points, etc....
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! I recently participated in a technical design meeting that was derailed by a single, fundamental question. “Why?” Despite the fact that I worked with the particular data source we were discussing for nearly two years, I fell into the common trap of going “on autopilot” and failing to question the initial need for the data. At this point, you would think asking “why” of years’ worth of work would...