Reaching 20k+ readers on Medium and nearly 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.
You Can Build A Data Pipeline In <90 Min.
Extract. Transform. Read.A newsletter from Pipeline: Your Data Engineering ResourceHi past, present and future data professional! Since today is a U.S. holiday, I won’t take much of your time; the good news is that, when conducted efficiently, building a data pipeline doesn’t have to take days, weeks or months. In fact, you can build a data pipeline in as little as 90 minutes. Accelerating pipeline development depends on a thorough read of the documentation, a familiarity with your scripting language’s requests library and patience dealing with pesky data structures. If you think, during this time, engineers are heads-down, you may have watched The Social Network too many times; personally, I like a little external stimuli while coding, which is how I ended up building a full dashboard during another American pastime–a baseball game. My secret? Distilling data with clean views, which I recommend over bloated source tables for both aesthetic and performance reasons. Even optimizations like views have their limitations, leading to optimization ceilings. The best way to break through, aside from stubbornness, is a combination of incremental problem-solving and “big picture” data modeling to reassess resources and attack the problem completely. Since I don’t want you to have to work any harder today, here are the embedded links as text:
If you’re celebrating America today, happy 4th! Thanks for ingesting, -Zach |
Extract. Transform. Read.
Reaching 20k+ readers on Medium and nearly 3k learners by email, I draw on my 4 years of experience as a Senior Data Engineer to demystify data science, cloud and programming concepts while sharing job hunt strategies so you can land and excel in data-driven roles. Subscribe for 500 words of actionable advice every Thursday.
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! I dreaded entering the job market after my data science master's. I felt like I knew more than a data analyst but less than a professional data scientist. I've since realized my program was more effective than I thought, but it couldn't prepare me for the key areas like cloud deployments and real-world problem-solving I had to learn on the job as a data engineer. And I’ve noticed these gaps in...
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! If you live in the U.S., this week marks the end of back to school season; though, if you’re like my southern relatives, you’ve been back since July. The closest feeling most adults get to back to school (aside from the teachers), is starting a new job. While a new org, title and compensation package represents new opportunities, it’s also easy to feel like the “new kid”, which can lead to being...
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! I once participated in a remote job interview in which the interviewer was on the video call while driving... and smoking. While that instance was among the most memorable interview experiences (for the wrong reasons), I’ve had just as many interviews that have blended together and faded into the recesses of my mind. The common denominator, however, was the insistence on asking one question. The...