Extract. Transform. Read.

A newsletter from Pipeline​

Hi past, present or future data professional!

For the first time since the birth of the Internet, the prevalence of AI summaries has damaged Google’s Search business, possibly irreparably. And while this might simply be a sign the times they are a changin’ (I just watched that new Bob Dylan movie), it points to a harsher reality.

These days search universally sucks; I’ve found this is especially true when readers, like yourself, want to dig deeper into my publication’s body of work.

My goal when beginning Pipeline was to create a sort of public notebook, recording the practical lessons I learned on the job. But like old notebooks, pages get torn, tossed aside and buried.

To help resurrect old data engineering content I believe is useful but overlooked, I went old school, making an index containing hundreds of searchable, scrollable hyperlinks in 1 post called, appropriately, “Every Pipeline Story Ever.” This effectively serves as the publication’s archive, allowing ready access to 250+ stories from November 2021 to present.

That means you can explore older content like my primer on GCP’s logging query language or working with streaming data in Python. If you’ve read my project guide you’ll know I’m a fan of unique projects like when I helped a Wall Street banker scrape data from 3,000 companies. And if you want to know what the day in the life of a working data engineer is, you can read about my typical work day, with a European twist.

As a bonus, I wrote a piece detailing the simple process I used to programmatically create this webpage. My only tools were Pandas, a few HTML tags and markdown syntax. And, to be honest, a bit of guidance from my pair programmer AI.

My hope is to fight the ever-picky algorithms by allowing you, the reader, to choose how and when you consume.

Take notes, Google.

Thanks for ingesting and enjoy your trek through the archives,

-Zach Quinn