Notes from
production.

8 years of data-engineering notes — Python, dbt, AWS, and the quiet habits that keep pipelines boring. No filler, no sponsored posts, no SEO bait.

73

posts · updated monthly

Category

All73 🤖 ML/AI5 🔌 API5 ⚙️ DE22 ☁️ Cloud/DevOps8 🔀 GIT3 🖥️ Hardware2 📦 Others2 🐍 Python11 🛠️ Tools/Utils11 🌍 Web4

All posts

Page 5 of 9 · newest first

Efficiently reading large volumes of data from redshift with pyarrow

⚙️ DE

Efficiently reading large volumes of data from redshift with pyarrow

08 Jun 2021 · 06 Mins read

Using regexs with python

🐍 Python

Using regexs with python

05 Apr 2021 · 06 Mins read

ING API with python

🔌 API

ING API with python

10 Mar 2021 · 03 Mins read

Java UDF with pyspark

⚙️ DE

Java UDF with pyspark

03 Nov 2020 · 04 Mins read

Setting up a Python environment

🐍 Python

Setting up a Python environment

20 May 2020 · 03 Mins read

Using Pre-commit to automate tasks

🛠️ Tools/Utils

Using Pre-commit to automate tasks

14 May 2020 · 01 Min read

Poetry python package manager

🛠️ Tools/Utils

Poetry python package manager

08 May 2020 · 03 Mins read

Cmder terminal for windows

🛠️ Tools/Utils

Cmder terminal for windows

29 Feb 2020 · 02 Mins read

Luigi orchestrator

🛠️ Tools/Utils

Luigi orchestrator

06 Jan 2020 · 05 Mins read