Reading Update

Data Engineering

• The new modern data stack Airbyte Airflow DBT - For the development of an ELT pipeline, the addition of airbyte to dbt and airflow is really interesting as it finally allows to have a full open source stack. It will be interesting to see how the developers will balance the need to monetize and open source it’s core
• From daily dashboards to enterprise grade data pipelines - linkedin tells the history on how they came from a Teradata + microstrategy warehouse to it’s current architecture to help answer questions and retrieve insights from their data
• Building Data Quality into the Enterprise Data Lake - paypal writes on how they built a Rule Execution Framework, a platform that provides multiple tools to increase data quality and lineage
• Announcing ksqlDB 0.23.1 - This is more of a release note but my interest on streaming SQL has increased with time and I think this will be a game enabler for future data platforms
• When Postgres blocks: 7 tips for dealing with locks - Citus Data gives 7 really good tips on some of the most common operations in postgres and should save some headaches
• Understanding the Metrics Store - again with metrics, what we are seeing is a another movement of separating something that was done before on BI tools and accept that we need a centralized way of defining the organization metrics (which can later be used by BI tools )

Engineering

• Kubernetes: The Documentary - This documentary on kubernetes was really interesting, especially knowing how pervasive the technology is nowadays
• Include diagrams in your Markdown files with Mermaid - As a github user this new feature is really great as we can start to define our architecture changes in diagrams directly on each repo and open discussions on it
• A non-exhaustive list of my favorite things about working at companies with a culture of writing - Not quite a technical article but writing is a really good skill that enables clearer understading inside a company and can be an enabler of new initiatives and open good career paths

Others

• Imitate, then Innovate - This isn’t a technical article but it resonates with my experience where I struggle first to learn and immitating help bring oneself to the next step and innovate on any given field