List: Data platform design | Curated by Data Saint Consulting Inc

Sep 10, 2023

24 stories

Data platform design
Cory Maklin
The Star Schema Is Dead Long Live Wide TablesIf you do a quick search for books data engineer must read, you’ll find 
two books:
1. Designing Data-Intensive Applications
2. The Data…
May 29, 2023
15
May 29, 2023
15
In
Picnic Engineering
by
Iliana Iankoulova
Scaling Business Intelligence with Python and SnowflakeThree big migration projects brought Picnic’s self-service Business Intelligence to new heights — speeding up the development cycle with…
May 18, 2021
5
May 18, 2021
5
Rui Li
How Bilibili Builds OLAP Data Lakehouse with Apache IcebergIntroduction
Jun 14, 2023
6
Jun 14, 2023
6
Lackshu Balasubramaniam
Design Patterns for Data LakesData Lake is the heart of big data architecture, as a result there needs to be careful planning in designing and implementing a Data Lake.
Apr 5, 2020
1
Apr 5, 2020
1
In
Geek Culture
by
Rahul Kapoor
System Design — Scaling from Zero to Millions Of UsersNote: I have read this great book System Design Interview — An insider’s guide by Alex Xu in depth. So most of my definitions and images…
Jan 2, 2023
9
Jan 2, 2023
9
In
Geek Culture
by
Priyal Walpita
Managing Data in Microservices ArchitectureEveryone knows that microservices architecture has lots of advantages like scalability, easy maintenance, and frequent deployments. But…
Jun 3, 2021
2
Jun 3, 2021
2
In
Coupang Engineering Blog
by
Coupang Engineering
Data platform 2022: Transforming bytes to business insightsPart II of a two-part series about Coupang’s data platform, focusing on the analytics platform
Jun 24, 2022
Jun 24, 2022
 This story is no longer available
In
Towards Data Science
by
Kaya Kupferschmidt
Big Data Engineering —Declarative Data FlowsThis is part 3 of a series on data engineering in a big data environment. It will reflect my personal journey of lessons learnt and…
Sep 30, 2020
Sep 30, 2020
Ankush Singh
Feather vs Pickle: A Comparative Analysis of Data StorageIntroduction
Jun 11, 2023
Jun 11, 2023
Ankush Singh
Comparing Data Storage: Parquet vs. ArrowData storage formats have significant implications on how quickly and efficiently we can extract and process data. In today’s blog, we’re…
Jun 11, 2023
3
Jun 11, 2023
3
In
Data Engineer Things
by
Khairina
Fundamentals of Data Engineering Book — Key Learning PointsHere are  my key learning points from the “Fundamentals of Data Engineering” book by Joe Reis and Matt Housley
May 14, 2023
8
May 14, 2023
8
Chenglong Wu
The Spark 5S Optimization Series, Part 4: Excelling at Serialization Optimization for Stellar PerfOptimizing Serialization in Apache Spark: Strategies to Improve Performance Using the 5S Optimization Framework
Apr 23, 2023
Apr 23, 2023
In
Level Up Coding
by
Arslan Ahmad
16 System Design Concepts I Wish I Knew Before the Interview.Mastering System Design Interview: Essential Concepts for Every Software Engineer
Apr 3, 2023
20
Apr 3, 2023
20
Philip S Bell
Run Presto at Meta scale with these lessons learned from a Meta Production EngineerPresto is a free, open source SQL query engine. We’ve been using it at Meta for the past ten years, and learned a lot while doing so…
Apr 15, 2023
Apr 15, 2023
In
Teads Engineering
by
Quentin Fernandez
How we made our reporting engine 17x fasterData relocation, the key to our new reporting solution
Apr 13, 2023
Apr 13, 2023
In
Pinterest Engineering Blog
by
Pinterest Engineering
Large Scale Hadoop Upgrade At PinterestYongjun Zhang | Software Engineer; William Tom | Software Engineer; Shaowen Wang | Software Engineer; Bhavin Pathak | Software Engineer…
Mar 30, 2022
Mar 30, 2022
In
Pinterest Engineering Blog
by
Pinterest Engineering
Scalable and reliable data ingestion at PinterestYu Yang | Pinterest engineer, Data
May 19, 2017
2
May 19, 2017
2
In
Building Carta
by
Troy Harvey
Carta Spotlight: Data EngineeringIn this post I will share some stories highlighting data engineering and the role we played in building Carta’s data platform.
Jan 25, 2023
3
Jan 25, 2023
3
In
Towards Data Science
by
Christianlauer
Five Best Practices for stable Data ProcessingWhat you should keep in Mind when setting up Processes like ETL or ELT
Jan 19, 2021
Jan 19, 2021

Data platform design

The Star Schema Is Dead Long Live Wide Tables

If you do a quick search for books data engineer must read, you’ll find two books: 1. Designing Data-Intensive Applications 2. The Data…

Scaling Business Intelligence with Python and Snowflake

Three big migration projects brought Picnic’s self-service Business Intelligence to new heights — speeding up the development cycle with…

How Bilibili Builds OLAP Data Lakehouse with Apache Iceberg

Introduction

Design Patterns for Data Lakes

Data Lake is the heart of big data architecture, as a result there needs to be careful planning in designing and implementing a Data Lake.

System Design — Scaling from Zero to Millions Of Users

Note: I have read this great book System Design Interview — An insider’s guide by Alex Xu in depth. So most of my definitions and images…

Managing Data in Microservices Architecture

Everyone knows that microservices architecture has lots of advantages like scalability, easy maintenance, and frequent deployments. But…

Data platform 2022: Transforming bytes to business insights

Part II of a two-part series about Coupang’s data platform, focusing on the analytics platform

Big Data Engineering —Declarative Data Flows

This is part 3 of a series on data engineering in a big data environment. It will reflect my personal journey of lessons learnt and…

Feather vs Pickle: A Comparative Analysis of Data Storage

Introduction

Comparing Data Storage: Parquet vs. Arrow

Data storage formats have significant implications on how quickly and efficiently we can extract and process data. In today’s blog, we’re…

Fundamentals of Data Engineering Book — Key Learning Points

Here are my key learning points from the “Fundamentals of Data Engineering” book by Joe Reis and Matt Housley

The Spark 5S Optimization Series, Part 4: Excelling at Serialization Optimization for Stellar Perf

Optimizing Serialization in Apache Spark: Strategies to Improve Performance Using the 5S Optimization Framework

16 System Design Concepts I Wish I Knew Before the Interview.

Mastering System Design Interview: Essential Concepts for Every Software Engineer

Run Presto at Meta scale with these lessons learned from a Meta Production Engineer

Presto is a free, open source SQL query engine. We’ve been using it at Meta for the past ten years, and learned a lot while doing so…

How we made our reporting engine 17x faster

Data relocation, the key to our new reporting solution

Large Scale Hadoop Upgrade At Pinterest

Yongjun Zhang | Software Engineer; William Tom | Software Engineer; Shaowen Wang | Software Engineer; Bhavin Pathak | Software Engineer…

Scalable and reliable data ingestion at Pinterest

Yu Yang | Pinterest engineer, Data

Carta Spotlight: Data Engineering

In this post I will share some stories highlighting data engineering and the role we played in building Carta’s data platform.

Five Best Practices for stable Data Processing

What you should keep in Mind when setting up Processes like ETL or ELT

Data Saint Consulting Inc