System design interview: how to design a video platform (e.g, Youtube, Netflix)

System design interview: how to design a video platform (e.g., Youtube, Netflix)

Methodology: READ MF!

[Originally from the Post: System design interview: how to design a chat system (e.g., Facebook Messenger, WeChat or WhatsApp)]

Remind ourselves with the "READ MF!" methodology.

Key designs and terms 

Let's first admit this is a hard problem. Youtube has hundreds of smart engineers keep improving it and it's almost impossible to design a good one in 60 mins. This solution is a pretty decent one from Grokking the system design interviews where it covers the backbone about the system.

 

You can also divided the system as usual into three major layers, presentation, service and data layer.

A few key terminologies. If you are not familiar, brush up a bit

• CDN: a distributed cache for videos
• Codec: encoding and decoding, a video would be encoded to different resolutions and different formats, e.g, 16:9 mp4, 4: 3 avi
• Video processing is slow, normally done asynchronously via queue and workflows 
• Video de-duplication, hash content (SHA, MD5), phase correlation, block matching. If you forget about Fourier Transformation, watch 3Blue1Brown
• Metadata sharding, pros vs con. If shard videos by user uuid (hot partition for popular users). If shard videos by video uuid (hot partition for popular videos)
• To solve hot hot partition / hot key: Consistent Hashing (original paper 1997: Consistent Hashing and Random Trees:Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web)  + Load balancing + Replicate data (Need to automatically discover the hotkey first, redis has built-in support). Alibaba Cloud Read
• Comment/Reply design, Likes and View counts (all sorts of counters) would be explained in a separate post.

 Baozi Youtube Video

• Youtube Video Link
• B站

References (Credits to original authors)

• Grokking the system design interviews
• Alibaba Cloud Read