scaling software

13 posts
Issue40

Scalability problems: Hidden challenges of growing a system

Two main challenges of scaling distributed systems: centralization and synchronization. When scaling up, the system can run into computational limitations, storage limitations, and network limitations.…

Issue27

What Google taught me about scaling engineering teams

Create reusable training material & use it to onboard new engineers. Agree on a set of standard coding conventions as early as possible.…

Issue26

Notes on scalability

Plan for problems. Design each layer of an application to work independently and redundantly. Use distributed architecture patterns: load balancers, web farms, clustering, mirroring and read-only…

Issue26

5 things toxic to scalability

ORMs (object relational mappers) abstract away the SQL difficulties of interacting with the backend but result in complex queries that the database cannot optimize easily. They…

Issue26

Scalability: growing a system in different directions

A scalable distributed system continues to perform effectively as its users and/or resources grow in different directions.Usually, a system grows in terms of more…

Issue25

Taming the rate of change

Stability concerns amidst high change frequency is a new reality for us to accept and adapt to. During the normal course: Design & build for redundancy, Build pipelines to release safely & for rollback…

Issue23

How to start a software project with a quality mindset

Start documentation from Day 1. Begin with a README file for other developers: introduce the project, explain how to run it locally and/or contribute to it. Develop a separate product manual for end users.…

Issue19

How is software developed at Amazon?

Decompose a monolithic organization into small, autonomous teams that own each service or product end to end. Automate as much as possible. Deploy in a pessimistic fashion, constantly looking for problems.…

Issue18

How Shopify manages petabyte-scale MySQL backup and restore

Use incremental snapshots of data for backup after one initial full snapshot to reduce both storage and recovery times. Save on storage costs by deleting all but the last two copies for recovery purposes.…

Issue18

Tips for high availability

Use testing & canary analysis to validate code before it is pushed to production. Deploy to production exactly what you tested. Know how to roll back any deploys quickly.…