DB/SQL history since college
3 November, 2026
- tech
This is the endpoint of a late nitght discussion with chatgpt about the evolution of sql since I was in the industry with fun links at the end. Not my own writing.
Twenty Years of Database Wars: How SQL Survived NoSQL
From Oracle and MySQL to Bigtable, MongoDB, and PostgreSQL’s unexpected dominance.
In 2005, the database world seemed relatively settled. Relational databases dominated, SQL was the universal language, and choosing a database largely meant choosing between expensive commercial products and increasingly capable open-source alternatives.
Twenty years later, we’ve witnessed an explosion of new database architectures, the rise of NoSQL, globally distributed transactions, and the emergence of cloud-native storage. Yet the biggest winner might be a database that was already nearly a decade old in 2005: PostgreSQL.
How did we get here?
2000–2005: Open Source Challenges the Database Giants
For decades, enterprise databases were dominated by Oracle, IBM DB2, and Microsoft SQL Server. These were sophisticated systems, but commercial licensing could be extraordinarily expensive.
The internet changed the economics. MySQL, first released in 1995, became a foundation of the LAMP stack (Linux, Apache, MySQL, PHP). It was free, relatively simple to administer, and fast enough for the websites being built by an emerging generation of developers.
PostgreSQL, released under that name in 1996, offered a more ambitious alternative, emphasizing SQL standards, advanced data types, transactions, and extensibility. SQLite, released in 2000, demonstrated that an entire transactional SQL engine could be embedded in an application without a database server.
By 2005, open-source relational databases had become credible alternatives to commercial products. SQL’s dominance seemed secure.
But trouble was brewing at Google and Amazon.
2005–2010: Google and Amazon Reinvent Storage
As Google scaled its services across thousands of servers, traditional relational databases presented difficulties. Scaling writes generally meant buying bigger machines or manually partitioning data across independent database servers.
Google’s 2006 Bigtable paper described a different architecture: an enormous, sparse, sorted map distributed across machines, automatically dividing data into manageable ranges called tablets.
Bigtable abandoned relational queries and general-purpose transactions in favor of scalability and predictable performance.
Amazon’s 2007 Dynamo paper approached the problem differently. Its distributed key-value system prioritized availability, allowing operations to continue through server failures and network partitions, with conflicts resolved when necessary.
These papers inspired an entire generation of databases. Apache HBase followed Bigtable’s model. Cassandra combined Bigtable-like data structures with Dynamo-inspired distribution. Riak explored highly available key-value storage.
The underlying challenge wasn’t that SQL was fundamentally wrong. It was that the prevailing implementations of relational databases weren’t designed to distribute writes automatically across hundreds of machines.
Nevertheless, the emerging movement became known as NoSQL.
2009–2015: The NoSQL Gold Rush
MongoDB, introduced in 2009, brought NoSQL to ordinary application developers.
Instead of managing relational tables, developers stored JSON-like documents. Schemas could evolve without traditional migrations, and application objects mapped naturally to database records.
MongoDB’s appeal was as much about developer productivity as scalability. During this period, JSON APIs, JavaScript frameworks, and rapidly changing web applications made document databases particularly attractive.
Other systems flourished. Redis made in-memory key-value storage ubiquitous. CouchDB offered document replication. Cassandra promised distributed write scalability. Amazon launched DynamoDB in 2012 as a managed NoSQL service, while Microsoft’s Azure DocumentDB, later Cosmos DB, expanded the cloud database market.
For a while, it seemed relational databases might become legacy technology.
But NoSQL came with tradeoffs.
Applications frequently needed relationships between records, transactions spanning multiple objects, and queries that hadn’t been anticipated when the data model was designed. Without joins or sufficiently expressive transactions, developers often reimplemented database functionality in application code.
And the promised simplicity of schema-free development could become difficult to maintain as applications matured.
Meanwhile, relational databases weren’t standing still.
2012–2020: SQL Strikes Back
Two developments helped change the direction of database technology.
The first was Google’s Spanner, described in a landmark 2012 paper. Spanner demonstrated that globally distributed databases could provide strong transactional guarantees rather than requiring applications to accept eventual consistency.
This helped inspire a new generation of distributed SQL systems, including CockroachDB, TiDB, and YugabyteDB.
These systems borrowed heavily from NoSQL architectures—automatic partitioning, distributed replication, and consensus protocols—but retained SQL and transactions.
The second, perhaps more consequential development for everyday developers, happened inside PostgreSQL.
In 2014, PostgreSQL 9.4 introduced JSONB, allowing efficient, indexable storage of JSON documents alongside traditional relational data.
Suddenly, developers didn’t necessarily have to choose between a document database and a relational database. PostgreSQL could provide both.
PostgreSQL also benefited from an unusually extensible architecture. PostGIS made it a powerful geospatial database. TimescaleDB expanded its usefulness for time-series workloads. Citus introduced distributed sharding capabilities.
Rather than being displaced by specialized databases, PostgreSQL increasingly absorbed their functionality.
2020–2026: PostgreSQL Becomes the Default
By the 2020s, PostgreSQL had become the preferred general-purpose database for an enormous range of new applications.
In the 2026 Stack Overflow Developer Survey, 57.9% of respondents reported working with PostgreSQL, versus 35.1% for MySQL and 19.0% for MongoDB. These are developer usage figures, not measures of database installations or revenue, but they illustrate the change in sentiment.
Several factors explain PostgreSQL’s success.
First, it is genuinely capable. Its query optimizer, transactional guarantees, indexing options, and extensibility make it a strong default for most applications.
Second, its ecosystem has become enormous. Nearly every cloud provider offers managed PostgreSQL, virtually every development framework supports it, and companies can change hosting providers without necessarily changing databases.
Third, PostgreSQL continues to assimilate competing technologies. The emergence of pgvector brought vector similarity search into ordinary SQL applications, allowing developers to experiment with AI workloads without adopting another database.
But PostgreSQL’s popularity conceals some architectural weaknesses.
It still generally depends on a single writable primary. Read replicas can distribute queries, but distributing writes requires sharding, extensions, or independently managed database instances. Its MVCC implementation also creates dead tuples that must be cleaned up through VACUUM.
For individual applications, modern hardware is powerful enough that these limitations may never matter. For platforms hosting hundreds of applications, shared write bottlenecks and resource contention are less attractive.
The rise of PostgreSQL-compatible distributed databases suggests that developers increasingly value PostgreSQL’s interface and ecosystem independently of its underlying storage architecture.
The Next Chapter: Distributed SQL, Analytics, and Separation of Storage
While PostgreSQL was winning developer mindshare, several other important developments were unfolding.
Distributed SQL systems such as CockroachDB, YugabyteDB, TiDB, and Google’s Spanner attempt to preserve SQL and strong transactions while distributing data and write ownership across machines. Their architectures are attractive, although distributed coordination brings latency, operational complexity, and cost.
Analytical databases have followed a different path. Snowflake, BigQuery, ClickHouse, and DuckDB demonstrate the advantages of column-oriented storage and specialized execution engines. DuckDB is particularly interesting because it brings high-performance analytical SQL to a single embedded process, much as SQLite did for transactions.
Storage and compute are separating. Cloud-native architectures increasingly place durable data in shared or object-based storage while allowing compute resources to scale independently. Systems such as Neon and Amazon Aurora illustrate different approaches to decoupling database compute from storage.
Open-source licensing is becoming a battleground. Companies including MongoDB, Elastic, Redis, and Cockroach Labs have moved important products away from conventional permissive open-source licensing. PostgreSQL’s permissive license and independent development community have consequently become competitive advantages of their own.
And there’s a surprising continuity underneath all these changes: many modern databases, including distributed and analytical systems, still expose SQL.
Did SQL Win?
The last twenty years of database development weren’t simply a story of SQL defeating NoSQL.
NoSQL solved real problems. Bigtable and Dynamo fundamentally changed how distributed storage systems are designed. Cassandra, MongoDB, DynamoDB, and Cosmos DB continue to serve workloads for which their architectures make sense.
But NoSQL also revealed just how valuable relational queries and transactions were. Developers discovered that eliminating database complexity frequently meant moving that complexity into applications.
PostgreSQL’s resurgence illustrates another lesson: a mature, extensible platform can outlast specialized competitors by incorporating their best features.
The next generation of databases may distribute writes automatically, separate compute from storage, or use entirely different storage engines. But many will still speak SQL—and an increasing number will speak PostgreSQL’s dialect of it.
Perhaps the real winner of the database wars wasn’t PostgreSQL itself, but the relational abstraction it helped preserve.
Further reading
1. Michael Stonebraker and Andy Pavlo, What Goes Around Comes Around… And Around…, 2024.
2. Fay Chang et al., Bigtable: A Distributed Storage System for Structured Data, 2006.
3. Giuseppe DeCandia et al., Dynamo: Amazon’s Highly Available Key-value Store, 2007.
4. James C. Corbett et al., Spanner: Google’s Globally-Distributed Database, 2012.
5. Andy Pavlo and Bohan Zhang, The Part of PostgreSQL We Hate the Most, 2023.
6. Stack Overflow Developer Survey 2026 — Databases.
Comments

Nuc based k8s cluster

22 October, 2026

Three-Node Talos Kubernetes Cluster

Goal

Build a self-contained, three-node Kubernetes appliance for developers to experiment with distributed systems, high availability, and cluster administration.

Target retail price: ~$1,000

The prototype will use commodity hardware inside a desktop cube, with one AC power connection and one Ethernet uplink.

Hardware

Component	Specification	Estimated cost
Compute	3× Beelink SER3, Ryzen 3 3200U, 16GB RAM, 500GB SSD	$660
Network	Waveshare Gigabit-Switch-5P, 5-port Gigabit	$25
Power	Shared regulated 19V/250W DC supply	$100–185
Distribution	Four fused DC outputs, cables, connectors	$20–40
Enclosure	2020 aluminum extrusion desktop cube	$50–90
Cooling/mounting	2× 120mm fans, brackets, shelves	$25–50
Total		$880–1,050

Optional: 600–850VA AC UPS (~$100).

Reference hardware:

Power Architecture

All three SER3 computers and the Ethernet switch run from one 19V DC supply.

Each SER3: 19V × 3.42A = 65W adapter rating.
Three SER3s: 195W combined adapter rating.
Waveshare switch: accepts 5–36V DC, including 19V.
Fans: use 19V-compatible fans or a DC-DC converter for 12V fans.
Supply: regulated 19V/250W, subject to startup and full-load testing.

Use a fused distribution block with four independently protected outputs: three barrel connectors for the Beelinks and one screw-terminal connection for the switch.

Verify barrel dimensions, polarity, wire ratings, and power consumption before assembling.

No power redundancy is required.

Enclosure

Build an approximately 300mm desktop cube using 2020 aluminum extrusion.

Three independently mounted Beelinks, retaining their original cases initially.
Waveshare switch mounted internally.
Two 120mm fans providing front-to-back airflow.
Shared DC supply and distribution.
One external Ethernet uplink and one AC power connection.
Accessible node power buttons and replaceable components.

Final dimensions depend on thermal and mechanical testing.

Kubernetes Software

Install Talos Linux on all three nodes.

Component	Configuration
Kubernetes	Talos-managed
Control plane	3 schedulable control-plane nodes
etcd	3 members, tolerates 1 node failure
Networking	Talos default CNI initially
Ingress	Traefik
Load balancer	MetalLB or equivalent
Storage	Local SSD initially
Observability	Optional OpenTelemetry/Grafana

The cluster provides 6 physical CPU cores, 48GB total RAM, and 1.5TB raw SSD capacity.

Storage is not automatically replicated. Distributed storage such as Longhorn or Ceph can be installed experimentally.

Prototype Milestones

Compute: Acquire three identical SER3s and install Talos.
Cluster: Establish a working HA Kubernetes control plane and test node failure/recovery.
Power: Measure consumption and implement the shared 19V supply with four fused outputs.
Enclosure: Assemble the cube, mount the computers and switch, and install cooling.
Validation: Test simultaneous startup, thermal behavior, sustained load, power recovery, upgrades, and node replacement.
Productization: Develop repeatable setup, LAN discovery, secure credential provisioning, factory reset, and recovery documentation.

Path to a $1,000 Product

The prototype uses retail hardware and may cost as much as the intended selling price.

For production, target $600–650 hardware and assembly cost by:

Sourcing complete mini PCs directly from an OEM.
Eliminating redundant cases and individual power adapters.
Purchasing the switch as a bare PCB.
Standardizing shared DC power distribution.
Replacing the extrusion frame with an inexpensive manufactured enclosure.
Automating Talos provisioning and hardware testing.

The product’s primary differentiator is a ready-to-use, physically distributed Kubernetes environment, not raw CPU performance.

Success criterion: A developer can connect one power cable and one Ethernet cable, initialize the cluster, and immediately begin running and breaking real distributed workloads.

Comments

Smith Brothers Locality
15 October, 2026
To be clear, I think America’s food distribution system is amazing. It’s also October, which is probably past peak produce box season in the Pacific Northwest.
That said, I wanted to see just how local one of our bigger grocery delivery services, Smith Brothers Farms, actually is. I’m arbitrarily defining “local” as anywhere in the Pacific Northwest.
Here’s where my groceries came from.
Pacific Northwest
- Sausage: Carlton Farms, Oregon.
- Kale: Carnation, Washington.
- Delicata squash: Bay Baby Produce, Skagit Valley, Washington.
- Milk: Kent, Washington.
- Apples: SweeTango apples are mostly grown in Washington state, although the exact origin of these wasn’t listed.
California
- Potatoes: California.
- Kiwi berries: California.
- Lemons and limes: Origin unclear, but let’s assume California.
Mexico
- Green onions: Mexico.
- Cherry tomatoes: Mexico.
Other or Unclear Origins
- Chicken: Sourced through Porter & York, which seesm to be woodinville, wa compman though meat source unclear.
- Cucumbers: Origin unclear.
- Pears: Origin unclear.
- Yogurt: Chobani, a national brand. I missed that Zoi appears to be a local alternative, so that’s my fault.
What Am I Cooking?
I’m using this order to make four Careme meals, including Chili-Lime Chicken & Delicata Squash and Sausage & Broccoli Bulgur Bowls.
Sadly, I wasn’t able to source ginger from Smith Brothers. I already had tahini on hand, though.
I’ll come back with reviews of the produce. The big question: Can you actually taste a difference compared with QFC?
The Order
Here’s an older screenshot of my Smith Brothers cart. This wasn’t the final order and doesn’t exactly reflect what arrived. For example, cucumbers were substituted for eggplant.
Also copy editing and formatting assistited by chatgpt see original here
Comments
Bootalicious
2 October, 2026
Near the oxbow trail in middle fork of snoqualmie. Only one other person on the train early parts. Apparently secret mountain biking den where they don’t want yo recording gps to give out their trails!

Comments
Fresh Home Hop Cascade Pale Ale
1 October, 2026
- homebrew
- beer
Fresh Cascade Pale Ale

First beer using fresh hops from the yard. I believe they’re Cascade, though hop variety obviously can’t be confirmed from appearance alone.
The goal was to make the fresh hops the interesting part and keep everything else simple and difficult to screw up: neutral American malt, clean yeast, known-alpha bittering hops, and all of the homegrown hops late in the process.
Recipe
Batch size: ~5 gallons
Style: Fresh-hop American Pale Ale
Grain
- 9.5 lb Rahr Pale Malt
- 1 lb Weyermann Vienna Malt
Total grain: 10.5 lb
Hops
- 0.5 oz Magnum pellets — 60 minutes
- 6 oz fresh Cascade — 5 minutes
- 10 oz fresh Cascade — whirlpool
Total fresh hops: 16 oz
The fresh hops were used only late in the boil because their alpha-acid percentage is unknown. Magnum provided the predictable bittering charge.
Yeast
- Imperial A07 Flagship
No dry yeast bullshit.
Water
Mash water:
- 4 gallons
- 1/2 tsp gypsum
- 1/4 tsp calcium chloride
- 1/4 Campden tablet
Sparge water was prepared separately and used to bring the kettle up to pre-boil volume.
Mash
Strike water went in at 167°F, a little higher than the planned 164°F.
Mash target was approximately 152°F for 60 minutes.
Grain was mashed in a removable grain tube, then the tube was lifted and sparged.
Boil
60-minute boil.
Time remaining Addition
60 min 0.5 oz Magnum
10 min Whirlfloc + yeast nutrient
5 min 6 oz fresh Cascade
0 min Flameout
After flameout I started chilling toward the planned 170°F whirlpool temperature.
The wort got down to 165°F before I managed to get the remaining hops in. Rather than reheating it, I added:
- 10 oz fresh Cascade at 165°F
The full fresh-hop charge was then allowed to steep for approximately 20 minutes.
The original 6 oz five-minute addition stayed in the wort during the whirlpool.
Cooling
Immersion chilling was slower than expected.
- 158°F → 100°F: ~20 minutes
- 100°F → 89°F: another ~20 minutes
The Anvil kettle has electronics in the base, so putting the kettle into an ice bath wasn’t an option.
Eventually chilled to approximately 67°F.
Original Gravity
Refractometer:
OG: ~1.070
This is considerably higher than the recipe’s predicted ~1.052, so I want to verify the refractometer calibration and/or reading before treating 1.070 as definitive.
If 1.070 is correct, either efficiency was much higher than expected, final wort volume was lower than planned, or both.
Fermentation
Pitched one pouch of Imperial A07 Flagship at approximately 67°F.
Plan:
- Hold around 65–67°F during active fermentation
- Allow to rise toward 68–70°F as fermentation finishes
- Package once gravity is stable
Targets
Original recipe targets:
- OG: 1.050–1.054
- FG: 1.010–1.012
- ABV: ~5.2–5.5%
- IBU: ~30–35
If the measured 1.070 OG is real and A07 finishes around 1.012–1.014, this may end up closer to 7.3–7.6% ABV than the intended pale ale.
Brew Day Notes
The biggest unknown in this batch is the homegrown fresh Cascade.
I deliberately avoided using it for bittering because I don’t know the alpha-acid percentage. The majority went into a relatively cool 165°F whirlpool in an attempt to preserve the fresh citrus/floral/resin character without extracting excessive bitterness or vegetal character.
Sixteen ounces of wet hops looked ridiculous going into five gallons of beer, but because freshly harvested hops are mostly water, that’s roughly equivalent to only a few ounces of dried hops.
Now we wait.
Comments