At PeerDB, we are building a fast and a cost-effective way to replicate data from Postgres to Data Warehouses such as Snowflake, BigQuery, ClickHouse, Postgres and so on. All our customers run Postgres at the heart of the data stack, running fully managed or self-hosted Postgres databases.

We often get asked about the preferred managed service for PostgreSQL. In that spirit, we are writing this blog to compare four popular options incl. AWS RDS Postgres, Azure Flexible Server Postgres, GCP Cloud SQL for Postgres, and Supabase Postgres, across Performance, Costs and Features. We also acknowledge other providers like Tembo, Crunchy Bridge, Neon and TimescaleDB which we'll cover in a future post.

Note that this this comparison aims to serve as a helpful "first" checklist for developers choosing a managed service. There may be something we missed, and we apologize for those oversights. We are happy to adjust our analysis based on feedback.

Setup

To ensure an apples-to-apples comparison, we aimed to match the four options as closely as possible in terms of RAM, vCores, disk space, PostgreSQL version, region, etc. The table below captures the details of the initial setup.

Performance

Benchmark Setup

All the performance tests were conducted using a VM (client) with the same compute capacity and collocated in the same region as the PostgreSQL database. We did 3 main performance tests:

1. pgbench representing a typical Transactional (OLTP) workload

2. COPY command to Batch Insert (Upload) data to Postgres

3. SELECT command to Batch download data from Postgres

pgbench

Across all the 4 managed PostgreSQL providers, pgbench was run for 24 hours with 8 parallel connections and 4 jobs pgbench -c 8 -j 4 -P 30. The graphs below capture a comparison of average throughput i.e. transactions per second (TPS), average latency and average CPU utilization for all the services.

AWS RDS PostgreSQL led the pack with an average of 2.7K TPS and 2.884 ms average latency. Azure Flexible Server PostgreSQL ranked second, closely trailing AWS RDS by just ~12%. It recorded an average of 2.4K TPS and an average latency of 3.260 ms. Supabase and GCP Cloud SQL PostgreSQL followed. Average CPU utilization across all the services was almost the same i.e. around 80%, except for Supabase. This could be because Supabase uses ARM processors compared to others who use x86.

Batch Upload and Download

For batch uploads, we used the COPY command to insert 1GB and 5GB files from the client to PostgreSQL. For batch downloads, we executed a SELECT query that retrieved 1GB and 5GB of data from a table in PostgreSQL to the client. The graphs below illustrate how each service performed in these tests:

In terms of batch upload with COPY command, AWS RDS was again the leader taking around ~105s to ingest 5GB of data. GCP Cloud SQL was second with 113s. Azure Flexible Server and Supabase followed.

In terms of batch download using SELECT, the numbers were close across AWS, GCP, and Azure, with GCP slightly ahead, taking 51 seconds to download 5GB data. It was interesting to note that Supabase took longer than the others, requiring 160 seconds to download 5GB of data.

CPU utilization peaks during the COPY command were almost consistent across AWS and GCP, at around ~45-50%. Supabase was at approximately 57%. However Azure peaked at 85%.

Costs

Below table captures costs across all the 4 managed services for a Postgres Database with 2vCPU, 8GB RAM and 100GB disk. More details regarding the infra can be found in this sheet.

If you notice, Supabase is the most cost-effective compared to other managed services, at $113. This could be because Supabase uses machines with ARM processors, which are more cost-effective compared to x64. GCP Cloud SQL comes in second at $116 per month. AWS RDS and Azure Flexible Server are tied at $141.44 per month.

Database Features

Postgres Managed Services typically support various important features for running production and enterprise-grade Postgres deployments. A few important features include:

Availability and Reliability:

1. High Availability (HA) to minimize downtime during DB failures/crashes.

2. Backups / Point-In-Time-Recovery to handle Disaster Recovery (DR) scenarios

3. Cross region read replicas for enterprise-grade DR

Performance and functionality:

1. Out of the box feature to help performance tuning of queries.

2. Read-replicas to segregate and scale read workloads

3. Out of the box connection pooling

4. Extension to enhance Postgres functionality

Security and Compliance:

1. SOC2 and HIPAA

2. Private Access

The table below compares each of the four managed services based on the above features:

Conclusion

Below is a summary of the results from the analyses conducted across the four managed services.

1. AWS RDS Postgres was the most mature Postgres offering of all the other managed services.

   1. Performance-wise, it surpassed Azure by just 12% and exceeded the others by over 45% in pgbench throughput and latency.

   2. Feature-wise, it supports almost all of them in the Availability and Reliability, Performance, and Security and Compliance categories.

   3. It supports the highest number of extensions, i.e., 92 of them.

2. Azure Flexible Server takes second place in performance. It was very close to AWS, being only about 12% lower in performance. It matches AWS RDS Postgres in terms of features.

3. Managed services across all three clouds offer robust support for features related to Availability & Reliability and Security & Compliance, which are important for enterprise-grade workloads.

4. Supabase and GCP Cloud SQL Postgres are the most cost-effective of all the managed services.

5. Special mention to Supabase for supporting features that make the lives of app developers incredibly easy.

Hope you enjoyed reading this blog. In future blogs we will add a few other managed services to this comparison and aim to go deeper in a few categories such as Performance.

References

Excel sheet capturing all our raw analysis to come up with this blog

NOTE: The blog was updated on April 17, 2023. The primary modification involved changing the Azure Flexible Server VM type from ARM (Standard_D2ads_v5) to Intel (Standard_D2s_v5). This change can be easily configured through radio buttons while provisioning and is set as the default across various regions. Therefore, we deemed it a fair modification in the comparison.