Postgres’ Breakout Era: From Budding Database to AI Infrastructure Backbone

Written by
Justin Yue

Postgres is having a moment. It began as a reliable, open-source database, and has now become the go-to database for a wide range of workloads, from transactional systems to emerging use cases such as vector search. In this post, I will share my thoughts on why Postgres is becoming an increasingly central part of conversations among developers and data infrastructure decision makers.

As a refresher, Postgres was developed in the 1980s at UC Berkeley to address functionality shortcomings of the incumbent INGRES solution, namely an inability to handle complex data types and relationships. Recently, key data infrastructure players such as Databricks and Snowflake have announced acquisitions of managed Postgres offerings. Moreover, according to DB Engines, Postgres is one of the top 5 databases based on popularity, with its ranking increasing year-over-year.

Why is its adoption momentum picking up four decades later? It is because of market dynamics and competitive dynamics trending in favor of Postgres.

Market Dynamics

Below are the key market tailwinds that are driving broader adoption of Postgres on the enterprise and developer level:

Postgres powers core transactional use cases, for which demand is growing

As companies scale operations and are tasked with addressing the needs of more customers, the demand for data infrastructure to store additional transactional data increases. This is because customers demand lightning-fast and responsive front-end applications, which requires high-throughput, low-latency back-end infrastructure that can efficiently process transactional workloads. Postgres’s ability to ensure low-latency performance for customer-facing transactional applications positions it well for adoption. 

Postgres also serves as the core memory store for AI agents and LLM orchestration framework workloads, which is forecasted to increase 

Organizations are investing in – and are expected to increase spending on – AI agents. According to a survey PwC conducted of 300 senior executives, AI adoption is budding and continues to have massive adoption potential. Consider the following data points: 17% of organizations today have full adoption of AI agents throughout the company. However, 88% of individuals say they plan on increasing AI budget in response to agentic AI developments. As companies build more customized AI agents, they will require databases to store and manage the underlying data that powers these agents.

Developers building AI agents today are using Postgres as the de facto storage and search database, which offers native integrations to other widely used AI infrastructure tools such as LlamaIndex and LangChain. Ultimately, Postgres is serving as the core transactional database for GenAI applications. 

Developers are increasingly adopting open-source Postgres 

Postgres’ vibrant open-source offering drives developer adoption. The open-source ecosystem around Postgres understands developers’ current demands and continues to enhance its offering to cater to those needs. For example, the community added extensions such as pgvector for vector search, expanding Postgres’ ability to handle new workloads. As a result, Postgres ranked as the #1 most commonly used database by developers in 2024, according to the Stack Overflow Developer Survey, for the second consecutive year. This marks a significant rise from its third-place ranking in 2018, when it first appeared in the survey. 

Competitive Dynamics

Let’s dive into the competitive differentiation of Postgres versus the other top-ranked OLTP (Online Transaction Processing) databases to understand how they are unique. Below is a comparison of Postgres and competing vendors:

 Ultimately, Postgres wins against peers given:

    • It is both open-source and community-run: Unlike peer solutions, this allows organizations to adopt Postgres with virtually no upfront cost (e.g. expensive licensing fees). Moreover, developers (who understand first-hand the current needs of Postgres users) are continuing to evolve and contribute to product innovation.
  • ACID Adherence: Compared to NoSQL vendors, Postgres adheres to the ACID framework. This ensures that database transactions are processed reliably and meet a standard set of criteria.
  • Object Storage Support: Postgres supports advanced indexing capabilities and JSON queries, supporting more detailed and flexible searches beyond just SQL querying. 
  • Performance: While not mentioned in the chart above, Postgres delivers lower-latency transactions than its peers. For organizations looking for best-in-class performance, Postgres is an ideal choice.
  • Ranking: Among the top 5 databases, Postgres is the only database that is experiencing year-over-year growth in popularity, while the others are declining.

These competitive and market dynamics suggest that Postgres will continue to be adopted for more future OLTP workloads.

Considerations When Using Postgres

There are a few considerations when adopting Postgres.

Postgres was not designed for horizontal scaling 

Postgres is not natively built for horizontal scaling (i.e. being able to add more servers to the database system). This can create bottlenecks when compute demands rapidly grow across multiple regions. Vendors such as PgDog offer a solution to address this limitation, providing a routing layer on top of Postgres to enable multi-region deployment. This makes it easier for engineering teams to scale Postgres without requiring significant manual effort.

Postgres has limited analytical capabilities 

Postgres was purpose-built for transactional workloads rather than analytical ones. Many database vendors specialize by workload, as organizations often prefer using the best database for each specific task. Thus, Postgres’s focus on transactional workloads is not a limitation unique to Postgres, but a reflection of the broader trend of organizations seeking workload-specific infrastructure. There are, however, complementary analytical databases that offer native integrations with Postgres. For example, ClickHouse (a Geodesic portfolio company), a real-time analytics database, acquired PeerDB, which cuts down the time to connect data in transactional databases to analytical databases like ClickHouse. 

In-house engineering teams need to manage Postgres

Similar to other open-source projects, teams must manage the underlying infrastructure for Postgres’s compute needs. In the next section, I will share a number of companies (“Managed Postgres” vendors) that are offering managed Postgres instances, which abstract this work.

Opportunities Within the Postgres Ecosystem 

There are a few solutions and extensions within the Postgres ecosystem that expand Postgres’ adoption across a wider range of use cases, offering capabilities such as managed Postgres, horizontally scalable Postgres, vector search and observability, addressing adjacent needs not fully met by open-source Postgres alone. 

Managed Postgres

The managed Postgres ecosystem has evolved into a diverse landscape of vendors, with solutions differentiating across performance, developer experience, scalability and ability to address specific use cases. Rather than listing every vendor, this overview spotlights a few notable vendors in each category to show how the market is expanding and where companies are emerging to address various use cases.  

Among the earliest and most mature offerings, Amazon Aurora is AWS’s solution of managed Postgres, optimized for high availability and throughput. This is an attractive offering for organizations embedded in the AWS ecosystem that seek performance improvements over open-source Postgres. 

Neon (acquired by Databricks) is an up-and-coming solution that has benefited from widespread developer adoption given its proprietary method of disaggregating storage and compute. Its platform allows compute resources to scale cost effectively, which is ideal for AI workloads with unpredictable compute demands. 

Finally, Crunchy Data (acquired by Snowflake) is an attractive solution for government agencies and organizations in regulated industries, given its focus on security, compliance and reliability.

Horizontally Scalable Postgres

As previously noted, Postgres was not built for horizontal scaling, but several vendors are addressing this limitation. pgEdge is a distributed Postgres platform that lets developers run Postgres across multiple global locations at once. By keeping data closer to the end user, its solution is tailored for running latency-sensitive, globally scaled applications. 

Another vendor is PgDog– a distributed Postgres platform designed to make it easier to scale Postgres across multiple nodes and regions. It extends standard Postgres with features like built-in replication, fault tolerance, and multi-region coordination, enabling high availability and horizontal scalability without requiring complex manual setup.

Vector Storage

Given the increased demand for infrastructure for agentic AI workloads, Postgres open-source developers created pgvector, an extension on top of Postgres that enables vector similarity search. This allows developers to store and query high-dimensional embeddings (e.g. from machine learning models) directly in a Postgres database, rather than having to store the embeddings in a separate database.

Observability

Observability continues to be top of mind for developers, given application downtime can have adverse effects on organizations’ top and bottom line. pganalyze is a performance monitoring and optimization tool for Postgres. It helps developers and database teams diagnose slow queries, track index usage, and monitor system health, offering insights on Postgres query efficiency and database behavior over time.

Conclusion

Postgres continues to gain market adoption, as it is the de facto database that powers core transactional workloads within organizations and is increasingly being adopted for GenAI use cases. We at Geodesic are excited about the growing role of Postgres and the broader ecosystem around it. If you are building in this category or just enjoy chatting about database solutions, I would love to connect.