{"id":2002,"date":"2024-06-28T09:26:48","date_gmt":"2024-06-28T09:26:48","guid":{"rendered":"https:\/\/www.w3computing.com\/articles\/?p=2002"},"modified":"2024-06-28T09:26:54","modified_gmt":"2024-06-28T09:26:54","slug":"how-to-use-postgresqls-brin-indexes-for-large-tables","status":"publish","type":"post","link":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/","title":{"rendered":"How to Use PostgreSQL&#8217;s BRIN Indexes for Large Tables"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Introduction<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">PostgreSQL is a powerful, open-source object-relational database system known for its extensibility and standards compliance. One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases. Among these, the Block Range INdex (BRIN) is particularly useful for large tables where conventional B-tree indexes may not be efficient.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In this tutorial, we will dive deep into BRIN indexes, understanding their structure, use cases, advantages, and how to implement them effectively. This tutorial assumes that you have a basic understanding of PostgreSQL and indexing concepts.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is a BRIN Index?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">BRIN indexes, introduced in PostgreSQL 9.5, are designed for handling large tables where the data is naturally ordered. BRIN stands for Block Range INdexes. Unlike B-tree indexes, which index individual rows, BRIN indexes summarize a range of blocks of the table. This summarization makes BRIN indexes much smaller and more efficient for certain types of queries, particularly those that involve scanning large portions of a table.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How BRIN Indexes Work<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">BRIN indexes work by summarizing the data stored in consecutive blocks of a table. A block, also known as a page, is the smallest unit of data storage in PostgreSQL, typically 8KB in size. A BRIN index stores the minimum and maximum values for a column (or columns) for each block range.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When a query is executed, PostgreSQL uses the BRIN index to quickly eliminate blocks that cannot contain the desired rows, significantly reducing the amount of data that needs to be scanned.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Advantages of BRIN Indexes<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Space Efficiency<\/strong>: BRIN indexes are much smaller than B-tree indexes because they store summaries of block ranges rather than individual row pointers.<\/li>\n\n\n\n<li><strong>Performance on Large Tables<\/strong>: For very large tables, BRIN indexes can be more efficient than B-tree indexes, especially when the data is naturally ordered.<\/li>\n\n\n\n<li><strong>Maintenance<\/strong>: BRIN indexes require less maintenance and are faster to create than B-tree indexes, which can be beneficial for large, write-heavy tables.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">When to Use BRIN Indexes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">BRIN indexes are particularly useful in scenarios where:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Large Tables<\/strong>: The table contains a very large number of rows.<\/li>\n\n\n\n<li><strong>Naturally Ordered Data<\/strong>: The data is inserted in a natural order, such as timestamps in a log table or sequential IDs.<\/li>\n\n\n\n<li><strong>Range Queries<\/strong>: Queries frequently involve scanning ranges of data, such as time ranges or sequential numeric ranges.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">However, BRIN indexes are not ideal for:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Highly Random Data<\/strong>: If the data is not ordered or is highly random, BRIN indexes may not provide significant performance benefits.<\/li>\n\n\n\n<li><strong>Point Queries<\/strong>: For queries that frequently search for individual rows, B-tree indexes may be more efficient.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Creating BRIN Indexes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Creating a BRIN index in PostgreSQL is straightforward. The syntax is similar to creating other types of indexes but with the addition of the <code>USING BRIN<\/code> clause.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Basic Syntax<\/h3>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> index_name <span class=\"hljs-keyword\">ON<\/span> table_name <span class=\"hljs-keyword\">USING<\/span> BRIN (column_name);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Example<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Consider a table <code>logs<\/code> with a timestamp column <code>created_at<\/code>. We can create a BRIN index on the <code>created_at<\/code> column as follows:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> brin_logs_created_at_idx <span class=\"hljs-keyword\">ON<\/span> <span class=\"hljs-keyword\">logs<\/span> <span class=\"hljs-keyword\">USING<\/span> BRIN (created_at);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Specifying Page Ranges<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">By default, PostgreSQL uses a page range size of 128 blocks (1MB, assuming an 8KB block size) for BRIN indexes. You can specify a different page range size using the <code>WITH<\/code> clause:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-3\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> brin_logs_created_at_idx <span class=\"hljs-keyword\">ON<\/span> <span class=\"hljs-keyword\">logs<\/span> <span class=\"hljs-keyword\">USING<\/span> BRIN (created_at) <span class=\"hljs-keyword\">WITH<\/span> (pages_per_range = <span class=\"hljs-number\">16<\/span>);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-3\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">A smaller <code>pages_per_range<\/code> can make the index more precise but larger in size.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Using BRIN Indexes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Once a BRIN index is created, PostgreSQL will automatically use it to optimize queries that can benefit from it. Let&#8217;s look at some examples of how queries can take advantage of BRIN indexes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Range Queries<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Consider the following query that retrieves logs created within a specific time range:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-4\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">SELECT<\/span> * <span class=\"hljs-keyword\">FROM<\/span> <span class=\"hljs-keyword\">logs<\/span> <span class=\"hljs-keyword\">WHERE<\/span> created_at <span class=\"hljs-keyword\">BETWEEN<\/span> <span class=\"hljs-string\">'2023-01-01'<\/span> <span class=\"hljs-keyword\">AND<\/span> <span class=\"hljs-string\">'2023-01-31'<\/span>;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-4\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">With a BRIN index on the <code>created_at<\/code> column, PostgreSQL can quickly identify the block ranges that contain the desired rows and skip over irrelevant blocks, significantly reducing the amount of data scanned.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Combining BRIN and B-tree Indexes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">In some cases, combining BRIN and B-tree indexes can provide the best of both worlds. For example, if you frequently query a large table by a range of timestamps and also perform point lookups by a different column, you can create both BRIN and B-tree indexes:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-5\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> brin_logs_created_at_idx <span class=\"hljs-keyword\">ON<\/span> <span class=\"hljs-keyword\">logs<\/span> <span class=\"hljs-keyword\">USING<\/span> BRIN (created_at);\n<span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> btree_logs_user_id_idx <span class=\"hljs-keyword\">ON<\/span> <span class=\"hljs-keyword\">logs<\/span> (user_id);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-5\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">This way, range queries on <code>created_at<\/code> benefit from the BRIN index, while point lookups on <code>user_id<\/code> benefit from the B-tree index.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Maintenance of BRIN Indexes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">BRIN indexes require less maintenance compared to B-tree indexes, but there are still some maintenance tasks to consider.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Reindexing<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">As with any index, it&#8217;s a good practice to periodically reindex BRIN indexes to ensure optimal performance. This can be done using the <code>REINDEX<\/code> command:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-6\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\">REINDEX INDEX brin_logs_created_at_idx;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-6\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Vacuuming<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Regularly vacuuming your tables is important to keep BRIN indexes efficient. Vacuuming helps to remove dead tuples and update the index summaries. You can use the <code>VACUUM<\/code> command to vacuum a table:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-7\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\">VACUUM <span class=\"hljs-keyword\">ANALYZE<\/span> <span class=\"hljs-keyword\">logs<\/span>;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-7\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Monitoring BRIN Indexes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">PostgreSQL provides several ways to monitor the effectiveness of BRIN indexes.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Using <code>pg_stat_user_indexes<\/code><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>pg_stat_user_indexes<\/code> view provides statistics about index usage. You can query this view to see how often your BRIN index is being used:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-8\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">SELECT<\/span> indexrelname, idx_scan\n<span class=\"hljs-keyword\">FROM<\/span> pg_stat_user_indexes\n<span class=\"hljs-keyword\">WHERE<\/span> indexrelname = <span class=\"hljs-string\">'brin_logs_created_at_idx'<\/span>;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-8\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h4 class=\"wp-block-heading\">Using <code>EXPLAIN<\/code><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">The <code>EXPLAIN<\/code> command shows the execution plan of a query, including whether the BRIN index is being used. For example:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-9\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">EXPLAIN<\/span> <span class=\"hljs-keyword\">SELECT<\/span> * <span class=\"hljs-keyword\">FROM<\/span> <span class=\"hljs-keyword\">logs<\/span> <span class=\"hljs-keyword\">WHERE<\/span> created_at <span class=\"hljs-keyword\">BETWEEN<\/span> <span class=\"hljs-string\">'2023-01-01'<\/span> <span class=\"hljs-keyword\">AND<\/span> <span class=\"hljs-string\">'2023-01-31'<\/span>;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-9\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">If the BRIN index is being used, you will see it in the output.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Advanced BRIN Index Features<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Multi-Column BRIN Indexes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">BRIN indexes can be created on multiple columns, which is useful when you frequently query by ranges on multiple columns. For example, if you have a table <code>sales<\/code> with columns <code>sale_date<\/code> and <code>amount<\/code>, you can create a BRIN index on both columns:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-10\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> brin_sales_idx <span class=\"hljs-keyword\">ON<\/span> sales <span class=\"hljs-keyword\">USING<\/span> BRIN (sale_date, amount);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-10\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Including Non-Summarized Columns<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You can include additional columns in a BRIN index without summarizing them. This can be useful if you want the index to cover more columns for query performance but only need summaries for certain columns:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-11\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> brin_sales_partial_idx <span class=\"hljs-keyword\">ON<\/span> sales <span class=\"hljs-keyword\">USING<\/span> BRIN (sale_date) <span class=\"hljs-keyword\">INCLUDE<\/span> (amount);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-11\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Operator Classes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">BRIN indexes support different operator classes that define how the summaries are created. For example, the <code>minmax<\/code> operator class (the default) stores the minimum and maximum values. Other operator classes include <code>inclusion<\/code> for range types and <code>bloom<\/code> for approximate indexing.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Specifying Operator Classes<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">You can specify the operator class when creating a BRIN index:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-12\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> brin_logs_inclusion_idx <span class=\"hljs-keyword\">ON<\/span> <span class=\"hljs-keyword\">logs<\/span> <span class=\"hljs-keyword\">USING<\/span> BRIN (created_at inclusion_ops);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-12\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Custom BRIN Operator Classes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">PostgreSQL allows you to create custom BRIN operator classes for specialized use cases. This is an advanced topic that involves defining your own summarization functions and strategies.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Performance Considerations<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Choosing the Right Page Range Size<\/strong> &#8211; The <code>pages_per_range<\/code> parameter is crucial for BRIN index performance. A smaller <code>pages_per_range<\/code> makes the index more precise but larger. Conversely, a larger <code>pages_per_range<\/code> makes the index smaller but less precise.\n<ul class=\"wp-block-list\">\n<li><strong>Experimentation<\/strong> &#8211; It&#8217;s often necessary to experiment with different <code>pages_per_range<\/code> values to find the optimal balance for your workload. You can create multiple BRIN indexes with different page range sizes and compare their performance.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong>Impact of Data Distribution<\/strong> &#8211; The effectiveness of BRIN indexes depends on the data distribution. If the data is not naturally ordered, the summaries may not be effective, and the index may not provide significant performance benefits.<\/li>\n\n\n\n<li><strong>Impact on Write Performance<\/strong> &#8211; BRIN indexes have a minimal impact on write performance compared to B-tree indexes. However, it&#8217;s still important to monitor the impact, especially on highly write-heavy tables.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Case Study: Using BRIN Indexes in a Real-World Scenario<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s walk through a real-world scenario where BRIN indexes can significantly improve performance.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Scenario<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose you have a table <code>sensor_data<\/code> that records data from IoT sensors. The table has the following structure:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-13\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">TABLE<\/span> sensor_data (\n    <span class=\"hljs-keyword\">id<\/span> <span class=\"hljs-built_in\">SERIAL<\/span> PRIMARY <span class=\"hljs-keyword\">KEY<\/span>,\n    sensor_id <span class=\"hljs-built_in\">INT<\/span>,\n    recorded_at <span class=\"hljs-built_in\">TIMESTAMP<\/span>,\n    <span class=\"hljs-keyword\">value<\/span> <span class=\"hljs-built_in\">FLOAT<\/span>\n);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-13\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">The table contains billions of rows, and new data is continuously inserted in chronological order.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Query Pattern<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You frequently run queries to analyze data for specific time ranges and specific sensors:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-14\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">SELECT<\/span> sensor_id, <span class=\"hljs-keyword\">AVG<\/span>(<span class=\"hljs-keyword\">value<\/span>) \n<span class=\"hljs-keyword\">FROM<\/span> sensor_data\n<span class=\"hljs-keyword\">WHERE<\/span> recorded_at <span class=\"hljs-keyword\">BETWEEN<\/span> <span class=\"hljs-string\">'2023-01-01'<\/span> <span class=\"hljs-keyword\">AND<\/span> <span class=\"hljs-string\">'2023-01-31'<\/span> \n<span class=\"hljs-keyword\">GROUP<\/span> <span class=\"hljs-keyword\">BY<\/span> sensor_id;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-14\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Solution<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Create a BRIN Index on <code>recorded_at<\/code><\/strong>:<\/li>\n<\/ol>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-15\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">CREATE<\/span> <span class=\"hljs-keyword\">INDEX<\/span> brin_sensor_data_recorded_at_idx <span class=\"hljs-keyword\">ON<\/span> sensor_data <span class=\"hljs-keyword\">USING<\/span> BRIN (recorded_at);<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-15\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<ol class=\"wp-block-list\" start=\"2\">\n<li><strong>Create a B-tree Index on <code>sensor_id<\/code><\/strong>:<\/li>\n<\/ol>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">CREATE INDEX btree_sensor_data_sensor_id_idx ON sensor_data (sensor_id);<\/code><\/span><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Performance Analysis<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Without BRIN Index<\/h4>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-16\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">EXPLAIN<\/span> <span class=\"hljs-keyword\">ANALYZE<\/span>\n<span class=\"hljs-keyword\">SELECT<\/span> sensor_id, <span class=\"hljs-keyword\">AVG<\/span>(<span class=\"hljs-keyword\">value<\/span>) \n<span class=\"hljs-keyword\">FROM<\/span> sensor_data \n<span class=\"hljs-keyword\">WHERE<\/span> recorded_at <span class=\"hljs-keyword\">BETWEEN<\/span> <span class=\"hljs-string\">'2023-01-01'<\/span> <span class=\"hljs-keyword\">AND<\/span> <span class=\"hljs-string\">'2023-01-31'<\/span> \n<span class=\"hljs-keyword\">GROUP<\/span> <span class=\"hljs-keyword\">BY<\/span> sensor_id;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-16\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<ul class=\"wp-block-list\">\n<li>The query may result in a full table scan, which is expensive for large tables.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">With BRIN Index<\/h4>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-17\" data-shcb-language-name=\"SQL (Structured Query Language)\" data-shcb-language-slug=\"sql\"><span><code class=\"hljs language-sql\"><span class=\"hljs-keyword\">EXPLAIN<\/span> <span class=\"hljs-keyword\">ANALYZE<\/span>\n<span class=\"hljs-keyword\">SELECT<\/span> sensor_id, <span class=\"hljs-keyword\">AVG<\/span>(<span class=\"hljs-keyword\">value<\/span>) \n<span class=\"hljs-keyword\">FROM<\/span> sensor_data \n<span class=\"hljs-keyword\">WHERE<\/span> recorded_at <span class=\"hljs-keyword\">BETWEEN<\/span> <span class=\"hljs-string\">'2023-01-01'<\/span> <span class=\"hljs-keyword\">AND<\/span> <span class=\"hljs-string\">'2023-01-31'<\/span> \n<span class=\"hljs-keyword\">GROUP<\/span> <span class=\"hljs-keyword\">BY<\/span> sensor_id;<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-17\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">SQL (Structured Query Language)<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">sql<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<ul class=\"wp-block-list\">\n<li>The query planner uses the BRIN index to quickly eliminate irrelevant blocks, significantly reducing the amount of data scanned.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Results<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">By adding the BRIN index, the query performance improves dramatically, making it feasible to analyze large data ranges efficiently.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">BRIN indexes are a powerful tool in PostgreSQL for optimizing queries on large tables with naturally ordered data. They offer significant advantages in terms of space efficiency and performance for range queries. By understanding how BRIN indexes work and how to implement and maintain them effectively, you can leverage their benefits to enhance the performance of your PostgreSQL database.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction PostgreSQL is a powerful, open-source object-relational database system known for its extensibility and standards compliance. One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases. Among these, the Block Range INdex (BRIN) is particularly useful for large tables where conventional [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[14,22],"tags":[],"class_list":["post-2002","post","type-post","status-publish","format-standard","category-database-development","category-postgresql","entry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>How to Use PostgreSQL&#039;s BRIN Indexes for Large Tables<\/title>\n<meta name=\"description\" content=\"One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Use PostgreSQL&#039;s BRIN Indexes for Large Tables\" \/>\n<meta property=\"og:description\" content=\"One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-06-28T09:26:48+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-28T09:26:54+00:00\" \/>\n<meta name=\"author\" content=\"w3compadmin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"w3compadmin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"TechArticle\",\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/\"},\"author\":{\"name\":\"w3compadmin\",\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/#\\\/schema\\\/person\\\/a550b3e20d78bb4f79b7c6b7b53f0561\"},\"headline\":\"How to Use PostgreSQL&#8217;s BRIN Indexes for Large Tables\",\"datePublished\":\"2024-06-28T09:26:48+00:00\",\"dateModified\":\"2024-06-28T09:26:54+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/\"},\"wordCount\":1352,\"articleSection\":[\"Database Development\",\"PostgreSQL\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/\",\"url\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/\",\"name\":\"How to Use PostgreSQL's BRIN Indexes for Large Tables\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/#website\"},\"datePublished\":\"2024-06-28T09:26:48+00:00\",\"dateModified\":\"2024-06-28T09:26:54+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/#\\\/schema\\\/person\\\/a550b3e20d78bb4f79b7c6b7b53f0561\"},\"description\":\"One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/how-to-use-postgresqls-brin-indexes-for-large-tables\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Articles Home\",\"item\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Database Development\",\"item\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/database-development\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"How to Use PostgreSQL&#8217;s BRIN Indexes for Large Tables\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/#website\",\"url\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/\",\"name\":\"Developer Articles Hub\",\"description\":\"\",\"alternateName\":\"Developer Articles\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/#\\\/schema\\\/person\\\/a550b3e20d78bb4f79b7c6b7b53f0561\",\"name\":\"w3compadmin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/wp-content\\\/litespeed\\\/avatar\\\/bd481d404e42caa2763662a3bfe825f8.jpg?ver=1780141266\",\"url\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/wp-content\\\/litespeed\\\/avatar\\\/bd481d404e42caa2763662a3bfe825f8.jpg?ver=1780141266\",\"contentUrl\":\"https:\\\/\\\/www.w3computing.com\\\/articles\\\/wp-content\\\/litespeed\\\/avatar\\\/bd481d404e42caa2763662a3bfe825f8.jpg?ver=1780141266\",\"caption\":\"w3compadmin\"},\"sameAs\":[\"http:\\\/\\\/w3computing.com\\\/articles\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to Use PostgreSQL's BRIN Indexes for Large Tables","description":"One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/","og_locale":"en_US","og_type":"article","og_title":"How to Use PostgreSQL's BRIN Indexes for Large Tables","og_description":"One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases.","og_url":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/","article_published_time":"2024-06-28T09:26:48+00:00","article_modified_time":"2024-06-28T09:26:54+00:00","author":"w3compadmin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"w3compadmin","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"TechArticle","@id":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/#article","isPartOf":{"@id":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/"},"author":{"name":"w3compadmin","@id":"https:\/\/www.w3computing.com\/articles\/#\/schema\/person\/a550b3e20d78bb4f79b7c6b7b53f0561"},"headline":"How to Use PostgreSQL&#8217;s BRIN Indexes for Large Tables","datePublished":"2024-06-28T09:26:48+00:00","dateModified":"2024-06-28T09:26:54+00:00","mainEntityOfPage":{"@id":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/"},"wordCount":1352,"articleSection":["Database Development","PostgreSQL"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/","url":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/","name":"How to Use PostgreSQL's BRIN Indexes for Large Tables","isPartOf":{"@id":"https:\/\/www.w3computing.com\/articles\/#website"},"datePublished":"2024-06-28T09:26:48+00:00","dateModified":"2024-06-28T09:26:54+00:00","author":{"@id":"https:\/\/www.w3computing.com\/articles\/#\/schema\/person\/a550b3e20d78bb4f79b7c6b7b53f0561"},"description":"One of the features that make PostgreSQL stand out is its variety of index types that can optimize query performance for different use cases.","breadcrumb":{"@id":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.w3computing.com\/articles\/how-to-use-postgresqls-brin-indexes-for-large-tables\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Articles Home","item":"https:\/\/www.w3computing.com\/articles\/"},{"@type":"ListItem","position":2,"name":"Database Development","item":"https:\/\/www.w3computing.com\/articles\/database-development\/"},{"@type":"ListItem","position":3,"name":"How to Use PostgreSQL&#8217;s BRIN Indexes for Large Tables"}]},{"@type":"WebSite","@id":"https:\/\/www.w3computing.com\/articles\/#website","url":"https:\/\/www.w3computing.com\/articles\/","name":"Developer Articles Hub","description":"","alternateName":"Developer Articles","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.w3computing.com\/articles\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.w3computing.com\/articles\/#\/schema\/person\/a550b3e20d78bb4f79b7c6b7b53f0561","name":"w3compadmin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.w3computing.com\/articles\/wp-content\/litespeed\/avatar\/bd481d404e42caa2763662a3bfe825f8.jpg?ver=1780141266","url":"https:\/\/www.w3computing.com\/articles\/wp-content\/litespeed\/avatar\/bd481d404e42caa2763662a3bfe825f8.jpg?ver=1780141266","contentUrl":"https:\/\/www.w3computing.com\/articles\/wp-content\/litespeed\/avatar\/bd481d404e42caa2763662a3bfe825f8.jpg?ver=1780141266","caption":"w3compadmin"},"sameAs":["http:\/\/w3computing.com\/articles"]}]}},"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":"w3compadmin","author_link":"https:\/\/www.w3computing.com\/articles\/author\/w3compadmin\/"},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/posts\/2002","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/comments?post=2002"}],"version-history":[{"count":1,"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/posts\/2002\/revisions"}],"predecessor-version":[{"id":2003,"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/posts\/2002\/revisions\/2003"}],"wp:attachment":[{"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/media?parent=2002"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/categories?post=2002"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.w3computing.com\/articles\/wp-json\/wp\/v2\/tags?post=2002"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}