Basin

Basin SQL is a serverless, distributed SQL engine for querying Apache Iceberg tables in Basin Catalog. It now supports OFFSET, which lets you skip a specified number of rows when retrieving query results. Use LIMIT to set the page size and OFFSET to skip earlier rows. With a page size of 100, use OFFSET 0 for the first page, OFFSET 100 for the second, and increase the offset by 100 for each later…

Each Basin Pipelines stream can now ingest up to 1 GB/s, increased from 5 MB/s. The higher per-stream limit gives high-volume application events, telemetry, and logs more room to grow without splitting ingestion across streams solely to stay within the previous limit. For the full list of stream, sink, and pipeline limits, refer to Basin Pipelines limits.

Basin, formerly the Cloudflare Data Platform, is now generally available. Basin brings an end-to-end analytics platform to the Developer Platform, enabling you to collect data from a variety of sources, such as apps, infrastructure, devices, and other Cloudflare services, then query it to answer analytical questions. Basin Pipelines Basin Pipelines, formerly Cloudflare Pipelines, ingests events f…

R2 Data Catalog now provides table-level maintenance visibility and manual compaction queueing in the Cloudflare dashboard. These updates make it easier to understand when maintenance is eligible to run, inspect completed operations, and request maintenance without leaving the table view. To view table maintenance details: In the Cloudflare dashboard, go to R2 Data Catalog. Select a catalog, then…

Billing is now enabled for R2 Data Catalog on non-enterprise accounts. R2 Data Catalog usage beyond the included free tier will appear on your next invoice. R2 Data Catalog charges based on two dimensions, in addition to standard R2 storage and operations: Catalog operations: $9.00 / million operations for metadata requests such as creating tables, reading table metadata, and updating table prope…

Billing is now enabled for Cloudflare Pipelines on non-enterprise accounts. Pipelines usage beyond the included free tier will appear on your next invoice. Pipelines charges based on two usage dimensions. Ingress into a Pipeline stream remains free regardless of volume: SQL transforms: $0.04 / GB for stateless transforms (filter, reshape, unnest, cast, compute). Sinks (egress): $0.03 / GB for JSO…

Billing is now enabled for R2 SQL on non-enterprise accounts. R2 SQL usage beyond the included free tier will appear on your next invoice. R2 SQL charges based on a single dimension: Data scanned: $0.0025 / GB ($2.50 / TB) of compressed data read from R2 to execute your query. All plans include 10 GB of data scanned per month. Each query is billed for a minimum of 10 MB of data scanned. R2 SQL pr…

R2 Data Catalog now accepts read-only API tokens, so query engines and clients that only read data no longer need a read-write token. Previously, every catalog operation required an Admin Read & Write token, which granted read-only clients more access than they needed. You can now authenticate your Iceberg engine based on your workload: Read-only operations (such as listing namespaces, loading ta…

R2 Data Catalog, a managed Apache Iceberg ↗︎ catalog built into R2, now automatically optimizes manifest files as part of compaction. Manifest files track the data files that make up an Iceberg table. As a table accumulates many small or fragmented manifests, query engines must read more metadata during query planning, which slows down queries even before any data is scanned. When compaction runs…

You can now query your R2 Data Catalog tables with R2 SQL directly from the Cloudflare dashboard, without installing a CLI or wiring up a client. This makes it easy to explore your Apache Iceberg ↗︎ data, validate queries, and inspect results in one place. To get started, go to R2 Data Catalog ↗︎ in the Cloudflare dashboard and select Query data to launch the built-in SQL editor. From there you c…

R2 Data Catalog is a managed Apache Iceberg ↗︎ catalog built directly into your R2 bucket. Iceberg tracks your data through a tree of metadata files, so every insert, update, and delete must go through a catalog transaction. Manually adding, modifying, or deleting objects outside the catalog can leave pointers referencing files that no longer exist, corrupting the table into an inconsistent state…

R2 SQL now supports window functions, SELECT DISTINCT, set operations, and additional aggregates, making it easier to write analytical queries without preprocessing your data elsewhere. R2 SQL is Cloudflare's serverless, distributed SQL engine for querying Apache Iceberg ↗︎ tables stored in R2 Data Catalog. New capabilities Window functions — ROW_NUMBER, RANK, DENSE_RANK, PERCENT_RANK, CUME_DIST,…

R2 SQL now supports set operations (UNION, INTERSECT, EXCEPT) and SELECT DISTINCT, expanding the range of analytical queries you can run directly on Apache Iceberg ↗︎ tables in R2 Data Catalog. Set operations Combine the results of multiple SELECT statements: UNION — returns all rows from both queries, removing duplicates UNION ALL — returns all rows from both queries, including duplicates INTERS…

The pipeline field inside the pipelines binding configuration in your Wrangler configuration file has been renamed to stream. The old field is deprecated but still accepted. Update your configuration to use stream to avoid the deprecation warning. Before (deprecated): { "$schema": "./node_modules/wrangler/config-schema.json", "pipelines": [ { "binding": "MY_PIPELINE", "pipeline": "<STREAM_ID>" }…

R2 Data Catalog is a managed Apache Iceberg ↗︎ data catalog built directly into R2 buckets, queryable by any Iceberg-compatible engine such as Spark, Snowflake, and DuckDB. R2 Data Catalog now has published pricing for catalog operations and table compaction, in addition to standard R2 storage and operations. Billing is not yet enabled. We will provide at least 30 days notice before we start char…

R2 Data Catalog is a managed Apache Iceberg ↗︎ data catalog built directly into your R2 bucket. It exposes a standard Iceberg REST catalog interface so you can connect query engines like Spark, Snowflake, DuckDB, and R2 SQL to your data in R2. R2 Data Catalog now has a dedicated section in the Cloudflare dashboard, replacing the previous settings panel embedded in R2 bucket configuration. The new…

Cloudflare Pipelines is a streaming data platform that ingests events, transforms them with SQL, and writes to R2 as JSON, Parquet, or Apache Iceberg ↗︎ tables. Pipelines now has published pricing based on two usage dimensions: the volume of data processed by SQL transforms and the volume of data delivered to sinks. Ingress into a Pipeline stream is free. Billing is not yet enabled. We will provi…

R2 SQL is a serverless, distributed query engine that runs SQL against Apache Iceberg ↗︎ tables stored in R2 Data Catalog. R2 SQL now has published pricing based on a single dimension: the volume of compressed data scanned to execute your queries. At $2.50 / TB ($0.0025 / GB), R2 SQL is priced at half the cost of AWS Athena and less than half of Google BigQuery on-demand. Billing is not yet enabl…

R2 SQL is Cloudflare's serverless, distributed SQL engine for querying Apache Iceberg ↗︎ tables stored in R2 Data Catalog. R2 SQL runs directly on Cloudflare's global network with no infrastructure to manage, so you can analyze data in R2 without exporting it to an external warehouse. R2 SQL now supports joining multiple Iceberg tables in a single query. You can combine tables with JOINs, filter…

R2 Data Catalog is a managed Apache Iceberg data catalog built directly into your R2 bucket that allows you to connect query engines like R2 SQL, Spark, Snowflake, and DuckDB to your data in R2. You can now query analytics for your R2 Data Catalog warehouses via Cloudflare's GraphQL Analytics API. Two new datasets are available: r2CatalogDataOperationsAdaptiveGroups tracks Iceberg REST API reques…

Cloudflare Pipelines ingests streaming data via Workers or HTTP endpoints, transforms it with SQL, and writes it to R2 as Apache Iceberg tables. R2 Data Catalog manages those Iceberg tables, compaction, and compatibility with query engines like R2 SQL, Spark, and DuckDB. You can now create and manage both products using Terraform, supported in the Cloudflare Terraform provider v5.19.0 ↗︎. This ad…

R2 Data Catalog, a managed Apache Iceberg ↗︎ catalog built into R2, now removes unreferenced data files during automatic snapshot expiration. This improvement reduces storage costs and eliminates the need to run manual maintenance jobs to reclaim space from deleted data. Previously, snapshot expiration only cleaned up Iceberg metadata files such as manifests and manifest lists. Data files that we…

R2 SQL is Cloudflare's serverless, distributed, analytics query engine for querying Apache Iceberg ↗︎ tables stored in R2 Data Catalog. R2 SQL now supports functions for querying JSON data stored in Apache Iceberg tables, an easier way to parse query plans with EXPLAIN FORMAT JSON, and querying tables without partition keys stored in R2 Data Catalog. JSON functions extract and manipulate JSON val…

R2 SQL now supports an expanded SQL grammar so you can write richer analytical queries without exporting data. This release adds CASE expressions, column aliases, arithmetic in clauses, 163 scalar functions, 33 aggregate functions, EXPLAIN, Common Table Expressions (CTEs),and full struct/array/map access. R2 SQL is Cloudflare's serverless, distributed, analytics query engine for querying Apache I…

Cloudflare Pipelines ingests streaming data via Workers or HTTP endpoints, transforms it with SQL, and writes it to R2 as Apache Iceberg tables. Today we are shipping three improvements to help you understand why streaming events get dropped, catch data quality issues early, and set up Pipelines faster. Dropped event metrics When stream events don't match the expected schema, Pipelines accepts th…

R2 Data Catalog now supports automatic snapshot expiration for Apache Iceberg tables. In Apache Iceberg, a snapshot is metadata that represents the state of a table at a given point in time. Every mutation creates a new snapshot which enable powerful features like time travel queries and rollback capabilities but will accumulate over time. Without regular cleanup, these accumulated snapshots can…

R2 SQL now supports aggregation functions, GROUP BY, HAVING, along with schema discovery commands to make it easy to explore your data catalog. Aggregation Functions You can now perform aggregations on Apache Iceberg tables in R2 Data Catalog using standard SQL functions including COUNT(*), SUM(), AVG(), MIN(), and MAX(). Combine these with GROUP BY to analyze data across dimensions, and use HAVI…

You can now enable compaction for individual Apache Iceberg ↗︎ tables in R2 Data Catalog, giving you fine-grained control over different workloads. # Enable compaction for a specific table (no token required) npx wrangler r2 bucket catalog compaction enable <BUCKET> <NAMESPACE> <TABLE> --target-size 256 This allows you to: Apply different target file sizes per table Disable compaction for specifi…

Today, we are launching the new Cloudflare Pipelines: a streaming data platform that ingests events, transforms them with SQL, and writes to R2 as Apache Iceberg ↗︎ tables or Parquet files. Pipelines can receive events via HTTP endpoints or Worker bindings, transform them with SQL, and deliver to R2 with exactly-once guarantees. This makes it easy to build analytics-ready warehouses for server lo…

Today, we are launching the open beta for R2 SQL: A serverless, distributed query engine that can efficiently analyze petabytes of data in Apache Iceberg ↗︎ tables managed by R2 Data Catalog. R2 SQL is ideal for exploring analytical and time-series data stored in R2, such as logs, events from Pipelines, or clickstream and user behavior data. If you already have a table in R2 Data Catalog, running…

You can now enable automatic compaction for Apache Iceberg ↗︎ tables in R2 Data Catalog to improve query performance. Compaction is the process of taking a group of small files and combining them into fewer larger files. This is an important maintenance operation as it helps ensure that query performance remains consistent by reducing the number of files that needs to be scanned. To enable automa…

Today, we are launching R2 Data Catalog in open beta, a managed Apache Iceberg catalog built directly into your Cloudflare R2 bucket. If you are not already familiar with it, Apache Iceberg ↗︎ is an open table format designed to handle large-scale analytics datasets stored in object storage, offering ACID transactions and schema evolution. R2 Data Catalog exposes a standard Iceberg REST catalog i…

Cloudflare Pipelines is now available in beta, to all users with a Workers Paid plan. Pipelines let you ingest high volumes of real time data, without managing the underlying infrastructure. A single pipeline can ingest up to 100 MB of data per second, via HTTP or from a Worker. Ingested data is automatically batched, written to output files, and delivered to an R2 bucket in your account. You can…