Skip to main content

​
Summary

This release introduces major new capabilities for tenant operations, security, and auditing, along with important performance enhancements, guardrails, and stability improvements across Apache Pinot and StarTree components.

​
🚀 New Features

​
Tenant Rebalance

You can now rebalance resources at the tenant level.
  • Applies automatically to all tables within a tenant.
  • Internally orchestrates table-level rebalances to provide a seamless experience.
  • Helps optimize resource distribution across large multi-tenant clusters.
📖 Read the documentation

​
Row-level RBAC

Row-level access control (RLAC) enables fine-grained data security.
  • Define row-level filters using SQL expressions.
  • Access is restricted to roles authorized for the corresponding filter.
  • Enforce per-user or per-team visibility rules without duplicating datasets.
📖 Read the documentation

​
Pinot Audit Logs

Pinot now supports auditing of cluster operations.
  • Capture audit events from Controller and Broker components.
  • Persist logs in Pinot’s deep store for long-term archival.
  • Configurable options for logging headers and payloads to meet compliance and debugging needs.
📖 Read the documentation

​
⚡ Improvements

​
Text Index Pinning for Tiered Storage

Users can now configure a text index when using cloud tiered storage and be able to pin it locally for accelerating query performance. Read full documentation here: https://docs.startree.ai/corecapabilities/manage-data/set-up-tiered-storage/use-text-index

​
Table Deletion & Creation

Enhanced validations and cleanup improve operational reliability:
  • Deletion:
    • Automatically removes Minion tasks that are not IN_PROGRESS.
    • Fails gracefully with clear error messages if tasks are still running.
  • Creation:
    • Detects dangling task data from previous tables.
    • Provides actionable error messages guiding users to retry deletion or manually clean up.

​
Table Rebalance Manager

  • Introduces a single manager to handle rebalance jobs across the cluster.
  • Prevents multiple jobs from running simultaneously on the same table.
  • Tracks and reports progress of ongoing rebalances.

​
Pinot Server Restart Speedup

Server restart time is now 3x faster or more:
  • Reuses on-disk upsert metadata instead of rebuilding from snapshots.
  • Typical restart time is now minutes per server, even with billions of primary keys.
  • Improves operational ease for StarTree Upserts.
  • Note: Requires clean shutdown. In case of unclean shutdown, servers fall back to snapshots.

​
🛠 Bug Fixes & Stability

​
Pinot Guardrails

New safeguards to protect clusters against ingestion and query instability:
  • Block ingestion if primary key count exceeds thresholds.
  • Validate conflicting indexing configurations that can cause ingestion failures.
  • Validate multi-value column compatibility.
  • Add throttling to prevent CPU spikes during ingestion or compaction.
  • Prevent index build failures caused by malformed or bad data.

​
Stability Improvements

  • Increased frequency of Upsert snapshot tasks for faster recovery.
  • Improved filesystem (FS) call resilience: timeouts and retries added for GCS, ADLS, and S3.
  • Changed default max length strategy: no more silent string truncation (previously caused ingestion failures with JSON columns).