Databricks Community watch · 536tech Databricks

Platform and governance posts from the Databricks Community Technical Blog, filtered and accumulated. Community posts are practitioner guidance, not product documentation; verify product claims against the linked official docs before relying on them.

uv run refresh.py community reads https://community.databricks.com/tmcxu86974/rss/board?board.id=technical-blog and appends new relevant posts.

Do not edit.

Added 2026-08-18#

Triggered vs. Continuous Mode: A Deep Dive into Serverless Lakeflow Spark Declarative Pipelines — Aug 10, 2026 One pipeline woke up every five minutes. The other stayed running. Using the same e-commerce streaming workload, median end-to-end data freshness improved from 4.0 minutes to 33 seconds, while P95 latency dropped from 9.3 minutes to just 60 seconds. Those resu
How to Build, Test, and Ship SQL Pipelines on Databricks — Aug 10, 2026 Spark Declarative Pipelines (SDP) give SQL developers a first-class ETL experience. It is purpose-built for SQL, and it handles batch as well as streaming. There is now a full dev loop for SQL pipelines: you can either author in the new Databricks pipelines ed
Building a Production LangGraph Agent on Databricks - NorthStar Brand Copilot — Aug 10, 2026 We built NorthStar Brand Copilot, an AI assistant for brand managers and field-sales reps at a fictional multi-category CPG company (Snacks, Beverages, Personal Care). It's a LangGraph custom agent that routes each question to the right Databricks-native capab
Building Custom Apps on Lakehouse — Aug 07, 2026
Plug & Play: Zerobus Ingest Now Supports Apache Kafka® Compatible APIs (Beta) — Aug 07, 2026 Learn how to use your existing Apache Kafka producer clients to ingest data directly into your Delta lakehouse with new beta support for Kafka-compatible APIs.
Load Testing Databricks SQL Warehouses with JMeter — Part 3: Running and Analyzing Results — Aug 05, 2026
Load Testing Databricks SQL Warehouses with JMeter — Part 2: Concurrency and the Test Plan — Aug 05, 2026
Load Testing Databricks SQL Warehouses with JMeter — Part 1: Inputs and Configuration — Aug 05, 2026
Try the new Zerobus Rescue Column: Choose-Your-Own-Adventure Schema Management (Beta) — Jul 29, 2026 It is Friday afternoon. An upstream team ships a harmless-looking change: they add a region field to the events your service already emits, and somewhere along the way, a sensor starts reporting temperature as "72F" instead of 72. You did not write that produc
From Experiment to Prod: LakeFlow Spark Declarative Pipelines Testing Blueprint — Jul 28, 2026 Part 2 of the testing blueprint: how to test SDP pipelines end-to-end. Decouple your logic for fast unit tests, then use pytest, the Databricks SDK, and Databricks Connect to run the real pipeline against ephemeral, isolated Unity Catalog schemas, all from you
Serve Tableau Reports Directly from Databricks SQL — Jul 28, 2026 Connect Tableau to Databricks SQL to eliminate data warehouse sync pipelines, cut compute costs, and serve dashboards directly from your Lakehouse.
When Milliseconds Matter: Databricks Powers the Next Generation of Operational Workloads — Jul 27, 2026 Databricks redefined real-time data processing with Real-Time Mode (RTM) in Spark Structured Streaming, hitting sub-300ms P99 latency for mission-critical operational workloads. Lakeflow Spark Declarative Pipelines let both pro-coders and SQL/Python developers
One Pipeline, Any Destination: The ForEachBatch Sink in Spark Declarative Pipelines is now GA — Jul 24, 2026 A Spark Declarative Pipeline used to stop at the edge of the lakehouse. Streaming to a system with no first-party connector meant bolting a standalone Structured Streaming job onto the declarative one, and operating a second framework to do it. The ForEachBatc
Azure DevOps with Databricks DABs: Step-by-Step Integration Guide — Apr 16, 2026 A practical CI/CD walkthrough using Declarative Automation Bundles, Databricks Git folders, Azure DevOps service connections, staged deployment, and production approvals.