DatabricksCostMonitoringandOptimization
Identify your most expensive Databricks workloads in minutes — and take action.
Dashboard
Welcome back. Here's your Databricks cost overview.
Last updated 2m ago
Retail prices
-5.3%Retail prices
-3.8%Retail prices
-7.1%Connected
analytics-prod
ml-training
etl-pipeline
data-science
reporting
JOB
UI
Azure Data Factory
PIPELINE
| Workspace | Activity / Cluster name | Source | Date | Duration | VM cost | DBU cost | Total cost |
|---|---|---|---|---|---|---|---|
| analytics-prod | daily_aggregation_pipeline | JOB | 2026-08-08 | 3h 27m | $1,245.30 | $1,487.20 | $2,732.50 |
| ml-training | model_training_xgboost | JOB | 2026-08-07 | 5h 7m | $980.40 | $1,156.80 | $2,137.20 |
| analytics-prod | customer_segmentation | Azure Data Factory | 2026-08-08 | 2h 47m | $720.15 | $892.30 | $1,612.45 |
| etl-pipeline | incremental_load_events | JOB | 2026-08-08 | 1h 55m | $485.60 | $612.40 | $1,098.00 |
| data-science | feature_store_refresh | JOB | 2026-08-07 | 1h 21m | $310.20 | $398.50 | $708.70 |
| ml-training | experiment_tracking | Databricks UI | 2026-08-08 | 4h 15m | $280.10 | $345.60 | $625.70 |
| analytics-prod | data_quality_checks | JOB | 2026-08-08 | 52m | $195.40 | $248.30 | $443.70 |
| reporting | weekly_exec_dashboard | Azure Data Factory | 2026-08-05 | 37m | $142.80 | $178.90 | $321.70 |
| etl-pipeline | schema_evolution_check | PIPELINE | 2026-08-07 | 27m | $98.50 | $124.30 | $222.80 |
| data-science | notebook_exploration | Databricks UI | 2026-08-08 | 2h 6m | $85.20 | $102.40 | $187.60 |
The problem
Databricks costs lack visibility
An incomplete cost picture
Databricks reports DBU consumption, but the underlying VM infrastructure — billed separately by Azure, AWS, or GCP — represents a significant share of the total cost. Without combining both, you're missing part of the picture.
Building cost tracking is a project in itself
Querying billing tables, joining cluster events with cloud pricing APIs, computing per-run costs across workspaces... it's a full engineering effort that takes weeks to build, and ongoing work to maintain.
Lakesight solves both. Get a complete cost breakdown — VM infrastructure and DBU combined — down to every job run, ready in minutes with no development effort.
Solution
Purpose-built for Databricks
Connect workspaces and get full cost visibility in minutes — down to every job, run, cluster, and tag.
Plug & Play Setup
Connect any Databricks workspace with just a URL and a PAT token or OAuth service principal.
Real-time monitoring
Monitor jobs and UI clusters actively consuming resources. Catch runaway workloads early before costs accumulate.
Multi-workspace support
Manage all your Databricks workspaces from a single dashboard. Per-workspace cost breakdowns, ingestion schedules, and team access — all in one place.
Per-run cost breakdown
Costs computed per job run and per cluster, based on actual cluster events. Drill into any job to see cost trends over time and evaluate the impact of switching node types.
How it works
Up and running in under 5 minutes
1. Connect your workspace
Provide a Databricks workspace URL and authenticate with a Personal Access Token (PAT) or an OAuth M2M service principal.
2. Data is ingested automatically
Job runs, cluster events, and compute usage are fetched every 5 minutes. No manual exports, no pipelines to maintain, no code to write.
3. Explore your cost breakdown
See all your costs in one place — by job, cluster, workspace, or custom tag. Drill down from workspace to job to individual run in a few clicks.
Lakesight only performs read operations. It doesn't require access to your cloud subscription, IAM roles, or any infrastructure changes.
Features
One place to explore all Databricks costs
Cost Breakdown
Break down Databricks costs by node type, billing type, Photon usage, or custom tags across all your workspaces.
Run History
Inspect every job run with per-cluster cost detail — VM cost, DBU cost, and duration. Drill down from job to individual run.
Job X — executions analysis over time
Benchmark instance types and keep the most cost-effective
Running Jobs & UI Clusters
Monitor running jobs and UI clusters in real time before costs accumulate unexpectedly.
UI Cluster History
Track interactive cluster session costs separately from automated job costs, with full session detail.
Analyze a Job
Drill into any job to see cost trends over time, task-level breakdown, and evaluate the impact of switching node types.
Cost Breakdown
Break down Databricks costs by node type, billing type, Photon usage, or custom tags across all your workspaces.
Run History
Inspect every job run with per-cluster cost detail — VM cost, DBU cost, and duration. Drill down from job to individual run.
Job X — executions analysis over time
Benchmark instance types and keep the most cost-effective
Analyze a Job
Drill into any job to see cost trends over time, task-level breakdown, and evaluate the impact of switching node types.
Running Jobs & UI Clusters
Monitor running jobs and UI clusters in real time before costs accumulate unexpectedly.
UI Cluster History
Track interactive cluster session costs separately from automated job costs, with full session detail.
Alerts & Notifications
Automated alerts and scheduled reports
Set failure and duration alerts for all jobs in a few clicks — including submitted runs from Azure Data Factory, Airflow, or any external orchestrator that Databricks can't alert on natively.
Alert Rules
Configure alerts for job run failures and abnormal durations. Email notifications are sent as soon as something goes wrong — before it impacts the pipeline.
Scheduled Cost Reports
Daily, weekly, or monthly cost reports delivered by email. Break down costs by job or custom tag — on a fully configurable schedule.