Databricks Use Case Logistik
Databricks Use Case Logistik

At a glance

  • Industry

    Transport & Logistics (International)

  • Division

    Air & Sea, Data Platform Department

  • Platform

    Azure Databricks, Delta Lake, Power BI 

  • Results

    3× more frequent data refreshes, 90% less security data

Initial Situation

The client is a global transport and logistics company whose Air & Sea division relies on data for day-to-day operations and informed business decisions. Thousands of employees across the organisation use Power BI reports to monitor and analyse shipments, operational performance metrics and commercial KPIs.

The existing data platform was built on Azure SQL Server. All data transformations were executed through stored procedures and orchestrated using Azure Data Factory. The processed data was then loaded into semantic models in Azure Analysis Services, which served as the foundation for the Power BI reports.

The architecture had evolved organically over several years, becoming increasingly complex and difficult to maintain. At the same time, the existing platform could no longer adequately meet the division's growing demands for data freshness, reliability and availability.

The challenges

  • Long processing times: Only one data load per day was possible, meaning data was often up to 24 hours old.

  • Redundant logic: Duplicate implementations of data transformations increased processing times and the risk of inconsistent results.

  • Hidden business rules: Critical business rules were embedded in lengthy SQL stored procedures, making them difficult to understand and trace.

  • Slow semantic models and inefficient row-level security: Extremely large security tables significantly slowed down report performance.

  • Limited scalability as data volumes and user numbers continued to grow.

The Solution: Migrating from Azure SQL Server to Azure Databricks

Migration with Added Value

Rather than translating the stored procedures line by line, we analysed each procedure, identified the underlying business rules and restructured the pipelines in Databricks. Redundancies were eliminated and recurring transformations consolidated. The result is a leaner, faster data processing layer built on Delta Lake, along with a documented overview of the client's business rules.

Seamless Integration into the Existing Azure Ecosystem

The new Databricks pipelines are orchestrated using Azure Data Factory and Azure Synapse Pipelines. This allows the solution to integrate seamlessly into the existing operational environment without requiring additional tools.

Optimised Semantic Models

We conducted a comprehensive review of the Azure Analysis Services models and optimised their design. This reduced refresh times and improved report responsiveness in Power BI.

Redesigned Power BI Row-Level Security

In a later phase, we redesigned the row-level security model and reduced security data volume by 90%, delivering significant performance improvements for all users.

Empowering the Client's Team

Working closely with the internal team, we established best practices for Databricks development and cost-efficient pipelines. Today, the team can independently maintain and further develop the platform.

Utilized technologies

    • Azure Databricks - Core platform for data processing and transformation
    • Delta Lake - Storage layer for reliable and high-performance data processing
    • Azure Data Factory and Azure Synapse Analytics (Pipelines) - Orchestration and management of data pipelines
    • Azure SQL Server - Legacy data platform and source of the migrated processing logic
    • Azure Analysis Services (Tabular) - Provision of semantic data models
    • Power BI - Reporting and self-service analytics, including row-level security (RLS)
Databricks Use Case Logistik
Databricks Use Case Logistik

The results

  • 3× more frequent data refreshes: Three data refreshes per day instead of just one.

  • 90% less security data: Faster, more responsive Power BI reports.

     

  • Stability at scale: Reliable performance for thousands of users.

     

  • Maintainable code: Clearly defined business rules instead of nested stored procedures.

     

  • Cost-efficient operations: Right-sized compute resources.

     

  • Self-sufficient team: Consistent standards for new pipelines.

Best Practices for Your Databricks Migration

  • Understand first, migrate second: Avoid carrying legacy issues over to the new platform.

  • Eliminate redundancy: Ensure each piece of logic is implemented in only one place.

  • Consider costs from the start: Right-size compute resources and schedule jobs efficiently.

  • Optimise end-to-end: Consider data transformation, semantic models, reporting and security as a whole.

     

  • Ensure knowledge transfer: Enable the internal team to work independently.

Frequently Asked Questions About Databricks

Databricks is a cloud-based data lakehouse platform that brings together data processing, analytics and AI in a unified environment. On Microsoft Azure, it is available as Azure Databricks and uses Delta Lake as a reliable storage layer.

A migration is particularly beneficial when long processing times, growing data volumes or complex stored procedures limit data freshness. The key is to review and optimise the existing logic during migration rather than simply copying it.

Yes. Databricks handles data processing and transformation, while Power BI provides reporting capabilities, including row-level security for thousands of users.

Updated: 28.10.2026