Skip to main content

Next Generation Analytics with Microsoft Fabric

Next Generation Analytics with Microsoft Fabric
Written By: Marketing Team

Blog

Next Generation Analytics with Microsoft Fabric

May 25, 2023 7-Minute read

We are thrilled with Microsoft’s launching of Fabric for public preview! It has been an exciting journey of over 6 months for the Fabric team at Sonata. We have had an opportunity to learn the new paradigm of end-to-end analytics SaaS platform, to work closely with the product engineering teams at Microsoft to share our findings and feelings about the product that has been in the making.

As we applied and evaluated this technology in solving various use cases in the new ways and shared feedback, observations, and suggestions, the Microsoft team was very responsive even when under the tremendous timeline pressures. We are fascinated by their engineering collaboration and integrated approach towards realizing the product vision and roadmap.

We at Sonata are very thankful to the Microsoft team for all the support and partnership in this journey and looking forward to transforming our customers’ analytics capabilities with Fabric. We are also very thankful for the tremendous support and guidance we received from the leadership teams on both sides throughout this journey.

arun-quote raj-quote
arun-quote
shushma-quote rehan-quote

Data Estate Modernization with Fabric

Sonata's modernization and digital transformation offering involves data platform modernization for every business function at its core, given that information and intelligence the foundational components for organizational transformation.

The first five "dimensions" are referred to as ABCDE of modernization.

Agility

Modern businesses and processes require extreme agility, which can be achieved by engineering agility, rapidly gathering information and insights, and quickly matching them.

Breadth

Modern businesses need data from markets, customers, competitors, public perspectives, policies, and news to keep up with the wide-spread data universe, as today their own data is too limited.

Cost

This journey requires cost efficiency to keep up with the increasing scope, speed, and use of data.

Domain alignment

The efficacy of analytics is largely contingent upon the sector, vertical, and specialized domain in which a company operates.

Ease of use

Making data discovery and utilization simple is the first step in enabling a data driven organizational culture.

Fabric’s capabilities enable every dimension of our modernization offering. End to end and well-integrated capabilities of Fabric truly enables these aspects of modernization.

microsoft-fabric-capabilities

Here is are some of our most common modernization scenarios and Fabric’s capabilities nicely align to those:

Increasing the platform's agility and adaptability to address business transformation requirements. Engineering agility and the speed of pipeline are core themes of Fabric.

Constructing or optimizing a platform from disparate applications. Data mesh and OneLake architecture of Fabric is a perfect for this.

no-data-movement

Upgrading technology for improved maintainability and interoperability. Open data standards, End to end auditability & governance, various data connectors and inherently integrated services on common SaaS foundation are good examples of how Fabric aims to achieve this.

pen

Migration to the cloud. SaaS analytics platform helps move many overheads of on-premise and even the IaaS based data and analytics systems.

Our Modernization Offering and Fabric Enablers Matrix

While the art and science of modernization involves in-depth understanding of the organizational structure and processes, culture, business processes, current systems and perspectives of all stakeholders involved, the technology that is built to enable the modernization takes us a few steps ahead in the journey.

In fact, we see an enabler of our modernization dimensions in every workload of Fabric. Below matrix has few of those examples.

our-modernization-offering
  Agility Breadth Cost Domain Ease
Data Engineering Data mesh
Lakehouse Architecture
Auto scale
Lakehouse integrated
Common storage for structured or unstructured data
Streaming & batch workload unified
 Lakehouse - reduce storage & hops
More self managed spark cluster
Shared Capacity Units
Industry Templates
Workspace Domain Mapping
Workspaces experience
Mount/unmount APIs
CI-CD support
Data Integration Low-code (Gen2) data pipeline
Integrated Spark Engine
Open storage format
135+ connectors
Cross-cloud data mesh
Low-code
Shared Capacity Units    
Power Query experience
More built-in transformation functions
Flexibility - Ex: Embedded Spark scripts in pipeline
Data Warehousing On Unified data foundation Physical & logical relationship Shared Capacity Units Intelligent Data modeling
Auto Partitioning
Real Time Analytics Integrated experience - dev productivity
Kusto DB integration
IoT, Event streaming unified No separate provisioning of streaming services
Shared Capacity Units
Streaming services integrated
Power BI Direct Lake (See Through) mode
One-Click Report
More seamless integrations Pause option
Shared Capacity Units
Quick measures
Data Science Auto-ML
Data Wrangler
Templates
Multi-model comparison & selection Shared Capacity Units
No ML Studio needed
Semantic link - better integration with PBI, Lakehouse, etc.
Administration & Governance End-to-end audit logs
Data Lineage
Broadly applicable - in cross-platform data mesh Integrated logging, no separate cost
Auto-Tune
Built-in capabilities for job monitoring, usage & errors
Pervasive security

Early adoption experience at Sonata

Prior to advocating our customers, we wanted to ensure we had first-hand experience of Fabric for our own global workforce. As Sonata continues to transform with data driven organization culture, we took on two challenges to ensure everyone had access to insights.

Enhancing project analytics:

Provide interactive insights into project delivery and financial performance for all roles and responsibilities, to drive considerable advantages in terms of efficiency, cost savings, and improved decision-making.

Consolidating financial reporting:

Provides a centralized finance view to enable better financial decision making, increase efficiency, reduce the duplication of effort and costs while facilitating improved collaboration and communication across the organization

Project Analytics:

Some of our project delivery and financial performance reporting involved manual effort and delays causing a loss of valuable time to fix the real issues. Profitability report for instance would take almost a month to arrive.

Directors and Project Managers had limited options to slice & dice, explore the patterns and trends, create tailored scorecards and accountabilities based on the situation at hand. We built an interactive and near-real time project analytics solution using Fabric (Fabric) see through mode reports.

This has improved the agility of the project teams to react and resolve the project performance challenges faster than ever before. This saves cost of opportunity and improves experience for our customers and the project team as well.

project-analytics
bharath-quote

Here is a recap of our modernization aspects that we applied to project analytics solution and how Fabric enabled it:

Agility: Data mesh avoided replication efforts and significantly reduced the data integration effort as well as the data latency.

Breadth: we used this opportunity to bring together some of the data points such as CX and people experience metrics that were not well integrated with project performance before. This has expanded our perspective of project performance.

saran-quote

Cost: We eliminated 2 layers of storage and pipeline with direct lake design. With a shared capacity option, we save the cost of provisioning various services for storage, pipeline and analytics. Team tried out one-click reports and developer productivity features and we see the value of Fabric in team collaboration, version control and improved productivity of the team that would lead to optimization of cloud service and IT services cost for data estate and analytics with Fabric.

Domain: Our use cases were led by functional teams. Keeping the ownership of respective functional domain expert teams and surfacing them in a unified lake has definitely helped our solution to be much closer to every user and their day-to-day use cases – both in finance team and project delivery teams.

Ease: Ability to slice and dice the insights, tailored scorecards and unified experience across the company has improved our internal user experience tremendously.

Integrated Finance reporting

Although the final financial report of the parent organization had a consolidated figures, reporting and analysis of individual subsidiaries and acquired entities were varied in terms of structure, process, and systems used. This remained distributed at the granular level and combined at the top level.

OneLake with Data mesh architecture approach of Fabric made a lot of sense to address this problem. We built consolidated data model with One Lake approach and unified reports in Power BI that are interactive and analytical in nature, instead of a pre-canned style.

So, our analysts, managers and leaders across the company have a unified experience and can collaborate across the organizations.

lake-first-saas niranjan-quote

Customer Pilot Experiences

Having experienced the benefits of Fabric within our organization, we wanted to learn from customer problem scenarios and showcase the new technology benefits to our customers through early prototypes.

Key objectives of the early prototypes include but not limited to:

  • Early exposure for our customers on the upcoming technology and the new paradigm of analytics
  • Showcase potential benefits
  • Provide inputs for customer’s capability & solution roadmap
  • Product evaluation and feedback

Agile Insights for a Technology Sales team

Faced challenges with current macroeconomic situation, stiff competition from new players and dynamic strategies, this customer’s sales team has a huge gap to achieve their goals. Current reporting solution is not timely and insightful enough to act in this agile sales process.

We developed a near-real time reports and scorecards to demonstrate how Fabric can help deliver more agile analytics.

agile-insights

Key improvement:

  • Workspaces enable multiple engineering teams to own data integrations, model and business logic aligned to specific functional domains
  • Shortcuts enable consolidated data model brought together from functional models
  • Power BI see-through mode helped minimize number of data flow hops and improved latency significantly
  • Workspaces, shortcuts and reduced layers of storage and pipeline significantly increased delivery agility in this pilot project
  • Estimated cost of running the entire solution is much lower compared to the previous solution – considering the Azure cost as well as build & sustain costs.

BPO Customer Experience Analytics

This is a case of customer experience and support domain with increasing competition and the need for meeting and exceeding customer expectations. We developed a solution on Fabric to showcase an integrated analytics solution for reactive and proactive customer experience improvements.

Solution involved leveraging multiple signals of customer engagement experiences – quantitative and qualitative. This solution brought together customer experience metrics, business performance metrics and text insights from customer interactions and sentiment and key messages derived from the text data using language models – all in Fabric. Clusters of customers was created based on the status of various aspects of customer engagement derived as described above.

Result was an actionable customer experience insights solution that customer services team, process excellence team as well as the sales team could use in their day-to-day functions.

bpo-customer-exp

Key benefits showcased:

  • Consolidated customer experience signals using quantitative and text analytics. Useful for strategic and operational improvements.
  • More actionable insights – reactive actions based on specific pain or concern areas and proactive actions for service improvement based on common patterns
  • Technical integration – Semantic link that brought Power BI, Data Science, Transactional data (evidences / samples) and KPI business logic together.

Legacy Data Warehouse Migration

This is a case study of modernizing On-Prem SQL based data warehouse system that was unable to keep up with the growing data volumes and faster data refresh needs.

SQL server based legacy BI solution for a retail-like business model with millions of transactions daily, was upgraded to Azure VM based solution in the past. However, this was not scaling to the increasing volumes and data pipelines were often failing and metrics were stale for several days at times.

We took a subset of this large legacy solution to prototype with Fabric. Initial round of migration involved building a data lake to bring together the source data, Lakehouse based solution to model the data and deliver discrete datasets and Power BI reports to fulfill current reporting needs as well as providing an opportunity for business users and analysts to create their own reports and data visualizations. We also worked around row level security limitation of the current version using mapping table and current user context from power BI. This new solution has solved the data pipeline performance and data latency issues.

legacy-data-warehouse-migration

Sonata’s Journey Ahead with Fabric

  • We continue to evaluate the product, work with Microsoft to share continuous feedback and contribute to make this product better.
  • We will increase our efforts with customer pilots and evaluations to shape next generation analytics solution for them and share feedback from customer experience to product engineering team
  • critical for customer success Develop complimenting framework and tools for implementing and migrating to Fabric

To engage with our experts and take your data story to the next level, contact us