ВИЗУАЛИЗАЦИЯ ДАННЫХ В РЕАЛЬНОМ ВРЕМЕНИ С ИСПОЛЬЗОВАНИЕМ POWER BI И ИСКУССТВЕННОГО ИНТЕЛЛЕКТА

УДК 004.89:61

ABSTRACT

Organizations increasingly rely on real-time analytics dashboards to support operational decision-making, yet integrating artificial intelligence capabilities directly into business intelligence (BI) platforms remains a non-trivial engineering challenge. This paper proposes and evaluates an architecture for real-time data visualization that combines Microsoft Power BI's streaming datasets and DirectQuery mode with AI-powered anomaly detection, natural language querying (NLQ), and predictive analytics, delivered as embedded Power BI reports within a web portal. The architecture was implemented and validated in a case study at a mid-sized Kazakhstani retail company processing over 1.2 million transactions per month. Benchmarks demonstrate end-to-end dashboard refresh latency of 2.4 seconds at the 95th percentile, anomaly detection precision of 91.3%, and a 37% reduction in analyst query time attributable to the NLQ module. The paper details the data pipeline, AI model integration patterns, and lessons learned.

АННОТАЦИЯ

Организации всё чаще полагаются на аналитические панели в реальном времени для поддержки операционного принятия решений, однако интеграция возможностей искусственного интеллекта непосредственно в платформы бизнес-аналитики (BI) остаётся сложной инженерной задачей. В данной работе предлагается и оценивается архитектура визуализации данных в реальном времени, которая объединяет потоковые наборы данных и режим DirectQuery в Microsoft Power BI с интеллектуальными методами обнаружения аномалий, обработкой запросов на естественном языке (NLQ) и предиктивной аналитикой, реализованными в виде встроенных отчётов Power BI в веб-портале. Предложенная архитектура была реализована и протестирована на примере средней казахстанской розничной компании, обрабатывающей более 1,2 миллиона транзакций в месяц. Результаты бенчмаркинга показывают задержку обновления дашбордов 2,4 секунды на 95-м перцентиле, точность обнаружения аномалий 91,3%, а также снижение времени выполнения аналитических запросов на 37% благодаря модулю NLQ.

В работе подробно рассматриваются конвейер обработки данных, подходы к интеграции моделей искусственного интеллекта и полученные практические выводы.

Keywords: Power BI, real-time visualization, artificial intelligence, anomaly detection, natural language querying, streaming analytics, business intelligence.

Ключевые слова: Power BI, визуализация в реальном времени, искусственный интеллект, обнаружение аномалий, обработка запросов на естественном языке, потоковая аналитика, бизнес-аналитика.

Introduction

The ability to monitor and act on business data in near-real time has shifted from a competitive advantage to a baseline expectation across industries including retail, finance, logistics, and manufacturing [1]. Business intelligence (BI) platforms such as Microsoft Power BI, Tableau, and Qlik have become the de facto standard for delivering interactive dashboards to business users, with Power BI alone reporting over 250,000 organizational customers as of 2024 [2]. However, traditional BI deployments operate in a batch-refresh paradigm — data is extracted, transformed, and loaded at scheduled intervals, introducing latency that ranges from minutes to hours — fundamentally incompatible with operational use cases such as fraud detection, supply chain exception handling, or live event monitoring [3].

The integration of artificial intelligence into BI workflows — often branded as Augmented Analytics [4] — promises to address several pain points simultaneously. AI-powered anomaly detection can surface statistically unusual patterns in streaming data without requiring analysts to define explicit threshold rules. Natural language querying (NLQ) interfaces, exemplified by Power BI's Q&A feature and its underlying language model, lower the technical barrier for business users to interrogate data in plain English or local languages [5]. Predictive analytics modules can extend historical dashboards with forward-looking forecasts, enabling proactive rather than reactive decision-making [6].

Despite this potential, practical integration of AI with real-time Power BI deployments introduces substantial engineering complexity. Power BI's real-time streaming capabilities (Push Datasets, Streaming Datasets, and DirectQuery against live sources) each impose different latency, data model, and DAX expression constraints [7]. Embedding AI-generated insights — anomaly scores, forecast bands, NLQ responses — into live dashboards requires careful consideration of refresh coordination, data model design, and user experience.

This paper makes three contributions: (1) a reference architecture for integrating AI capabilities into a real-time Power BI deployment; (2) an implementation and case study at a Kazakhstani retail organization; and (3) quantitative benchmarks covering latency, AI model accuracy, and analyst productivity impact. The remainder is structured as follows: Section 2 covers the system architecture; Section 3 describes the AI model components; Section 4 presents evaluation results; Section 5 concludes.

System Architecture and Methods

The proposed architecture follows a Lambda architecture pattern [8], adapted for real-time BI. The three-layer design: the data ingestion layer, the AI processing layer, and the visualization layer.

Data ingestion layer: Transactional data from the retail company's point-of-sale (POS) system, inventory management system (IMS), and e-commerce platform is streamed into Azure Event Hubs at a throughput of approximately 800 events/second during peak hours. Azure Stream Analytics performs windowed aggregations (tumbling windows of 30 seconds) and routes data to two sinks: (i) an Azure SQL Database for historical storage, enabling DirectQuery from Power BI; and (ii) a Power BI Push Dataset API endpoint for real-time KPI tiles. An Azure Data Factory pipeline handles daily full-refresh ETL for the analytical data model.

AI processing layer: Three AI sub-components were integrated. First, an Isolation Forest anomaly detection model [9] trained on 14 months of historical transaction data scores each incoming window for statistical deviation. Anomaly scores and binary labels are written back to the Azure SQL Database within the 30-second window budget. Second, an ARIMA-based time series forecasting service (retrained weekly) provides 24-hour and 7-day revenue forecasts via an Azure Functions endpoint, consumed by Power BI using the Web connector with scheduled refresh. Third, Power BI's native Q&A engine was extended with a custom synonym dictionary and a domain-specific linguistic schema covering retail terminology in Kazakh, Russian, and English.

Table 1.

Technology Stack and Component Responsibilities

Visualization layer: Power BI reports were developed using a composite model combining the Push Dataset (for real-time KPI cards and sparklines) with DirectQuery against Azure SQL (for interactive drill-through across historical data). The dashboard was embedded into the company's internal portal using Power BI Embedded with an A2 SKU. Five dashboard pages were designed: Executive Summary, Real-Time Sales Monitor, Anomaly Feed, Forecast Explorer, and Store Comparison. All pages were reviewed in three rounds of usability testing with six retail analysts using the System Usability Scale (SUS) [10].

Performance benchmarking was conducted over a 30-day production window. End-to-end latency was measured as the time between a POS transaction event and its appearance in the Power BI dashboard tile, sampled at 1,000 random events per day. Anomaly detection performance was evaluated against a labeled ground-truth set of 420 known fraudulent or erroneous transactions from the past year. Analyst productivity was measured by comparing query resolution times before and after NLQ deployment using a log analysis of 1,847 analyst interactions.

Results and Discussion

Table 2 summarizes the system performance metrics collected over the 30-day benchmarking period. End-to-end dashboard refresh latency averaged 1.8 seconds with a 95th percentile of 2.4 seconds, well within the 5-second threshold identified as acceptable for operational dashboards in prior studies [11]. Tail latency spikes above 4 seconds correlated with Azure SQL DTU saturation events, suggesting that upgrading to the S6 tier or migrating to Azure SQL Hyperscale would further improve tail performance.

Table 2.

System Performance Benchmarks (30-day production window)

The Isolation Forest anomaly detection model achieved a precision of 91.3% and recall of 87.6% (F1 = 89.4%) on the labeled ground-truth set. False positives (8.7% of flagged events) were predominantly concentrated in legitimate promotional pricing events not captured in the model's training features — a known limitation of unsupervised approaches that will be mitigated in future iterations by incorporating a supervised classification layer post-flagging. False negatives (12.4% of true anomalies missed) were mainly low-magnitude deviations that fell within normal variance bounds, posing limited operational risk.

The NLQ analysis revealed a 37% reduction in mean analyst query resolution time (from 29.2 seconds to 18.4 seconds) following deployment of the enhanced Q&A module with the domain-specific synonym dictionary. Queries in Kazakh accounted for 18.3% of total NLQ volume, validating the value of multilingual schema support. Qualitative analyst feedback highlighted the ability to phrase queries in natural Kazakh or Russian as a significant adoption driver. The SUS usability score of 78.3 classifies the dashboard as Good on the standard SUS scale, with analysts rating real-time anomaly notifications and forecast bands as the most valuable new features (Figure 2).

Figure 2. Analyst Feature Value Ratings by Dashboard Component (mean scores, N=6 analysts)

The 7-day revenue forecast achieved a mean absolute percentage error (MAPE) of 6.1%, slightly above the 6% target, primarily due to poor performance during public holidays not fully captured in the seasonal decomposition. The 24-hour horizon MAPE of 4.2% was within target. Incorporating a holiday calendar feature into the ARIMA model is planned for the next iteration.

Conclusion

This paper presented a production-validated architecture for real-time data visualization in Power BI augmented with AI capabilities including anomaly detection, time-series forecasting, and natural language querying. The case study at a Kazakhstani retail company demonstrated that the proposed architecture achieves sub-3-second end-to-end dashboard latency, near-90% anomaly detection F1-score, and a meaningful 37% reduction in analyst query time, all while maintaining acceptable usability scores.

The work highlights several practical lessons for practitioners: composite Power BI data models combining Push Datasets with DirectQuery require careful cardinality management to avoid query performance degradation; Isolation Forest models benefit from feature engineering that distinguishes promotional from anomalous price deviations; and multilingual NLQ schema investment yields measurable adoption gains in non-English speaking organizations. Future directions include migrating the anomaly detection pipeline to an online learning model capable of adapting to drift without periodic full retraining, exploring Power BI Copilot integration for generative AI-driven narrative summaries, and extending the architecture to support IoT sensor streams for the company's warehouse operations.

References:

1. Davenport T. H., Harris J. G. Competing on Analytics: The New Science of Winning. Boston: Harvard Business School Press, 2007. 240 p.
2. Microsoft Corporation. Power BI Product Overview and Customer Statistics. Redmond: Microsoft, 2024. URL: https://powerbi.microsoft.com (accessed: 15.03.2025).
3. Kimball R., Ross M. The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling. 3rd ed. Indianapolis: Wiley, 2013. 600 p.
4. Gartner Inc. Augmented Analytics Is the Future of Data and Analytics. Stamford: Gartner, 2021. 14 p.
5. Srinivasan A., Setlur S. Natural language interfaces for data analysis with visualization // Proceedings of the ACM CHI Conference on Human Factors in Computing Systems. 2019. P. 1–12.
6. Hyndman R. J., Athanasopoulos G. Forecasting: Principles and Practice. 3rd ed. Melbourne: OTexts, 2021. 442 p.
7. Microsoft Documentation. Real-time streaming in Power BI. Redmond: Microsoft, 2024. URL: https://docs.microsoft.com/power-bi/connect-data/service-real-time-streaming (accessed: 20.03.2025).
8. Marz N., Warren J. Big Data: Principles and Best Practices of Scalable Real-Time Data Systems. Shelter Island: Manning, 2015. 328 p.
9. Liu F. T., Ting K. M., Zhou Z.-H. Isolation forest // Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008). 2008. P. 413–422.
10. Brooke J. SUS: A retrospective // Journal of Usability Studies. 2013. Vol. 8, No. 2. P. 29–40.
11. Tableau Research. Dashboard Performance and User Satisfaction: A Survey Study. Seattle: Tableau Software, 2022. 22 p.
12. Abramova V., Bernardino J. NoSQL databases: MongoDB vs. Cassandra // Proceedings of the International C* Conference on Computer Science & Software Engineering. 2013. P. 14–22.
13. Zaharia M., Xin R. S., Wendell P. et al. Apache Spark: A unified engine for big data processing // Communications of the ACM. 2016. Vol. 59, No. 11. P. 56–65.