Traditional vs Deep Learning Algorithms in the Telecom Industry — Cloud Architecture and Algorithm Categorization
Google Cloud Architecture for Machine Learning Algorithms in the Telecom Industry
The unprecedented growth of mobile devices, applications, and services have placed the utmost demand on mobile and wireless networking infrastructure. Rapid research and development of 5G systems have found ways to support mobile traffic volumes, real-time extraction of fine-grained analytics, and agile management of network resources, so as to maximize user experience.
Moreover, inference from heterogeneous mobile data from distributed devices experiences challenges due to computational and battery power limitations. As a result, models employed in the edge-based scenario are constrained to light-weight to achieve a trade-off between model complexity and accuracy. Also, model compression, pruning, and quantization are largely in place.
In this blog, we try to understand the different use-cases, problems, and solutions that can be leveraged with ML as follows:
Use cases of traditional Machine Learning algorithms
In this section, let’s look at the different use cases in the telecom industry where different ML and AI algorithms have played a significant role in network traffic prediction, customer retention, and fraud analysis.
Smart traffic prediction and path optimization
The network and service control layer contains multi-dimension convergent management and control functions to manage and control traditional and SDN/NFV cloud networks.
Adding AI reasoning capability would allow Intelligent Network Operations and Management. Network performance data can help identify sleeping cells and trigger an automatic restart, Network Optimization (coverage optimization, capacity optimization, Massive MIMO optimization) RCA (Root Cause Analysis), and Intelligent Transmission Route Optimization and Network Strategy Optimization, etc.
Features governing Network security include:
Sentiment analysis with social media
As network operators have turned to Machine Learning to analyze brand coverage and customer sentiment, social posts help them to monitor language patterns and sentiment to identify trends like the factors driving new customers to subscribe or when do subscribers seek out a competitor.
System design and architecture for highly accurate customer churn modeling
Customer Service Recommendation and Business Personalization
Service recommenders may also be used to boost existing services or to identify why users do not adopt some services and, in turn, suggest them value-added services based on their profile and choice. In addition, they also predict churn based on the usage patterns of past churners and changes in other usage profiles.
The below figure illustrates an SVM (Support Vector Machine)-based music recommendation system that extracts personal user-level information, timing, location, activity records, along with musical context to suggest suitable music services.
Music Recommendation System a VAS leveraged by telecom operators—
With customer-generated network data, it is easier to automate the process of grouping customers into segments, like profiling customers based on their calling and messaging behavior.
Operators try to present product/service advertisements that are tailored to an individual, situation and device. This type of target-advertising, when directed at right intended customer bases, helps operators and advertisers to zero in on customers with ads that fit their needs and interests.
Customer segmentation on call-records
Different clustering techniques and classification techniques like K-means and others cluster mobile customers based on their call detail records and analyze their consumer behavior. PCA-based dimensionality reduction techniques can be used for the identification of relevant and recurrent patterns (e.g. location to identify common presence patterns) among the CDRs of a given user. Further, matrix factorization is employed to infer location preferences on sparse CDR data and generate location-based recommendations.
Clustering and Classification phase for predicting user-churn
Customer Churn Prediction
The above figure illustrates the application of SVM, Naive Bayes, Decision Tree, Boosting, Bagging, Random Forest in Customer Churn Prediction through supervised/unsupervised (clustering) techniques.
Traffic Flow Prediction
Profiling by association: PBA takes as input an IP-to-IP connectivity graph and information about a small subset of IP hosts and produces a prediction about the class of all the flows (edges) in the graph.
Topic Models for Mobile Short Message Service Communication
Latent Dirichlet Allocation (LDA), a generative topic modeling technique, is used to extract latent features arising from mobile Short Messaging Service (SMS) communication for automatic discovery of user interest. The mobile SMS documents are partitioned into segments, wherein the discovered topics in each segment are propagated to influence the discovery of latent features. This technique filters malicious mobile SMS communication. Topic models can effectively detect distinctive latent features to support automatic content filtering and remove security threats to mobile subscribers and operators.
Clustering to segment customer profiles requires complex multivariate time series analysis-based models, that have limitations around scalability and ability to accurately represent temporal behavior sequences (TBS) of users, illustrated in the figures below. TBS may be short, noisy, and non-stationary, where the LDA model serves as the best to represent the temporal behavior of mobile subscribers as compact and interpretable profiles, relaxing the strict temporal ordering of user preferences.
The model generating subscriber behavior documents. (left) , LDA model to generate interpretable subscriber profiles (right), MTS-Multivariate TimeSeries
Categorization of Deep Learning algorithms and their use cases in the Telecom Industry
Advantages of Deep Learning in Mobile and Wireless Networking
The Telecom industry acknowledges several benefits of employing Deep Learning to address network engineering problems:
PointNet++ Architecture (left) and Graph CNN Architecture (right)
Despite the challenges posed by Deep Learning models, emerging tools and technology make them tangible in mobile networks, as illustrated in the figures below: (i) Advanced Parallel Computing, (ii) Distributed Machine Learning Systems, (iii) Dedicated Deep Learning libraries, (iv) Fast optimization algorithms, and (v) Fog Computing.
CPU/GPU/Processing capability to support Deep Learning Architectures
Deep Learning has a wide range of applications in mobile and wireless networks.
Now, let’s take a quick look at the different Deep Learning platforms available, mobile hardware supported along with its speed and mobile compatibility.
Comparison of Mobile Deep Learning Models
Cloud Architecture with mobile data ingestion and Model Training, Prediction
The figure below depicts the different components involved in building the ML platform —
Network Monitoring/Optimization, Media Settlement, Advertising, Audience Orientation, Pattern Recognition, Sensor Data Mining and Mobility Analytics
Cloud Architecture with GCP for telecom Machine Learning and AI algorithms
Network Monitoring and Optimization
Network State Prediction refers to inferring mobile network traffic or performance indicators, given historical cellular measurements of EnodeB, Sector and Carrier data. MLPs and Deep Learning LSTM-based techniques are used to predict users’ QoE, and evaluate the best-beam for transmission based on:
By leveraging sparse coding and max-pooling, semi-supervised Deep Learning models have been developed to classify received frame/packet patterns and infer the original properties of flows in a WiFi network.
Mobility metrics based for Network Capacity Estimation
Further, AI-capable 5G networks aid in:
Predicting Mobile traffic at city scale
AE-LSTM for traffic flow prediction
Hybrid Multimodal Deep Learning framework for traffic flow forecasting
Multiple 3D CNN architecture
ST-DenNetFus based Deep Learning framework is used to predict network demand (i.e. uplink and downlink throughput) in every region of a city as illustrated in the figure below. The ST-DenNetFus architecture captures unique properties (e.g., temporal closeness, period, and trend) from Spatio-temporal data, through various branches of dense neural networks (CNN). ST-DenNetFus also introduces extra branches for fusing external data sources (e.g., crowd mobility patterns, temporal functional regions, and the day of the week) that have not been considered before in the network demand prediction problem of various dimensionalities.
GAN operating principle in the MTSR problem. The generator is employed in the prediction phase.
CDR Analysis Pipeline
RNN-based predictors significantly outperform traditional ML methods, including Naive Bayes, SVM, RF and MLP.
Deep Learning-Driven App-level Mobile Data Analysis
Analysis of mobile data, therefore, becomes an important and popular research direction in the mobile networking domain, as rapid emergence of IoT sensors and its data collection strategies have been able to provide a powerful solution for app-level data mining.
App-level mobile data analysis include: (i) Cloud-based computing and (ii) Edge-based computing. In the former, mobile devices act as data collectors and messengers that constantly send data to cloud servers, via local points of access with limited data pre-processing capabilities. In Edge-based computing, pre-trained models are offloaded from the cloud to an individual. The primary applications include mobile healthcare, mobile pattern recognition and mobile Natural Language Processing (NLP), and Automatic Speech Recognition (ASR).
Mobile Health: Wearable health monitoring devices being introduced in the market, incorporates medical sensors that capture the physical conditions of their carriers and provide real-time feedback (e.g., heart rate, blood pressure, breath status, etc.), or trigger alarms to remind users of taking medical actions.
Deep Learning-driven MobiEar to aid deaf people’s awareness of emergencies operates efficiently on smart phones and only requires infrequent communication with servers for updates. UbiEar, a lightweight CNN architecture designed for acoustic event sensing and notification system, operates on the Android platform and is able to assist hard-to-hear sufferers in recognizing acoustic events, without requiring location information.
Deep Learning-based (DL) models (CNNs and RNNs) are able to classify lifestyle and environmental traits of volunteers, different types of Human Activity Recognition with heterogeneous and high-dimensional mobile sensor data, including accelerometer, magnetometer, and gyroscope measurements. ConvLSTMs are known for fusing data gathered from multiple sensors and perform activity recognition.
Mobile motion sensors collect data via video capture, accelerometer readings, motion — Passive Infra-Red (PIR) sensing, specific actions and activities that a human subject performs. Such models trained on server for domain-specific tasks through federated learning, finally serve a broad range of devices.
Mobile Pattern Recognition based on patterns observed in the output of the mobile camera or other sensors. All these DL models demonstrate superior prediction accuracy over RFs and logistic regression.
Object Classification finds huge applications in mobile devices as devices take photos and rely on photo-tagging process. One such DL-based framework is the DeepCham that generates high-quality domain-aware training instances for adaptation from in-situ mobile photos. It has a distributed algorithm which identifies qualifying images stored in each mobile device for training and a user labeling process for recognizable objects identified from qualifying images using suggestions automatically generated by a generic deep model.
Mobile classifiers can also assist Virtual Reality (VR) applications, where Deep Learning object detectors are incorporated into a mobile Augmented Reality (AR) system. Object detectors use CNN-based frameworks used for facial expression recognition when users wear head-mounted displays in the VR environment.
The figure below demonstrates a lightweight Deep Learning-based object detection framework that combines spatial relations for:
Mobile Outdoor Augmented Reality method
The figure below demonstrates app-level data collection and transfer from edge devices to the cloud for algorithm training and prediction.
Deep Learning-Driven Mobility Analysis
Mobility data is usually subject to stochasticity, loss, and noise, which creates a problem in precise modeling. As Deep Learning is able to perform automatic feature extraction, it becomes a strong candidate for human mobility modeling. CNN's and RNNs are the most successful architectures in such applications as they can effectively exploit spatial and temporal correlations.
Hierarchical framework with coarse model and fine models, suited for spatial mobile data in an online learning system
An attention candidate generator to generate the candidates, which are exactly the regularities of the mobility and an attention selector to match the candidate vectors with the query vector, i.e., the current mobility status.
Architecture of DeepMove
ST-ResNet architecture. Conv: Convolution; ResUnit: Residual Unit; FC: Fully-connected.
Deep Learning Driven User Localization
Location-based services and applications (e.g. mobile AR, GPS) demand precise individual positioning technology. Deep Learning can enable high localization accuracy with both device-free and device-based localization services.
Limitations of Deep Learning in Mobile and Wireless Networking
Although Deep Learning has unique advantages when addressing mobile network problems, it also has several shortcomings, which partially restrict its applicability in this domain. Specifically:
In this blog, we discussed different traditional vs Deep Learning algorithms, DL-based architectures, their pros and cons, and applications in the telecom industry. We also explored the data ingestion, categorization, and model deployment architecture in production. We looked at the recent advances in ML driver mobile-app development (in object detection, speaker identification, emotion recognition, stress detection, and ambient scene analysis), in-built technologies to sustain limited mobile battery by building memory-energy efficient apps and model compression techniques.