Subscribe to DSC Newsletter

Much more devices held, much more messages sent, much more data up in the air.

Upon the arrival of IOT (Internet of Things) era, number of connected devices grows rapidly, the signals of which stack up mines containing valuable, hidden insights.

We used to analyze log files for risk management, spotting the anomalies and exceptions based on outlined records. Via the ever-richer meta data and context, we are entitled to weave more story from the strings now. By union or intersecting user data and log records, we can segment user groups by distinct behaviors and find their particular patterns. By comparing the information of each webpage we can further find the association among each genre of websites.

The idea of log mining is not disruptive; it inherits the long-lasting data mining methodology. Making log files more productive, however, the workload can be quite challenging. Using traditional database technologies, with no doubt, can jeopardize the system stability due to its capacity limit while using distributed approach the performance still cannot improve for the heavy shuffling among data components.

BigObject, designed to compute up to a few billions data records under an extended relational model, works ideally for this use case. We worked with a service provider who offers portal for educational institutes. It tries to identify what types of user role dominate in each campus. There are on average over 66 million records per month, with BigObject the multidimensional calculations can be done in 15 seconds. Simply embedding BigObject with it web-service, each website administrator can better monitor the service utilization now. 

BigObject delivers 2 to 3 orders of magnitude improvement in speed. Only when the analysis can be done quickly enough, the information can be truly actionable. Make your analysis actionable, visit here.

Views: 347

Tags: analysis, data, log, mining

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Videos

  • Add Videos
  • View All

© 2020   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service