Can Skype for Business Contribute to Big Data Research?

Analysis has been part of the Lync Server family pretty much since the beginning. How else will you know to troubleshoot a SIP trunk or softphone, if you don’t have some form of data on where the problem is?

In recent years though, we’ve seen the arrival of Big Data. Going beyond server logs to huge data warehouses. Algorithms poring through exabytes. Insights teased out for many a scientific or commercial purpose.

Some Big Data research goals I’ve seen include:

Analyzing marketing campaigns
Predicting customer behavior
Improving scientific research processes
Finding inefficiencies in operations

It got me wondering. Skype for Business generates a fair amount of data. What kind of value could it contribute to Big Data research?

More Than Monitoring

In a sense there’s already a Big Data engine within Skype for Business Server – the Monitoring Server. It collects data on the other servers and Skype4B clients. Call Detail Recording (CDR)? Quality of Experience (QoE)? The Call Quality Dashboard? All a type of data analysis.

I’m thinking bigger than Monitoring though. Every bit of data generated & stored by Skype for Business and its dependencies. Conversation logs, call data, SIP traffic, bandwidth monitoring, server use records, and so on.

If you put all these together, what would be the best Big Data purpose for which to use them?

I’d have to say the last one strikes me as the one best suited for Skype for Business data: Finding inefficiencies in operations.

Analyzing these data can tell you if:

There’s a correlation between your website’s uptime and customer support calls?
Your recent beta launch took off, according to call workflow activity?
Inter-department conversations alert you to an internal management issue?
Marketing’s claim of 20% brand awareness growth matches to Sales’ incoming call volume?

And so on.

So there’s at least a business case for adding Skype4B data into Big Data research. The question now is, how feasible is it?

How Skype for Business Could Contribute Data

Skype for Business is a Microsoft server app. So of course its default database server is MS SQL Server. Running Skype4B Enterprise Edition requires multiple SQL servers in fact – Front End, Persistent Chat, Archiving, Monitoring, and any mirroring you want. (Standard Edition uses SQL Express.)

SQL Server databases are relational. In some Big Data articles I’ve read, relational databases are considered inferior for Big Data applications. The reason being that Big Data takes multiple disparate data sources, tosses them in its blender, and whips up some results. The inherent tabular structure of relational databases complicates making matches to other data types.

Not sure I believe this. Besides, it doesn’t mean SQL Server data is unusable. Far from it! You’d just need the proper tools to either convert the relational data, or analyze it as-is.

In terms of tools, I went looking, but quickly got lost among the many Big Data technologies! Hadoop, Odata, Spark, R…suffice to say there’s options out there to make use of your Skype for Business data.

(In fact SQL Server 2016 will include R, which should make Big Data analysis much easier.)

I did find a third-party solution called PowerView from UnifySquare. PowerView collects Lync Server 2013 data (website says it’s also Skype for Business 2015 Ready) into a central warehouse usable for Big Data queries. It also generates visual reports & dashboards, to make use of the insights you’ll find.

There’s also the PowerBI initiative from Microsoft. Among PowerBI’s listed connections are “SQL Database Auditing,” “Azure SQL Database” and “SQL Server Analysis Servies.” I see no reason why it couldn’t use Skype for Business databases as a data source.

I’m sure many other third-party Big Data platforms exist, which can incorporate Skype for Business data sets. If you know of any, please share!

Have you employed Skype for Business data in a Big Data query? What tools did you use? Please comment or email.