big data in social networks

Challenges in analyzing Big data in social networks

There are social interactions everywhere.  According to the Global Web Index, as of Jan 2016, there are about 3.4 billion internet users in the world.  And within that, there are about 2.3 billion active social media users.  Interactions through social media have become ubiquitous and so is the immense amount of data that is generated through it.

Many popular social networks like Facebook have begun to use this data to know about their users to deliver personalized feed to suit their interests and behavior and the situation is no different in Enterprise social networks as well.

Even today with the enormous amount of data that is generated through social media channels, leaders will have to struggle with the implications of big data.  Analyzing and gleaning information from the data will become key factor for competition as well as rise in productivity, innovation and increase in consumer surplus says the Mckinsey in their report  “Big data: The next frontier for innovation, competition, and productivity”.

Many people from senior leadership teams to the people in the technology world talk about ‘Big Data’.  Big data is not a buzzword for smarter data analysis to gain insight.  Therefore, what exactly is Big data, what does is it mean for us and how can we use the insights gained from such analysis in the realm of social networks.  The data analysis and insights gained is significantly different from what managers might generate from regular analytics.

Big data in social media

Big data is all the voluminous and unstructured data from a wide ranging sources in the form of click stream data from  websites, social media data like ‘Likes’, Tweets and ‘Blog posts’ etc. and from video entertainment as well.  Just to give you an idea, Google processes about 24 petabytes of data and not all of this in rows and columns.  Sometimes organizations also take into account the real time information as it occurs in radio frequency identification systems and make changes as they happen.

The consumers as well as working professionals in the organizations have begun to realize the potential value and the intelligence that can be derived from the vast amount of data that is generated through social media conversations.

Big data applications largely depend on their ability to analyze this large and unstructured data and handle the scale of the geometric growth of the social networks.  Social networks generate conversations and there is context attached to these conversations.  It is this context to information from various expert users is what makes knowledge sharing through social media tools so invaluable.  Finding specific information in this vast sea of billions of conversational messages is no easy task.  Big data in social networks together with social analytics need to go hand in hand in finding out the specific information we need.

The challenges

The challenges for mining such huge data from today’s social media systems are of two kinds.  Firstly, this requires use of emerging technology such data mining grid and Map reduce infrastructures such as Hadoop and a non-linear and non-deterministic software architecture.  This actually changes the way we think about data capture and processing.

Secondly, it is known fact that ‘what we measure is what we manage’.  We need to know ‘What we are looking for’ and the timing ‘When to ask the question’ is important.  ‘Spotting trends’ is one emerging area in social media analytics.  Then the question, Do you know what you are looking for?  Still lingers on.

With these challenges in the view, the future of big data technologies will blur both consumer and internal organizational data.  Big data in social network solutions will bring in a comprehensive view of entire social media footprint for the individual and for the organization.  Social networks bring enormous volumes of data.  Its nature is self-organizing, evolving.  Cloud computing with its cloud infrastructure can be a workable and an effective solution for big data analytics in the future.

Ramkumar Yaragarla
Available at

Ramkumar Yaragarla

I am 42 years old. I have spent double digit years working as a business analyst and a program manager in Human resources and IT functions in several Fortune 100 companies in India and the UK.
Completed Masters in IT for manufacturing at the University of Warwick, UK and a PRINCE 2 certified practitioner.

My interests include collaborative innovation, group dynamics, Idea hubs and work life balance. I am open to your suggestions.
Ramkumar Yaragarla
Available at

Leave a Comment

%d bloggers like this:
Read previous post:
Office environment
How Office Environment Influences Productivity and Collaboration

A certain portion of productivity and collaboration comes from within the stellar team members you hire for the job. If...

Close