When Big Data Means Bad Analytics

When analytics delivers disappointing results, it is often because there is not enough analytic expertise, and/or lack of understanding of a business objectives for using Big Data in the first place. To avoid failure, insist on high standards.

Forrester: Big Data market to grow three times faster than tech overall

Led by newer kinds of databases and software for distributed storage and computing, the Big Data market is set to grow at three times the rate of the entire technology market, according to a new report from Forrester Research Inc.
Forrester’s report, “Big Data Management Solutions Forecast 2016 to 2021,” which the analyst firm claims is the first of its kind, divvies up the Big Data market into six separate chunks – namely, enterprise data warehousing, NoSQL, Hadoop, big data integration, data virtualization and in-memory data fabric. Of these, it reckons that Hadoop, NoSQL and in-memory data fabric are expected to see the most significant growth in the next six years.

Topic Modeling: Deriving Insight From Large Volumes of Unstructured Data

The rise of social networks has led to an increase in unstructured data available for analysis, with a large proportion of this data being in text format such as tweets, blog posts, and Facebook posts. This data has a wide range of applications, for example it is often used in marketing to understand people’s opinions on a new product or campaign, or to learn more about the target market for a particular brand.

When dealing with large volumes of unstructured text data, it can be difficult to extract useful information efficiently and effectively. There is almost always too much data to read through manually, so a method is needed that will extract the relevant information from the data and summarise it in a useful way.
Topic modelling is one method of doing this. Topic modelling is a technique that can automatically identify topics (groups of commonly co-occurring words) within a set of documents (e.g. tweets, blog posts, emails).

An effective topic model should output a number of very distinct groups of related words, which are easily identifiable as belonging to the same subject. For example, if the topic model was trained on thousands of tweets related to diet, one group of words might include “gluten”,”glutenfree”, “coeliac”, “intolerance”, which would correspond to a “gluten free diet” topic. Another group of words might be “vegan”, “dairyfree”, “meatfree”, which would represent a “vegan diet” topic.

Beware of the gaps in Big Data – E & T Magazine

As we entrust ever more of our lives to ‘big data’, how can we protect against the gaps and mistaken assumptions used to handle the information?

When the municipal authority in charge of Boston, Massachusetts, was looking for a smarter way to find which roads it needed to repair, it hit on the idea of crowdsourcing the data. The authority released a mobile app called Street Bump in 2011 that employed an elegantly simple idea: use a smartphone’s accelerometer to detect jolts as cars go over potholes and look up the location using the Global Positioning System. But the approach ran into a pothole of its own.

“Real-time” is getting real | DBMS 2 : DataBase Management System Services

“Real-time” is getting real

I’ve been an analyst for 35 years, and debates about “real-time” technology have run through my whole career. Some of those debates are by now pretty much settled. In particular:

Yes, interactive computer response is crucial.Into the 1980s, many apps were batch-only. Demand for such apps dried up.Business intelligence should occur at interactive speeds, which is a major reason that there’s a market for high-performance analytic RDBMS.

Big Data And A Shocking Waste Problem

It’s a shocking fact that in the 21st century more than three quarters of a billion people do not have access to enough food to keep themselves healthy, while 30% of the food produced around the world goes to waste.  

It’s partly a problem of logistics – it’s expensive to keep food fresh and get it to the people who need it most. This of course means its ripe for tackling with technology and Big Data.