IT is part of the business

Recently, during one of the classes I am teaching on data management, one of the students asked me about the difference between big data and a simple database.
Neither the students, nor I think anyone else, is unaware that in today’s digital era, data has become one of the most valuable assets in the world, and that it has become one of the most valuable assets in the world. The ability to collect, analyze and use data effectively can make the difference between success and failure when it comes to decision making. Within this context we can find concepts such as big data, but also small data. What are they and in which situations to use each one?

Before delving into the differences between big data and small data, it is important to have an understanding of what is meant by the term data in general. The term “data” refers to any type of information or set of facts that is collected and stored. Data is the basis for all business decision making; it can be quantitative or qualitative and can be presented in a variety of formats, such as numbers, text, images, videos or other formats. This data can be structured or unstructured, meaning that it can be organized in tables or lack a clear structure.

For example, a retailer that collects sales data. This data could include information on the number of products sold, sales prices, dates of purchase, and store location. In this case, the data would be the raw information about sales transactions, but could be organized into a defined structure. On the other hand, product catalog documents would not have a defined structure.

Small data

Small data refers to data sets that are relatively small in size, but are highly relevant and meaningful to an enterprise. Unlike big data, which focuses on large volumes of data, the focus of small data is on accuracy and quality. In this case, it can be analyzed without the need for advanced data processing tools. It is about extracting valuable information from smaller, more specific data sets. Small data has a focus on accuracy and quality.

For example, a service company using customer satisfaction surveys to collect data on customer satisfaction with its services. Instead of analyzing large amounts of data indiscriminately, the company could focus on a specific set of customer responses that reveal important trends. For example, they might identify that customers who mention the speed of service response are more likely to recommend the company to others. Or a restaurant might use small data to analyze customer orders during a specific week to adjust its menu.

Another example might be an online media company that uses small data to analyze the reading and viewing habits of its users. From specific data, such as which news sections a user visits frequently, it can customize recommended content to improve user retention and interaction with the site.

Big Data

Big data, on the other hand, refers to extremely large and complex data sets that may come from various sources, in different formats and without defined structure. In this case, the data cannot be processed efficiently with traditional analysis tools. The main focus of big data is the ability to process, analyze and derive valuable information from these vast data sets; i.e., the power of scale and variety. This data is often characterized by the typical three “Vs”: volume, velocity and variety (although we now talk about 7 Volume, Velocity, Variety, Veracity, Value, Variability and Visualization).

For example, a global supply chain can leverage big data to predict product demand in multiple regions. By analyzing large volumes of data, such as sales history, weather events, and economic trends, they can optimize their manufacturing and distribution operations to meet demand more efficiently.

A social media platform such as Facebook or Twitter handles massive amounts of data on a daily basis. This includes not only user posts and messages, but also demographic data, geographic location, content preferences and more. Using big data, we can analyze behavioral patterns, identify content trends and personalize the user experience based on the data collected.

Main Differences Between Small Data and Big Data

Now that we have explored the concepts of data, small data and big data, it is important to highlight the main differences between them.

Data Set Size: The most obvious difference lies in size. Small data refers to small and manageable sets, while big data is characterized by its immensity and complexity.

Granularity: Small data is highly detailed and specific, allowing in-depth analysis of individual elements. It focuses on accuracy and data quality. Big data, on the other hand, typically focuses on general patterns and trends rather than specific details. It focuses on the scale and variety of the data.

Processing Tools: Small data can be analyzed with conventional analysis tools such as spreadsheets. In contrast, big data requires distributed data management systems and advanced processing tools such as Hadoop or Spark.

Cost: Big data can be expensive to manage due to the need for infrastructure and specialized personnel. Small data tends to be cheaper to manage and analyze.

Use: The choice between small data and big data depends largely on a company’s objectives and resources. Small data is useful for detailed and specific analysis, such as customer feedback or sales tracking. Other applications could be:

• Customer segmentation: Using specific data to target particular customer segments.
• Internal process optimization: Identify specific areas to improve operational efficiency.
• Product or service customization: Tailor products or services to individual preferences.
• Etc.

Big data is used in market analysis, fraud detection, social network analysis and many other applications such as:

• Market analysis: identifying large-scale market trends and predicting changes.
• Supply chain optimization: Improve inventory management and large-scale distribution.
• Research and development: Using big data for innovation and discovery of new opportunities.
• Etc.

Benefits and Challenges

Each category of data offers unique benefits and challenges for companies. Small data enables quick and targeted decision making, but can miss broader opportunities. On the other hand, big data offers a more complete view, but can be costly and complex to analyze.

There is no one-size-fits-all approach; each category has its place and purpose, and the choice between them depends on a company’s objectives and resources. Whether it’s through small, highly meaningful data sets or big data analytics, organizations can make the most of their data investment and gain valuable insights that drive growth and success.