1.0 Introduction

The phrasal word ‘Big Data’ means literally an enormously huge volume of data. In the present information era, terabytes of data are generated and petabytes of data are processed everyday with the outreach of ubiquitous social media, online shopping and many other mobile and internet based applications. The traditional methodologies became inadequate to process, analyze and extract information from these huge data volumes from diverse sources and of new and different data types. So, new ‘Big Data’ methodologies have been developed in dealing with this barrage of data and to help governments, businesses and industries in taking informed decisions for optimal results. Though big data tools can be used to achieve a number of objectives from cost reduction, offering new services to developing new products, it is discussed in the following sections that how ‘Big Data’ is being used in the optimization of a business process, product performance and new product development.

1.2 Statement of the problem

One of the classic examples of the big data model is the ORION project undertaken by UPS. The world’s largest shipping services company UPS, Inc. is one of the earliest players in handling huge volumes of data, by tracking almost 16 million packages, every day, with almost 40 million tracking requests from more than 8 million customers and by storing petabytes of data. As part of the ORION (On-Road Integrated Optimization and Navigation) project, UPS installed a telematics framework in their 46,000 delivery vehicles, that basically comprised different vehicle sensors and GPS tracking devices which in turn connected to the driver’s mobile device so that they could capture different types of data related to vehicle movement, delivery routes and the time taken for the delivery (UPS Pressroom, 2016, pp. 1). UPS had developed a proprietary algorithm to process and analyze all that data and translate it into an optimized delivery route instruction for their driver to save fuel, time and hence cost.

1.3 Objective

-The objective of the study is to identify the role of business analytics in managing big data

1.3.1 Specific objectives

-To access the issues around big data with respect to product optimization

-To access big data analysis for process improvement with the help of real data

1.4 Significance of the study

According to UPS, a mile saved by each driver everyday would yield them an annual financial benefit of $50 million, which is an ultimate case of process optimization achieving a great cost reduction in the big data model. UPS is now aiming to offer a new range of personalized services to their customers by leveraging this model and to apply this same model to its air fleet as well. Bank of America has adopted a behavioral analytics platform from Sociometric Solutions, an MIT based startup (UPS Pressroom, 2016, pp. 1). They issued their call center employees some special ID cards equipped with sensors to capture data on how they move around their workplace and the way they talk. All this data has been integrated onto a big data analytics platform to identify some behavioral patterns of the best performing employees. They found that people taking breaks together are doing great and implemented that concept of group breaks only to see a 20% increase in productivity.



2.0 Introduction

The best example of the product optimization using bigger data model is demonstrated by the Xcel Energy’s project in Boulder, Colorado. We have these traditional electricity meters installed in our homes and all other places to monitor the power consumption. And the billing is usually done by taking these meter readings manually. Xcel Energy installed a new product called ‘smart electric meter’ at 23,000 places in Boulder City of Colorado. This smart meter is fully automated and transfers the information of energy consumption to an internet based platform called ‘My account’ portal, where users can log into their individual accounts and monitor their electricity consumption even over 15 minute duration. This smart metering lets them know where energy is being consumed more and how they can reduce it empowering them to come up with an effective energy management. This also helps energy companies in responding quickly to the power outages and also in forecasting the electricity consumption needs in that area. It provides numerous other secondary benefits like paperless billing and reduced number of visits by the personnel of service providers. If we imagine the volumes of data that would be created, processed and stored if this ‘Smart Grid City’ platform of Xcel Energy is extended to an entire state and country, it can easily be concluded that only a ‘Big Data’ framework can enable this change. Another great example is General Electric’s model of monitoring the health of their jet engine’s blades. The blades of a gas turbine are equipped with sensors that collect information of health indicators like ‘stress cracks’ which help in assessing the breaking point of the blades so that they can be prepared in advance to avoid the break. Each gas turbine’s sensor would create an amount of 500 GB of data on a daily basis, and they have 12,000 such gas turbines in total. Only a big data analytics platform is capable of extracting insights from such huge volumes of data.

Big data is sometimes referred to as “3Vs”, this is because it basically entails more volume, higher rates of velocity and more variety which the current technology and techniques may not be able to handle the storage and the processing of the data (John, 2014, pp.352). In terms of volume, there is more data than ever been before. A lot of factors contribute to the volume of data, increasing this include increasing quantities of data sensor being collected and also the text data, which is often stream from social media. Issues emerge on to create value from the data that is relevant given that there is increasing cost of storage and determining relevance amidst the large volume of data. On the other hand, velocity is how fast it would take for data to be produced and processed to meet demands. It is a combination of data management and infrastructure that addresses different concerns that are valuable in creating and adding big data objects in the system. This covers the factors like website response, fast updates across all data stores and execution time of transaction. When dealing with variety, data structures, database and excel tables has changed so much in terms of structure and formats. Most structures can no longer be imposed like in the past in order for them to keep over analysis. There are however more than the 3Vs in data management nowadays due to increase in complexity in organizations (Matheson, 2014, pp. 456).

2.1 Using big data to improve manufacturing

In order to assess and improve various practices, analytics alludes to the utilization of insights and other scientific apparatuses to the information in administration in business. Operations managers can apply the use of advanced analytics so as to know historical process data, know the patterns and relationships among the discrete steps of processing and optimizing the factors that have the highest effects on the produce. Most of the global manufacturers in many industries and places have the capability to conduct advanced statistical assessments. They are making use of the previously isolated data sets thus aggregating them and analyzing the data as to reveal the important steps and insights to be taken. Even within the manufacturing operations that are considered best, using advanced analytics may increase the opportunities for better yields. This case was experienced at one already established European company for making specialty and functional chemicals for a number of industries, including detergents and paper industries. The average produce was remarkably higher than the industries benchmark and staffers were even so skeptical that there was ability for more improvement. Several unexpected insights came up when the company used advanced analytics that are based on how the brain of human can process given data or information to quantify, and value the impact of different inputs of productions on the results or yields. Efforts to focus on using data to more effectively manage the relationship with the customers is key for improvement (Davenport, 2012, pp.208). Meanwhile, a precious metal mine increased its profitability and yields by keenly assessing data of production that were not even complete. The company was going through a time in which its ores grade were significantly declining, hence a strategy that it would use to maintain its level of production was speeding up or optimizing its process of extraction and refining. The process and production data that were being used at the mine were so much fragmented hence the initial step for the analytics team was cleaning it up and accounting for information gaps.

2.2 Capitalizing on big data

For manufacturers who want to use advanced analytics to improve their production, they should first consider the amount of data the company has at its disposal. The majority of the companies gets a lot of process data, but only use the data for purposes of tracking and for the reason of improving its operations. For these companies, the challenge is investing in systems and skills that would likely allow them to make optimum use of the available process information, that is, they index or centralize data from different sources so that they can be easily analyzed or hiring data analysts who have vast knowledge in drawing and spotting insights of information. Companies, especially with production cycles that go yearly or monthly long have very little data to be meaningful statistically when put under the scrutiny of analysts. The management at these companies has the challenges of taking long-term focus and also putting investments in systems and the practices that gather more data. These companies can, for instance, gather information about a certain important or complex process steps within the activities of the larger chain, therefore applying more complex analysis the various parts of the process. Even with the emergence of big data, the advanced analytics practice is rooted in more time for research mathematics and scientific application. The companies that set up their capabilities in quantitative assessment successfully can put themselves far ahead of their competitors.

Optimizing data driven automated feedbacks is possibly one of the biggest potential big data benefits in enterprises. Optimization activities without any innovation that is disruption driven can lead to very strategic outcomes. On the other hand, getting into a different or a new market altogether is always risky, especially when one is creating a market which was not in existence before. They do not have much information about the potential customers and they also lack extensive relevant data about the customer behavior of buying and the related markets. Lacking useful optimization in terms of poor data, lack of enough analytics or even poor management strategies may pose some very big risks for innovation. It is a mistake for one to believe that they can innovate without proper insight of what they want to venture into. For the companies that understand balancing optimization luck is a factor when determining the level of success of a particular innovation, this is because the few people who manage in venturing into innovation without much insight are simply lucky since most of these people fail. Making use of the big data driven optimization to the maximum level is a way that companies make much in their widgets, the same way that it is for any company that wants establish innovation is its basis of competency. For good success the companies should know the time they should optimize and when to disrupt not forgetting knowing when to optimize and when to disrupt.

2.3 Big data structures and skills

The most used structures to accommodate or even initiate the big data technologies in the current organization day to day operations are grouped with IT organizations or analytics groups or groups for innovation. The companies that have the approaches that are effective and seemingly easily succeed build very close relationship between the IT organizing teams and the business groups addressing the big data. Most of the big firms augment their analytical staffs that are proficient in data analysis and have a vast ability to manipulate the big data technologies compared to the traditional analysts. The skills might include text mining, video analytics and also the visual analysts. An important skill involves the ability to explain the outcomes of big data to the senior executives either in verbal or visual terms. In the operational processes the prescriptive analytical models are useful, and those who need to work with front line workers or the owners of processes to give the required changes in the roles and skills. The people who do the work on big data management and control should have the skills needed and a team based approach in assembling the given data. The background of most of them is supposed to be stronger and much relevant scientific knowledge and others who are experienced programmers with skills in data analysis.


3.0 Implementing big data for process and product optimization

To aggregate, analyze and also visualize big data, a variety of technologies and technologies have been developed. They are drawn from many different fields that include economics, mathematics, statistics and computer science. The techniques and technologies have been successfully adapted and are applied to a large set of more complex and diverse data. They can be applied to datasets that are more diverse and large. Some of the technology and techniques include association rule learning which consists of a variety of algorithms to test and generate the possible rules that can be used to identify strong rules for database using many measures. An application is where the retailer can determine the products that are mostly bought and hence use the acquired information for determining that practices to be put in the market. Another technique to use is the method of identifying categories which the new observations belong. This calls for a lot training to accurately identify the required observations. This can be applied in predicting specific customer behavior where there is a clear hypothesis outcome (John, 2014, pp.354). Also, data, integrating and fusion technique helps in analyzing data from multiple sources so as to develop insights in the ways that are more accurate than using a single source of data to analyze.

Moreover, data mining is the set of technique used to extract various patterns from large sets of data by integrating methods from machine learning and statistics with management in the database. The main reason for the data mining process is to get information from a data set and transform the information into structures that can be easily understandable for further or even later use. This can be used in mining the data of the customer so that one can know the segments to respond to an offer, or the resources data to identify characteristics of the employees who are most successful in their duties. On the other hand, machine learning deals with the development and design of algorithms that enables the computers to evolve characters based on data.

3.1 Problems associated with big data

Many applications have problems that come as a result of big data, these include, business forecasting, geospatial classification and risk analysis due to network traffic. Network detection and intrusion are very sensitive on time and thus require much efficient big data techniques to tackle the problems when they arise. Big data often have many data points that are meaningless hence gives the analysts very hard time to distinguish the information that are needed and those that are not. Privacy problems arise when dealing with big data in product and process optimization, for instance, in separating what can be seen on the social networks. Big data is much complicated and the huge size requires services from high performance computers and systems.

3.2 Advantages

When using real time in the processing of data, errors that are within the organization are very easy to identify. This helps the companies to react quickly to the problems, hence can save the company from failing. It is also very easy to notice the new strategies that have been imposed by the competitors, the analytics easily get notified the moment that the competitors are putting in new strategies or even lowering the prices of their products for example. When frauds are about to happen or have happened already, it can be detected and the necessary steps taken to limit the damages that it can cause. Also helps in keeping the trends of the customers by providing the valuable information needed concerning the customer trend. One of the main reason of big data using the analytics is time and cost saving and enabling the users to deal with seemingly larger datasets even in environments with minimal resources (Wen-Chen, 2013, pp.26). The big data usage is sometimes also very predicative.



Week1 Week2 Week3 Week4 Week5 Week6 Week7 Week8 Week9 Week10
Gathering of Data for big data
Gathering of product optimization data
Process improvement
Using real data for analysis
Conclusions and new developments



Big data is bringing about drastic changes in improving a process or the performance of a product across the spectrum of both transactional businesses and manufacturing industries. Even within an industry or business big data analytics are applied for different purposes ranging from cost reduction and increasing productivity by reducing the carbon emissions. Big data analytics can be aligned with each phase of a six sigma methodology like DMAIC and flawless process performance can be achieved.




UPS Pressroom, 2016. ORION Fact Sheet. [Online] Available at:              <            Factsheets&id=1426321623549-553> [Accessed 4 Jul. 2016].

Matheson, R., 2014. Moneyball for Business. [Online] Available at: <>

[Accessed 4 Jul. 2016].

John, W. 2014. Encyclopedia of Business Analytics and Optimization. Pennsylvania: IGI Global.

Davenport, T. 2012. Enterprise Analytics: Optimize Performance, Process, and Decisions Through Big Data. Upper Saddle River, New Jersey: FT Press.

Wen-Chen, H. 2013. Big Data Management, Technologies, and Applications. Pennsylvania: IGI Global.