|“||is processed and analyzed in real time, or near real time, and has to be handled in a very different way than data at rest (i.e., persistent data). Data in motion tends to resemble event-processing architectures, and focuses on real-time or operational intelligence applications.||”|
"Data-in-motion comprises traffic moving over LANs, WANs, the Internet, etc. Another form of data-in-motion is transport of data via mobile media (e.g. flash drives, portable hard drives, laptops, etc.)."
Big data Edit
"Typical characteristics of data in motion that are significantly different in the era of Big Data are velocity and variability. The velocity is the rate of flow at which the data is created, stored, analyzed, and visualized. Big Data velocity means a large quantity of data is being processed in a short amount of time. In the Big Data era, data is created and passed on in real time or near real time. Increasing data flow rates create new challenges to enable real- or near real-time data usage. Traditionally, this concept has been described as streaming data. While these aspects are new for some industries, other industries (e.g., telecommunications) have processed high volume and short time interval data for years. However, the new in-parallel scaling approaches do add new Big Data engineering options for efficiently handling this data.
"The second characteristic for data in motion is variability, which refers to any change in data over time, including the flow rate, the format, or the composition. Given that many data processes generate a surge in the amount of data arriving in a given amount of time, new techniques are needed to efficiently handle this data. The data processing is often tied up with the automatic provisioning of additional virtualized resources in a cloud environment. Detailed discussions of the techniques used to process data can be found in other industry publications that focus on operational cloud architectures.20 21 Early Big Data systems built by Internet search providers and others were frequently deployed on bare metal to achieve the best efficiency at distributing I/O across the clusters and multiple storage devices. While cloud (i.e., virtualized) infrastructures were frequently used to test and prototype Big Data deployments, there are recent trends, due to improved efficiency in I/O virtualization infrastructures, of production solutions being deployed on cloud or Infrastructure-as-a-Service (IaaS) platforms. A high-velocity system with high variability may be deployed on a cloud infrastructure, because of the cost and performance efficiency of being able to add or remove nodes to handle the peak performance. Being able to release those resources when they are no longer needed provides significant cost savings for operating this type of Big Data system. Very large implementations and in some cases cloud providers are now implementing this same type of elastic infrastructure on top of their physical hardware. This is especially true for organizations that already have extensive infrastructure but simply need to balance resources across application workloads that can vary."