When it comes to managing traffic, the collection and use of data is a key component in the deployment of intelligent transportation systems. In recent years, the availability of data and the quantity of data retrieval points has given transportation agencies access to more data than ever before. Armed with this large amount of data and an increasing number of other, non-traditional data points, the transportation industry has entered the world of Big Data.
Big Data refers to the capture, analysis and utility of several advanced data points, the volume of which is so large and wide-ranging it surpasses the capabilities of more traditional data management and analysis tools. Within the transportation industry, two forms of Big Data have gained attention: crowdsourced data, where users freely provide information through applications like Waze; and traffic flow data like the information gathered by tracking cell phones or other Bluetooth®-enabled devices. Many agencies see these forms of data collection as a less expensive way to gather traffic analytics, and while this is true for certain applications, many of these same agencies are seeing holes in the techniques they are employing to fulfill key traffic metrics.
Both crowdsourced and flow data collection techniques gather all data and are blind to sampling, making it possible for agencies to place fewer collection points at random locations rather than try and guess where to put them. The raw information provided can measure large populations of vehicles within a transportation system and can help us understand the “big picture” of a traffic system; unfortunately, these data sources miss many critical points of information that more traditional statistical samples provide.
Effective ITS aims to provide traffic applications that offer value to the driving public. In the ITS industry, the approach to data collection is often determined by the type of application being used. In other words, different applications have different detection needs, often at a more granular level than Big Data techniques can provide. If applications suffer due to inappropriate data collection methods, the public perception of the transportation agency’s value is diminished.
For example, when collecting lane-by-lane data for managed motorway, wrong-way detection or ramp metering applications, statistical sampling data is required. This allows for small fractions or granular data points of the population to be sampled. Per vehicle, per lane and other more granular data points allow systems to make specific decisions for specific locations; whereas Big Data includes nearly all the population without knowing how that population is affecting the point at which the data is being collected.
With flow data, a vehicle can be detected but speed and direction cannot be determined. There is no way of knowing if this vehicle is traveling the wrong way or if the vehicle’s speed is affecting other traffic on the motorway. That kind of information is only available with statistical sampling, and that can only be done with spot sensor detection.
Watch a short video showing how the graphics for this story were created using data from SmartSensor HD
Spot sensor detection is exactly what its name implies – detection of certain conditions at a specific spot or location. For many years, spot detection was accomplished by embedding coiled wires, or loops, in the roadway to create a magnetic field that was disrupted when a vehicle passed over. By placing loops sequentially, it was possible to know the number of vehicles passing that location, their speeds, and the sizes or classes of those vehicles. Today, many traffic agencies accomplish the same thing non-intrusively, with video or radar devices installed above the ground. These sensors provide a wealth of information beyond the “big picture” data available from Big Data, making them more appropriate for a variety of traffic applications.
A good example of this is the use of ramp meters for motorways. Ramp metering systems can change instantly depending on the data coming from both the mainline of traffic and traffic queuing on the ramp. While Big Data collects large, electronic “breadcrumbs,” they require a much larger analytical approach to analyze the data, incorporating data science of pattern recognition and machine learning. This type of data is simply not specific or nimble enough to handle the moment-to-moment reality of a ramp metering system. Hence, all Big Data systems must be supplemented with more granular sampling data to both verify the overall findings of Big Data, and to meet the moment-to-moment needs of the application.
Big Data systems must be supplemented with more granular sampling data to both verify the overall findings of Big Data, and to meet the moment-to-moment needs of the application.
Additionally, many agencies are involved in improving traffic situations through experimentally designed solutions. In these cases, granular data provides statistical validity of the design. This allows immediate feedback on the impact these solutions are having on drivers. While Big Data does a better job of collecting data for larger purposes and is often used to analyze and address new questions, it is unable to determine whether a specific solution is working in the moment.
In the future, we will see the large-scale implementation of connected vehicles and vehicle-to-infrastructure communications. These connected systems will produce an ocean of new data and will be, of course, a game changer. However, even as data becomes bigger, the need for granular spot detection data will remain. Yes, Big Data is important and will become more important over time, but to ignore the benefits of the per lane and per vehicle data provided by spot data points will put the effectiveness of the traffic applications drivers depend on at risk and lower the value of the transportation agency that fails to provide.
