Avoiding Data Overload
In addition to understanding what raw RFID tag data looks like, it is important to understand the truth behind the common concern of RFID data overload and how it can easily be avoided. Many people believe that large RFID systems with hundreds of thousands of tags, and potentially hundreds of readers sending the raw data from these tags into a system can easily become overloaded. The truth is that only a very small percentage of this raw data can be classified as useful. The real goal for avoiding data overload lies in eliminating useless data, sending only valuable information further into the system. Techniques to reduce data volumes include smoothing, filtering, or simply distributing the data load over a number of locations through local processing at the edge of the network
As a data overload example, suppose a warehouse is looking to have a near real-time view of changing inventory levels. To do this, assume the warehouse holds 10,000 items tagged with 96-bit RFID tags. Readers take inventory every two minutes. Further, assume that 1,000 items are removed from the warehouse every day, while 1,000 new items are added. In this scenario, approximately 117 kilobytes (10,000 x 96 bits) of data gets generated every two minutes, which is about 3.5 megabytes (30 x 117 kilobytes) of data every hour, or about 85 megabytes (24 x 3.5 megabytes) per day. But because the only data the company is concerned is changes in inventory, only 192,000 bits ((the 1,000 added +1,000 removed) x 96 bits) is valuable, equaling less than 1 percent of the data potentially generated by the system.
Smoothing and filtering are two very effective ways of eliminating useless data before it propagates into enterprise systems and applications. Smoothing is the ability to ignore temporary data inconsistencies to avoid generating meaningless events. For example, if a reader reads a specific RFID tag, then the tag isn't read during a 5 second span only to re-appear thereafter, was that tag ever really gone, or did some external interference simply get in the way? RFID system designers must balance the system's tolerance for temporary data inconsistencies by ignoring useless data and events while still maintaining system performance.
Filtering is the process of reporting only on tags that match a certain criteria. For example, a simple filter might only generate activity reports for RFID-tagged pallets that enter a warehouse, and might discard any item- or case-level tags picked up by the RFID readers. So, although you may often hear about RFID systems generating and collecting massive amounts of data, only a very small amount of that data needs to be propagated into enterprise systems if smoothing and filtering are used correctly.
The location in the network where filtering or smoothing occurs can greatly impact the effectiveness of a filtering or smoothing implementation, as well as the overall flexibility of a system. If the filtering occurs at the RFID reader level, such as by ignoring all except pallet-level reporting, the RFID system will be difficult to modify if the company decides to use case or item tracking in the future.
Another problem is that you can dramatically increase the ROI of an RFID system if multiple applications share the RFID data. Each application may potentially care about different items than the other applications. Thus, if the data for some items gets discarded at the reader level, new applications will not be able to get access to it.
A second option could be to perform smoothing and filtering far away from the readerat the enterprise layer. However, filtering or smoothing at the enterprise or at some central location wastes all the bandwidth involved in moving useless data through the network, only to have it discarded. Another option is to put the filtering and smoothing logic for an application right into the application code. This seems like a reasonable idea, until you consider the task of building and maintaining a filtering and smoothing engine in your application code; not a recommended approach for any application design team.
A better solution is to perform filtering and smoothing operations at the RFID middleware layer. Some RFID middleware platforms contain filtering and smoothing capabilities, discarding useless data at the edge of the network so that application developers work only with important events or processed reports. This ability to perform filtering, smoothing and other logic at the edge of the network is often referred to as edge processing.
Edge Processing to Expose Valuable Information
One common method of exposing valuable RFID information to enterprise applications is through Application-Level Events (ALE). The ALE specification, published by EPCGlobal Inc., provides a standards-based method for applications to request and receive tag data for EPC tags. ALE implementations provide the ability to filter and consolidate data from multiple sources, and then expose the data in the form of periodic, XML-based event cycle reports. These reports summarize the tag activity during the event cycle, and often have configurable options for duration, along with filtering and grouping based on the fields of a URI representation. Because these reports contain only the URI or raw HEX representation of each tag, it is up to the application that receives the report to process the XML document and add the business context. For example, an application might be responsible for mapping a company prefix to the actual company name for display or storage.
It is important to note that ALE by itself is not business logic. ALE processing is largely infrastructural, in that it only reports the specific details of data gathering. No business semantics are expressed in event cycle reports; only the consumers of ALE reports truly implement the business logic of an application. As an example, an ALE event cycle report might contain the information that at Location L, within the time interval Time1-Time2, 100 case-level EPC tags, and 1 pallet-level EPC tag were read. A business application would be required to express the business-level semantics of this report: at Location L, at time Time2, 100 cases were loaded onto a specific pallet.
Although the ALE standard has been ratified and incorporated into a number of applications and RFID middleware offerings, it has some limitations that make it unsuitable for a variety of RFID implementations. These limitations include:
- ALE reports expose the URI representation of EPC tags; therefore application developers must understand how to decode each URI to extract the meaningful fields. For example, if an application receives an ALE report with the URI representation urn:epc:tag:sgtin-96:3.0037000.06542.773346595, it would need to know that the representation stores the Company Prefix between the first two periods after the last colon. This logic must be implemented by the application because the ALE event cycle report does not provide direct mapping between field names and values.
- ALE uses pattern matching against the URI representation for filtering and grouping, meaning that the person configuring reports needs to know the format of the URI representation.
- The current ALE standard does not provide the flexibility to report on custom data potentially stored in EPC generation 2 tags, meaning that an application processing the report must manually decode the custom data fieldand also that you can't perform filtering or grouping on specific sections of the custom data field.
- Most importantly, the ALE specification only reports on EPC tags, so systems that use ISO tags cannot leverage the ALE specification.