Caching NetFlow 9 Templates

Processing NetFlow 9 data requires caching the templates used to process the messages. When you configure NetFlow 9 properties, you can specify the maximum number of templates to cache and how long to allow an unused template to remain in the cache. You can also configure the stage to allow an unlimited number of templates in the cache for an unlimited amount of time.

When you configure caching limitations, templates can be ejected from the cache under the following conditions:
  • When the cache is full and a new template appears.
  • When a template exceeds the specified idle period.

Configure NetFlow 9 caching properties to allow the stage to retain templates for processing in a logical way. When a record requires the use of a template that is not in the cache, the record is passed to the stage for error handling.

For example, say you use the UDP Source origin to process NetFlow 9 data from five servers. Each server sends data using a different template, so to process data from these five servers, you can set the cache size to five templates. But to allow for additional servers that might be added later, you might set the template cache to a higher number.

Most servers resend templates periodically, so you might take this refresh interval into account when you configure the cache timeout.

For example, say your server resends templates every three minutes. If you set the cache timeout for two minutes, then a template that hasn't been used in two minutes gets evicted. If the server sends a packet that requires the evicted template, the stage generates an error record because the template is not available. If you set the cache timeout for four minutes and an unlimited cache size, then the templates from all servers remain in the cache until replaced by a new version of the template.

Note: Data Collector keeps the cached templates in memory. If you need to cache large numbers of templates, you might want to increase the Data Collector heap size accordingly.Data Collector heap size accordingly. For more information, see Java Heap Size in the Data Collector documentation.