Evaluation of the ngd cache performance and types

[erikyao]

Status Quo

Current implementation of NGD caches is in https://github.com/biothings/pending.api/blob/master/web/utils/cache.py. It's made due to that functools.lru_cache cannot decorate async def functions.

Alternative: cachetools

• Github: https://github.com/tkem/cachetools
• Docs: https://cachetools.readthedocs.io/en/latest/

Performance Comparison

Is cachetools.LRUCache faster than our vanilla implementation?

Which type is best for the NGD scenario?

cachetools provided the following types of caches:

• cachetools.FIFOCache: First In First Out
• cachetools.LFUCache: Least Frequently Used (will be discarded)
• cachetools.LRUCache: Least Recently Used (will be discarded)
• cachetools.MRUCache: Most Recently Used (will be discarded)
• cachetools.RRCache: Random Replacement (item selected by random.choice())
• cachetools.TTLCache: LRU Cache with per-item time-to-live (TTL)
• cachetools.TLRUCache: Time-aware LRU; similar to cachetools.TTLCache but the TTL calculation can be customized

Which type works best for NGD?

Side Note: cachetools thread-safety

All the above cache classes are not thread-safe. However, cachetools provides thread-safe decorators; see cachetools.func - functools.lru_cache() compatible decorators.

Since we are using corotines, thread-safety is not an issue so far.