There's levels to this
And I'm heavy as it get - Meek Mill
Kapell’s enrichment pulls together many sources. Each one takes time to fetch. This latency cost has been offset with a per-process, in-memory least recently used (LRU) cache which is lovely if a lot of people are listening to the same things, or if you never restart the API!
I expect, however, that folks will listen to a lot of different music. This will put pressure on the backend to spend a lot of time fetching data from the likes of Wikipedia over and over again. To improve performance the API now includes hierarchy of caches with the inclusion of a new Postgres-based second level (L2) cache. This cache aggressively holds slow-moving data like album intro text, avoiding a need to refetch. Caching this data both improves performance by huge margins and makes us a more polite client that consumes fewer of our sources… resources. Query durations to Wikipedia have a lot of variability, but the addition of this cache seems to shave a few hundreds of milliseconds — sometimes muliple seconds! — off enrichment queries. One of my test albums shows consistent responses in the 150ms range with network latency.
The downside is stale articles, but they don’t change much! We also use negative caching to avoid re-fetching a missing resource, with a shorter TTL so we can retry sooner than we would for a positive response.
Happy enriching!