Class AbstractDynamicMetadataResolver

All Implemented Interfaces:
Component, DestructableComponent, IdentifiableComponent, IdentifiedComponent, InitializableComponent, Resolver<EntityDescriptor,​CriteriaSet>, ClearableMetadataResolver, DynamicMetadataResolver, MetadataResolver
Direct Known Subclasses:
AbstractDynamicHTTPMetadataResolver, LocalDynamicMetadataResolver

public abstract class AbstractDynamicMetadataResolver
extends AbstractMetadataResolver
implements DynamicMetadataResolver, ClearableMetadataResolver
Abstract subclass for metadata resolvers that resolve metadata dynamically, as needed and on demand.
  • Field Details

    • METRIC_TIMER_FETCH_FROM_ORIGIN_SOURCE

      public static final String METRIC_TIMER_FETCH_FROM_ORIGIN_SOURCE
      Metric name for the timer for fetchFromOriginSource(CriteriaSet).
      See Also:
      Constant Field Values
    • METRIC_TIMER_RESOLVE

      public static final String METRIC_TIMER_RESOLVE
      Metric name for the timer for resolve(CriteriaSet).
      See Also:
      Constant Field Values
    • METRIC_RATIOGAUGE_FETCH_TO_RESOLVE

      public static final String METRIC_RATIOGAUGE_FETCH_TO_RESOLVE
      Metric name for the ratio gauge of fetches to resolve requests.
      See Also:
      Constant Field Values
    • METRIC_GAUGE_NUM_LIVE_ENTITYIDS

      public static final String METRIC_GAUGE_NUM_LIVE_ENTITYIDS
      Metric name for the gauge of the number of live entityIDs.
      See Also:
      Constant Field Values
    • METRIC_GAUGE_PERSISTENT_CACHE_INIT

      public static final String METRIC_GAUGE_PERSISTENT_CACHE_INIT
      Metric name for the gauge of the persistent cache initialization metrics.
      See Also:
      Constant Field Values
    • log

      private final org.slf4j.Logger log
      Class logger.
    • metricsBaseName

      @NonnullAfterInit private String metricsBaseName
      Base name for Metrics instrumentation names.
    • timerResolve

      @Nullable private com.codahale.metrics.Timer timerResolve
      Metrics Timer for resolve(CriteriaSet).
    • timerFetchFromOriginSource

      @Nullable private com.codahale.metrics.Timer timerFetchFromOriginSource
    • ratioGaugeFetchToResolve

      @Nullable private com.codahale.metrics.RatioGauge ratioGaugeFetchToResolve
      Metrics RatioGauge for count of origin fetches to resolves.
    • gaugeNumLiveEntityIDs

      @Nullable private com.codahale.metrics.Gauge<Integer> gaugeNumLiveEntityIDs
      Metrics Gauge for the number of live entityIDs.
    • gaugePersistentCacheInit

      @Nullable private com.codahale.metrics.Gauge<AbstractDynamicMetadataResolver.PersistentCacheInitializationMetrics> gaugePersistentCacheInit
      Metrics Gauge for the persistent cache initialization.
    • taskTimer

      @Nullable private Timer taskTimer
      Timer used to schedule background metadata update tasks.
    • createdOwnTaskTimer

      private boolean createdOwnTaskTimer
      Whether we created our own task timer during object construction.
    • minCacheDuration

      @Nonnull private Duration minCacheDuration
      Minimum cache duration.
    • maxCacheDuration

      @Nonnull private Duration maxCacheDuration
      Maximum cache duration.
    • negativeLookupCacheDuration

      @Nonnull private Duration negativeLookupCacheDuration
      Negative lookup cache duration.
    • refreshDelayFactor

      @Positive private Float refreshDelayFactor
      Factor used to compute when the next refresh interval will occur. Default value: 0.75
    • maxIdleEntityData

      @Nonnull private Duration maxIdleEntityData
      The maximum idle time for which the resolver will keep data for a given entityID, before it is removed.
    • removeIdleEntityData

      private boolean removeIdleEntityData
      Flag indicating whether idle entity data should be removed.
    • expirationWarningThreshold

      @Nonnull private Duration expirationWarningThreshold
      Impending expiration warning threshold for metadata refresh. Default value: 0 (disabled).
    • cleanupTaskInterval

      @Nonnull private Duration cleanupTaskInterval
      The interval at which the cleanup task should run.
    • cleanupTask

      The backing store cleanup sweeper background task.
    • persistentCacheManager

      private XMLObjectLoadSaveManager<EntityDescriptor> persistentCacheManager
      The manager for the persistent cache store for resolved metadata.
    • persistentCacheKeyGenerator

      private Function<EntityDescriptor,​String> persistentCacheKeyGenerator
      Function for generating the String key used with the cache manager.
    • initializeFromPersistentCacheInBackground

      private boolean initializeFromPersistentCacheInBackground
      Flag indicating whether should initialize from the persistent cache in the background.
    • backgroundInitializationFromCacheDelay

      @Nonnull private Duration backgroundInitializationFromCacheDelay
      The delay after which to schedule the background initialization from the persistent cache.
    • initializationFromCachePredicate

      private Predicate<EntityDescriptor> initializationFromCachePredicate
      Predicate which determines whether a given entity should be loaded from the persistent cache at resolver initialization time.
    • persistentCacheInitMetrics

      Object tracking metrics related to the persistent cache initialization.
    • indexes

      private Set<MetadataIndex> indexes
      The set of indexes configured.
    • initializing

      private boolean initializing
      Flag used to track state of whether currently initializing or not.
  • Constructor Details

    • AbstractDynamicMetadataResolver

      public AbstractDynamicMetadataResolver​(@Nullable Timer backgroundTaskTimer)
      Constructor.
      Parameters:
      backgroundTaskTimer - the Timer instance used to run resolver background management tasks
  • Method Details

    • isInitializeFromPersistentCacheInBackground

      public boolean isInitializeFromPersistentCacheInBackground()
      Get the flag indicating whether should initialize from the persistent cache in the background.

      Defaults to: true.

      Returns:
      true if should init from the cache in background, false otherwise
    • setInitializeFromPersistentCacheInBackground

      public void setInitializeFromPersistentCacheInBackground​(boolean flag)
      Set the flag indicating whether should initialize from the persistent cache in the background.

      Defaults to: true.

      Parameters:
      flag - true if should init from the cache in the background, false otherwise
    • getBackgroundInitializationFromCacheDelay

      @Nonnull public Duration getBackgroundInitializationFromCacheDelay()
      Get the delay after which to schedule the background initialization from the persistent cache.

      Defaults to: 2 seconds.

      Returns:
      the delay
      Since:
      3.3.0
    • setBackgroundInitializationFromCacheDelay

      public void setBackgroundInitializationFromCacheDelay​(@Nonnull Duration delay)
      Set the delay after which to schedule the background initialization from the persistent cache.

      Defaults to: 2 seconds.

      Parameters:
      delay - the delay
      Since:
      3.3.0
    • getPersistentCacheManager

      @Nullable public XMLObjectLoadSaveManager<EntityDescriptor> getPersistentCacheManager()
      Get the manager for the persistent cache store for resolved metadata.
      Returns:
      the cache manager if configured, or null
    • setPersistentCacheManager

      public void setPersistentCacheManager​(@Nullable XMLObjectLoadSaveManager<EntityDescriptor> manager)
      Set the manager for the persistent cache store for resolved metadata.
      Parameters:
      manager - the cache manager, may be null
    • isPersistentCachingEnabled

      public boolean isPersistentCachingEnabled()
      Get the flag indicating whether persistent caching of the resolved metadata is enabled.
      Returns:
      true if enabled, false otherwise
    • getPersistentCacheKeyGenerator

      @NonnullAfterInit public Function<EntityDescriptor,​String> getPersistentCacheKeyGenerator()
      Get the function for generating the String key used with the persistent cache manager.
      Returns:
      the key generator or null
    • setPersistentCacheKeyGenerator

      public void setPersistentCacheKeyGenerator​(@Nullable Function<EntityDescriptor,​String> generator)
      Set the function for generating the String key used with the persistent cache manager.
      Parameters:
      generator - the new generator to set, may be null
    • getInitializationFromCachePredicate

      @NonnullAfterInit public Predicate<EntityDescriptor> getInitializationFromCachePredicate()
      Get the predicate which determines whether a given entity should be loaded from the persistent cache at resolver initialization time.
      Returns:
      the cache initialization predicate
    • setInitializationFromCachePredicate

      public void setInitializationFromCachePredicate​(@Nullable Predicate<EntityDescriptor> predicate)
      Set the predicate which determines whether a given entity should be loaded from the persistent cache at resolver initialization time.
      Parameters:
      predicate - the cache initialization predicate
    • getMinCacheDuration

      @Nonnull public Duration getMinCacheDuration()
      Get the minimum cache duration for metadata.

      Defaults to: 10 minutes.

      Returns:
      the minimum cache duration
    • setMinCacheDuration

      public void setMinCacheDuration​(@Nonnull Duration duration)
      Set the minimum cache duration for metadata.

      Defaults to: 10 minutes.

      Parameters:
      duration - the minimum cache duration
    • getMaxCacheDuration

      @Nonnull public Duration getMaxCacheDuration()
      Get the maximum cache duration for metadata.

      Defaults to: 8 hours.

      Returns:
      the maximum cache duration
    • setMaxCacheDuration

      public void setMaxCacheDuration​(@Nonnull Duration duration)
      Set the maximum cache duration for metadata.

      Defaults to: 8 hours.

      Parameters:
      duration - the maximum cache duration
    • getNegativeLookupCacheDuration

      @Nonnull public Duration getNegativeLookupCacheDuration()
      Get the negative lookup cache duration for metadata.

      Defaults to: 10 minutes.

      Returns:
      the negative lookup cache duration
    • setNegativeLookupCacheDuration

      public void setNegativeLookupCacheDuration​(@Nonnull Duration duration)
      Set the negative lookup cache duration for metadata.

      Defaults to: 10 minutes.

      Parameters:
      duration - the negative lookup cache duration
    • getRefreshDelayFactor

      @Nonnull public Float getRefreshDelayFactor()
      Gets the delay factor used to compute the next refresh time.

      Defaults to: 0.75.

      Returns:
      delay factor used to compute the next refresh time
    • setRefreshDelayFactor

      public void setRefreshDelayFactor​(@Nonnull Float factor)
      Sets the delay factor used to compute the next refresh time. The delay must be between 0.0 and 1.0, exclusive.

      Defaults to: 0.75.

      Parameters:
      factor - delay factor used to compute the next refresh time
    • isRemoveIdleEntityData

      public boolean isRemoveIdleEntityData()
      Get the flag indicating whether idle entity data should be removed.
      Returns:
      true if idle entity data should be removed, false otherwise
    • setRemoveIdleEntityData

      public void setRemoveIdleEntityData​(boolean flag)
      Set the flag indicating whether idle entity data should be removed.
      Parameters:
      flag - true if idle entity data should be removed, false otherwise
    • getMaxIdleEntityData

      @Nonnull public Duration getMaxIdleEntityData()
      Get the maximum idle time for which the resolver will keep data for a given entityID, before it is removed.

      Defaults to: 8 hours.

      Returns:
      return the maximum idle time
    • setMaxIdleEntityData

      public void setMaxIdleEntityData​(@Nonnull Duration max)
      Set the maximum idle time for which the resolver will keep data for a given entityID, before it is removed.

      Defaults to: 8 hours.

      Parameters:
      max - the maximum entity data idle time
    • getExpirationWarningThreshold

      @Nonnull public Duration getExpirationWarningThreshold()
      Gets the impending expiration warning threshold used at refresh time.
      Returns:
      threshold for logging a warning if live metadata will soon expire
    • setExpirationWarningThreshold

      public void setExpirationWarningThreshold​(@Nullable Duration threshold)
      Sets the impending expiration warning threshold used at refresh time.
      Parameters:
      threshold - the threshold for logging a warning if live metadata will soon expire
    • getCleanupTaskInterval

      @Nonnull public Duration getCleanupTaskInterval()
      Get the interval at which the cleanup task should run.

      Defaults to: 30 minutes.

      Returns:
      return the interval
    • setCleanupTaskInterval

      public void setCleanupTaskInterval​(@Nonnull Duration interval)
      Set the interval at which the cleanup task should run.

      Defaults to: 30 minutes.

      Parameters:
      interval - the interval to set
    • getMetricsBaseName

      @NonnullAfterInit public String getMetricsBaseName()
      Get the base name for Metrics instrumentation.
      Returns:
      the Metrics base name
    • setMetricsBaseName

      public void setMetricsBaseName​(@Nullable String baseName)
      Set the base name for Metrics instrumentation.
      Parameters:
      baseName - the Metrics base name
    • getIndexes

      @Nonnull @NonnullElements @Unmodifiable @NotLive public Set<MetadataIndex> getIndexes()
      Get the configured indexes.
      Returns:
      the set of configured indexes
    • setIndexes

      public void setIndexes​(@Nullable Set<MetadataIndex> newIndexes)
      Set the configured indexes.
      Parameters:
      newIndexes - the new indexes to set
    • indexesEnabled

      protected boolean indexesEnabled()
      Return whether secondary indexing is effectively active.
      Returns:
      true if active, false if not.
    • clear

      public void clear() throws ResolverException
      Attempt to clear all data from the internal cache of the resolver.
      Specified by:
      clear in interface ClearableMetadataResolver
      Throws:
      ResolverException - if the clear operation was unsuccessful
    • clear

      public void clear​(@Nonnull String entityID) throws ResolverException
      Attempt to clear data from the internal cache of the resolver for the specified entityID.
      Specified by:
      clear in interface ClearableMetadataResolver
      Parameters:
      entityID - the target entityID
      Throws:
      ResolverException - if the clear operation was unsuccessful
    • resolve

      @Nonnull public Iterable<EntityDescriptor> resolve​(@Nonnull CriteriaSet criteria) throws ResolverException
      Specified by:
      resolve in interface Resolver<EntityDescriptor,​CriteriaSet>
      Throws:
      ResolverException
    • resolveEntityID

      @Nullable protected String resolveEntityID​(@Nonnull CriteriaSet criteria)
      Attempt to resolve the single entityID for the operation from the criteria set.

      If an EntityIdCriterion is present, that will be used. If not present, then a single entityID will be resolved via the secondary index manager of the backing store.

      Parameters:
      criteria - the criteria set on which to operate
      Returns:
      the resolve entityID, or null if a single entityID could not be resolved
    • resolveEntityIDs

      @Nonnull protected Set<String> resolveEntityIDs​(@Nonnull CriteriaSet criteria)
      Attempt to resolve all the entityIDs represented by the criteria set.

      If an EntityIdCriterion is present, that will be used. If not present, then entityIDs will be resolved via the secondary index manager of the backing store.

      Parameters:
      criteria - the criteria set on which to operate
      Returns:
      the resolved entityIDs, may be empty
    • resolveFromOriginSource

      @Nonnull @NonnullElements protected Iterable<EntityDescriptor> resolveFromOriginSource​(@Nonnull CriteriaSet criteria, @Nullable String entityID) throws ResolverException
      Fetch metadata from an origin source based on the input criteria, store it in the backing store and then return it.
      Parameters:
      criteria - the input criteria set
      entityID - the previously resolved single entityID
      Returns:
      the resolved metadata
      Throws:
      ResolverException - if there is a fatal error attempting to resolve the metadata
    • resolveFromOriginSourceWithEntityID

      @Nonnull @NonnullElements protected Iterable<EntityDescriptor> resolveFromOriginSourceWithEntityID​(@Nonnull CriteriaSet criteria, @Nonnull String entityID) throws ResolverException
      Fetch metadata from an origin source based on the input criteria when the entityID is known, store it in the backing store and then return it.
      Parameters:
      criteria - the input criteria set
      entityID - the entityID known to be represented by the criteria set
      Returns:
      the resolved metadata
      Throws:
      ResolverException - if there is a fatal error attempting to resolve the metadata
    • resolveFromOriginSourceWithoutEntityID

      @Nonnull @NonnullElements protected Iterable<EntityDescriptor> resolveFromOriginSourceWithoutEntityID​(@Nonnull CriteriaSet criteria) throws ResolverException
      Fetch metadata from an origin source based on the input criteria when the entityID is not known, store it in the backing store and then return it.
      Parameters:
      criteria - the input criteria set
      Returns:
      the resolved metadata
      Throws:
      ResolverException - if there is a fatal error attempting to resolve the metadata
    • lookupCriteria

      @Nonnull @NonnullElements protected Iterable<EntityDescriptor> lookupCriteria​(@Nonnull CriteriaSet criteria) throws ResolverException
      Lookup and return all EntityDescriptors currently available in the resolver cache which match either entityID or secondary-indexed criteria.
      Parameters:
      criteria - the input criteria set
      Returns:
      the resolved metadata
      Throws:
      ResolverException - if there is a fatal error attempting to resolve the metadata
    • processNonEntityIDFetchedEntittiesDescriptor

      @Nullable protected List<EntityDescriptor> processNonEntityIDFetchedEntittiesDescriptor​(@Nonnull EntitiesDescriptor entities) throws ResolverException
      Process an EntitiesDescriptor received from a non-entityID-based fetch.
      Parameters:
      entities - the metadata to process
      Returns:
      the resolved descriptor(s)
      Throws:
      ResolverException - if there is a fatal error attempting to resolve the metadata
    • processNonEntityIDFetchedEntityDescriptor

      @Nullable protected List<EntityDescriptor> processNonEntityIDFetchedEntityDescriptor​(@Nonnull EntityDescriptor entity) throws ResolverException
      Process an EntityDescriptor received from a non-entityID-based fetch.
      Parameters:
      entity - the metadata to process
      Returns:
      the resolved descriptor(s)
      Throws:
      ResolverException - if there is a fatal error attempting to resolve the metadata
    • fetchFromOriginSource

      @Nullable protected abstract XMLObject fetchFromOriginSource​(@Nonnull CriteriaSet criteria) throws IOException
      Fetch the metadata from the origin source.
      Parameters:
      criteria - the input criteria set
      Returns:
      the resolved metadata root XMLObject, or null if metadata could not be fetched
      Throws:
      IOException - if there is a fatal error fetching metadata from the origin source
    • lookupEntityID

      @Nonnull @NonnullElements protected List<EntityDescriptor> lookupEntityID​(@Nonnull String entityID) throws ResolverException
      Get list of descriptors matching an entityID.
      Overrides:
      lookupEntityID in class AbstractMetadataResolver
      Parameters:
      entityID - entityID to lookup
      Returns:
      a list of descriptors
      Throws:
      ResolverException - if an error occurs
    • processNewMetadata

      @Nonnull protected void processNewMetadata​(@Nonnull XMLObject root, @Nonnull String expectedEntityID) throws FilterException
      Process the specified new metadata document, including metadata filtering, and store the processed metadata in the backing store.

      Equivalent to processNewMetadata(XMLObject, String, boolean) called with false.

      Parameters:
      root - the root of the new metadata document being processed
      expectedEntityID - the expected entityID of the resolved metadata
      Throws:
      FilterException - if there is a problem filtering the metadata
    • processNewMetadata

      @Nonnull protected void processNewMetadata​(@Nonnull XMLObject root, @Nonnull String expectedEntityID, boolean fromPersistentCache) throws FilterException, ResolverException
      Process the specified new metadata document, including metadata filtering, and store the processed metadata in the backing store.

      In order to be processed successfully, the metadata (after filtering) must be an instance of EntityDescriptor and its entityID value must match the value supplied as the required expectedEntityID argument.

      Parameters:
      root - the root of the new metadata document being processed
      expectedEntityID - the expected entityID of the resolved metadata
      fromPersistentCache - whether the entity data was loaded from the persistent cache
      Throws:
      FilterException - if there is a problem filtering the metadata
      ResolverException - if there is a problem processing the metadata
    • prepareForFiltering

      @Nonnull protected XMLObject prepareForFiltering​(@Nonnull XMLObject input)
      Prepare the object for filtering: If persistent caching is enabled, return a clone of the object in case the configured filter mutates the object.
      Parameters:
      input - the XMLObject on which to operate
      Returns:
      the XMLObject instance to be filtered
    • preProcessEntityDescriptor

      protected void preProcessEntityDescriptor​(@Nonnull EntityDescriptor entityDescriptor, @Nonnull AbstractMetadataResolver.EntityBackingStore backingStore)
      Pre-process the specified entity descriptor, updating the specified entity backing store instance as necessary.
      Overrides:
      preProcessEntityDescriptor in class AbstractMetadataResolver
      Parameters:
      entityDescriptor - the target entity descriptor to process
      backingStore - the backing store instance to update
    • logMetadataExpiration

      private void logMetadataExpiration​(@Nonnull EntityDescriptor descriptor, @Nonnull Instant now, @Nonnull Instant nextRefresh)
      Check metadata for expiration or pending expiration and log appropriately.
      Parameters:
      descriptor - the entity descriptor being processes
      now - the current date/time
      nextRefresh - the next refresh trigger time for the entity descriptor
    • computeExpirationTime

      @Nonnull protected Instant computeExpirationTime​(@Nonnull EntityDescriptor entityDescriptor, @Nonnull Instant now)
      Compute the effective expiration time for the specified metadata.
      Parameters:
      entityDescriptor - the EntityDescriptor instance to evaluate
      now - the current date time instant
      Returns:
      the effective expiration time for the metadata
    • computeRefreshTriggerTime

      @Nonnull protected Instant computeRefreshTriggerTime​(@Nullable Instant expirationTime, @Nonnull Instant nowDateTime)
      Compute the refresh trigger time.
      Parameters:
      expirationTime - the time at which the metadata effectively expires
      nowDateTime - the current date time instant
      Returns:
      the time after which refresh attempt(s) should be made
    • shouldAttemptRefresh

      protected boolean shouldAttemptRefresh​(@Nonnull AbstractDynamicMetadataResolver.EntityManagementData mgmtData)
      Determine whether should attempt to refresh the metadata, based on stored refresh trigger time.
      Parameters:
      mgmtData - the entity'd management data
      Returns:
      true if should attempt refresh, false otherwise
    • createNewBackingStore

      @Nonnull protected AbstractDynamicMetadataResolver.DynamicEntityBackingStore createNewBackingStore()
      Create a new backing store instance for EntityDescriptor data. Subclasses may override to return a more specialized subclass type. Note this method does not make the returned backing store the effective one in use. The caller is responsible for calling AbstractMetadataResolver.setBackingStore(EntityBackingStore) to make it the effective instance in use.
      Overrides:
      createNewBackingStore in class AbstractMetadataResolver
      Returns:
      the new backing store instance
    • getBackingStore

      Get the EntityDescriptor backing store currently in use by the metadata resolver.
      Overrides:
      getBackingStore in class AbstractMetadataResolver
      Returns:
      the current effective entity backing store
    • initMetadataResolver

      protected void initMetadataResolver() throws ComponentInitializationException
      Subclasses should override this method to perform any initialization logic necessary. Default implementation is a no-op.
      Overrides:
      initMetadataResolver in class AbstractMetadataResolver
      Throws:
      ComponentInitializationException - thrown if there is a problem initializing the provider
    • initializeMetricsInstrumentation

      private void initializeMetricsInstrumentation()
      Initialize the Metrics-based instrumentation.
    • initializeFromPersistentCache

      protected void initializeFromPersistentCache()
      Initialize the resolver with data from the persistent cache manager, if enabled.
    • processPersistentCacheEntry

      protected void processPersistentCacheEntry​(@Nonnull String currentKey, @Nonnull EntityDescriptor descriptor)
      Process an entry loaded from the persistent cache.
      Parameters:
      currentKey - the current persistent cache key
      descriptor - the entity descriptor to process
    • removeByEntityID

      protected void removeByEntityID​(String entityID, AbstractMetadataResolver.EntityBackingStore backingStore)
      Remove from the backing store all metadata for the entity with the given entity ID.
      Overrides:
      removeByEntityID in class AbstractMetadataResolver
      Parameters:
      entityID - the entity ID of the metadata to remove
      backingStore - the backing store instance to update
    • doDestroy

      protected void doDestroy()
      Overrides:
      doDestroy in class AbstractMetadataResolver
    • indexEntityDescriptor

      protected void indexEntityDescriptor​(@Nonnull EntityDescriptor entityDescriptor, @Nonnull AbstractMetadataResolver.EntityBackingStore backingStore)
      Index the specified entity descriptor, updating the specified entity backing store instance as necessary.
      Overrides:
      indexEntityDescriptor in class AbstractMetadataResolver
      Parameters:
      entityDescriptor - the target entity descriptor to process
      backingStore - the backing store instance to update