Package org.opensaml.storage.impl.memcached

Class MemcachedStorageService

  • All Implemented Interfaces:
    Component, DestructableComponent, IdentifiableComponent, IdentifiedComponent, InitializableComponent, StorageService
    public class MemcachedStorageService
extends AbstractIdentifiableInitializableComponent
implements StorageService
    Memcached storage service. The implementation of context names is based on the implementation of simulated namespaces discussed on the Memcached project site:

    https://code.google.com/p/memcached/wiki/NewProgrammingTricks#Namespacing

    This storage service supports arbitrary-length context names and keys despite the 250-byte limit on memcached keys. Keys whose length is greater than 250 bytes are hashed using the SHA-512 algorithm and hex encoded to produce a 128-character key that is stored in memcached. Collisions are avoided irrespective of hashing by using the memcached add operation on all create operations which guarantees that an entry is created if and only if a key of the same value does not already exist. Note that context names and keys are assumed to have single-byte encodings in UTF-8 (i.e. ASCII characters) such that key lengths are equal to their size in bytes. Hashed keys naturally meet this requirement.

    An optional context key-tracking feature is available to support updateContextExpiration(String, Long). Key tracking is disabled by default, but can be enabled by setting the enableContextKeyTracking parameter in the MemcachedStorageService(net.spy.memcached.MemcachedClient, int, boolean) constructor. While there is modest performance impact for create and delete operations, the feature limits the number of keys per context. With the default 1M memcached slab size, in the worst case 4180 keys are permitted per context. In many if not most situations the value is easily double that. The limitation can be overcome by increasing the slab size, which decreases overall cache memory consumption efficiency. When key tracking is disabled, there is no limit on the number of keys per context other than overall cache capacity.

    Limitations and requirements

    1. The memcached binary protocol is strong recommended for efficiency and full versioning support. In particular, deleteWithVersion(long, String, String) and deleteWithVersion(long, Object) will throw runtime errors under the ASCII protocol.
    2. Memcached server 1.4.14 or later MUST be used with binary protocol for proper handling of cache entry expiration values. See the 1.4.14 release notes for details.

    • Field Detail

      • CTX_KEY_LIST_SUFFIX

        @Nonnull
@NotEmpty
protected static final String CTX_KEY_LIST_SUFFIX
        Key suffix for entry that contains a list of context keys.
        See Also:
        Constant Field Values

      • CTX_KEY_DELETED_SUFFIX

        @Nonnull
@NotEmpty
protected static final String CTX_KEY_DELETED_SUFFIX
        Key suffix for entry that contains a list of deleted context keys.
        See Also:
        Constant Field Values

      • MAX_KEY_LENGTH

        private static final int MAX_KEY_LENGTH
        Maximum length in bytes of memcached keys.
        See Also:
        Constant Field Values

      • logger

        @Nonnull
private final org.slf4j.Logger logger
        Logger instance.

      • stringTranscoder

        @Nonnull
private final net.spy.memcached.transcoders.Transcoder<String> stringTranscoder
        Handles conversion of strings to bytes and vice versa.

      • memcacheClient

        @Nonnull
private final net.spy.memcached.MemcachedClient memcacheClient
        Memcached client instance.

      • operationTimeout

        @Positive
private int operationTimeout
        Memcached asynchronous operation timeout in seconds.

      • trackContextKeys

        private boolean trackContextKeys
        Flag that controls context key tracking.

    • Constructor Detail

      • MemcachedStorageService

        public MemcachedStorageService​(@Nonnull
                               net.spy.memcached.MemcachedClient client,
                               @Positive
                               int timeout)
        Creates a new instance.
        Parameters:
        client - Memcached client object. The client MUST be configured to use the binary memcached protocol, i.e. BinaryConnectionFactory, in order for deleteWithVersion(long, String, String) and deleteWithVersion(long, Object) to work correctly. The binary protocol is recommended for efficiency as well.
        timeout - Memcached operation timeout in seconds.

      • MemcachedStorageService

        public MemcachedStorageService​(@Nonnull
                               net.spy.memcached.MemcachedClient client,
                               @Positive
                               int timeout,
                               boolean enableContextKeyTracking)
        Creates a new instance with optional context key tracking.
        Parameters:
        client - Memcached client object. The client MUST be configured to use the binary memcached protocol, i.e. BinaryConnectionFactory, in order for deleteWithVersion(long, String, String) and deleteWithVersion(long, Object) to work correctly. The binary protocol is recommended for efficiency as well.
        timeout - Memcached operation timeout in seconds.
        enableContextKeyTracking - True to enable context key tracking, false otherwise. NOTE this flag must be set to true in order for updateContextExpiration(String, Long) to work. If that capability is not needed, the flag should be set to false for better performance. The feature is disabled by default.

    • Method Detail

      • setCapabilities

        public void setCapabilities​(@Nonnull
                            MemcachedStorageCapabilities capabilities)
        Sets the storage capabilities. This method should be used when the default 1M slab size is changed; the MemcachedStorageCapabilities.getValueSize() should be set equal to the chosen slab size.
        Parameters:
        capabilities - Memcached storage capabilities.

      • create

        public boolean create​(@Nonnull @NotEmpty
                      String context,
                      @Nonnull @NotEmpty
                      String key,
                      @Nonnull @NotEmpty
                      String value,
                      @Nullable @Positive
                      Long expiration)
               throws IOException
        Creates a new record in the store with an expiration.
        Specified by:
        create in interface StorageService
        Parameters:
        context - a storage context label
        key - a key unique to context
        value - value to store
        expiration - expiration for record, or null
        Returns:
        true iff record was inserted, false iff a duplicate was found
        Throws:
        IOException - if fatal errors occur in the insertion process

      • create

        public <T> boolean create​(@Nonnull @NotEmpty
                          String context,
                          @Nonnull @NotEmpty
                          String key,
                          @Nonnull
                          T value,
                          @Nonnull
                          StorageSerializer<T> serializer,
                          @Nullable @Positive
                          Long expiration)
                   throws IOException
        Creates a new record in the store with an expiration, using a custom serialization process for an arbitrary object.
        Specified by:
        create in interface StorageService
        Type Parameters:
        T - type of record
        Parameters:
        context - a storage context label
        key - a key unique to context
        value - object to store
        serializer - custom serializer for the object
        expiration - expiration for record, or null
        Returns:
        true iff record was inserted, false iff a duplicate was found
        Throws:
        IOException - if fatal errors occur in the insertion process

      • create

        public boolean create​(@Nonnull
                      Object value)
               throws IOException
        Creates a new record in the store using an annotated object as the source.

        The individual parameters for the creation are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

        Specified by:
        create in interface StorageService
        Parameters:
        value - object to store
        Returns:
        true iff record was inserted, false iff a duplicate was found
        Throws:
        IOException - if fatal errors occur in the insertion process

      • read

        public <T> StorageRecord<T> read​(@Nonnull @NotEmpty
                                 String context,
                                 @Nonnull @NotEmpty
                                 String key)
                          throws IOException
        Returns an existing record from the store, if one exists.
        Specified by:
        read in interface StorageService
        Type Parameters:
        T - type of record
        Parameters:
        context - a storage context label
        key - a key unique to context
        Returns:
        the record read back, if present, or null
        Throws:
        IOException - if errors occur in the read process

      • read

        public Object read​(@Nonnull
                   Object value)
            throws IOException
        Returns an existing record from the store, if one exists, and uses it to update the annotated fields of a target object.

        The context and key to look up are obtained from the target object, and the value and expiration are written back, using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

        Specified by:
        read in interface StorageService
        Parameters:
        value - object to look up and populate
        Returns:
        the updated object passed into the method, or null if no record was found
        Throws:
        IOException - if errors occur in the read process

      • read

        public <T> Pair<Long,​StorageRecord<T>> read​(@Nonnull @NotEmpty
                                                  String context,
                                                  @Nonnull @NotEmpty
                                                  String key,
                                                  @Positive
                                                  long version)
                                           throws IOException
        Returns an existing record from the store, along with its version.

        The first member of the pair returned will contain the version of the record in the store, or will be null if no record exists. The second member will contain the record read back. If null, the record either didn't exist (if the first member was also null) or the record was the same version as that supplied by the caller.

        Specified by:
        read in interface StorageService
        Type Parameters:
        T - type of record
        Parameters:
        context - a storage context label
        key - a key unique to context
        version - only return record if newer than supplied version
        Returns:
        a pair consisting of the version of the record read back, if any, and the record itself
        Throws:
        IOException - if errors occur in the read process

      • update

        public boolean update​(@Nonnull @NotEmpty
                      String context,
                      @Nonnull @NotEmpty
                      String key,
                      @Nonnull @NotEmpty
                      String value,
                      @Nullable @Positive
                      Long expiration)
               throws IOException
        Updates an existing record in the store.
        Specified by:
        update in interface StorageService
        Parameters:
        context - a storage context label
        key - a key unique to context
        value - updated value
        expiration - expiration for record, or null
        Returns:
        true if the update succeeded, false if the record does not exist
        Throws:
        IOException - if errors occur in the update process

      • update

        public <T> boolean update​(@Nonnull @NotEmpty
                          String context,
                          @Nonnull @NotEmpty
                          String key,
                          @Nonnull
                          T value,
                          @Nonnull
                          StorageSerializer<T> serializer,
                          @Nullable @Positive
                          Long expiration)
                   throws IOException
        Updates an existing record in the store using a custom serialization strategy.
        Specified by:
        update in interface StorageService
        Type Parameters:
        T - type of record
        Parameters:
        context - a storage context label
        key - a key unique to context
        value - updated value
        serializer - custom serializer
        expiration - expiration for record, or null
        Returns:
        true if the update succeeded, false if the record does not exist
        Throws:
        IOException - if errors occur in the update process

      • update

        public boolean update​(@Nonnull
                      Object value)
               throws IOException
        Updates an existing record in the store, using an annotated object as the source.

        The individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

        Specified by:
        update in interface StorageService
        Parameters:
        value - object to update from
        Returns:
        true if the update succeeded, false if the record does not exist
        Throws:
        IOException - if errors occur in the update process

      • updateWithVersion

        @Nullable
public <T> Long updateWithVersion​(@Positive
                                  long version,
                                  @Nonnull @NotEmpty
                                  String context,
                                  @Nonnull @NotEmpty
                                  String key,
                                  @Nonnull
                                  T value,
                                  @Nonnull
                                  StorageSerializer<T> serializer,
                                  @Nullable @Positive
                                  Long expiration)
                           throws IOException,
                                  VersionMismatchException
        Updates an existing record in the store, if a version matches, using a custom serialization strategy.
        Specified by:
        updateWithVersion in interface StorageService
        Type Parameters:
        T - type of record
        Parameters:
        version - only update if the current version matches this value
        context - a storage context label
        key - a key unique to context
        value - updated value
        serializer - custom serializer
        expiration - expiration for record, or null
        Returns:
        the version of the record after update, null if no record exists
        Throws:
        IOException - if errors occur in the update process
        VersionMismatchException - if the record has already been updated to a newer version

      • updateWithVersion

        @Nullable
public Long updateWithVersion​(@Positive
                              long version,
                              @Nonnull
                              Object value)
                       throws IOException,
                              VersionMismatchException
        Updates an existing record in the store, if a version matches, using an annotated object as the source.

        The individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

        Specified by:
        updateWithVersion in interface StorageService
        Parameters:
        version - only update if the current version matches this value
        value - object to update from
        Returns:
        the version of the record after update, null if no record exists
        Throws:
        IOException - if errors occur in the update process
        VersionMismatchException - if the record has already been updated to a newer version

      • updateExpiration

        public boolean updateExpiration​(@Nonnull @NotEmpty
                                String context,
                                @Nonnull @NotEmpty
                                String key,
                                @Nullable @Positive
                                Long expiration)
                         throws IOException
        Updates expiration of an existing record in the store.
        Specified by:
        updateExpiration in interface StorageService
        Parameters:
        context - a storage context label
        key - a key unique to context
        expiration - expiration for record, or null
        Returns:
        true if the update succeeded, false if the record does not exist
        Throws:
        IOException - if errors occur in the update process

      • updateExpiration

        public boolean updateExpiration​(@Nonnull
                                Object value)
                         throws IOException
        Updates expiration of an existing record in the store, using an annotated object as the source.

        The individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

        Specified by:
        updateExpiration in interface StorageService
        Parameters:
        value - object to update from
        Returns:
        true if the update succeeded, false if the record does not exist
        Throws:
        IOException - if errors occur in the update process

      • delete

        public boolean delete​(@Nonnull @NotEmpty
                      String context,
                      @Nonnull @NotEmpty
                      String key)
               throws IOException
        Deletes an existing record from the store.
        Specified by:
        delete in interface StorageService
        Parameters:
        context - a storage context label
        key - a key unique to context
        Returns:
        true iff the record existed and was deleted
        Throws:
        IOException - if errors occur in the deletion process

      • delete

        public boolean delete​(@Nonnull
                      Object value)
               throws IOException
        Deletes an existing record from the store, using an annotated object as the source.

        The individual parameters for the deletion are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

        Specified by:
        delete in interface StorageService
        Parameters:
        value - object to delete
        Returns:
        true iff the record existed and was deleted
        Throws:
        IOException - if errors occur in the deletion process

      • deleteWithVersion

        public boolean deleteWithVersion​(@Positive
                                 long version,
                                 @Nonnull
                                 Object value)
                          throws IOException,
                                 VersionMismatchException
        Deletes an existing record from the store, using an annotated object as the source, if it currently has a specified version.

        The individual parameters for the deletion are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

        Specified by:
        deleteWithVersion in interface StorageService
        Parameters:
        version - record version to delete
        value - object to delete
        Returns:
        true iff the record existed and was deleted
        Throws:
        IOException - if errors occur in the deletion process
        VersionMismatchException - if the record has already been updated to a newer version

      • reap

        public void reap​(@Nonnull @NotEmpty
                 String context)
          throws IOException
        Manually trigger a cleanup of expired records. The method MAY return without guaranteeing that cleanup has already occurred.
        Specified by:
        reap in interface StorageService
        Parameters:
        context - a storage context label
        Throws:
        IOException - if errors occur in the cleanup process

      • updateContextExpiration

        public void updateContextExpiration​(@Nonnull @NotEmpty
                                    String context,
                                    @Nullable
                                    Long expiration)
                             throws IOException
        Updates the expiration time of all records in the context.
        Specified by:
        updateContextExpiration in interface StorageService
        Parameters:
        context - a storage context label
        expiration - a new expiration timestamp, or null
        Throws:
        IOException - if errors occur in the cleanup process

      • deleteContext

        public void deleteContext​(@Nonnull @NotEmpty
                          String context)
                   throws IOException
        Forcibly removes all records in a given context along with any associated resources devoted to maintaining the context.
        Specified by:
        deleteContext in interface StorageService
        Parameters:
        context - a storage context label
        Throws:
        IOException - if errors occur in the cleanup process

      • lookupNamespace

        protected String lookupNamespace​(String context)
                          throws IOException
        Looks up the namespace for the given context name in the cache.
        Parameters:
        context - Context name.
        Returns:
        Corresponding namespace for given context or null if no namespace exists for context.
        Throws:
        IOException - On memcached operation errors.

      • createNamespace

        protected String createNamespace​(String context)
                          throws IOException
        Creates a cache-wide unique namespace for the given context name. The context-namespace mapping is stored in the cache.
        Parameters:
        context - Context name.
        Returns:
        Namespace name for given context.
        Throws:
        IOException - On memcached operation errors.

      • memcachedKey

        private String memcachedKey​(String... parts)
        Creates a memcached key from one or more parts.
        Parameters:
        parts - Key parts (i.e. namespace, local name)
        Returns:
        Key comprised of 250 characters or less.

      • handleAsyncResult

        private <T> T handleAsyncResult​(net.spy.memcached.internal.OperationFuture<T> result)
                         throws IOException
        Handle async result.
        Type Parameters:
        T - type of result
        Parameters:
        result - the result
        Returns:
        the result
        Throws:
        IOException - if an error occurs

      • updateContextKeyList

        private boolean updateContextKeyList​(String suffix,
                                     String namespace,
                                     String key)
                              throws IOException
        Update context key list.
        Parameters:
        suffix - the suffix
        namespace - the namespace
        key - the storage key
        Returns:
        whether the update was a success
        Throws:
        IOException - if an error occurs