Dynamic metadata error caching

[Kristof Bajnok]

On 2017-06-20 15:42, Cantor, Scott wrote:
>> I assume that this is implemented in order to prevent resource hogging.
> 
> It's to prevent constant attempts to query what could be a hung metadata source that might be causing requests to block. The system has no way to know that the error is transitory so it resets itself to try again after a minimal duration.

IMO Hung dynamic metadata sources should block the requests the same way
as hung attribute sources do. The current implementation favours configs
with permanently faulty metadata sources instead of ones with temporary
errors, even though the latter is more common, IMO.

FWIW my tests show immediate continuation of the request if the query is
rejected or a blocking for 5 seconds if there is no response.

>> I can't think of any normal configuration where it could happen
> 
> Failure is just an expected possibility. It could maintain a counter I suppose, but I don't think endless retry is really the right approach.

Current implementation retries endlessly once in every 10 minutes. I
think rate limiting makes sense but not in this magnitude.

>> simplest would be to just remove lines 346-362 from
>> DynamicMetadataProvider.cpp, but if you still want to deal with this, I
>> think a new variable like retryDelay should be used instead, defaulting
>> to something like 1 second. I don't think that lowering minCacheDuration
>> would be the proper way to avoid sticking metadata errors.
> 
> It is the proper way, that's what it's for. I don't think it really has much of an impact on anything else.

I thought it was meant primarily for controlling the caching time if the
response does not contain a cacheDuration.

Anyway, as my MDX server always supplies cacheDuration what has
precedence until it is a higher value than minCacheDuration, hence I can
instruct my SPs to set their minCacheDuration to 1 without it having an
effect on anything else than retries.

Thanks for the tip,
Kristof