Move (back?) to a select-based PMDA with custom main.
An observed problem with doing log reads only during the
metric fetch callback, is that we start getting behind in
events once the volume starts ramping up from small files
to the point that the PMDA is overwhelmed. Caused by two
issues:
only reading new events based solely on client fetch
intervals means for relatively long intervals (say once
every few minutes) the consumption doesn't keep up with
the event generation.
only reading once (i.e. one read(2) call) per fetch,
which made the above even more severe and noticable.
We address these issues by using a custom PMDA loop which
is awoken on either readable file descriptors or expiry of
a timer (iow server side driven event reading, not client).
Whenever we wake, we consume all available events at that
time for each file descriptor.
Move (back?) to a select-based PMDA with custom main.
An observed problem with doing log reads only during the
metric fetch callback, is that we start getting behind in
events once the volume starts ramping up from small files
to the point that the PMDA is overwhelmed. Caused by two
issues:
intervals means for relatively long intervals (say once
every few minutes) the consumption doesn't keep up with
the event generation.
which made the above even more severe and noticable.
We address these issues by using a custom PMDA loop which
is awoken on either readable file descriptors or expiry of
a timer (iow server side driven event reading, not client).
Whenever we wake, we consume all available events at that
time for each file descriptor.