Re: [Clamav-devel] libmilter's workers pool in clamav-milter

That's already done inside libmilter :

With sendmail 8.14.0, you shall compile libmilter with :

APPENDDEF(`conf_libmilter_ENVDEF',`-D_FFR_WORKER_MODEL')

This will switch the libmilter thread model to a pool of workers instead
of one thread per sendmail connection. And the number of threads in the
filter will be the number of threads really doing something...

This makes a very big difference on systems handling a big number of
simultaneous connections.

Here are some old slides (2003) from an old presentation I've done at
sendmail about that :

http://j-chkmail.ensmp.fr/papers/workermilter.pdf

Well... I've got an award from sendmail... 8-)

http://www.j-chkmail.org/wiki/doku.php/award


Ernest Wypierowski wrote:
> Good evening,
>
> version:
> libmilter - sendmail-8.14.2
> clamav-milter - clamav-0.91.2
>
>
> If you configure libmilter library with using poll() rather than
> select() and use workers pool you might hit problem related with locking
> up your clamav-milter.
>
> To reproduce this kind of behavior you should ran clamav-milter in 32
> bit mode, lower max-children and set timeout to some rather high value.
> When number of simultaneous connections exceed max children then every
> new connection will equal one thread. Going step further, when threads
> allocate all virtual memory available for clamav-milter workers from
> libmilter will not have space to execute (clamav-milter: LAUNCH_WORKER
> error: Cannot allocate memory).
>
> Daemon clamav-milter will not answer with temporary error but will
> keep going with blocking threads/connections.
>
> How to solve this kind of problem? Does not clamav-milter is prepare for
> workers from libmilter?
>
> Most interesing is how to limit clamav-milter's resource based on thre-
> ads. In described situation number of threads are not equal number of
> connections, limiting it with dont-wait threads is not a good idea.
> Maybe there is a way to limit numbers of threads that workers can
> consume?
>
> Best regards,
> Ernest Wypierowski
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>

-- 
  ---------------------------------------------------------------
  Jose Marcio MARTINS DA CRUZ           http://j-chkmail.ensmp.fr
  Ecole des Mines de Paris
  60, bd Saint Michel
  75272 - PARIS CEDEX 06      mailto:Jose-Marcio.Martins@ensmp.fr
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net