tac_plus forks multiple perl childs

[OlexanderH]

Hello Marc.
Thank you for a great tac_plus daemon.
I understand that tac_plus is deprecated, but I could not compile the tac_plus-ng on debian yet.
My question is - after switching from the very old version 201705210930 to the last one, I noticed that the main tac_plus service forks 4 child .pl processes (there was only one in the old version).

root        7262       1  0 14:34 ?        00:00:00 tac_plus: 0 connections, accepting up to 600 more
root        7263    7262  0 14:34 ?        00:00:00 tac_plus: 0 connections
root        7264    7263  0 14:34 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root        7265    7263  0 14:34 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root        7266    7263  0 14:34 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root        7267    7263  0 14:34 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl

Is this a feature or something I have configured incorrectly? If it is a feature, what is its role and how can it be used and managed?

[MarcJHuber]

Hi Olexander,

these Perl processes form a helper process pool that handles authentication/authorization via an external database. Generally, the number of those processes can be controlled via a "child (min|max) = ..." directive, and the default for minimum is indeed 4, so your process listing looks fine to me. I don't remember whether the 2017 defaults were different.

Also, mavis_tacplus_mysql_ldap.pl isn't part of my code base. I think that's from TACACSGUI (by Aleksey Mochalin).

Cheers,

Marc

[OlexanderH]

Hi Marc.
Sorry, I didn’t clarify, I certainly understand the essence of the perl process, the question concerned precisely the number of childs.
mavis_tacplus_mysql_ldap.pl is your mavis_tacplus_ldap.pl script modified to suit our own realities to obtain additional data from the DB.
By the way, in the tac_plus daemon directive "child (min|max) = ..." did not work, but directive "childs (min|max) = ..." worked, thanks.
In the old version and one perl process could handle the load, does a larger number affect performance?
In standard mode, the load reaches tens of tacacs requests per second; during mass work there can be hundreds per sec of them.

[MarcJHuber]

Hi Olexander,

the Perl processes are single-threaded and blocking, so running more than one will allow for concurrent backend queries (authentication, authorization).

Depending on the speed of the backend (mySQL/LDAP) this may or may not be noticeable. If your older installation did get along with just one Perl process running, then the more recent version will, too. There's been no change in behavior.

tac_plus by default (this can be disabled) uses in-memory caching, so typically only a small fraction of TACACS+ requests will need the help of the Perl scripts.

Cheers,

Marc

[OlexanderH]

Yes, I am aware of the cache, and in the old version I even had difficulties - when one user could have different rights to different devices, although Perl defined a group for the user, the cache stubbornly returned the last known entry of the user group for all devices.

I had to either disable the cache (which is not rational) or distribute different services across different entities of the tac_plus daemon.

In the new version, at first glance, this has been eliminated, perhaps several perl processes played a role in this. Thanks for this again.

[OlexanderH]

Hi Marc

One more question - I also noticed that when the service is restarted (systemctl restart tac_plus), the child processes are not interrupted immediately, but only after 1-2 minutes.
The main inconvenience of this is that during an OS restart it waits for more than a minute for the interruption of those processes.
Is this expected behavior and is there a directive to control child interruption?
OS - Debian.

ps -ef | grep tac
root       72101       1  0 Feb20 ?        00:00:02 tac_plus: 0 connections, accepting up to 600 more
root       72102   72101  0 Feb20 ?        00:00:01 tac_plus: 0 connections
root       72103   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       72104   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       72105   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       72106   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
systemctl restart tac_plus
ps -ef | grep tac
root       72102       1  0 Feb20 ?        00:00:01 tac_plus: 0 connections
root       72103   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       72104   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       72105   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       72106   72102  0 Feb20 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       92011       1  0 09:27 ?        00:00:00 tac_plus: 0 connections, accepting up to 600 more
root       92012   92011  0 09:27 ?        00:00:00 tac_plus: 0 connections
root       92013   92012  7 09:27 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       92014   92012  7 09:27 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       92015   92012  7 09:27 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl
root       92016   92012  7 09:27 ?        00:00:00 perl /usr/local/lib/mavis/mavis_tacplus_mysql_ldap.pl

For myself (I only have one daemon on host), I eliminated this by editing function stop in /etc/init.d/tac_plus changing

'kill -9 $PID' to 'kill -9 $(pidof tac_plus)'

stop () {
        PID=`cat $PIDFILE 2>/dev/null`
        /bin/echo -n "Stopping $NAME: "
        if [ "x$PID" = "x" ]
        then
                echo "failed ($NAME is not running)"
        else
                kill -9 $(pidof tac_plus) 2>/dev/null
                #kill -9 $PID 2>/dev/null
                rm -f $PIDFILE
                echo "done."
        fi
}

[MarcJHuber]

Hi Olexander,

having the child daemons terminate when the master reloads is currently not an option as the individual worker processes don't remember connections, they just add them to the event processing logic. I might change that behavior, eventually, but that will take some time.

Cheers,

Marc

[MarcJHuber]

Hi Olexander,

adfe834 reduces wait time to 2 seconds, I think that's fine for now.

Cheers,

Marc

[OlexanderH]

Hi Marc
Thank you very much for the solution and for your support.

Have a good day :)