Pilot manager handler simplifications

[fstagni]

First part for #8597

BEGINRELEASENOTES

*WMS
CHANGE: Removed several RPCs and DB calls for PilotManager

ENDRELEASENOTES

[aldbr]

ddev: just need to make sure that htcondor does not remove the outputs from the spool once you requested it once.

[iueda]

Checking the BelleDIRAC code following the question at the last Dops+Ddev meeting, we don't have any extension of PilotManagerHandler nor PilotAgentsDB, but I see some RPC calls with RPCClient('WorkloadManagement/PilotManager') such as getGroupedPilotSummary()

[fstagni]

Checking the BelleDIRAC code following the question at the last Dops+Ddev meeting, we don't have any extension of PilotManagerHandler nor PilotAgentsDB, but I see some RPC calls with RPCClient('WorkloadManagement/PilotManager') such as getGroupedPilotSummary()

That has been moved to DB-only call, no need to go through the RPC.

[fstagni]

Some info related to the removal of PilotOutput table and related methods:

Historically, running condor_transfer_data did not affect the job's presence in the queue, so it was possible to invoke it multiple times on the same job and retrieve the same output files each time.
However, this behavior changed recently in the HTCondor 25.x series. A successful condor_transfer_data now causes the job to become eligible to leave the queue shortly afterwards. This change was introduced because many users expected the job to be automatically cleaned up after retrieving the output, rather than having to run condor_rm separately.
So the answer depends on the HTCondor version in use: older versions allow repeated retrievals, while newer versions may remove the job from the queue after a successful transfer, preventing subsequent condor_transfer_data invocations.
Jamie (James Frey) also mentioned that the HTCondor team would be open to adding an option to preserve the old behavior if there is interest in such a use case.

This puts for the moment a halt on the attempt to simply remove the DIRAC functionality.

[fstagni]

Some info related to the removal of PilotOutput table and related methods:

Historically, running condor_transfer_data did not affect the job's presence in the queue, so it was possible to invoke it multiple times on the same job and retrieve the same output files each time.
However, this behavior changed recently in the HTCondor 25.x series. A successful condor_transfer_data now causes the job to become eligible to leave the queue shortly afterwards. This change was introduced because many users expected the job to be automatically cleaned up after retrieving the output, rather than having to run condor_rm separately.
So the answer depends on the HTCondor version in use: older versions allow repeated retrievals, while newer versions may remove the job from the queue after a successful transfer, preventing subsequent condor_transfer_data invocations.
Jamie (James Frey) also mentioned that the HTCondor team would be open to adding an option to preserve the old behavior if there is interest in such a use case.

This puts for the moment a halt on the attempt to simply remove the DIRAC functionality.

Functionality restored.