ITS: new CPU + GPU seeding vertexer

[cima22]

Adds an optional seeding vertexer that runs as a prepended tracker pass (diamond trackleting -> cells -> lines -> parallel seeding), on both the CPU and GPU traits.
Additionally, unsorted clusters are loaded to GPU and then sorted on GPU.

To set it:
ITSCATrackerParam.seedingVertexIteration=1;
ITSVertexerParam.useParallelSeeding=1;

Tuned parameters for Pb--Pb:

ITSCATrackerParam.seedingVertexIteration=1;ITSVertexerParam.useParallelSeeding=1;ITSVertexerParam.useTruthSeeding=0;ITSCATrackerParam.diamondTrackletingPVres=1.5102214480735774;ITSCATrackerParam.diamondTrackletingNSigmaCut=2.976946815605669;ITSCATrackerParam.diamondTrackletingCellDeltaTanLambdaSigma=0.005566295924057422;ITSCATrackerParam.diamondCellTanLambdaNSigma=2.5;ITSCATrackerParam.diamondTrackletingCellDeltaPhiMinPt=0.12;ITSCATrackerParam.cellLineSharedClusterCut=1;ITSVertexerParam.clusterCut=0.07;ITSVertexerParam.pairCut=0.025684202919354766;ITSVertexerParam.phiCut=0.008875191490279988;ITSVertexerParam.clusterContributorsCut=3;ITSVertexerParam.lineMinPt=0.10;ITSVertexerParam.nSigmaCut=0.0016;ITSVertexerParam.goodLineChi2Cut=9.814679066181695;ITSVertexerParam.goodLinePtCut=0.12;ITSVertexerParam.goodContributorsSignificance=0.070;ITSVertexerParam.suppressLowMultDebris=13;ITSVertexerParam.fineZWindow=0.010;ITSVertexerParam.fineMinDensity=8;ITSVertexerParam.fineMaxDrift=0.005;ITSVertexerParam.duplicateZScale=0.7;ITSVertexerParam.duplicateZCut=0.1388966993405415;

Tuned parameters for pp:

ITSCATrackerParam.seedingVertexIteration=1;ITSVertexerParam.useParallelSeeding=1;ITSVertexerParam.useTruthSeeding=0;ITSCATrackerParam.diamondTrackletingPVres=3.74344532796542;ITSCATrackerParam.diamondTrackletingNSigmaCut=5.352263543811316;ITSCATrackerParam.diamondTrackletingCellDeltaTanLambdaSigma=0.002563403956702525;ITSCATrackerParam.diamondCellTanLambdaNSigma=3.257473162361808;ITSCATrackerParam.diamondTrackletingCellDeltaPhiMinPt=0.09407482537938365;ITSCATrackerParam.cellLineSharedClusterCut=3;ITSVertexerParam.clusterCut=0.05557266167329535;ITSVertexerParam.pairCut=0.05176031785438137;ITSVertexerParam.phiCut=0.017639175978851104;ITSVertexerParam.clusterContributorsCut=2;ITSVertexerParam.lineMinPt=0.1;ITSVertexerParam.nSigmaCut=0.0016;ITSVertexerParam.goodLineChi2Cut=9.814679066181695;ITSVertexerParam.goodLinePtCut=0.12;ITSVertexerParam.goodContributorsSignificance=0.07;ITSVertexerParam.suppressLowMultDebris=13;ITSVertexerParam.fineZWindow=0.013014502308220061;ITSVertexerParam.fineMinDensity=4;ITSVertexerParam.fineMaxDrift=0.004003953018135234;ITSVertexerParam.duplicateZScale=0.1230148343342053;ITSVertexerParam.duplicateZCut=0.06571512005272727;

[shahor02]

Hi @cima22 , are there some slides or plots on the new algo., particularly its performance? Perhaps you could present it next Wed., either at the ITS meeting at 11 (if it is planned, @fprino ?) or at sim/rec WP12/13 at 15:30?

[f3sch]

Hi @cima22, thanks! Some preliminary comments although I have to admit to not understand the logic in its entirety yet.

[f3sch]

Do we need to duplicate the already existing ClusterLines math, can we not have only one of these classes?

[cima22]

Could we promote the GPU class to be the common one? The CPU one uses ROOT, I don't think it is available for GPUs?

[f3sch]

sure, you can adapt the CPU class however you like to make it GPU compatible.

[cima22]

@shahor02 This algorithm should be tested, but I have some preliminary results.

On 50 simulated PbPb TFs at 50kHz:

On a PbPb TF with embedded pp collisions:

PbPb:

pp:

pp track efficiency:

[shahor02]

Thanks, looks good! Note that there are conflicts in the PR.
So, will you be available to present it next Wed.?

[cima22]

Thanks, looks good! Note that there are conflicts in the PR. So, will you be available to present it next Wed.?

Yes absolutely

[fprino]

Thanks, looks good! Note that there are conflicts in the PR. So, will you be available to present it next Wed.?

Yes absolutely

Sorry for replying late: next week we are considering to move the ITS WP2 meeting from Wednesday 2 september to Thursday 3 September, starting at 10 am.
Would this work for you (Gabriele, Ruben, Felix)?

[f3sch]

@fprino I will be on vacation from Thursday onwards.

[f3sch]

Some things from an initial test:
Timing:

[PROD][fschlepp@epn000 pvtest]$ python3 time.py gabriele/reco.log
Processing:
Original count: 925
Trimmed count:  649
Discarded:      138 low + 138 high
Total:          336405.15 ms (336.41 s)
Average:        518.34 ms
Std deviation:  194.22 ms

Vertex seeding:
Original count: 925
Trimmed count:  649
Discarded:      138 low + 138 high
Total:          60625.25 ms (60.63 s)
Average:        93.41 ms
Std deviation:  13.94 ms

Tracking:
Original count: 919
Trimmed count:  645
Discarded:      137 low + 137 high
Total:          231575.46 ms (231.58 s)
Average:        359.03 ms
Std deviation:  86.99 ms
[PROD][fschlepp@epn000 pvtest]$ python3 time.py dev/reco.log

Processing:
Original count: 925
Trimmed count:  649
Discarded:      138 low + 138 high
Total:          511572.46 ms (511.57 s)
Average:        788.25 ms
Std deviation:  239.93 ms

Vertex seeding:
Original count: 925
Trimmed count:  649
Discarded:      138 low + 138 high
Total:          222618.46 ms (222.62 s)
Average:        343.02 ms
Std deviation:  89.72 ms

Tracking:
Original count: 924
Trimmed count:  648
Discarded:      138 low + 138 high
Total:          240011.05 ms (240.01 s)
Average:        370.39 ms
Std deviation:  70.58 ms

note this timing from MI50 EPNs. the new vertexing seems to be 3.7 times faster which in total means a speedup of 1.5 in total processing time (-34.2%).

23 kHz Pb-Pb.
red:dev, blue:this PR, green:this PR but relaxing ITSVertexerParam.clusterContributorsCut from 3 to 2:
pv_cont.pdf
There seems be recovery of PVs below 80 contributors and a slight degradation for higher mult vertices.

black:dev, red:this PR, blue:this PR but relaxing ITSVertexerParam.clusterContributorsCut from 3 to 2:
K0sFitOverlay.pdf
K0sFitVsPt.pdf
K0s yield decreases by 0.2% although this is dominated by the 1-1.5 GeV/c bin.

Note that there is currently no second iteration for UPC flagging.

[shahor02]

Then let's have it presented at the wp12/13 meeting next Wed. 15:30.
@cima22 Was suppressLowMultDebris=13 added to pp settings intentionally? Usually, we don't remove debris there.
@f3sch thanks, looks good. I assume for the UPC you left default settings?

[f3sch]

@f3sch thanks, looks good. I assume for the UPC you left default settings?

@shahor02 there is only one iteration done in this PR, if you mean the one from dev then yes.

[shahor02]

OK, I saw https://github.com/AliceO2Group/AliceO2/pull/15733/changes#diff-27fc8aaeb7769a496f838182d30a759ba63b3163bc7f458dd9bbbc895d3c7211R258-R260, but then the ITSVertexerParam are irrelevant in that mode, and there is no ITS trackfinding UPC iteration?
If the answer is long, don't waste time, will learn at the presentation.

[f3sch]

OK, I saw https://github.com/AliceO2Group/AliceO2/pull/15733/changes#diff-27fc8aaeb7769a496f838182d30a759ba63b3163bc7f458dd9bbbc895d3c7211R258-R260, but then the ITSVertexerParam are irrelevant in that mode, and there is no ITS trackfinding UPC iteration?

There is no track finding since no ROFs are flagged by a vertex carrying the UPC flag in the UPC mask, all ROFs are automatically masked in UPC iteration

[cima22]

Then let's have it presented at the wp12/13 meeting next Wed. 15:30.

@cima22 Was suppressLowMultDebris=13 added to pp settings intentionally? Usually, we don't remove debris there.

@f3sch thanks, looks good. I assume for the UPC you left default settings?

Hi, sorry today I am travelling so I won't be very responsive. I didn't put yet the UPC iteration in the PR, I will test one in the next days.

suppressLowMultDebris=13 slipped when copying the parameters, but in any case it is not used by this algorithm by default, thanks for noticing.

[cima22]

Fixed comments by @f3sch and added UPC iteration. For PbPb:

ITSCATrackerParam.diamondTrackletingCellDeltaPhiMinPt=0.12;ITSCATrackerParam.diamondCellTanLambdaNSigma=2.5;ITSCATrackerParam.seedingVertexIteration=1;ITSCATrackerParam.doUPCIteration=1;ITSCATrackerParam.cellLineSharedClusterCut=1;ITSCATrackerParam.diamondTrackletingPVres=1.5102214480735774;ITSCATrackerParam.diamondTrackletingNSigmaCut=2.976946815605669;ITSCATrackerParam.diamondTrackletingCellDeltaTanLambdaSigma=0.005566295924057422;ITSVertexerParam.suppressLowMultDebris=13;ITSVertexerParam.goodLineChi2Cut=9.814679066181695;ITSVertexerParam.goodLinePtCut=0.12;ITSVertexerParam.clusterContributorsCut=3;ITSVertexerParam.clusterCut=0.07;ITSVertexerParam.duplicateZCut=0.1388966993405415;ITSVertexerParam.pairCut=0.025684202919354766;ITSVertexerParam.nSigmaCut=0.0016;ITSVertexerParam.phiCut=0.008875191490279988;ITSVertexerParam.goodContributorsSignificance=0.070;ITSVertexerParam.fineZWindow=0.010;ITSVertexerParam.fineMinDensity=8;ITSVertexerParam.fineMaxDrift=0.005;ITSVertexerParam.lineMinPt=0.10;ITSVertexerParam.duplicateZScale=0.7;

For PbPb UPC iteration, clusterContributorsCut is set to 2, but it may need other tuning.