Yup, I figured out how to do it. The mysterious use is to use the beta function to fit the out-of-sample scores during cross-validation. Why the beta function? Because it naturally lives on an interval, and comes with skew and excess kurtosis, unlike the Normal distribution which does not live on an interval and has no skew or excess kurtosis. We can map the beta function to the interval of correlations, [-1,1]. Once you have used maximum likelihood to fit the distribution of scores you can derive any kind of estimator you like from it. In particular maximum likelihood estimates are robust against spurious fluctuations that are statistically guaranteed to occur during parameter optimization. See how nicely it fits our data:
I also considered the Logit-Normal, but in the limit that the standard deviation goes to zero, the skew and excess kurtosis also go to zero and that is contrary to what is observed. I am exploring the Beta-Ratio, the ratio of the areas of the fitted beta distribution above and below some threshold, in some sense similar to the sortino ratio, but it remains finite no matter where you set the threshold.