Rayleigh-Gauss-Newton

pynqs.optim.grad.rgn.RGN_grad implements the regularized Rayleigh-Gauss-Newton (RGN) update described by Peng and Chan, Phys. Rev. Research 7, 043351 (2025). It follows the same stochastic objects used by pynqs.optim.grad.lm.LM_grad:

\[O_i(n) = \frac{1}{\Psi(n)}\frac{\partial \Psi(n)}{\partial \theta_i}, \qquad h_i(n) = \partial_i E_{\rm loc}(n) + O_i(n)E_{\rm loc}(n).\]

The sampled gradient, overlap, and approximate Hessian are

\[\begin{split}G_i &= \operatorname{Cov}(E_{\rm loc}, O_i),\\ S_{ij} &= \operatorname{Cov}(O_i, O_j),\\ H^{\rm eff}_{ij} &= \operatorname{Cov}(O_i, h_j) - G_i\langle O_j\rangle - \langle E_{\rm loc}\rangle S_{ij}.\end{split}\]

RGN minimizes the second-order expansion with the SR overlap penalty, which gives the linear equation

\[\left(H^{\rm eff} + \delta I + \frac{S+\delta' I}{\epsilon}\right)c = -G.\]

In PyNQS the optimizer stores the preconditioned gradient

\[d\theta = \left(H^{\rm eff} + \delta I + \frac{S+\delta' I}{\epsilon}\right)^{-1}G,\]

and the optimizer step applies theta <- theta - lr*dtheta. For the paper’s convention use lr=1.

Usage in VMCOptimizer:

vmc_opt = VMCOptimizer(
    nqs=model,
    opt=opt,
    sampler_param=sampler_param,
    electron_info=electron_info,
    rgn=True,
    RGN_epsilon=1.0,
    RGN_delta=0.0,
    RGN_damping_lambda=1.0e-3,
)

RGN_epsilon is the overlap-penalty scale \(\epsilon\). Small values approach SR, while math.inf gives the approximate Newton limit based on \(H^{\rm eff}\). RGN_delta is the H-side shift, analogous to the \(\delta I\) added to L in LM. RGN_damping_lambda is the S-side shift, analogous to the \(\delta' I\) added to R in LM and to damping_lambda in SR/minSR. For finite \(\epsilon\), RGN_damping_lambda enters the final matrix as \(\delta'/\epsilon\). These parameters can be replaced by RGN_epsilon_sh(step), RGN_delta_sh(step), and RGN_damping_lambda_sh(step) schedulers.