Description

This veritable scattercannon of a PR needs to be reworked into multiple PRs. Not for merging as is
This PR is a superset of #2637 and breaking it up will commence after that one is merged.

Todos

Deletions

• Delete ael.cc (Approximate Excitation Level, never called and not PSI_API) (merged Remove unused approximate excitation level (AEL) code #2672)
• Delete Wabei_RHF_FT2_a.cc (never called and not PSI_API) (merged Remove unused Wabei_RHF_FT2_a function and file #2677)
• Delete unused functions/overloads from occ: Array2d::diagonalize, Array2d::davidson, Array2d::cdsyev, Array2d::cdgesv, Array2d::lineq_flin, Array2d::lineq_pople, SymBlockMatrix::davidson, SymBlockMatrix::diagonalize, SymBlockMatrix::cdsyev, SymBlockMatrix::cdgesv, SymBlockMatrix::lineq_flin, SymBlockMatrix::lineq_pople (never called and not PSI_API) (merged Remove unused diagonalizers and linear solvers from occwave::Array2d and occwave::SymBlockMatrix #2679)
• Delete unused functions/overloads from dfocc: Array2d::diagonalize, Array2d::davidson, Array2d::cdsyev, Array2d::cdgesv, Array2d::lineq_flin, Array2d::lineq_pople and Tensor2d::davidson, Tensor2d::cdsyev, Tensor2d::cdgesv, Tensor2d::lineq_flin, Tensor2d::lineq_pople (PR Remove unused diagonalizers and linear solvers from dfoccwave::Tensor2d and dfoccwave::Array2d #2684)
• Delete unused function overloads from libmints: Matrix::diagonalize (PR Remove unused and pointer-argumented overloads of Matrix::diagonalize #2693)
• Delete commented out code, an unused extern FILE* and unused #includes from mcscf::SCF::energy (PR Remove commented out code from mcscf #2678)

New DSYEV wrappers, anti-sq_rsp(...) crusade

• Implement two new wrapper functions around DSYEV to replace sq_rsp(...) with. The new wrappers have a much cleaner interface, and no longer require the allocation of an eigenvector array if the caller only needs eigenvalues. They also no longer swallow the return value of DSYEV, in fact their return value is marked as [[nodiscard]]. In short, checking for diagonalization failure went from impossible to mandatory.
• Replace sq_rsp(...) calls in mcscf (mcscf::SCF::energy and mcscf::MatrixBase::diagonalize) with new wrapper. Add checks for diagonalization failure.
• Replace sq_rsp(...) calls in detci (detci/h0block.cc, detci/sem.cc and detci/sem_test.cc) with new wrapper. Add checks for diagonalization failure.
• Replace sq_rsp(...) call in libmints/matrix.cc (Matrix::diagonalize) with new wrapper. Add checks for diagonalization failure.
• Replace sq_rsp(...) calls in ccenergy (ccenergy/d1diag.cc, ccenergy/d2diag.cc and ccenergy/new_d1diag.cc) with new wrapper. Add checks for diagonalization failure. This allows the removal of the eigenvector array, and the code that allocates/deallocates it.
• Replace sq_rsp(...) calls in the libqt Davidson solver with new wrapper. Add checks for diagonalization failure.
• Replace sq_rsp(...) call in the RHF and ROHF stability checks with new wrapper. Add checks for diagonalization failure.
• Replace sq_rsp(...) calls in libsapt_solver/sapt2.cc with new wrapper. Add checks for diagonalization failure.
• Replace sq_rsp(...) call in psi4/src/psi4/adc/diagonalize.cc with new wrapper. Add check for diagonalization failure.
• Replace sq_rsp(...) call in dfoccwave::Tensor2d::diagonalize with new wrapper. Add check for diagonalization failure.
• Remove vestiges of the diagonalization cutoff from dfocc. dfoccwave::Tensor2d::diagonalize used to take a "diagonalization cutoff" argument, which was passed down to sq_rsp(...). This used to be hardcoded to 1E-10, but went completely defunct when sq_rsp(...) started using DSYEV internally (~forever ago), as LAPACK offers no such parameter to adjust and always diagonalizes down to ~machine precision.

New sanity checks

• Add a sanity check for non-square matrices in mcscf::MatrixBase::diagonalize
• Add a sanity check for non-square matrices in dfoccwave::Tensor2d::diagonalize
• Add a sanity checks for non-square matrices and illegal values of "diagonalization order" in libmints/matrix.cc (Matrix::diagonalize)
• Check that all requested roots converge in the libqt Davidson solver used by adc/prepare_tensors.cc
• Mark the return value of the libqt Davidson solver as [[nodiscard]] to force callers to check how many of the requested roots actually converged.

Unrelated fixes

• Fix a memory leak in the ROHF stability check. (merged: Fix memory leak in ROHF stability #2671)

Changes to comments, formatting, etc.

• Fix typos in comments in adc/diagonalize.cc and occ/dpd.h
• clang-format some files
• Delete some empty lines between doc-comment-blocks and the function definitions so that VSCode can parse the docstrings automatically. (eg. d8cd1b9 )
• Use the more generic "LAPACK" as a term, instead of MKL/ACML/etc. when referring to the linear algebra library in comments found in occ/dpd.h.
• Add the default values of "diagonalization order" as comments at the definitions of Matrix::diagonalize in libmints/matrix.cc

API and deprecation

• Mark old diagonalizer functions rsp(...) and sq_rsp(...) as PSI_DEPRECATED. They are now no longer called anywhere from the C++ codebase, but these are marked as PSI_API so immediate deletion would be very harsh.
• Mark diagonalizer functions C_DSPEV(...) and C_DSYEV(...) as PSI_API, to serve as a potential replacement for the lost functionality. Other LAPACK functions are already PSI_API, so this should not be any more burden Psi4-side.

Archival

• Add ael.cc to psi4attic (merged Add AEL (approximate excitation level) code psi4attic#1)
• Add Wabei_RHF_FT2_a.cc to psi4attic

TODO

• Rework the remaining Matrix::diagonalize to only take references, and rework all callers to only pass references, eliminating pointer arguments. May touch: dct/dct_memory.cc, libmints/cdsalclist.cc, libmints/molecule.cc, ...
• adc/prepare_tensors.cc is now the only remaining caller of the Davidson solver in libqt. It has exactly one call site, and the solver is not PSI_API. Candidate for future cleanup.

Questions

• Question1

Checklist

• No new features
• Almost the entire test suite passes, sans MRCC due to unrelated issues. Some runtime-loaded plugins are not covered.

========================================================================================================== short test summary info ==========================================================================================================
> FAILED ../tests/mrcc/ccsd_t_/test_input.py::test_mrcc_ccsd_t_ - AssertionError: Using the `local_options` keyword argument is deprecated in favor of using `task_config`, in version 0.30.0 it will stop working.
> FAILED ../tests/pytests/test_composite.py::test_allen_focal_point - NameError: name 'psi4' is not defined
> FAILED ../tests/mrcc/optfreq/test_input.py::test_mrcc_optfreq - AssertionError: Using the `local_options` keyword argument is deprecated in favor of using `task_config`, in version 0.30.0 it will stop working.
> =========================================================================== 3 failed, 3726 passed, 173 skipped, 219 xfailed, 7068 warnings in 2238.06s (0:37:18) ============================================================================

Status

• Ready for review
• Ready for merge