I am not in favor of using multimeters on PSUs. See my canned text below on testing them. Sadly, there is often collateral damage when a PSU fails, and/or if the power anomaly is excessive. Let's hope it is only the PSU.
To properly and conclusively test a power supply unit (PSU), it must be tested under various realistic "loads" then analyzed for excessive ripple and other anomalies. This is done by a qualified technician using an oscilloscope or power analyzer - sophisticated (and expensive) electronic test equipment requiring special training to operate, and a basic knowledge of electronics theory to understand the results. Therefore, conclusively testing a power supply is done in properly equipped electronic repair facilities.
Fortunately, there are other options that are almost as good. I keep a FrozenCPU Ultimate PSU Tester in my tool bag when I am "in the field" and don't have a good spare power supply to swap in. While not a certain test, they are better than nothing. The advantage of this model is that it has an LCD readout of the voltage. With an actual voltage readout, you have a better chance of detecting a "failing" PSU, or one barely within specified ATX Form Factor Standard tolerances. Lesser models use LEDs to indicate the voltage is just within some "range". These are less informative, considerably cheaper, but still useful for detecting PSUs that have already "failed". Newegg has several testers to choose from. All these testers contain a "dummy load" to fool the PSU into thinking it is connected to a motherboard, and therefore allows the PSU to power on, if able, without being attached to a motherboard - great for testing fans, but again, it is not a true load or suitable for conclusive testing.
Swapping in a known good supply is a tried and trued method of troubleshooting, used for years even by pros. If you have access to a suitably sized, spare power supply, carefully remove the suspect supply and replace it with a known good one and see if the problem goes away.
I do not recommend using a multimeter to test power supplies. To do it properly, that is, under a realistic load, the voltages on all the pins must be measured while the PSU is attached to the motherboard and the computer powered on. This requires poking (with some considerable force) two hard and sharp, highly conductive meter probes into the main power connector, deep in the heart of the computer. One tiny slip can destroy the motherboard, and everything plugged into it. It is not worth the risk considering most multimeters, like plug-in testers, do not measure, or reveal any unwanted and potentially disruptive AC components to the DC voltages.
Note the required voltage tolerance ranges:

And remember, anything that plugs into the wall can
kill. Do not open the power supply's case unless you are a
qualified electronics technician. There are NO user serviceable parts inside a power supply.