These are common symptoms for the following overheating, bad PSU, a short circuit (possibly in the MB) and a bad MB in general.
Some questions if I may;
What is the computer model name or number (I note that the MB is a HP model)
Was there any problems leading up to this.
Any power outages or spikes whilst the computer was working or connected to the wall socket.
Any new hardware or software added/installed.
What are the replacement CPU and PSU details.
Was thermal paste applied to the replacement CPU.
Do you have an add on video card fitted.
Can you please check that all other internal connections are secure including the front panel header that includes the power on button, check for any damaged wires that could be shorting out and any stray screws for the same reason.
Inspect the capacitors on the MB for any signs of damage such as bulging or leaking, some images to help you http://cquirke.mvps.org/badcaps.htm
I realise that I have asked a lot of questions but this will hopefully help to identify the issue quickly for you.