you dont need to remove the battery as well, it is an extra measure of safety to clear the CMOS.
as long as you unplug the box, moving the jumper for 10-20 seconds is sufficient. feel free to pull the battery as well, if you like. it certainly wont hurt.
after doing that, you will need to go into BIOS and reconfigure settings. make sure the cpu is seen correctly (should be automatic), ram is set correctly (go with "BY SPD" if available, this configures the RAM settings by what the ram tells the machine it can do).
there should be a screen to monitor system voltages and temperatures in BIOS, have a look there. let it run for while, checking the CPU temp occasionally. using the numbers we posted before, if the chip is running hot, then redoing the thermal paste will be needed.
also I would suggest setting the boot sequence for floppy, CD, HDD.
is this a new hard drive, or being moved from another system? fresh install of XP, or a reload, or ?