ISP: Verizon FiOS
Router: Actiontec MI424WR (came with FiOS install)
Wired devices (3): 1 Dell Dimension 4600C, 1 HP Photosmart 2710 Multifunction, 1 Network-connected external hard drive
Wireless devices (6): 1 Apple Powerbook G4 (10.4), 1 Apple MacBook (10.5), 1 Dell laptop (unknown make; only about 6 months old, runs Vista), 1 Apple iPhone, 1 XBox360 with Wireless XBox Live adapter
Additionally, I have been trying to set up a Slingbox PRO but with my network as balky as it has been, I have been hesitant (plus I need to get a longer CAT5 cable

It seems, at least from an entirely anecdotal point of view, that most of the problems occur after I shut off the XBox after playing on XBox Live. The wireless stops working other times, but it seems most frequent after this activity.
When I say "the wireless stops working," I should clarify. Every device still shows a strong connection to the router, but, for instance, when opening a web page, it will say that I am not connected to the internet. At this point I usually shut off the router, then turn it back on after 30-40 seconds. I do notice that the router feels warm when I shut it off, but I don't believe the heat buildup is the culprit in the internet failure over wireless, since it still works for the wired devices, and there is no steady degradation of performance as the router heats up: it just stops working, sometimes randomly, sometimes after doing certain things (the XBox Live issue is the only one that stands out).
I have been looking into upgrading the wireless capabilities of the network, but unfortunately, setting up and maintaining even small networks was never my area of expertise when computers are involved. My question, therefore, is this: is my problem rooted in the configuration of my network or do I need the extra bandwidth a separate wireless AP or a switch of some sort or just a more powerful router to solve the problem? Any advice would be appreciated, since I leave for school on Wednesday night and would like to have the situation settled before then.