they can be quite loud though.
Well, compared to some aftermarket coolers that claim to be quiet. But both Intel and AMD have put some effort into making their OEM coolers quieter over the last few years too. Unlike aftermarket coolers, AMD and Intel are restricted by size restraints - that is, the OEM cooler is designed to fit in virtually any case, with any motherboard that supports that CPU. Even small crowded cases with big graphics cards and tall RAM.
Aftermarket coolers often use huge fans that run slower, thus much quieter. Some fans also blow across the CPU instead of downward firing and can they can be very tall off the motherboard surface. So they don't fit in many cases.
I think this happens because side fans can cause turbulence, interfering with good airflow.
This is true in some cases/situations. It depends on the the size and placement of the side fan. For example, some use tunnels to direct the side fan flow directly onto the CPU heatsink and they tend to work great, without disrupting the desired front to back flow. Some tunnels direct the flow towards the graphics solution. But most side fans just blast into the interior without tunnels.
My general recommendation is to try it without a side fan first and see if your temps are controlled. If more cooling is needed, add a side fan (if no room for another rear or front fan) then carefully check your temps and if temps are not better, remove it.
Another problem with side fans is most cases do not support removable, washable air filters. So they tend to suck in a lot of heat trapping dust.
Note too that blow-hole (top) fans are great at extracting rising heat, but - you don't want air to flow directly from the front intake to the top output without providing sufficient cool air to the heat sensitive components at the rear of the case - like the graphics card. So if using a blow-hole fan, I generally run it at its slowest (quietest) speed. I prefer a rear fan (not counting the PSU fan) but blow-hole fans are nice. I have a "Bigboy" (200mm) top fan on my bottom PSU mount Antec case. It is located at the far back of the top - I am happy with that.