It’s tidier and easier to reconfigure. The patch cables can be any length, giving you flexibility on placing the switch and router. The cables in walls usually have solid wires in them, which is less flexible than the stranded wires in patch cables. Punching down a cable on a patch panel is easy, but terminating it on an RJ45 connector is a pain and is error-prone, and you can get more noise if it’s a sloppy job. Labels on a patch panel are easier to read and more durable than labels on a cable. Are those enough reasons?

Mostly, though, it’s the tidiness.