Yes, you can remove the one wall so you can access the hallway. It still makes it hard to do the downstairs hallway, but with a little crafty patience it can be done without having to tear the whole house apart. I had to do this on the Willowcrest finished last autumn...
Oddly, after all the torture you go through finishing the hallways, when you restore the wall and rooms it abuts, you still don't see the hallways...but it is a personal satisfaction!