When we got to Nanzenji we were two out of maybe a couple of dozen scattered around the extensive site, and after we'd paid the admission fee to walk through the sand gardens, there were a handful of other visitors, all of whom were careful to avoid cluttering up everyone else's photographic record.
There were, predictably, things you're not supposed to photograph. In this case, interior panels that wouldn't have shown up too well without artificial assistance in the lighting department, which is probably why the instruction is there.