I think they where called nerfuls. They at least match the above description. There is a page on this site for them if you want to look. It's under n fof nerfuls of corse. There's also an old tv add on YouTube the toy the header is Nerfuls 1986 commcial.

my cousin and i were just reminiscing about these over the weekend! we thought they were just called "nerfs." he had the stackable houses, one was a jail, the other just a home. they had trap doors in the bottom of each home too. definitely had the green slides. he had one ball that was blue and that was a dog and there was long shaggy blue ears/hair that went on top of the ball. he had a green chair for them to sit in without the bottom part of the bodies. also had the robber ball with the black eye mask across the face. we were cracking up, thinking of the hours we played with this fun toy!