Woolery stated to PEOPLE in 1991 that his childhood was " real happy and very simple" while residing in a two-story clapboard ...