Hyde Park is a wonderful old neighborhood. You will feel like you traveled 100 years back in time. The church is 1/2 block to iconic Avenue B Grocery (in continuous operation and largely unchanged since 1909). Great place to get a soft drink or a sandwich, and sit at picnic tables when the weather is nice.
It's also only three blocks to Shipe Park, with a very nice playscape for younger siblings. Elizabet Ney Museum is adjacent to the park, and would make a nice outing. It looks like a tiny castle and houses a collection of historic paintings and sculptures. Central Market is walking distance, and also has a great playscape, plus a cafe with both indoor and outdoor seating. Yarborough Library is a 5 min. drive away.