doing whatever makes you happiest is a nice philosophy...but it doesn't realy hold water in this situation.
the responsible thing to do is to discuss it with your wife and really think about how it is going to effect her and the kids. Once you have a family things do change significantly from the days of just being able to "do what makes you happy". I think that you are on the right path here scott. You seem to be more concerned with the family and the career. I would pass on this one and continue to do it for fun. also you might offer to do it for them on a more official basis when they are in town... that way you can get a bit of recognition for for th awesome work you do.