It is totally natural to feel fear, but if you're feeling terror, it's not time. I've read plenty of stories on hdforums written by guys who have gone through accidents similar to what you are describing, but haven't (and hope not to) experienced it myself. I am a relatively new rider, so the high level awareness is still ever present for me, but the fear is gone. It took some time and there came a point where I was just ready to let go. Remember that life is temporary, no matter what we do, and safety is often nothing more than an illusion. I bet that you will ride again, but it sounds like you might not be ready. Stop worrying about anyone who might deem you a "pussy", because chances are nearly 100% that whoever is administering the label is completely full of shit anyway. When considering that sort of judgment, I remind myself that 99% of people today=shit on feet, and poof, any worrying I did about their opinion magically disappears.

Take your time man, enjoy the little things, and get back on two wheels when you're ready, but not until you're ready. The time will come, just have faith in yourself.