This is a self-administered fear of being constrained by our fears. The idea of our fears suppressing us from reaching our full potential and life experience should invoke an even greater fear, the fear of fears.
Does this mean we need to embrace a never-say-no-philosophy? Although extremes can be useful to prove points they are seldom effective in real world application. This brings us to the next crucial aspect; know the difference between the fear of the unknown and hesitation based on valid concerns. This is the fear we need to face, we need to be empowered by our experiences, not be kept back from the lack of previous experience when an opportunity presents itself.
Be honest with yourself and face your fears aggressively, as you would an enemy. We have to live with ourselves our whole existence so the person you should strive to disappoint the least is yourself. Make a decision, yes or no and do not doubt yourself once it is made, the biggest failure is indecisiveness or the lack of resolve to commit to a decision (This points to the commitment to a choice not an inability to admit to failures).