You are definitely not alone. I spent years daydreaming about being a writer because if I admitted out loud to anybody that I wanted to be published then I was opening myself up to fail. 

I still haven't dared send anything to an agent yet but I am learning more all the time through Jericho Writers. On the one hand, the more I find out the more I realise how much work I need to do - why would an agent pick be as the 1 in a couple of thousand pitches? But on the other hand I am becoming more confident as a writer and realising that if I keep at it some of the stuff I produce is actually not completely crap! I still have lots of editing to do but I have an idea of how to do it now and of what direction I need to head in. And the journey will be long but it will be enjoyable.

Keep writing and believing in yourself.