For what it's worth, I have had this exact same issue for a long time. My wife has it far worse than I do.
I am finding the more time I spend in introspection about who I am and why I do the things I do, that more and more is coming back to me. It's difficult, but I think it does occur, you just have to be willing to give yourself time.
One thing that I've found very important is I have a wife and a chosen-brother who are on similar journeys as my own. Being able to talk to someone is very important, even if it's to a journal/diary, I think it could help. If you can't do that, a good therapist might be of use.
I guess my major point is, be patient with yourself. The memories are locked up in there, it is going to take quite some time for them to come back out.
"Beware of he who would deny you access to information, for in his heart he dreams himself your master." - Commisioner Pravin Lal