4 weeks on buspirone. I'm horribly tired and fatigued

I have a terrible health anxiety and just very bad anxiety in overall. My dr prescribed me buspirone 7.5 mg one tablet twice a day and it's been 4 weeks of me taking it. My anxiety is still shit and I really don't think I can continue this med. I'm so tired, I lost motivation to do anything around the house, I'm lazy. I know it takes a bit for the medicine to settle in but at 4 weeks I think I should feel at least a bit better? I will be contacting my dr on Monday to see If I can just stop cold turkey or stop it gradually. I'm so freaking sleepy all the time.

Please tell me, is it normal? Should I push through? Is it worth it?