College possibly being ableist?

I go to a small media oriented college that mainly focuses on media studies, creating films and stuff like that. It's only got a couple courses and I'm in my second year. Everyone is aware of my epilepsy (I have focal epilepsy) including the head of the entire college. It's just one of those places where everyone knows everyone. A couple weeks ago, I believe before Christmas actually, I was talking to the head about an incident that was transpiring regarding another friend and mental health, the specifics aren't important. What is important is that it was causing me to not be able to do work as I was very worried about them. I spoke with her about it as it was a serious situation and she said that I didn't have time to worry about this as I already have to work twice as hard as everyone else due to my epilepsy. At the time I didn't really think anything of it until I brought it up yesterday with some youth workers at a group I go to. They said that what she said was pretty ableist and unfair and that I shouldn't need to just do more because of something completely out of my control so, I wanted to ask here. Was this an ableist comment or am I now overthinking this?