I think there's much truth in what you say.
I've listened to classical on and off for forty years. I haven't been as rigorous a listener as some on this forum obviously are, but a turning point for me was when I came to the point of buying recordings then putting them aside and not listening to them for months (or even never listening to them). I realised that this risked making music into a burden. What solved the problem was only buying what I wanted to listen to straight away. Eventually I did a big cull of my collection (got rid of two thirds) which made me prioritise my listening further.
At times, with my focus on music bordering on the obsessive, I lost sight of that thing called life. Music is great, but its not a particularly social activity. Although I didn't plan it, that process of distilling what I wanted out of music allowed me the space to do other things.
Life also changes, compared to previous times these days I often don't have energy or inclination to listen to music. After a busy day, I'm often happy to rest without any canned music, just silence (or what amounts to it in reality). There's a calming aspect to simply doing things that need to be done, or even doing nothing, without any distraction.
Lessening the amount I listen to has had the advantage of being able to read more, which includes books on music. I'm getting into the habit of reading composer biographies and then listening to the music I have by them. Usually, it's limited to the handful of discs I have by a composer, and if needed I supplement that with youtube. It's allowed me to gain a deeper insight into the music I'm really interested in. Whenever I can, I contribute a review or survey including information I find interesting from these books on the relevant composer page on this forum.
The ideas of John Cage (who didn't own commercial recordings or a stereo system) and going further back, the stoics, have influenced me a bit. I do think that less can be more, and an individual can find some sort of a way out of all the FOMO around instant access to everything on the net. There are advantages to it of course, but the downside is a feeling of being overwhelmed and clogged up no matter how much you listen to. It makes sense to try and find your own way out of this, for one thing it's more balanced and sustainable on the long term.