I remarked in another thread that I was always prejudiced against Rigoletto because of the story. Funny thing is I recently had an experience the exact opposite to yours. Normally I always follow the libretto but recently I tried listening to it without it, just concentrating on the music and I began to appreciate the beauty of the music more. Nonetheless I find in most cases following the libretto greatly enhances the experience.
Verdi for me was an acquired taste but I've gradually become a passionate fan and he's now my favorite opera composer. But for a long time I was cool to Falstaff. Because of its reputation I was frustrated by my inability to appreciate it. I even bought the Dover score hoping following the score would help me appreciate it more. Unfortunately the score only gives the Italian text. Then recently I listened again to the Karajan-Gobbi recording and suddenly something ignited in me. This time I was following the libretto not the score. Falstaff is one opera where you really need to follow the words. Verdi follows every twist and turn of the text and the music by itself often lacks continuity.
La Traviata has also been growing on me. The sentimental story has always appealed to me less than more garish melodramas like Il Trovatore and La Forza.