I ended up scraping the window frames. That was a ton of work, but there were so many layers of old paint that just painting over was not going to look good. I tried to sand them at first, but there was simply too much to sand.
So I got out a heat gun. That looks and works like a blow dryer, but produces much more heat. So much that you can burn stuff, if you’re not careful – I curled the hair on my arm a time or two, but luckily didn’t get real burns. Warm up the paint just enough for it to soften and then scrape off the warmed piece, repeating the steps until most of the paint is gone.
It took about a week of evenings and the weekend to remove the paint, sand, spackle (and window putty), sand, spackle, sand, prime, sand, paint, sand and paint. But the windows look so much better now, I’m glad I put in the work! 🙂