I'm surprised
the magic eraser didn't work. Have you tried liquid dish soap, and a mild scrub pad? I'd suggest a car polish or scratch remover, but they are white, and I don't know if they'd also not stain?
You can try on a small test area.
I have used it on soft bumpers on my Eureka canister bumper gaskets with no problem.