Did brake cleaner not help at all? It's supposed to clean away dirt, rust, and brake fluid. I would soak them in CRC or your favorite brand of brake cleaner in a little tub for several minutes and then let them air dry.
In theory I bet they get a lot hotter than 400 degrees rolling down Pike's Peak or whatnot.
When in doubt get the credit card out... and buy new brakes
I have never had any drums or pads fail on me and I've owned and operated a lot of sketchy, old vehicles.
What I have had happen is when changing out brake pads, I was able to break the friction material right off of the backer plate by hand! I guess friction had just been holding it into place for who knows how long...
I have had mild luck spraying brake shoes soaked in oil (leaky axles on my Austin Healey Sprite) with carb cleaner and lighting them on fire and letting them burn themselves out over the course of 30 seconds or so. Oil is attracted to the hottest part and is spent as fuel. That's the theory any way. It seemed to work OK. Baking them in the oven might not actually get the oil out of the friction material. I'm not sure. If you ever spray something with carb or brake cleaner and then light it on fire, DO NOT BREATH THE FUMES. It can kill you.
I do know one thing.. it's no fun to drive a car with oil soaked brake drums