I'm going to post exactly what I did so anyone who has Windows XP (like no one, lol) doesn't have to sit around forever figuring it out....
1) Open the Control Panel and click Regional and Language Options.
2) Click on the Languages tab.
3) Click on Details, then Add.
4) Click on Keyboard Layout/IME.
5) Find "US - International"
After this, you should get an icon in the task bar or near the top of your window. You can toggle between standard English and the US-International by clicking on this icon. Hold down the right ALT key with the key. Umlauts are Y, P, and Q (depending on what they are close to. See - ¿Come estas, MarÃa? Was gibt es zum Frühstück? Etc etc. Ä, Ãœ, Ö.