'Mo Ichido Onegai Shimasu'
This next phrase builds on onegai shimasu and is one that I learned to use the hard way. Mo ichido means "once more." So, "Mo ichido onegai shimasu" means "Once more, please." It's a polite, simple way to ask for clarification if someone speaks too quickly, or if you had trouble understanding what was said. If you're in a pinch, "Mo ichido" can work as well!
While we're on the subject of having trouble understanding, wakarimasen means "I don't understand" or "I don't know." It can be used as a single-word response for a question that you don't know the answer to, or can be used in the structure X wakarimasen to mean "I don't understand X." For example, if someone comes up and starts speaking to you in rapid Japanese, or nihongo, you can say, "Sumimasen. Nihongo wakarimasen!"
'X wo Kudasai'
"X wo kudasai" is another way to ask for something, and basically translates to "please give me X." It's commonly used when ordering, and can be used interchangeably with onegai shimasu in restaurants. For example "Bi-ru wo kudasai" or, "I'll have beer, please."
Please note that although wo has a w in it when written, its pronunciation is closer to that of a single letter "o."
It's almost impossible to overstate the usefulness of sumimasen. Depending on the context in which it's used it can mean "excuse me," "I'm sorry" or "thank you." If you bump into someone on the sidewalk, you can offer sumimasen as a quick apology. If you drop something and someone goes out of their way to pick it up, sumimasen is way to thank them and acknowledge the trouble you inadvertently caused. Perhaps the usage that foreign visitors feel most self-conscious about, saying sumimasen loudly in a restaurant or bar, is also the best way to get the attention of the staff!
'X Arimasu ka'
Next, let's move on to a couple of useful questions. "X arimasu ka" literally translates as "Does X exist?" Thinking about it logically, the meaning is closer to "Are/is there X?" or "Do you have X?" For example, if you want to know if a restaurant has the popular dish okonomiyaki, you could ask, "Okonomiyaki arimasu ka?"
Keep in mind that it's grammatically incorrect to use "X arimasu ka" when asking about people or other living things, and sort of sounds like you're asking about their now-inanimate dead body.
'X wa Doko Desu ka'
Whether you're a tourist looking for the right train line or a desperate customer in need of a bathroom "X wa doko desu ka" ("Where is ~?") is your best friend. If you want to find out where Asakusa's famous Kaminarimon Thunder Gate is located, you could ask, "Kaminarimon wa doko desu ka?" Even with sophisticated map applications, the narrow, often winding streets of many of Japan's cities can be tough to navigate, and being able to ask where something is will save a lot of time and aimless wandering.
BONUS: 'Arigato Gozaimasu'
This one kind of goes without saying. But if someone is kind enough to give you directions, or brings that beverage you've been thirsting for, make sure to thank them with a heartfelt arigato gozaimasu!