How could I change the default language so that I get something else instead of "Table" or "Figure" before my captions?
Answer
You can change your document's language using babel package. The captions will be updated automatically.
As a bonus, you'll get the correct hyphenation for the chosen language.