Well, firstly, we don't use just Greek letters. Most commonly these days, it's Roman ("English") letters that are used, such as x, y, z… to denote unknowns, or a, b, c… to denote constants. (Including e and i to denote some particularly special constants — perhaps I should call them constant constants.) Further, some Hebrew letters are used as well, such as aleph (א) to denote cardinalities, and some Fraktur letters too. (Though you could say that's just a typeface, rather than another script.)
That said, indeed Greek letters are used, not only for constants such as π and φ, but also α, β, θ, λ. The answer to why they are used is just that Western civilization has the Greek civilization as one of its parents, and this usage has continued through the centuries. Specifically:
Aristotle and Diophantus used Greek letters to denote numbers or unknowns,
later mathematicians used letters from alphabets they were familiar with, and as both Greek and Latin were part of the education of everyone (until perhaps a century ago or so), they used those letters. In particular, using Greek letters has an advantage that they are less likely to be confused with the running text in the language of the mathematical work being written (Latin, or, these days, English), while still being familiar to everyone. (Consider "Let a be a number", where without proper typesetting on the variable a, you can go down an English-language garden path of "Let a…")
When you run out of Roman letters to use, you have to reach outside anyway :) (Tales are told of lectures in which the speaker ran out of both Roman and Greek letters, and started on other alphabets… clearly, other alphabets are less preferred.)
made-up symbols or letters in some other language would be distracting and unfamiliar, or even hard for printers to typeset. Indeed the symbol
was in use for representing unknowns for a while, but it has fallen out of use.