One of the most important conditions for a thing to become mubah (lawful) is that the khums for that thing (if wajib) must have been paid.
Ignoring the condition of mubah for salat is the same as taking salat lightly or insignificant (istekhfaf) and amounts to a great sin.
If khums was wajib on the clothes you are wearing and has not been paid, and if the salat has been offered in such clothes, then the salat is batil (invalid) even if you did not know this rule.