Why does it matter?

Why can't we let people freely love who they love and let them spend the rest of their lives with? It is like saying that someone can't eat cake just because I'm on a diet, and in America, there is freedom to religion, which is ususally the argument to only have marriage between a man and a woman.