Reality TV stars Kody Brown and his four wives say they just want one thing: to be left alone.

As authorities investigate them for bigamy, the TLC "Sister Wives" family is asking a federal judge to overturn part of Utah's bigamy law because it bans them from living together and criminalizes sexual relationships between unmarried consenting adults.

"What they are asking for is the right to structure their own lives, their own family, according to their faith and their beliefs," said Jonathan Turley, their attorney, adding that the lawsuit is about privacy — not polygamy.

The case in federal court in Utah, however, could open up the possibility that a way of life for tens of thousands of self-described Mormon fundamentalists could be decriminalized.

While all states outlaw bigamy, some like Utah have laws that both prohibit having more than one marriage license at a time and also ban adults from living together and having a sexual relationship.

The latter provision could include same-sex couples, unmarried heterosexual couples and those, like the Browns, who do not have licenses but have created within their homes a marriage-like relationship.