One should give up attachment for his wife – or in other words for sex life.

Every husband is too much attached to his wife. Therefore, to give up one’s connection with his wife is extremely difficult, but if one can somehow or other give it up for the service of the Supreme Personality of Godhead, then the Lord Himself, although not able to be conquered by anyone, comes very much under the control of the devotee. And if the Lord is pleased with a devotee, what is there that is unobtainable? Why should one not give up his affection for his wife and children and take shelter of the Supreme Personality of Godhead? Where is the loss of anything material? Householder life means attachment for one’s wife, whereas sannyasa means detachment from one’s wife and attachment to Krishna. If one is intelligent, he can think of his wife’s body as nothing but a lump of matter that will ultimately be transformed into small insects, stool or ashes. In different societies there are different ways of dealing with the human body at the time of the funeral ceremony. In some societies the body is given to the vultures to be eaten, and therefore the body ultimately turns to vulture stool. Sometimes the body is merely abandoned, and in that case the body is consumed by small insects. In some societies the body is immediately burned after death, and thus it becomes ashes. In any case, if one intelligently considers the constitution of the body and the soul beyond it, what is the value of the body? Antavanta ime deha nityasyoktah saririnah: the body may perish at any moment, but the soul is eternal. If one gives up attachment for the body and increases his attachment for the spirit soul, his life is successful. It is merely a matter of deliberation.

Source: A.C. Bhaktivedanta Swami Prabhupada (2014 edition), “Srimad Bhagavatam”, Seventh Canto, Chapter 14 – Text 12 & 13