Many devotees fall down due to illicit sex.
Material life is such that due to indulgence in illicit sex, gambling, intoxication and meat-eating, the conditioned soul is always in a dangerous condition. Meat-eating and intoxication excite the senses more and more, and the conditioned soul falls victim to women. In order to keep women, money is required, and to acquire money, one begs, borrows or steals. Indeed, he commits abominable acts that cause him to suffer both in this life and in the next. Consequently illicit sex must be stopped by those who are spiritually inclined or who are on the path of spiritual realization. Many devotees fall down due to illicit sex. They may steal money and even fall down from the highly honored renounced order. Then for a livelihood they accept menial services and become beggars. It is therefore said in the sastras, yan maithunadi-grhamedhi-sukham hi tuccham: materialism is based on sex, whether licit or illicit. Sex is full of dangers even for those who are addicted to household life. Whether one has a license for sex or not, there is great trouble. Bahu-duhkha-bhak: after one indulges in sex, many volumes of miseries ensue. One suffers more and more in material life. A miserly person cannot properly utilize the wealth he has, and similarly a materialistic person misuses the human form. Instead of using it for spiritual emancipation, he uses the body for sense gratification. Therefore he is called a miser.
Source: A.C. Bhaktivedanta Swami Prabhupada (2014 edition), “Srimad Bhagavatam”, Fifth Canto, Chapter 14 – Text 22