The subtle body ends if there is no material desire at the time of death.

At the time of death, fire burns the gross body, and if there is no more desire for material enjoyment the subtle body is also ended, and in this way a pure soul remains. This is confirmed in Bhagavad-gita (tyaktva deham punar janma naiti). If one is free from the bondage of both the gross and subtle material bodies and remains a pure soul, he returns home, back to Godhead, to be engaged in the service of the Lord. Tyaktva deham punar janma naiti mam eti: he goes back home, back to Godhead.

Source: A.C. Bhaktivedanta Swami Prabhupada (2014 edition), “Srimad Bhagavatam”, Ninth Canto, Chapter 6 – Text 54