The fire broke out on Tuesday, but there are still hot-spots as of Friday at a warehouse in Campbellsville.
The smoldering spots have delayed investigators from getting inside the building to begin their investigation.
The building used to be used by Fruit of the Loom, but they closed operations there in 1998. Now, it is used for storage by other companies.
Eight different fire departments responded to the fire and it took them hours to get the flames under control. About a quarter of the building was reportedly destroyed.