For years, we've been told that we needed Downy Fresh softness in our wash, or the convenience of Bounce dryer sheets. Fabric softeners take away the scratchiness of towel, and, well.....soften them.
Unfortunately they also take away some of the absorbancy. Have you ever had to use a hotel towel, or perhaps one from a gym? Those scratchy towels are much more absorbant because they don't use any fabric softener in the wash.
There are two other reasons not to use fabric softener. As a quilter, I've found that cottons which will be used in quilts shouldn't be softened. The chemicals stay in the fabric longer, because quilts are not frequently washed, and they can deteriorate the fibers.
And, for those of you who have asthma, the perfume added to fabric softeners can be a problem. Years ago, we lived in an apartment building that had laundry facilities. I had to go down a flight of stairs, out the door, over to the next entry and down another flight of stairs to get to the washers. I noticed as I was reversing the trip that I was wheezing by the time I got back to my front door, and even started an exercise program to get into shape.
When the wheezing didn't go away, it finally dawned on me that I could add fabric softners to the list of materials that I was allergic to. Now, whites, are the only clothes that I use softener on. I wash and bleach them, rinse them, start the washer a second time with a quarter of a cap of softener, and then rinse them once more. The rest of our clothes go softener-less, and no one has complained!
Try giving it up. It will save you money, and might make your towels more functional!