I'm tending more and more to cotton - are there ANY downsides??

yes, a quilt with cotton batting can be a bit stiffer than one with poly
batting and initially not feel as soft, to get round this, my suggestion
would be to choose a cotton batting that allows a large space between
lines of quilting and not quilt it densely, it will soften up with use
and washing.

Cheers
Anne