Hi Margaret,
Milk has an enzyme that breaks down protein, i.e., gluten. Perhaps
fresh milk has even more of it.
Try scalding the milk beforehand (hand removing the "skin" on top of
the milk). That should remove/deactivate the enzyme and improve your bread.
I've read that a little bit of oil in dough helps oven spring. I
don't know whether the fat in the milk is a good substitute for the
oil you added previously. At the very least, if you know the
butterfat content of the milk (typically 3.6%) you can do some
calculation on how much fat you've added and see if it's similar to
the amount of oil you've left out.
By the way, you shouldn't just substitute 1 cup of milk for each cup
of water you omit. Milk only has 87.4% water (regular whole
milk--don't know about the milk from your cow)! So for each cup of
water you omit, you need to add 1.14 cups of milk.
Allen