Yes he does write about paying in Confederate money (missed this earlier). Marye was at Hagerstown MD (p. 1227):
"General Hill's orders were very strict to the effect that we should pay for everything whether to friend or foe, particularly as we regarded Maryland as a friendly State. Of course, having nothing but Confederate money, we had to pay with that." He then tells of buying boots with Confederate money.
The story of impressing the "pet" horses comes soon after, though they had crossed into Pennsylvania; nothing explicit about paying or not paying for goods and horses taken.
John