What does your money pay for when you adopt

It depends on the type of adoption. For overseas adoption, some might go to the child's birth country's government to cover various fees, some might go to the agency who provides care for the child and other children as they wait for families, some goes to the home study agency, some goes to government fees in our own country.
For an adoption in the US, lots of the cost goes to the agency to pay the folks who work with you doing the home study, who work with the birth mother, etc. If you have ANY concerns about costs, a reputable agency will be able to give you a precise cost breakdown. If they cannot, they are NOT the agency to trust.