Jesus was asked should we pay taxes and basically answered YES but then (as usual) said more. Just as money has the portrait / inscription of government on it, and so is given to government, something else is to be "given up / given back" to God and the implication is it's the thing that has the portrait / inscription of God on IT.
He doesn't really say "pay back." It's "give up / give back." As if the one who has taken possession of the thing is not the rightful owner.
From Hungry Heart #4/15...
Christ lived on earth that He might show forth the image of God in His life; He lives in heaven that we may show forth the image of God in our lives
God has but one way of revealing Himself, it is 'Christ in you.' He has no other way of showing Himself to men except as Christ lives in us