In general, languages usually use accusative case for direct object after transitive verb. You won't mistake too much if you use accusative in Ukrainian either:
Сестра написала мені цей лист.
Still, Ukrainian often also allows something like genitive case to be used for direct object after transitive verb:
Сестра написала мені цього листа.
Here both variants are correct (though they may have some tricky differences in connotation).
But not under all circumstances you can replace accusative case with genitive case for direct object after transitive verb.
In fact, in most cases when you can, it is:
- either some rudiments of the partitive case, e.g. «випити води», «наїстися хліба»;
- or some almost-stable expressions.
Probably it is not even a real genitive case; it sometimes uses a form that is untypical for genitive case; for example if we consider the noun “приз” (“prize”) — typical genitive case for it is “при́зу” (e.g. “нема призу”), but under these special circumstances “приза” is used:
З-під самого носа у чоловіка приза витаскав, а ще й каже, що не винний.
Там вона взяла участь у позаконкурсній програмі, отримала приза глядацьких симпатій…
I don't think that it is related to the verb reflexiveness (i.e. “-ся” ending) — see the example with the word “наїстися” above. I think it more depends on the object rather than on the verb — and also on how well the phrase in general fits into some patterns where historically the partitive case was used (or maybe where some other extinct cases were used). Sadly I can't postulate the concrete criteria (I recommend you to use accusative when you are unsure; with optional usage of (pseudo?-)genitive (partitive?) in the phrases that you regularly hear said in that style from others).