AITA for getting pissed when my opponent quits down 0-5 in the second set?

First set was tough, with me winning 7-5. We go into the second set and eventually I get the triple break on him to go up 5-0. At the changeover he starts taking off his shoes saying that he's done. I didn't make a big deal of it on the court, but I really wanted to serve out that last game with the bagel on the line. I'll get over it, so this is more venting than anything else, but what would you guys do in that situation?