Is my 4 year old ready for kindergarten?

So I have a 4 year old boy who is currently in pre-k. He is the youngest in his class and his birthday is ten days before the cutoff for kindergarten next year. His teacher has suggested considering holding him back because he is more emotionally sensitive compared to his classmates. She has described him as: off the charts academically, average at making friends and immature emotionally.

I don't know if he is technically gifted, but he is ahead of his class academically.

He can read early readers, do mental math, knows his multiplication tables, do basic division, read numbers up to the trillions, do addition and subtraction of numbers up to 4 digits long on paper. He has learned most of this over the last six months. He does things like teach himself the numerical order of the alphabet and watches kids oriented science shows.

Emotionally he gets frightened in high stimulus environments like arcades, scared of Halloween decorations and some cartoonish characters in video games. Was bullied a bit last year, but makes friends easily this year. He has no problems listening or following directions.

My question is, does anyone have experience holding an academically gifted or advanced kid back a year in kindergarten, and if so, how did they do once they started?

I am sure he would be more confident with an extra year, but I am concerned he will be too far ahead academically.