Get my kid to behave at school?

I'm at a loss. My 6 year old son comes home everyday with a bad note. He's in a program where the school sends home a report every day on his behavior. He gets points on being safe, responsible, and respectful. Then it's added up in a percentage. He has to have like at least 80% to meet his goal for the day. They send the week's total percentage through email every week. His biggest thing is not wanting to do his work or listen to the teacher. I've tried rewarding him, taking away dessert and special treats, not letting him play outside with friends, housework. At school he doesn't even want to try doing his work and he has to stay in during recess to finish, then he still refuses or acts out/throws tantrums. He also has a problem hitting and biting when people make him made. He's been kicked off the bus like 5 times. He crumbles up his report. He was in an accident when he was 18 months old and had a concussion. His hearing was messed up a bit after, but we never knew for awhile. He forgot a lot of things he learned, he was behind in speech, he had to have occupational therapy, he's in language therapy, and has an IEP. I don't know whether it has something to do with the school, me, or if my son just doesn't care anymore. He lived with grandpa for a little while so I don't know if he was just spoiled or what. We struggle at home with him too. He has a lot of break downs and tantrums and hits his 2 year old brother (he's even charged and ed him when he was younger). When I talk to him about what happened at school he avoids the subject, sometimes even changing the subject randomly. Have any of you had similar struggles?