LAST EDITED ON 02-26-12 AT 11:40 PM (EST)To prove you wrong, I will start with the end where you write:
"Matt didn't do the math"
According to everyone, Colton was going to be the first boot so Matt didn't have the math wrong afterall. His four weren't in danger.
Next, Colton heard that there was no EI but that there were idols. They had to be hidden in camp! He quickly started making a plea to the women for help finding the idol. Sab didn't give him the idol out of the blue: He sold his case to her, playing the martyr in order to get the power that the idol held.
Even if there was desperation, Colton correctly calculated his move. Now, it seems that the Misfits are united in getting the muscle out. Before Colton's move, we don't even know if the Misfits were aligned or playing as individuals, trying to get in with the muscle.
He is smart enough to know that he can't stay with the Misfits, that it's a temporary alliance until he gets with the women. If the women's number go down even more, they will gladly invite him back to their camp, especially if he brings one or two votes along.
As for early moves being remembered, he probably knew he couldn't win the moment the twist was announced. Going as far as possible does earn a nice check.