This caught my eye from Jim's Insider clip:

I gotta go old school Survivor. I gotta go back and highlight the risks and show people who's running the game and who's gonna win this game and what their plan is. I tell you what, Coach coming back a third time, do you really think he came back to make friends and establish a cult. <shakes head> Coach is here for one reason and it's the same reason I'm here for. A million dollars. I don't trust people, I trust motives, and what are Coach's motives? A million dollars. None other."

(cut)

"I'm either going home tonight or I'm winning a million dollars. I'm going to pull Sophie and Albert aside, whether or not I win immunity today, and I'm gonna have it all laid out in the sand, covered up with a couple of palm fronds, lift it up, here's what's gonna happen, you know what's gonna happen."

I don't know if Jim did that with Albert and Sophie, but as some have spec'd, Jim likely makes the remark, and he tells the people at RI that Coached is running the show and they need to wake up.

Kind of like Shii Ann was the one who pointed to Amber as she (SA) exited the game and said, Amber is going to win.

If Jim says it before the duel, then Jim is still in the game and a motive to try to shake it up. If after the duel, he is perhaps trying to make his point now that his own self interest has been taken out of the equation, thinking they might listen to him now.