In a way, that over-the-top quality is the central freedom of a fictional BB season. You can go completely into the lunatic fringe and stay there because anything you could come up with is probably something the show's considered, starting with stupid twists, ranging through insane contestants, and wrapping it all up in a theme no reasonable person would have ever considered putting it on the air. No matter how far beyond the borders you think you're venturing, you're still in show territory: you just got there first.Of course, it might help to (unlike this season) have at least one person the viewers/readers could remotely stand. My first thought for a storyline would be 'one or more sane people struggling against the occupants of a madhouse, an uncaring show, an inattentive host, and a sadistic viewership mostly comprised of VFTW posters on their day off.' Imagine a first-person BB from the viewpoint of a puppeteered America's Player Part II, with the show, the other players, and especially America all working to make hir life as hard as possible -- then watching that player make it all work for hir. (Probably to lose at the end at the kicking feet of a jury that wants revenge. The show's funny like that.)
But BB does have problems if you translate it to fiction. For starters, the setting is -- limited. Very, very limited. (One of the central themes for a season is likely claustrophobia -- but it can be overplayed.) You have a house and a backyard. Better dress them up well, because you'll be seeing a lot of them.
The duration is arguably the biggest problem. Take a season all the way to the end, and you're covering ninety days of action. That's practically an order of magnitude over what's been attempted or accomplished so far: Survivor is thirty-nine days, the Race course typically takes about a month to run and the Mole contestants will likely work around that scale. Ninety days is a lot of material to work with.
How much do you show? You can't just base it around the most typical show events: competitions, nominations and votes, the occasional interview and contestant feature segment. Do you use a DAW Chorus of sorts to cover some of the live feed events? How much is seen through the insider viewpoint? No one can manage every hour of every day -- but even if the contestant is in the relative dark, there are times when the readers have to have a more complete idea of house events. And the writer can't go to the guinea pigs feed.
The host is her own minor hell. There are going to be competitions which get miscalled, questions that make no sense, interviews somehow attempting to remove insight, and the occasional outright forgetting of which player she's talking to. None of which even remotely stretches the character as she's already established herself. Whee!
Oh, and then there's the contestant pool. If you're doing a typical season, you probably have fourteen of them, and there ain't no such thing as tribes: pretty much everyone can be around at any given time, and typically, all of them are talking. Or more likely, screaming. That's a lot of people to track at any given time, especially in group scenes -- some of which will inevitably degrade into shoutfests.
As a fictional season, Big Brother has a lot of problems to overcome.
As a real series, it arguably has even more.