|
|
|
|
PLEASE NOTE: The Reality TV World Message Boards are filled with desperate
attention-seekers pretending to be one big happy PG/PG13-rated family. Don't
be fooled. Trying to get everyone to agree with you is like herding cats,
but intolerance for other viewpoints is NOT welcome and respect for other
posters IS required at all times. Jump in and play, and you'll soon find out
how easy it is to fit in, but save your drama for your mama. All members are
encouraged to read the
complete guidelines.
As entertainment critic Roger
Ebert once said, "If you disagree with something I write, tell me so, argue
with me, correct me--but don't tell me to shut up. That's not the American way."
|
|
|
"Statistical Rankings: TAR 1 thru 7"
sittem 4186 desperate attention whore postings
DAW Level: "Jerry Springer Show Guest"
|
05-17-05, 06:47 PM (EST)
 
  |
|
1. "RE: Statistical Rankings: TAR 1 thru 7" |
As one who loves statistics this is great information. Thanks for taking the time and doing the research coming up with this information. I've got to think a little more about methodology - sounds logical, but there's a nagging check for me that I have to identify. So what I add below is a bit tentative, but wth?I suspected that RA and KJ were right near the top of best racers and this confirms that. I am surprised at FM - I liked them, but didn't remember them being that solid and that competitive with RB. I also felt CC would be high up. The ranking of CK and UJ doens't surprise me - they were kind of average, but took care of business late. They seemed to "get it" as the season wore on. Surprises for me were how low Chris and Alex were rated - I know they had some low early finishes so maybe that affected their place. They are the lowest rated winners of the seven races, slightly behind FK. Only twice have the highest rated racers also won - in #3 it was FZ and in #4 it was RC. However, RC apparently didn't have much competition as their fellow racers finished 18th and 19th overall making TAR 4 the season with the lowest cumulative ranking. All the other winners were second ranked during their seasons. The winners finishing the lowest behind a higher rated second were FK in TAR 6 - not a surprise to me at all. They were a full 18.8 behind KJ. Overall the seasons ranked as: 1. TAR 1 - 206.6
2. TAR 7 - 199.1
3. TAR 5 - 191.8
4. TAR 3 - 180.9
5. TAR 2 - 178.4
6. TAR 6 - 170.6
7. TAR 4 - 169.7 Two more notes for me:
- TAR 4 was also my least favorite season.
- The top three rated seasons here happened to be the ones where African Americans finished first twice and second once. Just an interesting side note that really doens't mean a whole lot.  2002 IceCat Originals, Inc. All rights reserved.
|
|
|
Top |
| |
 |
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-18-05, 10:14 AM (EST)
|
|
8. "RE: Statistical Rankings: TAR 1 thru 7" |
<I am surprised at FM - I liked them, but didn't remember them being that solid and that competitive with RB.>I was surprised, too. RB and FM were the top two teams in each of the last 5 legs. FM finished first or second in the final 8 legs. <I also felt CC would be high up.> Do you remember when they were yielded and finished in last place? Fortunately for them, it was a non-elim leg. If they had finished, 2nd (as opposed to 4th) on that leg, their ranking would have #1 overall all-time. <Surprises for me were how low Chris and Alex were rated> They only won 3 legs (one of which was the final), and had several low finishes along the way. They had two 6th place finishes, and two 7th place finishes. After reviewing the data, one interesting thought jumped out at me. In the earlier seasons, there was a fast forward in every leg, which allowed mediocre teams to finish 1st more often than in later seasons. That's probably why FM rated #3 overall, but finished 1st in only 3 legs.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-18-05, 09:53 PM (EST)
 |
|
13. "Identifying that 'nagging check'" |
LAST EDITED ON 05-18-05 AT 10:52 PM (EST)ed. typo I've got to think a little more about methodology - sounds logical, but there's a nagging check for me that I have to identify. ...yeah, same here. I thought about it, and I'll try to phrase what follows in simple terms so that readers/posters here without a background in statistics can follow it. For us to say that the statistical results prove something, we also have to look at the sampling distribution. In order to say, "these statistics prove this or that", we also have to say that the population of racers from one TAR season to the next, and within each TAR season, come from an evenly distributed random sample. Case in point, if one TAR season consists entirely of stars, and another TAR season consists entirely of klutzes, we really can't compare the statistical performance of the stars relative to the performance of the klutzes. For all we know, the worst star finishing 11th in one TAR season is better than the best klutz finishing 1st in another TAR season. Comparing one TAR season's racers with another TAR season's racers would be like "comparing apples and oranges", except here it's "comparing stars and klutzes". Ideally, TAR seasons would have an evenly distributed mix of stars and klutzes and ranges in between. Another case in point, if one TAR season consists very evenly matched teams and another TAR season consists of a wide range of stars and klutzes, this will also affect the results. In a perfectly evenly matched race, we would likely see a lot of lead and position changes because the teams are so equal. In a race with widely distributed quality racers, we would likely see the stars on top and the klutzes on the bottom a lot more often. Needless to say, CBS does not draw TAR contestants from a random sample of the population. They are recruited and hand-picked with network ratings and entertainment value in mind. So, for us to say (more mildly) "these statistics show (not prove) this or that", we would have to also state the assumption that the population of racers (in terms of quality) from one TAR season to the next and within each TAR season is fairly evenly and equally distributed. This gives us a basis to work from, without introducing subjective opinions about the quality of the racers within a TAR season and from one TAR season to the next. As an aside, a TAR All-Stars episode would be interesting simply because it might help us to some extent identify which TAR season was more likely to consist of stars or klutzes. We could make some statements based on actual relative performance (rather than subjective measures) of racers from different TAR seasons. The post by Rasta was interesting simply because the result was not quite what we might expect. If you had told me that the top 5 out of 6 using his methodology were race winners, I would have said, "yup, that's what we'd expect to see". Instead, we find that the top 5 of 6 did not win the race, and that's interesting. It makes us ask why and want to investigate further. That's the fun of this exercise, and that's why some of us TAR fanatics like to play with the numbers. 
"It's all in the numbers"
|
|
|
Top |
| |
|
|
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-19-05, 09:58 AM (EST)
|
|
15. "RE: Identifying that 'nagging check'" |
<Comparing one TAR season's racers with another TAR season's racers would be like "comparing apples and oranges", except here it's "comparing stars and klutzes".>You make a good point. Paraphrased, you're saying that the skill level of the teams might not be a normal distribution, which might lead to skewed results when comparing seasons. You might be right; you might be wrong. There is no way to tell. And more importantly, there's no way to statistically include in the analysis. There are other factors that influence the results that haven't been factored in. Most obvious would be the Fast Forwards. In the first 5 seasons, the FF was available on every leg. This turned every leg into two distinct races: (1) the race for FF, and (2) the regularly scheduled roadblocks and detours. You could argue that finishing a leg in 2nd place to a team who went for the FF is superior to finishing 2nd in a leg WITHOUT a FF. Maybe we should change the methodology to reflect this. Another new twist that complicates the analysis is the YIELD, which didn't exist until TAR 5 (I think). Anyway, please consider my methodology a work in progress. And while it may indicate which teams are strong or weak, it is not perfect. It isn't intended to prove anything. Thanks for your post. I'm glad there are a few people out there interested in statistics, and I hope we can use this thread to share ideas and improve the analysis.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-19-05, 12:03 PM (EST)
|
|
16. "RE: Identifying that 'nagging check'" |
LAST EDITED ON 05-19-05 AT 12:44 PM (EST)ed. add comments All good points you raised there. Some comments follow. There is no way to tell {about normal distribution}. And more importantly, there's no way to statistically include in the analysis. Yup. That's why the suggestion right from the beginning to state the assumption that the distribution of racers within a TAR season and between TAR seasons is normally distributed. This stated assumption gives us a solid basis from which we can go ahead with further analysis. I'd also like to posit that this is the only assumption we need to make -- the fewer assumptions stated (and this one is already a necessary biggie), the better. Based on what I've viewed in TAR, I have a notion that this assumption is fairly true to a large extent anyway, but that's a subjective opinion. There are other factors that influence the results that haven't been factored in. {namely, the FF and Yield} You raised good points about the use of FFs and Yields and how they might affect the finish results. In addition to FFs and Yields, we might also want to look at what impact non-elimination rounds (NELs) have, too. What I might later propose is to treat the simple non-NEL finish stats as the "base case" (or more accurately, the null-hypothesis case). From there, we might be able to test the statistical significance of FFs, Yields, NELs and any other factors we'd like to consider. Other factors I'd also like to look into are some demographic factors (for example, age, gender, relationship of racers/teams). Another thing we can have a look at is sittem's aggregate seasonal stats based on your methodology to test if there really is any variation between the TAR seasons given the normal distribution assumption already made. Anyway, please consider my methodology a work in progress. Well, all of this is.  I hope we can use this thread to share ideas and improve the analysis. Likewise!
This one's old, but worthy... "statistics don't lie, but statisticians do" (source unknown)
|
|
|
Top |
| |
|
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-18-05, 02:14 AM (EST)
|
|
4. "RE: Statistical Rankings: TAR 1 thru 7" |
As a game theory & stats nut, I think the methodology to used arrive at the rankings is quite good.Very interesting results... you were probably on the edge of your seat working on the calculations to see how it'd turn out. These stats do bear out that R&A were the best overall racers in terms of finishing performance, love 'em or hate 'em or just watch 'em. The results also bear out something else -- even the best (top 3 to begin with) in ranking don't necessarily win. Winners turned out to rank 4th, 7th, 8th, 10th, 12th, 13th and 15th, leaving 8 other top-15 teams, beginning with the top 3, and next the two out of 6, who did not win. Only one in the top 6 actually won. It was kind of surprising to see the skewed results. Thanks for your work on this. It gives some substance to some of the various comments/arguments about who the best finish performance TAR racers might be, and that not always do the best win.
Some racers were standard deviants.
|
|
|
Top |
| |
|
|
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-18-05, 10:17 AM (EST)
|
|
9. "RE: More grain for the mill" |
If I have time, I'll do a complete run-down of all the teams in every season soon. I'll post the results soon.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-18-05, 01:08 PM (EST)
|
|
10. "RE: More grain for the mill" |
That would be great!I started some work on it and I had trouble getting the final positions of all teams in the first few episodes of TAR1 based on the CBS recaps. From TAR1 episode 5 onwards, it's clear. I thought maybe the DAW reviews here in the archives here might fill in the gaps. Some of the later TAR's have more complete CBS recaps, but there may be gaps in them, too. If you have complete raw data, I'd like to request a copy. Better yet, if you don't, I'll help you put it together or help you verify it -- if we do this, we'd probably have to do this offline through e-mail, and double check it before publicly posting the raw data. What do you think? Proposed format:
- on one axis, TAR# & Episode# (7 x 13);
- on the other axis the teams for each TAR# (11 x 7);
- within each cell (7 x 13 x 11) the place-rank of each team for that episode (about half will be empty due to eliminated teams). For raw data, it'd be good to include and mark the NEL's, but possibly exclude them later to make analysis simpler (just treating an NEL as an extended episode rather than a final result). Let's see if we can get accurate raw data put together first so we have something to play with.
I ain' one, but a gamblin' man's business is to know the odds.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-18-05, 01:48 PM (EST)
|
|
11. "RE: More grain for the mill" |
LAST EDITED ON 05-18-05 AT 01:51 PM (EST)I'd love to work on it together. I guess we can use this thread to compare notes. As for TAR 1, I had a tough time finding the leaderboard, but eventually found it here: http://members.cox.net/amazrace/leader1.html One other note, in case your results don't match mine, I changed the formula for allocating points for the final episodes. In a three team race, 1st place would get 100 points, 2nd place would normally get 50, and 3rd place would get 0. I felt this penalized the teams too harshly, so allocated points as if there were still 4 teams racing. Therefore, the points were 100, 67 and 33 respectively once the race is down to three teams. The only other suggestion I have is that you have to dig a little deeper than simply looking at the leaderboard to find the true results. For example, in TAR 7, the official CBS leaderboard doesn't show the non-elimination leg that Ray & Deena won (and that G/M lost), apparently since it was the first half of a two-hour episode. Still, it was a pit stop, and should be included.
|
|
|
Top |
| |
|
|
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-19-05, 01:40 PM (EST)
|
|
17. "Methodology: whether to apply points or zero to last place" |
In a three team race, 1st place would get 100 points, 2nd place would normally get 50, and 3rd place would get 0. I felt this penalized the teams too harshly, so allocated points as if there were still 4 teams racing.Therefore, the points were 100, 67 and 33 respectively once the race is down to three teams. I agree with your methodology, and I would have done the same thing as you have. I'm still undecided which is better, whether to allocate points for last place or not, and I see your reasoning. My suggestion is to apply the same methodology to all of the legs (not just the final one) for the sake of statistical consistency. Maybe you can try both methods (allocating points vs. allocating 0 points to last place in each leg) and compare the two and see which seems to give the more accurate result. If you decide to allocate points to last place, the zero result could be said to apply to already eliminated teams, which seems fair enough. We might want to look into this again.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-19-05, 03:20 PM (EST)
|
|
18. "RE: Methodology: whether to apply points or zero to last place" |
I'm slowly working my way through the seasons using this methodology. I'm up to TAR4. This time I'm including every team in the analysis.Once I have all the raw data, I'll be able to run different scenarios fairly easily. Here's a question for you: What do you think about scoring the teams based on when they arrive not only at the pit stop, but also at the clue markers, detours, and roadblocks? Another poster (Ratboy maybe?) has compiled all the data, but I'm not sure if it's meaningful. Any thoughts?
|
|
|
Top |
| |
|
Ratboy 79 desperate attention whore postings
DAW Level: "Hollywood Squares Square"
|
05-20-05, 01:15 AM (EST)
|
|
19. "RE: Methodology: whether to apply points or zero to last place" |
What's up. I wish you would clarify your methodology, because I've been running some numbers and can't get the same ones you do, so I would like to know what exactly you are doing. And it seems as if you are changing your allocation of numbers from leg-to-leg, and if you are doing that, that will skew your results very badly, perhaps to the extent that they will be meaningless. Here's what I believe you are doing, or along the lines of. (I could very easily have this wrong.)1st--100 points
2nd--90 points
3rd--80 points
4th--70 points
5th--60 points
6th--50 points
7th--40 points
8th--30 points
9th--20 points
10th--10 points
11th or below--00 points This will take care of the first few legs. But once there are less than 10 teams, you still must use these same numbers, however many there are. So if there are only eight teams left, the last place team should still get 30 points, because they are currently finishing ahead of the teams that have been eliminated. But above you mentioned using, when there are only three teams left, of 100, 66, and 33 respectively. And if that is true you are reducing a third place team, albeit a last place team, to the rank of an eighth place team, when they are clearly not that bad. But like I said, if you are using the 100, 66, 33, 00 points for each leg, regardless of how many teams are left, then you are all right. It is just essential that you do not change the rankings to fit the number of teams for each leg, because then you end up punishing teams for making it farther in the race, instead of rewarding them. >Here's a question for you:
>
>What do you think about scoring
>the teams based on when
>they arrive not only at
>the pit stop, but also
>at the clue markers, detours,
>and roadblocks?
>
>Another poster (Ratboy maybe?) has compiled
>all the data, but I'm
>not sure if it's meaningful.
> Any thoughts? Well, since I'm not entirely clear on your numbers, I'm not sure if it's meaningful. I kind of doubt that it would be, because my measurement is very simple, just the average arrival of each team at each new clue. Your numbers would just seem to apply a score to each team, instead of just an average. But if you want that info, I could get it to you. I don't exactly know how, but I'll get it to you. What you should be more worried about is the fast forward, and how it makes racers from seasons 5-7 appear to be better racers than the seasons 1-4 racers. With mine, they are skewed slightly because a superior team is pushed down at each new clue to make room for an inferior one. In season one, Frank and Margarita lose about 11 placements because of the fast forward, and don't gain any because they were already in first when they used theirs. On the other hand, Rob and Amber from the latest season, would only lose one placement from the use of the fast forward. When Ray and Deana used theirs, they were already ahead of Rob and Amber, and so they didn't get bumped any places. (And this can get complicated, because Rob and Amber, if not wasting their time going for the fast forward, may have placed ahead of some other teams in front of them, and maybe even first. But we can't assume placements. And since we saw R/A behind R/D at the F.F., we have to accept it that they would have stayed behind them.) With U/J, they got their fast forward late, and only bumped R/A from second to third at the pit stop. But I'm only losing one point per each placement. With your scoring, you are looking at the possibility of a team losing 33 points, and that would really skew your numbers between the seasons. And one more thing about the yield. It's barely a factor. Only four yields have been used, (thinking about that now, I may be wrong on that number) and three of them were meaningless from a placement perspective. Colin/Christie were already in last place when yielded, as were Adam/Rebecca, and Ron/Kelly stayed in second. (Once again, we can't assume that they would have passed anybody without having to endure the penalty of the yield, as we didn't see it.) The other one, Freddy/Kendra lost one placement, and then quickly regained it. So the yield is almost a non-factor. But hey, if you want that information, I can get it to you. It's always fun to screw around with numbers.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-20-05, 09:59 AM (EST)
|
|
20. "RE: Methodology: whether to apply points or zero to last place" |
It's interesting that you asked about my formula, because after thinking about it for a while, I've made a slight change to the formula.My old formula gave 100 points for 1st place, 0 points for last place, and pro rata points for everyone in between. For example, if 11 teams are racing, the points would be: 1 - 100
2 - 90
3 - 80
4 - 70
5 - 60
6 - 50
7 - 40
8 - 30
9 - 20
10 - 10
11 - 0 If there were only 5 teams racing, the points would be: 1 - 100
2 - 75
3 - 50
4 - 25
5 - 0 NOTE: Once the race is down to 3 teams, I decided to still assign points as if 4 teams were racing. After thinking about it, I decided that it was unfair that the last place team get 0 points, which is really the same as the teams already eliminated. Now, in a 5 team race, the points are: 1 - 100
2 - 80
3 - 60
4 - 40
5 - 20 Also, with this new formula, I don't have to tweak things in the final leg (as noted above). <So if there are only eight teams left, the last place team should still get 30 points, because they are currently finishing ahead of the teams that have been eliminated.> I understand what you're saying, and the reasoning behind it. You're suggesting that a 3rd place finisher should receive a certain score (80 pts, for example) regardless of how many teams were racing. I'm looking at it from a different perspective. To me, the value of finishing 3rd varies depending on how many teams are racing. For instance, finishing 3rd out of 11 teams is quite an accomplishment. However, finishing 3rd out of only 4 teams is below average. <because then you end up punishing teams for making it farther in the race, instead of rewarding them.> I disagree. Each team will earn points in every leg they compete in; teams already eliminated will earn 0 points. The teams' final score is their cumulative points divided by the total number of legs in the race (not the number of legs each team competed in). In other words, while a poor finish in a later leg might lower that team's AVERAGE, they are still distancing themselves from every team that's already been eliminated. On other issues, a agree with you about the YIELD. It's had no significant impact on the race. I need to research The Fast Forward further, but I'm leaning towards leaving the raw numbers alone, and not adjusting for FFs.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-20-05, 10:35 AM (EST)
|
|
23. "RE: Methodology: whether to apply points or zero to last place" |
About the Fast Forwards...Currently, I'm against adjusting the leg results to account for the change in the FF. I've waffled on this issue, and I definately understand why some people might want to make an adjustment. Here are my reasons for NOT adjusting for FF:
1. While the FF enabled some weaker teams to temporarily just to the lead, EVERY team had the option to use the FF. In fact, every team that actually finished the race used the FF at some point. 2. Often, two (or sometimes even three) teams would go for the FF, wasting valuable time in the process. These teams usually found themselves struggling to stay in the race. Therefore, while the FF teams almost always finished 1st, the remaining teams benefited from the wasted time spent by teams failing to get the FF. 3. If an adjustment were to be made for FF, it would have to take into account these offsetting circumstances. 4. If an adjustment were to be made, how could we objectively quantify it? Without clear evidence that an adjustment is necessary, AND that the adjustment is quantifiable, I think it's preferable to leave the raw data alone. Any thoughts?
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-20-05, 10:01 AM (EST)
|
|
21. "RE: Methodology: whether to apply points or zero to last place" |
Hi, Ratboy, glad to see you joined in. I took a look at your posts and stats on the other stats thread, and found it all quite interesting.I think Rasta's methodology is fairly clearly stated in his original post. For example, a leg with 4 racers would have points allocated as 100/66/33/0, a leg with 5 racers would have points allocated as 100/75/50/25/0 and 6 racers would have points allocated as 100/80/60/40/20/0, etc. Basically for each leg, he's assigning 0-100 points from worst to best, pro-rated in between. You make a valid point about the different number of total points allocated each leg (200, 250, 300 in the example above), but also note that in each leg the average points given to a team still in the race is always 50 (200/4, 250/5, 300/6), but it ignores racers already out of the race (default 0 points). He's debating whether to make an exception for the last leg. Your suggestion of 100/90/80/70/60/50/40/30/20/10/0 (that's 11 teams) throughout makes sense, but in the latter legs, there's not much differentiation between teams (final leg would be 100/90/80 for teams in the race and 70/.../0 for the eliminated teams). This is simpler, and as you say, basically reflects the overall average, a straight linear function. As an aside, a normal random-sample distribution (bell curve) of quality of racers might look more approximately like 100/80/65/60/55/50/45/40/35/20/0. I can give exact numbers later if I try this one out, given the normal distribution assumption mentioned earlier in this thread. I really don't know if it's valid or not until looking at the data and first doing some analysis to statistically test the normal distribution assumption. I do like the idea of sub-dividing each episode in to sort-of mini-episodes (is it usually 4 per leg?), with the clue box as the boundary (the mat at the end of a leg is essentially the first clue box for the next episode), since teams later departure times are separated by the same difference in arrival time to the mat. It gives a "finer grain" to the analysis and plot charts, along with 4 (?) times more result data to work with. I saw this on the other thread and liked it. Finally, if you have raw stats you'd like to share, kindly e-mail them to me. I'm just about done with simple leaderboard spreadsheet in Excel format, and I'd be glad to put your data into Excel format if you haven't already. It would be easy, then, to calculate and put the results of each methodology into adjacent columns and compare them. Also, I might later convert them a second time into SPSS (a stats package) format.
|
|
|
Top |
| |
|
Ratboy 79 desperate attention whore postings
DAW Level: "Hollywood Squares Square"
|
05-20-05, 11:31 AM (EST)
|
|
24. "RE: Methodology: whether to apply points or zero to last place" |
Gotta try to make this quick.>but also note that in
>each leg the average points
>given to a team still
>in the race is always
>50 (200/4, 250/5, 300/6), This is not true, and because of this, your numbers will not be consistent from episode to episode. Let me demonstrate... Here is the order of finish for the seventh leg of this past race, with only six teams left. Rob/Amber....100
Ron/Kelly....80
Uchenna/Joyce...60
Lynn/Alex.....40
Mere/Gretchen....20
Brian/Greg.....0 That indeed averages out to 50. However, Rasta said that (s)he divides each team's score not by how many legs they competed in, but by how many legs are in the race total, and because of that, this is what the above table should look like at the end of that seventh leg, under those conditions. Rob/Amber....100
Ron/Kelly....80
Uchenna/Joyce....60
Lynn/Alex....40
Mere./Gretchen....20
Brian/Greg.......0
Ray/Deana......0
Susan/Patrick.....0
Debbie/Bianca....0
Megan/Heidi.....0
Ryan/Chuck.....0 We have to give them zero points, because Rasta divides everyone's score by total number of legs, regardless of how many legs they competed in. So those teams are getting zero points for this leg. That averages out to 27.27. The next leg would have an average score of 22.73. And the final leg would have an average of 13.64. What I'm saying is that if you are giving eliminated teams a score of zero for each leg they are not in the race, you are bringing down each leg's average score, and thus, the expected number the remaining teams should hover around. To fix this, you could either have a fixed standard pro rated number set used for each leg, like the one I posted in my first post, or disregard handing out points to eliminated teams, or just divide by the number of legs each team ran instead of just total number of legs. Because as it stands now with the differing averages, these numbers can't even be compared from episode to episode, much less season to season.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-20-05, 12:28 PM (EST)
|
|
25. "RE: Methodology: whether to apply points or zero to last place" |
I see your point, and need to think it over. Fortunately, my spreadsheet calculates the results two ways: (1) points divided by 13 legs, and (2) points divided by legs run. I kind of intended the second calculation as a tie-breaker. Anyway, for TAR7 here are the rankings both ways... (1) Points / 13 legs:
1 Rob & Amber 80.2
2 Uchenna & Joyce 67.7
3 Ron & Kelly 66.6
4 Lynn & Alex 38.3
5 Meredith & Gretchen 31.0
6 Brian & Greg 28.8
7 Ray & Deana 24.7
8 Debbie & Bianca 12.4
9 Susan & Patrick 12.0
10 Megan & Heidi 2.9
11 Ryan & Chuck 0.7 (2) Points / legs run
1 Rob & Amber 80.2
2 Uchenna & Joyce 67.7
3 Ron & Kelly 66.6
4 Lynn & Alex 55.3
5 Debbie & Bianca 53.7
6 Ray & Deana 53.6
7 Brian & Greg 53.5
8 Susan & Patrick 38.9
9 Meredith & Gretchen 36.7
10 Megan & Heidi 18.6
11 Ryan & Chuck 9.1 The only significant changes are M/G falling from 5th to 9th, and D/B rising from 8th to 5th. Subjectively, this makes sense. D/B look formidable when they won leg #1, but bombed out in leg #3. Conversely, while M/G never finished highly, they moved up in the rankings due to their longevity. In any subjective ranking, I don't think D/B's 9th place exit could rank higher than M/G's 4th place exit (regardless of how annoying Gretchen was <G>). One other note, if I were to assign a constant value for each place (ie 3rd place always gets 80 pts), the results would very similar to simply averaging each team's finishing positions. What's the difference, then, of assigning 100-90-80-etc on every leg versus the much simpler 1-2-3-etc shown on cbs.com?
|
|
|
Top |
| |
|
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-20-05, 12:30 PM (EST)
|
|
26. "RE: Statistical Rankings: TAR 1 thru 7" |
I posted this in the other thread...
<There were 12 legs in both editions, 13 in all the others.>
TAR7 has 13 legs only if you count the unofficial pit stop in India, which I think is correct. TAR6 had 12 legs, but could have 13 if you consider the episode where Lori/Bolo missed the train as another "unofficial" pit stop. In part one of the "to be continued" episode, teams left the pit stop in Berlin, completed a Roadblock, flew to Budapest, completed a Detour, then made their way to an internet cafe (that opened at 10pm). In part two, teams started at the internet cafe. They then went to a nearby train museum, that unfortunately did not open until 10am the following morning. There, they received they next clue, which included a Fast Forward. Leaving the train museum, teams had to complete a Detour, then a Roadblock, before heading for the pit stop. Add it up, and you'll find 2 Detours, 2 Roadblocks, a Fast Forward, plus an artificial 12 hour delay. Sounds very much like TAR7's unofficial pit stop, right? Based on this, I think this need to be added to my analysis as two seperate legs.
|
|
|
Top |
| |
Ratboy 79 desperate attention whore postings
DAW Level: "Hollywood Squares Square"
|
05-21-05, 02:03 AM (EST)
|
|
27. "How does this sound?" |
I've been going through this all day, and here's where I've gotten in my thinking. I said there were three different things that we could do to "fix" the numbers from before, as the mean kept changing from leg to leg, and therefore, changing the standard deviation from each leg, making stats worthless. We could, 1) have a fixed, pro-rated score for each leg and for each team, regardless if they are still in the race or not. This would have a standard mean, but as Rasta asked, what would be the difference betweeen that, or just the average statistical rating that is at the CBS site. The answer is nothing. They would be almost the same, except we would be applying a positive number (that is, whoever has the highest number is the best) instead of a negative ranking (the lowest number is best). So this wouldn't work.So then I tried a fixed, non-standard score for each leg. For the first leg, points would be handed out as 100, 90, 80, 70, etc. all the way down to zero for the 11th place team. All of those numbers added up to 550. For the second leg, I had to make all of those numbers add up to 550 while having a standard deviation between each number, and I had 109, 97, 85, 73, etc., all the way down to 1 point for the last place team. That did add up to 550. For the next leg, it was 121, 106, 91, 76, etc, down to the number one again. But then it hit me: this flies in the face of logic. Why should a team that finishes first out of eleven opponents, get fewer points than a team that finishes first out of nine oppenents? Isn't finishing first with more competitors more of an accomplishment than finishing first out of fewer? I determined that it was, and since I was not reflecting that in my numbers, I threw them all out. We could then do 3) not hand out points to eliminated teams, and just divide by number of legs each team ran. To do this, we would have to keep raising the number of points each team gets the farther we get in the race. For instance, with five teams left, it would be 100, 200, 300, 400, and 500 for finishing fifth, fourth, third, etc. With four teams, 150, 250, 350, 450. This should work, with average teams being around 300 points for the race, and also giving away fewer points for each first place down the line while racing against fewer team. But doing this inspired me to take it one step further, with the inspiration that a first place finish, a last place finish, and a middle of the road finish, should always count the same. The problem that I always had with my data about average arrival at each clue, was that it didn't tell me anything. By looking at it, I could tell who were the good racers and who were the bad ones, but I had no idea what defined an average racer. So I took the idea of having a set average from the above paragraph, and assigned it a value of zero. From that, I set up a leg-by-leg continuum based on placings. Here it is for the first leg of this past race, and it is much easier to understand when on a horizontal line than presented here. 11th--(-50)--Ryan/Chuck
10th--(-40)--Ron/Kelly
9th---(-30)--Megan/Heidi
8th---(-20)--U/Joyce
7th---(-10)--Ray/Deana
6th---(00)--M/Gretchen
5th---(+10)--Lynn/Alex
4th---(+20)--Brian/Greg
3rd---(+30)--Rob/Amber
2nd---(+40)--Susan/Patrick
1st---(+50)--Debbie/Bianca From this snapshot of the first leg, we can clearly see the teams that finished above average, what team was average, and what teams were below average. For the next leg, there were ten teams, and so no team could be exactly in the middle, thus no team has an assignment of zero. Here it is for the second leg. 10th-(-50)--Megan/Heidi
9th--(-38.8)--Brian/Greg
8th--(-27.7)--Susan/Patrick
7th--(-16.6)--M/Gretchen
6th--(-5.5)--Debbie/Bianca
5th--(+5.5)--Lynn/Alex
4th--(+16.6)--U/Joyce
3rd--(+27.7)--Ray/Deana
2nd--(+38.8)--Ron/Kelly
1st--(+50)--Rob/Amber The top, average, and bottom stay the same, but the spaces between the points move because with one less team, we need a set standard between the teams. Going through the entire race, these are the final numbers I come up with, with a score of zero being average, any positive score meaning you are an above average racer, and any negative score meaning you are a below average racer. I simply take all of their points, add them together (whether it is a positive or negative number) and then divide by the total legs run for that team. The numbers... 11th--Ryan/Chuck--(-50)
10th--Megan/Heidi--(-40)
9th--M/Gretchen--(-25.67)
8th--Susan/Patrick--(-18.8)
7th--Brian/Greg--(-3.53)
6th--Lynn/Alex--(-2.97)
5th--Ray/Deana--(-2.72)
4th--Debbie/Bianca--(-1.83)
3rd--U/Joyce--(+10.15)
2nd--Ron/Kelly--(+11.42)
1st--Rob/Amber--(+21.54) Two things jump out at me that warns me something might be wrong with this data. First, I only have three teams that can be called above average, and a whopping eight teams below average. But upon further thought, I bet I have two very bad teams, two bad teams, four average teams, two good teams, and one great team. Someone mentioned on this thread about running a sqaure test, and I would assume that if someone ran a t-sqaure, z-square, or chi-square test on my data (I can't remember which one it is, or even really how to do it anymore) it would come up with a deviation of about five points allowable either way of zero for an average score. So those four teams slightly under zero should still be considered average. Second, the ranking of M/Gretchen worried me. But once again, after some thought, it doesn't worry me anymore. M/Gretchen consistently ran to the back of the pack, and these numbers back that up. Just because a team makes it far in the race does not mean they are good racers. I haven't done this with another season yet, so this method is far from tested. But I think we have such an odd distribution because of the flame-out of some early teams, and the outright domination of two teams on the race. The other races I don't think will have such a distribution. One more thing. I counted the clue that Phil gave out on the mat on the roof as the end of a leg in my figures. It adds one more data point, which is always preferable to less data.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-21-05, 03:14 AM (EST)
|
|
28. "RE: How does this sound?" |
Will have to make this quick, as I'm about to leave for an extended weekend holiday at the beach. Offline!  Will print a copy of your post, read it again and think through it a bit more over the weekend. Will also bring along my stats text to check on methods -- haven't looked at it in over a year.I'd been thinking along almost precisely the same lines you have... ESP? First, I think the leg-by-leg thing is good -- the same thought occured to me this morning. It allows you to refine the rankings as each episode occurs. It also has predictive power, especially if you're the kind who likes to place bets or make predictions as the season goes along. The 2nd chart is indeed a normal distribution. The 3rd chart comes very close to a normal distribution, with a skew to the left (negative). It seems that this skew is unavoidable, as the last place team will always have -50 (and for all we know, they're not that horrible a team, maybe just unlucky -- there's too little data with just one result to support saying how good they are, except they finished last of 12 in one run).
A quick question... if a team always finishes first, including the final leg as winners, is the result +50? I think it is, but maybe you can confirm. If so, this method seems sound, although it some tweaking might make it better. Will get back to you guys next week! Have a good one.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-23-05, 11:21 AM (EST)
|
|
29. "RE: How does this sound?" |
LAST EDITED ON 05-23-05 AT 11:32 AM (EST)Wow, this is cool. Great post. Your third methodology is similar to my first methodology. You're assigning +50 to the winner, -50 to the loser, 0 (or close to it) to the middle team. That's very similar to mine, except my continuum was from 0 to 100: winners always receive 100, losers always receive 0, middle teams 50 (or close to it). Just add 50 to your number assignments, and you've equaled my assignments. Basically, we're in agreement up to this point. The next step is whether to divide the point total by 13 total legs, or by the number of legs each team raced. Initially, I thought we should divide simply by the total legs, but you've convinced me otherwise. By using the actual legs raced as the denominator, the resulting rankings look very much like a normal distribution, with the mean very close to 50. Before showing you the list, I changed the methodology slightly. The first place team still receives 100, but now the last place teams receive a value higher than 0. The best way to illustrate is via an example, in a 5 team race... Original assignments:
1st=100, 2nd=75, 3rd=50, 4th=25, 5th=0 Revised assignments:
1st=100, 2nd=80, 3rd=60, 4th=40, 5th=20 I'm certainly not married to this methodology, but it does have some advantages over my original formula. The resulting mean is skewed slightly higher than 50 (or so I believed, until I saw the results). to be continued...
|
|
|
Top |
| |
|
|
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-23-05, 01:27 PM (EST)
|
|
31. "RE: How does this sound?" |
A few observations about this list...1. Adam/Rebecca and Meredith/Gretchen: Two weak teams that survived a long way, despite usually finishing towards the back of the pack. Also, include Nancy/Emily and Kami/Karli in this group. 2. Solin/Doyim, Alison/Donny, Amanda/Chris: Teams that finished first in an early leg, only to then be eliminated shortly thereafter. 3. Derek/Drew and Hayden/Aaron: Two VERY strong teams that made it to the final four. Bad luck/decisions at a crucial point lead to elimination. NOTE: I included the "unofficial pitstops" from TAR6 (Budapest) and TAR7 (India) in my database.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-24-05, 01:44 PM (EST)
|
|
32. "RE: How does this sound?" |
Ok, back from long beach weekend. And, yup, pondered these posts a lot during free time.About the skew: First, for both Ratboy and Rasta, the reason we see the heavy skew to the left is because of the elimination process. Those teams that got eliminated 12th/12 or 11th/11 are way down there. And here's why: They only got one chance (one result). I would wager if you ran ten races (not just one) with all 11 (or 12) racers, you wouldn't see them finish last all the time. Indeed, their average might be closer to 7 or 8 or 9. So, one conclusion is this: The left skew is an unavoidable artifact of the elimination process. On Rasta's most recent post, the results are very interesting, and even more so given the mean, median and SD. I think you're on to something here, as was Ratboy in his most recent post. What I'd like to do is to compare both of your methods in practice to see how good they are in predicting results, leg-by-leg, for each race. I'll throw in one base heuristic test method as a baseline: Prediction of team's result for next round = team's result from last round. Before I can do that, though, I want to be sure I'm working with the correct data. I will post the data separately, and kindly comment on its accuracy. It is this data set (with corrections or adjustments to be equal your data sets) that I'll use for later analysis.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-24-05, 05:14 PM (EST)
|
|
34. "RE: Data set" |
Those darn ties! I seem to have missed most of them! I'll check my data closely tonight and get back to you tomorrow.When ties occur, I think we should split the points between the teams. For example, when Blake/Paige and Hope/Norm tied for fourth in leg #1, I think we should allocated the points like this: (4th place pts + 5th place points)/2. Make sense? Also, I think we need to include the "unofficial" leg in TAR6 in our analysis. It occurred in Budapest, between what you've shown as legs 5 and 6. That "leg" spanned two episodes; it had 2 Detours, 2 Roadblocks, 1 Fast Forward, and an artificial 12 hour delay. Very much like what happened in India in TAR7. The order of finish in that leg was: Freddy/Kendra, Gus/Hera, Jonathan/Victoria, Hayden/Aaron, Kris/Jon, Adam/Rebecca, and Lori/Bolo.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-24-05, 06:18 PM (EST)
|
|
35. "RE: Data set" |
Glad you had a look at the data set.1. I agree that ties should be split. For example, being tied for 4th and 5th should be recorded as 4.5 for both teams, or the one instance of a 3-way tie for 1st should be recorded as 2.0 for all the three teams. I planned to do this anyway, but wanted to show the ties first. I checked the CBS site on the ties, and there were one or two that were still unclear, so please check your data, too. 2. I also agree to include the extra leg in TAR6, for the reasons you stated. Thanks for the order of finish for that! I'll add that in between legs 5 and 6. After this clean-up of the data, and waiting for your confirmation on the ties, I'll repost the data. And thinking ahead a bit... We might try a leg-by-leg test of your method and Ratboy's method, along with the heuristic last=next method I mentioned earlier, and compare which of the three is the best predictor. I'd be willing to do this, as I'm very curious about what the result will be. I'm still thinking about the best way to evaluate this, but will probably do a summation of averaged predicted-place errors. That's to say if prediction=1st, actual=3rd, then error=2, and vice versa with the error always positive. There is no prediction for leg 1, since there is no prior history. For each episode (from episode 2 onwards), determine the prediction error for each team, sum them up for all teams, and then divide by the number of teams to arrive at the average error per team for an episode. Then finally sum up average error per team per episode for all the episodes, and then divide by the number of episodes (which will all soon be 13, anyway). This will give us the average error per team per episode over a season. Then we do this for all seasons, and evaluate the results to see how to come up with a better prediction method. Sometime later, we could even venture a prediction of episode 1 results based on demographic factors (team relationship, age, gender), based on correlation analysis. For example, analysis of results might show that an older married team will finish lower on average than a younger unmarried team, right from the beginning (to the end, too!).
|
|
|
Top |
| |
|
Ratboy 79 desperate attention whore postings
DAW Level: "Hollywood Squares Square"
|
05-26-05, 00:15 AM (EST)
|
|
36. "RE: Data set" |
I'm back. I haven't gotten all of my data done yet, but I wanted to comment on these recent posts.1. Interesting numbers Rasta. What stands out to me is that with a mean of 49, you have 48 teams above it and 31 below. If plotted on a straight line, you wouldn't have a bell curve, but it would be slightly to the right of center, signalling that it isn't an eqaul distribution. But what I have found with the numbers that I have done is that they are slightly skewed to the left of center, so I have almost the opposite graph that you do. I think that is only a result of me using zero as a base, and you using a positive number. 2. If you use the median as the center of your graph, then we have something. Then there are 40 teams above it, one right at the center, and 38 below it. That's a good distribution. I don't know where the median for mine is yet. 3. About the prediction thing. That is something that should be done. Stats are all about predicting outcomes. But if we are going to do that, we need categories to classify each team. Just some examples to throw out there: married couple, dating couple, same-sex couple/female, same-sex couple/male, parent/child, older couple, etc. This isn't all of them, of course, just possibilities. And we'll have to decide where to put each team, as they could fit in different categories. For instance, Bob/Joyce could be married couple or older couple. I would put them in the older category. And also, some teams could be hard to slot. Should Avi/Joe be in the same category as Derek/Drew or Chris/Alex? I don't know, because they seem line totally different teams to be in the same category.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-26-05, 11:14 AM (EST)
|
|
38. "RE: Data set" |
LAST EDITED ON 05-26-05 AT 12:59 PM (EST)I ran the numbers for TAR7 using my original formula, then compared the results to your formula. I believe they statistically equivalent, as shown below. From your earlier post: 11th--Ryan/Chuck--(-50)
10th--Megan/Heidi--(-40)
9th--M/Gretchen--(-25.67)
8th--Susan/Patrick--(-18.8)
7th--Brian/Greg--(-3.53)
6th--Lynn/Alex--(-2.97)
5th--Ray/Deana--(-2.72)
4th--Debbie/Bianca--(-1.83)
3rd--U/Joyce--(+10.15)
2nd--Ron/Kelly--(+11.42)
1st--Rob/Amber--(+21.54) My comparison:
Team Rasta Rasta-50
Uchenna & Joyce 60.17 10.17
Rob & Amber 71.54 21.54
Ron & Kelly 61.43 11.43
Meredith & Gretchen 24.33 -25.67
Lynn & Alex 47.05 -2.95
Brian & Greg 46.45 -3.55
Ray & Deana 47.31 -2.69
Susan & Patrick 31.18 -18.82
Debbie & Bianca 48.15 -1.85
Megan & Heidi 10.00 -40.00
Ryan & Chuck 0.00 -50.00 We've both assigned the points over a 100-point spectrum. Yours: -50 to +50; mine: 0 to 100. The deviation between teams within each leg is identical. Therefore, I'd expect the results to be the same, except that my scale is 50 points higher than yours. The comparison shows this, but for an extremely slight rounding difference (about 2 / 1000ths per team). Based on this, I think we are in total agreement up to this point. However, I since then have changed my formula slightly. My reasoning went along these lines:
1. I considered the point allocation in the final legs too harsh on the 2nd and 3rd place teams. 2. I decided to allocate points in the final legs as if 4 teams were racing instead of only 3. Therefore, instead of 100-50-0, I assigned 100-67-33. 3. This bothered me, since I didn't think it was valid to have two different formulas in place for different sections of the race. (Cahaya pointed out this inconsistency earlier in this thread) 4. Therefore, I decided to change the formula across the board. For example, in a 5 team race, the original points would have been 100-75-50-25-0. Now, they are 100-80-60-40-20. The 1st place team still receives 100 points, and every other team receives points in proportion to their relative finish. However, the last place team no longer gets 0. As a result, the point allocation for 2nd place through last place has been increased. In early legs, this upward adjustment is very slight; in later legs more pronounced, and is most pronounced in the final leg. What do you think?
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-26-05, 01:29 PM (EST)
|
|
39. "RE: Data set" |
LAST EDITED ON 05-26-05 AT 02:11 PM (EST)ed. add comment Good observation there about your ending up with essentially the same results as Ratboy, with just a scale adjustment. I agree with your 'across the board' methdology, simply for consistency's sake; there's a risk of skewing or creation of artifacts (outliers) by making exceptions for single legs. Let's keep the original formula under our belt for later testing of predictions, since you both ended up with essentially the same method. It's one method that all three of us agree on in principle to try out. I'll test this perhaps this weekend, along with the last=next heuristic to compare it to for prediction purposes. We can try other formulas, too, as we go along. Taking a look at your new always-positive value idea, I'd like to suggest a slight change (or an additional change, as we can try both out). With 5 teams, I'd be tempted to give a scale of 90-70-50-30-10. This is 'pigeon holing', with a range of 10 free on each side of each team, on a scale of 0-100. Visually:
Team 1:{80-100} center=90
Team 2:{60-80} center=70
Team 3:{40-60} center=50
Team 4:{20-40} center=30
Team 5:{0-20} center=10 It's essentially the same as what you had (with an interval = 20 between teams, but -10 on the scale from your new suggestion. I think it might solve the mean/median = 50 scale problem that Ratboy just commented on. There is a difference in interval in your new idea (20, instead of 25 as originally done). Statistically, I think there's no difference in these methods, except the scale adjustment (just as yours was +/-50 with Ratboy's method) and the interval adjustment (20 or 25), neither of which (I think!) have a bearing on predicting outcomes. We'll have to check that assumption, though! Last night, I worked out the formula that will give the average prediction error (APE) for all permutations of 3-13 teams (actually, it applies to any number of teams). This was not a trivial task! I'll post the formula (preview: APE=((n*n)-1)/3 for n teams) and values soon with some explanations. From this work, I found that I have to revise my earlier statement about simply averaging the errors over each leg. Instead, either (a.) the error needs to be weighted with a function according to the number of teams in the leg, or, (b.) the human/method prediction error is expressed as a ratio/fraction of the average prediction error APE. For a leg with 5 teams, the average prediction error APE = ((5*5)-1)/3 = 8, and if your method gives a prediction with error = 4, then your prediction error ratio is 0.5, a decent value, with a ratio of 0/8 = 0.0 being a perfect prediction, while 8/8 = 1.0 being an average/random prediction. Perfect predictions with medium/large number of teams is almost impossible, with odds being 1/n!, a factorial (for example, with 6 teams, that's 1/6! = 1/720 odds). Ok, all for now. I'll clean up and repost the data set sometime tomorrow or day after for you to verify one last time.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-26-05, 02:48 PM (EST)
|
|
41. "Latest rankings" |
LAST EDITED ON 05-26-05 AT 02:49 PM (EST)I ran the numbers using both of my formulas: (1) my original, which I'll refer to as the 100-50 method, and (2) my revised, which I'll refer to as the 100-67 method. The results are remarkably consistent.
Rank - Team - 100-67 - 100-50 - Change in Rank
1 - Rob & Brennan - 78.5 - 74.5 - 0
2 - Frank & Margarita - 78.0 - 72.4 - 0
3 - Rob & Amber - 77.6 - 71.5 - -2
4 - Colin & Christie - 77.5 - 71.8 - 0
5 - Kris & Jon - 77.2 - 71.9 - 2
6 - Tara & Wil - 72.3 - 65.4 - -1
7 - Derek & Drew - 72.3 - 66.9 - 1
8 - Flo & Zach - 71.1 - 63.9 - -2
9 - Hayden & Aaron - 70.3 - 64.2 - 1
10 - Reichen & Chip - 70.2 - 63.9 - 1
11 - Ron & Kelly - 69.2 - 61.4 - -1
12 - Ken & Gerard - 69.2 - 60.3 - -2
13 - Chip & Kim - 69.0 - 63.6 - 2
14 - Uchenna & Joyce - 67.7 - 60.2 - -1
15 - Freddy & Kendra - 66.9 - 61.0 - 2
16 - Jonathan & Victoria - 65.4 - 59.6 - 0
17 - Drew & Kevin - 64.5 - 56.8 - -1
18 - Charla & Mirna - 64.3 - 58.6 - 1
19 - Bill & Joe - 63.2 - 51.4 - -9
20 - Jon & Al - 62.9 - 55.6 - -1
21 - Chris & Alex - 62.8 - 53.8 - -3
22 - Gary & Dave - 62.1 - 55.5 - 0
23 - Mary & Peach - 62.1 - 56.5 - 4
24 - Millie & Chuck - 62.1 - 55.3 - 1
25 - Brandon & Nicole - 61.9 - 52.5 - -2
26 - Shola & Doyin - 61.1 - 55.9 - 6
27 - David & Jeff - 59.1 - 50.2 - -2
28 - Pat & Brenda - 58.2 - 53.3 - 3
29 - Kelly & Jon - 57.8 - 47.8 - -4
30 - Blake & Paige - 57.7 - 48.0 - -2
31 - Steve & Josh - 57.5 - 52.7 - 5
32 - Teri & Ian - 55.5 - 47.5 - -3
33 - Lynn & Alex - 55.3 - 47.0 - -4
34 - Alison & Donny - 55.0 - 50.0 - 4
35 - Oswald & Danny - 54.1 - 44.1 - -9
36 - Debbie & Bianca - 53.7 - 48.1 - 5
37 - Ray & Deana - 53.6 - 47.3 - 1
38 - Brian & Greg - 53.5 - 46.5 - -1
39 - Lori & Bolo - 53.2 - 45.6 - -2
40 - Bob & Joyce - 53.0 - 47.0 - 2
41 - Heather & Eve - 52.8 - 47.5 - 7
42 - Monica & Sheree - 52.4 - 46.2 - 2
43 - Linda & Karen - 51.9 - 42.2 - -4
44 - Aaron & Arianne - 51.0 - 44.2 - 1
45 - Amanda & Chris - 50.4 - 45.5 - 3
46 - John Vito & Jill - 50.2 - 41.8 - -3
47 - Gus & Hera - 50.1 - 43.3 - 2
48 - Tian & Jaree - 49.1 - 41.9 - 0
49 - Michael & Kathy - 48.7 - 42.4 - 3
50 - Marshall & Lance - 47.5 - 40.1 - 0
51 - Andre & Damon - 45.1 - 37.5 - 0
52 - Adam & Rebecca - 44.5 - 32.3 - -4
53 - Steve & Dave - 44.3 - 37.4 - 1
54 - Cyndi & Russell - 43.0 - 35.1 - 1
55 - David & Margaretta - 40.9 - 33.6 - 1
56 - Hope & Norm - 39.1 - 32.5 - 1
57 - Susan & Patrick - 38.9 - 31.2 - 0
58 - Lena & Kristy - 37.6 - 30.4 - 0
59 - Paul & Aimee - 37.0 - 28.7 - -1
60 - Meredith & Gretchen - 36.7 - 24.3 - -4
61 - Jim & Marsha - 36.4 - 29.3 - 2
62 - Nancy & Emily - 35.9 - 24.8 - -1
63 - Kami & Karli - 34.1 - 22.4 - -2
64 - Russell & Cindy - 32.4 - 25.5 - 3
65 - Kim & Leslie - 32.3 - 25.0 - 3
66 - Lenny & Karyn - 30.4 - 19.6 - 0
67 - Dennis & Andrew - 26.3 - 18.8 - 0
68 - Meredith & Maria - 23.2 - 15.0 - 0
69 - Don & Mary Jean - 22.9 - 13.2 - 0
70 - Peggy & Claire - 19.8 - 10.7 - 0
71 - Megan & Heidi - 18.6 - 10.0 - 0
72 - Tramel & Talicia - 17.0 - 9.1 - 0
73 - Ryan & Chuck - 9.1 - 0.0 - 0
74 - Matt & Anna - 9.1 - 0.0 - 0
75 - Avi & Joe - 9.1 - 0.0 - 0
76 - Deidre & Hillary - 9.1 - 0.0 - 0
77 - Dennis & Erika - 9.1 - 0.0 - 0
78 - Debra & Steve - 8.3 - 0.0 - 0
79 - Gina & Sylvia - 8.3 - 0.0 - 0
Mean - 48.9 - 41.3
Standard Deviation - 19.3 - 20.2
Median - 53.0 - 46.2
58 out of the 79 teams had a change in rank of 2 positions or less. 73 out of 79 had a change of 4 positions or less. As I said, remarkably consistent. PS Interesting idea about pigeon-holing. I'll play around with it.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-26-05, 03:43 PM (EST)
|
|
42. "RE: Data set" |
That pigeon-holing idea was brilliant! I think we should use it.I calculated the point assignments based on my 100-67-33 method, and then pigeon-holed it as you suggested. The results look very promising. First, here are the point allocations for a race with 11 teams, 7 teams, and 3 teams: 11 teams: 95-86-77-68-59-50-41-32-23-14-5 7 teams: 93-79-64-50-36-21-7 3 teams: 83-50-17 Second, here are the points for 1st place and last place, beginning when there are 12 teams racing, all the way down to 3 teams racing: 1st place: 96-95-95-94-94-93-92-90-88-83 Last place: 4-5-5-6-6-7-8-10-13-17 Third, the benefits of using this method: 1. The points are still allocated relative to finishing position, and across a spectrum of initially 90+ points. 2. The points allocated always average exactly 50. 3. Winners in early legs are rewarded slightly more points than winners in later legs. I'll be curious to hear what you and Ratboy think of. I'm very excited about it.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-26-05, 04:22 PM (EST)
|
|
43. "RE: Data set" |
LAST EDITED ON 05-27-05 AT 08:00 AM (EST)Okay, last post of the day. I promise! I've re-run the numbers using the pigeon-hole method combined with my 100-67-33 method. Then, I compared them to my original 100-50-0 mthod (which is the same as Ratboy's method). The resulting rankings are incredibly, incredibly close. 38 of the 79 teams resulted in the exact same rank. 75 out of 79 shifted 2 positions or less. Basically, I think Ratboy and I were already very close, with very few variations between our rankings. Adding Cahaya's pigeon-hole idea brought the methods even closer together. It's probably as good of a marriage as we can hope for. Rank - Team - 100-50 - 100-50 Rank - 83-50 - 83-50 Rank
1 - Rob & Brennan - 74.5 - 1 - 69.0 - 1
2 - Kris & Jon - 71.9 - 3 - 68.8 - 2
3 - Colin & Christie - 71.8 - 4 - 68.6 - 3
4 - Frank & Margarita - 72.4 - 2 - 68.5 - 4
5 - Rob & Amber - 71.5 - 5 - 68.2 - 5
6 - Derek & Drew - 66.9 - 6 - 64.7 - 6
7 - Tara & Wil - 65.4 - 7 - 62.8 - 7
8 - Hayden & Aaron - 64.2 - 8 - 62.5 - 8
9 - Flo & Zach - 63.9 - 10 - 62.1 - 9
10 - Reichen & Chip - 63.9 - 9 - 61.1 - 10
11 - Ken & Gerard - 60.3 - 14 - 60.2 - 11
12 - Chip & Kim - 63.6 - 11 - 60.1 - 12
13 - Ron & Kelly - 61.4 - 12 - 59.8 - 13
14 - Jonathan & Victoria - 59.6 - 16 - 58.8 - 14
15 - Freddy & Kendra - 61.0 - 13 - 58.4 - 15
16 - Uchenna & Joyce - 60.2 - 15 - 58.3 - 16
17 - Charla & Mirna - 58.6 - 17 - 58.0 - 17
18 - Drew & Kevin - 56.8 - 18 - 56.3 - 18
19 - Shola & Doyin - 55.9 - 20 - 55.7 - 19
20 - Mary & Peach - 56.5 - 19 - 55.6 - 20
21 - Millie & Chuck - 55.3 - 23 - 55.3 - 21
22 - Jon & Al - 55.6 - 21 - 55.1 - 22
23 - Gary & Dave - 55.5 - 22 - 54.8 - 23
24 - Bill & Joe - 51.4 - 28 - 53.7 - 24
25 - Chris & Alex - 53.8 - 24 - 53.3 - 25
26 - Pat & Brenda - 53.3 - 25 - 53.2 - 26
27 - Brandon & Nicole - 52.5 - 27 - 53.0 - 27
28 - Steve & Josh - 52.7 - 26 - 52.7 - 28
29 - Alison & Donny - 50.0 - 30 - 50.2 - 29
30 - David & Jeff - 50.2 - 29 - 50.0 - 30
31 - Debbie & Bianca - 48.1 - 31 - 48.7 - 31
32 - Kelly & Jon - 47.8 - 33 - 48.6 - 32
33 - Lynn & Alex - 47.0 - 37 - 48.2 - 33
34 - Blake & Paige - 48.0 - 32 - 48.2 - 34
35 - Heather & Eve - 47.5 - 34 - 48.0 - 35
36 - Ray & Deana - 47.3 - 36 - 47.7 - 36
37 - Bob & Joyce - 47.0 - 38 - 47.6 - 37
38 - Brian & Greg - 46.5 - 39 - 47.2 - 38
39 - Monica & Sheree - 46.2 - 40 - 46.9 - 39
40 - Teri & Ian - 47.5 - 35 - 46.5 - 40
41 - Lori & Bolo - 45.6 - 41 - 46.4 - 41
42 - Amanda & Chris - 45.5 - 42 - 46.0 - 42
43 - Oswald & Danny - 44.1 - 44 - 45.9 - 43
44 - Aaron & Arianne - 44.2 - 43 - 45.5 - 44
45 - Gus & Hera - 43.3 - 45 - 44.2 - 45
46 - Linda & Karen - 42.2 - 47 - 43.7 - 46
47 - Michael & Kathy - 42.4 - 46 - 43.6 - 47
48 - Tian & Jaree - 41.9 - 48 - 43.3 - 48
49 - John Vito & Jill - 41.8 - 49 - 43.1 - 49
50 - Marshall & Lance - 40.1 - 50 - 41.6 - 50
51 - Andre & Damon - 37.5 - 51 - 39.3 - 51
52 - Steve & Dave - 37.4 - 52 - 39.2 - 52
53 - Cyndi & Russell - 35.1 - 53 - 37.3 - 53
54 - Adam & Rebecca - 32.3 - 56 - 36.1 - 54
55 - David & Margaretta - 33.6 - 54 - 35.5 - 55
56 - Hope & Norm - 32.5 - 55 - 34.3 - 56
57 - Susan & Patrick - 31.2 - 57 - 33.6 - 57
58 - Lena & Kristy - 30.4 - 58 - 32.6 - 58
59 - Jim & Marsha - 29.3 - 59 - 31.4 - 59
60 - Paul & Aimee - 28.7 - 60 - 31.3 - 60
61 - Nancy & Emily - 24.8 - 63 - 28.7 - 61
62 - Meredith & Gretchen - 24.3 - 64 - 28.6 - 62
63 - Russell & Cindy - 25.5 - 61 - 27.8 - 63
64 - Kim & Leslie - 25.0 - 62 - 27.5 - 64
65 - Kami & Karli - 22.4 - 65 - 26.7 - 65
66 - Lenny & Karyn - 19.6 - 66 - 24.0 - 66
67 - Dennis & Andrew - 18.8 - 67 - 21.7 - 67
68 - Meredith & Maria - 15.0 - 68 - 18.4 - 68
69 - Don & Mary Jean - 13.2 - 69 - 17.4 - 69
70 - Peggy & Claire - 10.7 - 70 - 14.7 - 70
71 - Megan & Heidi - 10.0 - 71 - 13.9 - 71
72 - Tramel & Talicia - 9.1 - 72 - 12.7 - 72
73 - Ryan & Chuck - 0.0 - 73 - 4.5 - 73
74 - Matt & Anna - 0.0 - 74 - 4.5 - 74
75 - Avi & Joe - 0.0 - 75 - 4.5 - 75
76 - Deidre & Hillary - 0.0 - 76 - 4.5 - 76
77 - Dennis & Erika - 0.0 - 77 - 4.5 - 77
78 - Gina & Sylvia - 0.0 - 78 - 4.2 - 78
79 - Debra & Steve - 0.0 - 79 - 4.2 - 79
Mean 42.2
Median 46.5
Std Dev 18.0
|
|
|
Top |
| |
|
|
|
|
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-27-05, 05:23 PM (EST)
|
|
47. "RE: Data set" |
Just a quick note here...I've re-posted the new data set (edited the original post) and also ran the ranking formula just as you did. I got precisely the same results you did, except for just one team -- Flo & Zach. I have a feeling one of us has got one of the placings wrong somewhere. Here are the placings and ratings for each of the rounds for Flo/Zach: Round
1 2 3 4 5 6n 7 8n 9 10n 11 12n 13 Avg.
Placing
2 5 3 2 4.5 2 1 2 5 3 2 3 1 2.73
Rating
87.5 59.1 75.0 83.3 50.0 78.6 91.7 70.0 10.0 50.0 62.5 16.7 83.3 62.9 (you had 62.1) Double check this, and then I think we're set to check how well this method predicts leg-by-leg results. Maybe we can get to that this weekend, as I mentioned in the previous post. All for now!
|
|
|
Top |
| |
|
|
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-26-05, 10:29 AM (EST)
|
|
37. "RE: Data set" |
I've finally had a chance to review the data. Sorry about the delay.A couple of notes: First, there are two ties that are shown in your data, but are not referenced in the footnotes. Tar3, Ep5: Flo/Zach tied John Vito/Jill for 4th. Tar4, Ep4: Jon/Al tied Millie/Chuck for 2nd. Second, on Tar3, Ep3, you have Flo/Zach tied for 4th with Aaron/Arianne. I think F/Z came in 3rd, then A/A came in 4th. Third, on Tar4, Ep5, you show Reichen/Chip 3rd, followed by Kelly/Jon in 4th. I think this is backward. R/C should be 4th, K/J should be 3rd. Also, thanks to your data, I've corrected a few errors in my database, and have now accounted for all of the ties (by sharing points). I'll post my updated rankings shortly.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-31-05, 02:23 PM (EST)
|
|
49. "During/post-season prediction power of 3 ranking methods" |
LAST EDITED ON 05-31-05 AT 03:29 PM (EST)Rasta and Ratboy, here are the results for the prediction tests for three ranking methods. The results were somewhat surprising at first, but upon closer inspection they make sense. Ranking/prediction methods The three ranking/prediction methods looked at were: 1. The RRC method (Racer Rating Calculation method, or for the mathematicians who like to name formulas, the Rasta-Ratboy-Cahaya method), based on ratings calculated at the end of each leg, up to the end of the race. The rating calculation for each leg is discussed elsewhere in this thread, but the formula is given here: RRC = 100-(((100/t)*((2*p)-1))/2), where t = number of teams and p = place result of the team for the leg. For example, with 5 teams finishing in a leg, 90-70-50-30-10 points are allocated to teams finishing 1-2-3-4-5. Points are accumulated for each leg and then divided by number of legs already run to arrive at an average RRC up to that point of the race until the end of the race. The final RRC ranking is determined by the end of the race. 2. The AVG method, or the running average method, based on average placing calculated at the end of each leg, up to the end of the race. The final average place ranking is determined by the end of the race. 3. The NEL or next=last method, where next leg's predicted placing equals the previous leg's actual placing. The final place order is determined at the end of the race, such as shown on the CBS leaderboard. Prediction power of methods and prediction errors Prediction power was based on the ratio of: method prediction error / average prediction error. The average prediction error for any leg is given by the formula APE=((n*n)-1)/3, where n equals the number of teams still in the race. For example, in a leg with three teams, and the finish is Team A first, Team B second, Team C third, the prediction errors are:
ABC prediction: PE=0 (no prediction error)
ACB prediction: PE=2 (1 place error each for B and C)
BAC prediction: PE=2 (1 place error each for A and B)
BCA prediction: PE=4 (2 place error for A, and 1 place error each for B and C)
CAB prediction: PE=4 (2 place error for C, and 1 place error each for A and B)
CBA prediction: PE=4 (2 place error each for A and C)
Average prediction error APE as shown = 16/6 = 2.67. The formula is APE=((3*3)-1)/3 = 8/3 = 2.67. The formula is much easier apply for a large number of teams because the number of possible permutations of finish order is equal to n! (a factorial), where 3! = 2*3 = 6. Even for just six teams this is 6! = 2*3*4*5*6 = 720 possible finish orders. Each method is calculated leg-by-leg, starting with the calculation for leg 1 to predict the results for leg 2, and so on, until the end of the race. Based on this calculation at each leg, teams are then ranked for the next's legs predicted finish order. Prediction errors are then determined by the results of the next leg. The numbers shown are methods prediction errors / average prediction errors. A result of 1.000 is an average prediction, while a result of 0.000 is a perfect prediction. The lower the number, the better. Pre-season prediction capability of each method during the course of the race Here are the prediction error results for "blind" (not knowing results in advance) leg-by-leg predictions given by each method, during the course of each TAR season: Mtd TAR 1 TAR 2 TAR 3 TAR 4 TAR 5 TAR 6 TAR 7 ALL
RRC: 0.740 1.057 0.783 1.147 0.857 0.811 0.899 0.900
AVG: 0.747 1.084 0.777 1.130 0.907 0.805 0.899 0.907
NEL: 0.634 0.919 0.755 1.069 0.882 0.833 0.835 0.850 These results show that the next=last method is the most effective prediction method of the three methods. At first, the may not seem to make sense, as you would think that an average based on previous legs' average placings or point-systems would work better. But, after some thought, teams finishing at the top of the order have an advantage going into the next leg, as well as the result reflecting their current skill level. Post-season ranking system fitness After the season is completed, each method can be applied to the season's race to determine its "fitness" to the actual results, using the predicted placing error based on the post-season rankings (not at each leg, as was done during the race). Instead of varying during the season (as was done above), the predicted placing for each team remains the same throughout the season based on their ranking. The results are far better: Mtd TAR 1 TAR 2 TAR 3 TAR 4 TAR 5 TAR 6 TAR 7 ALL
RRC: 0.542 0.780 0.630 0.852 0.646 0.664 0.540 0.668
AVG: 0.581 0.793 0.679 0.785 0.658 0.653 0.488 0.665
NEL: 0.555 0.747 0.728 0.768 0.671 0.743 0.642 0.697 The results are better simply because of "20/20 hindsight" given by the post-season ranking system. Both the RRC and running AVG methods give more accurate results than the final finishing order given by NEL. That is to say, for example, RRC and AVG rankings which both ranked Rob/Amber ahead of Joyce/Uchenna in TAR 7 gave better overall prediction results for the entire season, even if Rob/Amber did not win the final leg. Summary If you're going to try to predict the next leg's results, your best bet (so far determined during this study) is to follow the order of the previous leg, using the next=last method. If you're going to rank teams after the season is over, and want the rankings to reflect the best possible prediction (with fewest errors) in hindsight, the RRC and AVG methods are best. The results from RRC and AVG methods are too close to determine which is better, and there may be other equally good (if not better) methods not yet proposed. Other observations There is an optimum ranking of teams for each season which gives the best quantifiable prediction results (with least prediction error). I'll probably do this next, by playing with it through a little trial and error using Excel. Because the RRC and AVG methods are both different, yet give similar but also different results, I'm not still not sure what the best ranking methodology would be to arrive at the optimum ranking for each season. Some seasons were far more predictable than others. Season 4 was incredibly unpredictable, often with a complete flip-flop of top and bottom teams in the order. Only after the season was completed could a ranking system give better than average predictions. Rasta and Ratboy, if either of you guys want me to post HTML format Excel spreadsheets of any particular method and/or season, let me know, and I'll post what you request. They're rather large and if you'd like to have all methods/seasons posted, it might be better to put them up on a web site (I don't have one, but if you do, I'll e-mail it all to you). The other advantage of web site posting is that the Excel spreadsheets are intact with the formulas as well. For all the thought and work this has involved, it has been quite fascinating!
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
05-31-05, 04:50 PM (EST)
|
|
50. "RE: During/post-season prediction power of 3 ranking methods" |
Very interesting analysis. I had to read through your post twice just to understand it.I'm not surprised that the NEL had the highest in-race predictive results. I am surprised (and a little disappointed) that the RRC wasn't significantly different than the AVG. Then again, I never expected the RRC to be too effective as a predictor to start with. A couple of questions... On the post-race analysis, how did you determine the NEL? Was it basically just their final rank on the leaderboard? Also, you mentioned an optimal prediction method. Would it be some combination of NEL and RRC?
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
05-31-05, 07:33 PM (EST)
|
|
51. "RE: During/post-season prediction power of 3 ranking methods" |
Likewise, I had hoped the RRC method would do consistently better than the AVG method for post-race ranking fit with fewer prediction errors. I was kind of surprised it didn't overall (although it did for some seasons).To answer your questions: 1. Yes, the NEL is the final rank at the end of the season. It's no surprise it was weaker than RRC and AVG for post-race rankings (one reason we did them to begin with). 2. With some trial and error by switching around final post-race rankings (with Excel autocalculations), I think we can arrive at the optimal ranking that gives the least prediction errors. But it doesn't give us a methodology; it only gives us what the best ranking solution is for each season. By doing this, perhaps we can find some clue(s) about how to improve the RRC methodology. There may be some clues, too, in comparing why RRC did better some seasons and AVG other seasons. We may also have come close to reaching a limit of sorts, given the inherent unpredictability of race results (TAR 4 is a clear example of this). Let's see if we can come up with the optimal rankings first, and then perhaps we can work backwards to arrive at a logical and consistent methodology (possibly a modification to the RRC methodology) to achieve this. If I come up with the optimal rankings, I'll post them (along with the associated minimzed prediction error ratio) here.
|
|
|
Top |
| |
|
Ratboy 79 desperate attention whore postings
DAW Level: "Hollywood Squares Square"
|
06-01-05, 01:28 AM (EST)
|
|
52. "RE: During/post-season prediction power of 3 ranking methods" |
This is all very interesting stuff. A lot of great information here that I can use to follow up on something that I am trying to do with these numbers.The most interesting thing that I find is this. TAR seasons 5-7, for preseason and post-season prediction ability, is fairly uniform. TAR's 1-4 are all over the place. You even commented below that Season 4 is unpredictable. And I think that has to be due to the Fast Forward, especially considering the Next=Last leg predictor. Common sense would state that if a team finishes high, then the next leg, they have a good chance to finish high again. But the FF throws a wrench into that. Look at the FF from season 4 to see what I mean. Leg 1: Monica/Sharee--use FF to finish 4th, finish 10th next leg.
Leg 2: Steve Dave--use FF to finish 1st, finish 9th next leg.
Leg 3: Steve/Josh--use FF to finish 1st, finish 9th (last) next leg.
Leg 4: Tian/Jaree--use FF to finish 1st, finish 7th next leg.
Leg 5: Millie/Chuck--use FF to finish 1st, finish 4th next leg. And also, let's look at their numbers for the previous leg before the FF was used to see how far they moved up. M/S--(no place)-4-10
S/D--5-1-9
S/J--9-1-9
T/J--4-1-7
M/C--2-1-4 All right, so Millie and Chuck aren't a good example. (I realize that now after putting them in.) But you have four teams throwing off that prediction. And not only that, but they are also slightly throwing off everyone else's predicted finish by their displacement at the top, and through four legs, that is a heck of a lot of placings. Just looking at Steve/Josh's near last to first, they move up 8 spots, moving eight other teams down one each, resulting in a sixteen place skew on just one leg of the race. Season two has the same high number as season four, and here are the FF's for that season. Shola/Doyin--3-1-5
Oswald/Danny--1-1-6 After this, there were other FF's used, but they were much later in the race and didn't effect the standing as much. So I don't know what can account for the high number with the prediction scale. Possibly bunching, or evenly matched teams, I don't know what it is. Season's 1 and 3 have the lowest number, even with the FF, but looking at the results would explain why. First season, first leg, FF was taken by Rob/Brennan, who then stayed at the lead of the pack for the rest of the race. Third leg, Fester's took it and stayed ahead. Frank/Marg used it in first place, and stayed there. Guido's used it in fourth place, and stayed in fourth after using it. The only hiccup is Pat/Brenda, using it to go from 5-1-9. In season three, Ken/Gerard use FF on first leg, and stayed up front from there on out. Derek/Drew grab it on second leg, and stay up front. Dennis/Andrew use it in last place, and finish last. Teri/Ian go 6-1-4. Flo/Zach go 2-1-2. J.V./Jill go 5-1-5, but have other good marks. It seems that if good teams used them early, there would be less skew in all three prediction tools. I'm not saying there is anything we could do to possibly make it fair to adjust for the FF, it's just that we have to keep in mind that it is there, and may be throwing off some of these numbers very slightly, or in the case of season four, quite a bit. I don't want to end on a downer. Great stuff overall here. Superb in every way from a stats junkie's heart.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
06-02-05, 02:55 PM (EST)
|
|
53. "Demographics" |
I agree about the FF: the leg-by-leg predictive results have to be heavily skewed by the FF.Any in-race prediction therefore must take this into account. In addition, I think the teams' demographics should be taken into account as well. Gender is the easiest to measure. Here are the priliminary results, based on the RRC method: Count Mean Median Std Dev Avg Finish
FF 17 33.5 32.6 17.3 8.1
MF 41 43.6 46.4 17.8 5.7
MM 21 46.6 52.7 16.9 5.1 1/3 2/3 3/3
FF 3 5 9
MF 13 14 14
MM 10 8 3 1/3% 2/3% 3/3%
FF 18% 29% 53%
MF 32% 34% 34%
MM 48% 38% 14% The first chart shows the mean, median and standard deviation for each gender combo. The FF teams are clearly the worst combination. MM seems to be slightly more successful than MF. The second chart divides the rankings into thirds (ie top, middle, bottom), and shows how many teams finished in each third. The third chart illustrates the odds of a gnder combo finishing within each segment. For instance, of the 17 FF teams, 18% ranked in the top 3rd, 29% ranked in the middle 3rd, and 53% ranked in the bottom 3rd. Again, this shows that FF teams generally perform worse than other teams. Interestingly, the MM teams appear significantly more successful on average than MF teams. Next up: Age, although I'm wondering how many segments to divide into. For example, I could use... 1. Under 40 vs Over 40
2. Under 35 vs 35-50 vs Over 50
3. Average team age
4. etc, etc Any thoughts?
|
|
|
Top |
| |
|
|
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
06-03-05, 03:03 AM (EST)
|
|
56. "RE: Demographics" |
Interesting results there in the gender analysis.It might be interesting to split the MF into married/unmarried to see what impact marital status has on the MF result. Presumably, married MF teams would fare better because of their longer experience doing things together on a daily basis. That could be a false assmumption, so I'm just curious what the result would turn out to be. About the age groups... Because of the low sample size, ages have to be grouped, as often done in statistics. First, determine the number of groups. In your example
1. 2 groups
2. 3 groups
and also 4 or more groups. If you have 2 or 3 groups, you'll probably want to set the age for each group so that you have roughly an equal number witin each group. If a group is too small, the sample size is also too small and statistics could be misleading, given the sample error. With 4 or more groups, the interval between groups should be the same, for example, with 5 teams: <30 30-34 35-40 40-45 >45, with an interval of 5 and open at each end. Maybe a wider interval would give a better distribution. Since we have 79 teams, I'd probably go with either 3 or 4 groups. Another question you'll have to ask yourself is what if a team has a member each in a different age group (Ray and Deana come to mind). Do you average them? Or do you treat them separately (thus having not 79 teams, but 158 individuals in your sample). I'd probably try both ways to see which makes more sense after seeing the results. There are methods in statistics, correlation and regression, that can determine which factor has a greater effect (e.g., is age more a factor than gender, or vice versa?). Given that we're working with only a few variables, it'll probably be easy to tell just by looking at the data and results (similarly to how you presented the gender analysis above) anyway. But if we want to include demographics as factors to be included within a ranking system (right now, the only factor is previous placings which we apply a function/formula to), we'll need know the weight of each factor has on the result through correlation/regression analysis. As an aside, I worked out optimum ranking for TAR1 and TAR2. And guess what? Good news, RRC gives one optimum ranking for TAR1 that gives the least possible prediction error, although there are at least two ranking other solutions that give the same least prediction error (PE/APE ratio). For TAR1, anyway, we reached the best possible limit with RRC. TAR2 is trickier (being an unpredictable season), and the optimum ranking giving the least prediction error looks somewhat counter-intuitive, but it does give the least prediction error. I'll continue to do TAR3-TAR7 as I'm curious if RRC came up with an optimum ranking in any of these seasons as well. I'm also relooking into the prediction error thing. As it stands, each placing is of equal weight. So if you were to make bets, you'd be making the same bet (say, $10 each) for each position. I've since run stats for PE weighted by leg, not place. That is, what if you were given $100 to bet on each leg? You'd be be betting $10 for each place in a leg with 10 teams, and $20 for each place in a leg with 5 teams, so the weights of places in later legs are higher than in earlier legs. The results are interesting and I may post them later. Still haven't found a link for you that gives TAR1 racer ages, still looking.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
06-03-05, 10:02 AM (EST)
|
|
57. "RE: Demographics" |
That's good news on the RRC.I'd like to limit the number of age groups to 3 or 4. However, the population is really skewed towards the younger end of the spectrum, which makes it difficult to create balanced groups. For instance, there were 65 racers in their 20s, 39 racers in their 30s, 19 in their 40s, 6 in their 50s, and 7 in their 60s. (I have no data for TAR1 yet). The median age is only 30. I'm thinking about using average age too. This would affect Ray/Deana as well as all of the parent/child teams. The median age would be 32. The population is still skewed, but not as heavily. One interesting tidbit about parent/child teams: they really never work well. The seven P/C teams have finished 11th, 8th, 9th, 5th, 7th, 10th, 9th. Average finish is 8.43. I'll keep working on it.
|
|
|
Top |
| |
|
Ratboy 79 desperate attention whore postings
DAW Level: "Hollywood Squares Square"
|
06-07-05, 00:58 AM (EST)
|
|
58. "RE: Demographics" |
The demographics are what I've been working on. But I don't think that we can strictly do it based on age or gender of team, because we already have distinctions in teams. I have come up with seven designations of teams: parent/child, old couple team, married couple, dating teams, male/male teams, female/female teams, and male/female non-dating teams. Of course, a team could be included in more than one category, but for ranking them, we can only put them in one, so I chose the category that best defines them. Here is how I slipped them into each category.Parent/Child
1. Steve/Josh
2. Nancy/Emily
3. Gus/Hera
4. Jim/Marsha
5. Dennis/Andrew
6. Deidre/Hillary
7. Susan/Patrick Old Couple Teams
1. Teri/Ian (could also be Married)
2. Bob/Joyce (could also be dating or Married, whichever)
3. Steve/Dave (male/male)
4. Dave/Margheretta (married)
5. Don/MJ (Married)
6. Peggy/Claire (female/female)
7. Debra/Steve (married)
8. Meredith/Gretchen (married) Married Couples
1. Frank/Margarita
2. Tara/Will
3. Chip/Kim
4. Lori/Bolo
5. Hope/Norm (could possibly be old, don't know ages)
6. Uchenna/Joyce
7. Jon/Victoria Male/Male teams
1. Rob/Brennan
2. Joe/Bill
3. Ken/Gerard
4. Derek/Drew
5. Chris/Alex
6. Jon/Al
7. Reichen/Chip (could also be Dating Team)
8. Gary/Dave
9. David/Jeff
10. Kevin/Drew
11. Oswald/Danny
12. Shola/Doyin
13. Lance/Marshall
14. Andre/Damon
15. Avi/Joe
16. Lynn/Alex (dating team)
17. Ryan/Chuck
18. Brian/Greg Dating Teams
1. Colin/Christie
2. Kris/John
3. Flo/Zach
4. Brandon/Nicole
5. Hayden/Aaron
6. Millie/Chuck
7. Kelly/John
8. Freddy/Kendra
9. Alison/Donny
10. John Vito/Jill
11. Adam/Rebecca
12. Amanda/Chris (were they dating or married? I don't remember)
13. Lenny/Karyn
14. Cindy/Russel (dating or married again?)
15. Michael/Kathy
16. Paul/Amy
17. Russel/Cindy (dating or married again?)
18. Dennis/Erika
19. Matt/Ana (dating or married again?)
20. Rob/Amber
21. Ron/Kelly
22. Ray/Deana Female/Female Teams
1. Charla/Myrna
2. Lena/Kristy
3. Mary/Peach
4. Linda/Karen (possible old team?)
5. Monica/Sharee
6. Pat/Brenda (possible old team?)
7. Kami/Karli
8. Heather/Eve
9. Tian/Jaree
10. Kim/Leslie
11. Meredith/Maria
12. Gina/Sylvia
13. Debbie/Bianca
14. Megan/Heidi Male/Female Non-dating teams
1. Blake/Paige
2. Aaron/Arianne
3. Tramel/Talisha Now I have been trying to come up with a way to assign points to how each category finishes in each season, so we can predict how future teams are going to finish before each season. But I haven't come up with a good way of doing it. If I average it, then I get the same types of teams at the top, middle, and bottom. Plus, many of the categories have the same averages, or close to it, because the Male/Male teams are not equal. Rob/Brennan sure aren't the same as Ryan/Chuck or Avi/Joe. So I don't know if I need another category for those types of Male/Male teams. Then I thought I would look at all of our different results and rankings, and give out points for that. But that is dependent on race-by-race results, and not a full picture kind of thing. I've reached a brick wall here. But I just thought I would post this and show you guys where I'm at and what I'm trying to do. And you know what else sucks. If we reach a point where we think we can do a good job predicting the finish, we can't do it for this next race, as it is the family edition. That's a downer.
|
|
|
Top |
| |
|
Rasta 210 desperate attention whore postings
DAW Level: "Network TV Show Guest Star"
|
06-07-05, 11:02 AM (EST)
|
|
59. "RE: Demographics" |
I like the way you've tried to seperate the teams. I've realized that simply looking at age or gender isn't enough, and that we need to look deeper.I've started focusing on a combination of gender vs physical fitness, and have created a 3x3 matrix. The x-axis is gender (MM, MF, FF), which is self-explanatory. The y-axis is based on fitness level. Racers are either fit or unfit. The matrix therefore has three categories: fit/fit, fit/unfit, and unfit/unfit. Clearly, the main problem is how to subjectively decide who is "fit". Also, this matrix ignores the relationship aspect of the teams (married, dating, relatives), which can have an impact. On the positive side, the matrix places each team into only one of nine possible categories. No team can overlap into another category. I'll play around with it. Then we can compare results and go from there.
|
|
|
Top |
| |
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
06-10-05, 02:54 PM (EST)
|
|
60. "RE: Demographics" |
The categorization of teams by Ratboy looks interesting. As he said, there's some overlap and some subjective judgement made about which group to put a team in. Somehow, though, we've got to quantifiably rank teams even within each category, or else it all comes out almost the same, as Ratboy mentioned.Subjective categorization of 'fitness' might be interesting to look at, but let's keep in mind it's subjective. There's no way to rate or categorize fitness without a clear measure, and I think we'd be speculating. Still, it would be interesting to see what you come up with along this line. The three solid variables that are measured are age, gender, and relationship. Ratboy combined these to some extent. My own thought is to analyze each of these variables separately at first (similar to what you did for the gender variable earlier), and then analyze them in pairs and triplets, or categories (as Ratboy has just suggested). The advantage of separate variable analysis is the effect is more easily quantifiable. Grouping and categorizing is more difficult to quantify, and that's what Ratboy meant when he said he'd hit the wall with it. Rasta, could you put up a short list of racer/age (e.g., A/B/25/30 for each team? I could take the time to look them all up if you'd rather I do that and then we use that as a basis for data cross-checking. I'd be willing to do some straight-up correlation analysis with the data (for age, gender and relationship, separately), and then make a preliminary suggestion about how to incorporate it into the RRC (or any other) ranking system. As I noted earlier, RRC by itself reached optimum *post-season* prediction ranking for one season (and another to be verified), so I wonder (optimistically) how much demographics will actually help improve pre-season prediction capability in addition to the first leg of each season. It can't hurt to try, as there is still a lot of room for improvement in the pre-season prediction capability. I will make an additional post soon on the "betting" prediction (equal weight by leg, instead of equal weight for each position) as mentioned earlier. It really does make a difference (a lot!) how you distribute your bets during the season for each of the ranking systems.
|
|
|
Top |
| |
|
|
|
|
|
cahaya 19891 desperate attention whore postings
DAW Level: "Playboy Centerfold"
|
06-10-05, 03:50 PM (EST)
|
|
61. "RE: During/post-season prediciton power of 3 ranking methods" |
The previous prediction table placed equal weight for each and every position. In the event of betting, an equal bet would be placed on each position. Earlier legs with more teams would have a higher number of bets; later legs would have fewer bets. This places greater emphasis on predicting earlier legs correctly, because there are more positions to predict in these legs.This table places equal weight for each and every leg (not position). For betting, an equal sum of bets is placed for each leg and divided equally for each position within that leg. For example, $100 is placed for each leg. For a leg with 10 teams, $10 is placed on each team's predicted position. For a leg with 4 teams, $25 is placed on each team's predicted position. This places greater emphasis on predicted later legs correctly, because the weight for each position in later legs is greater. The results are quite suprising, relative to the previous table. Equal-weight Betting Per Leg Prediction Results Pre-season Prediction
Met TAR 1 TAR 2 TAR 3 TAR 4 TAR 5 TAR 6 TAR 7 ALL
RRC 0.703 1.062 0.980 1.140 0.797 0.812 0.872 0.901
AVG 0.749 1.082 0.957 1.042 0.913 0.809 0.852 0.908
NEL 0.541 0.911 0.939 1.048 0.932 0.895 0.847 0.887 Post-race Fit
Met TAR 1 TAR 2 TAR 3 TAR 4 TAR 5 TAR 6 TAR 7 ALL
RRC 0.430 0.860 0.744 0.881 0.646 0.726 0.635 0.703
AVG 0.560 0.832 0.848 0.798 0.754 0.712 0.606 0.726
NEL 0.432 0.775 0.739 0.782 0.608 0.727 0.673 0.672 What does this tell us, relative to the previous table? For pre-season predictions and during-season betting, it does make a difference how you distribute your bets (either by position or by leg), and NEL did surprisingly well for a post-race ranking system fit if results are weighted by leg instead of position. RRC shows up to nearly equal to or significantly better than AVG for each leg, and better overall. More can be inferred on closer inspection.
|
|
|
Top |
| |
|
p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e -
p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e -
p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e -
p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e - p l a c e h o l d e r t e x t g o e s h e r e -
|
|
Email this topic to a friend
Printer-friendly version of this topic
Bookmark this topic (Registered users only)
