New Rating System Status Update

Running Chess Tournaments
21

That’s a fair question, but holding the players hostage is not an acceptable solution. (And what makes you think that a disreputable organizer would care whether the games get rated or not? He’s already got the entry fees.) Your “leverage” should be the option of decertifying the TD and suspending the affiliate.

We should probably distinguish here between problems caused by willful misconduct (deliberate refusal to collect memberships, for example), and those resulting from the TD not doing as much work as you would like (missing IDs, say). In the first case, the tournament should be rated – perhaps after some delay during which you try to resolve the problem – and then the TD/affiliate squashed. In the latter, the office has to do more work, and a surcharge of some kind is certainly reasonable, but refusing to rate the tournament (or even threatening to do so) is not appropriate.

22

If all we know is that an ID is missing, do we really know if it is that TD’s fault, ie not collecting memberships, or some other TD’s fault?

I agree that in the latter case we need to have mechanisms to get around the ID issues, that’s why the membership exception request system was developed. It can be refined as we gather more experience with online reporting. However, any time there are missing IDs, it WILL delay how fast the event gets rated, because we have to allow some time for memberships from other events to arrive and be processed.

BTW, crosstables were posted about a half hour ago. They need a little cleaning up (some of the date fields are missing), but the important stuff looks good to me.

I have a list of things that need to be fixed or changed before we run the next rate batch, and I may not even have net access until the weekend, so there may not be another batch run until early next week.

23

Mike,

It’s good to see fresh data out on MSA. Thanks for all you hard work.

A question: The new crosstables on MSA are listed in the pairing order with which the report was submitted. Is this the new way we will see these? Or is putting them in score order on your list of changes?

Thanks,
Jeff

24

Sorting the results is on the to-do list, though some pairing programs can write the upload files in that order.

25

It was pretty sweet to get a tournament rated less than 12 hours after submitting it :wink:

26

We anticipate a lot of confusion and questions about the new policy of re-rating events to get them into chronological order.

Having the K-12 rated after several hundred events is certainly going to give people a harsh introduction to the concept that ratings are preliminary until they are re-rated into chronological order.

(Another way of thinking about it is that the K-12 actually IS in chronological order already, it’s all the events since then that are out of sync.)

27

Nolan:

The change in the way directors will send in online tournaments, has been the best change to happen in years. Since the age of online tournaments (ICC, yahoo, ect), for the first time over-the-board will be at par in the speed to rate events. It only will happen if and only if the directors use the new service. With faster turn around time, players will know what director is online and the other I’ll get to it when I feel like it.

If you ever run for the Executive Board you got my vote, President Nolan sounds right. :smiley:

28

I would heartily agree, except for one basic fact of life: People like Mike are far more effective NOT on the EB than on it. To put it another way, I shudder to contemplate wasting Mike’s talents by putting him on the EB.

Bill Smythe

29

Idea for the eventual future (probably not of topmost importance, though):

Choose a method for the default sort (either initial rating or final rank) and display it that way, but allow an option to sort by some other criterion. The crosstable could be stored internally in just one specific way, with the versions sorted other ways generated on the spot upon request, and then discarded.

“Other ways” could include:

→ Final rank (let’s leave tiebreaks out of the picture for the time being)
→ Pre-tournament rating
→ Alphabetical (could make it easier to find a player)
→ Score after N rounds, for each N from 1 through the number of rounds

The latter could be used to see easily how the pairings worked in round N+1.

Bill Smythe

30

With a disreputable organizer needs a disreputable director. When the affiliate and the director are in the same person, it can become more disreputable – as there are little checks and balance. In that case, its’ the players themselves that make or break the tournament.

Decertifying directors is not as simple as people think, as there are only 21 on the ‘TD Certification Revocations and Suspensions’ (February 2005 Supplement). The entry fees do cover the cost of the rating fee. There are greater reasons why directors do not send in a tournament report on time then the cost of the rating fee. If the director is so cheap not to use the online report at 18 cents for a dual rated G/60 tournament. Its’ more of a case of being lazy then being dirt cheap.

Will tell any player, check the tournament received list. If the tournament is received for rating, then the director did everything he could to get it rated. The days of sending tournament in the mail will be as so on common as seeing pairing cards in a few years.

31

I AGREE WITH SMYTHE

:laughing: :laughing: :sunglasses: :smiley:

By the way one question; the msa says they are gonna rate everything in ending date order; i understand that but is that include a rerate for the tournaments under the old system :question: :exclamation: :question: :exclamation: or just from now on??? :exclamation: :exclamation: :question: :question: :open_mouth: :smiley: :laughing:

32

Multiple sort orders probably aren’t much more work than sorting them into results order, so that sounds like a reasonable thing to look at when we get to that item on the list.

To be honest, I don’t know how far back we’ll be able to re-rate events yet. I was hoping to be able to go back to about May or June of 2004, which should take care of most of the straggler events, but right now I’m not sure if I can even go back to September 2004 because there are about 100 events from the old ratings system with data issues in them between September and December.

One of my first tasks in TN is to set up the new database server there, once I get it loaded and synced with the master server in NY, I may be able to try running a test bulk re-rate on it going all the way back to June and see how many problems that uncovers.

33

One other topic that has fallen by the wayside recently:

Colors.

It would be informative to be able to see, in the crosstable, what color each player had in each game. Will it eventually be possible for organizers to add this information to crosstables they submit, and for the colors to be displayed when the crosstable is viewed?

To reduce confusion, I think a separate column for color history, such as “wbbwb”, would be more readable than adding the color to each result, as in “W12w D27b L32b” etc.

Bill Smythe

34

I’m thinking of the problem with a re-rate with a match. Under the current rules, two players cannot have a rateable match with more then 400 points. If they play with each other and its’ less then 400 points, the match would be rated. If one or both have their ratings re-rated from a past event, they could have their ratings greater then 400 points between each other. How will the match program work under the new rating system?

35

I think the rules should be interpreted as saying that matches are ratable if the current PUBLISHED ratings of the two players are no more than 400 points apart, that should be easy to check in the ratings system without getting into the re-rate issue at all, and it is also something players can check before playing a match.

36

When will the ‘dual-rated’ crosstables be placed into the data of the MSA? Its true that new software programing takes time to get all the bugs out. I’m glad that part of the programing is done, as players can still see their updates of quick in their general.

Im not sure what the long term problem will happen with re-rating events. There are a number of directors with old events not sent in and some with events not let rated. Like myself have a event received for rating (11.29.04) Spring Lake Blitz. It will cause three tournaments needing to be rerated, two being my own events and one some other director, that will change the post rating of the seven other players, and whoever they were paired with.

Since it was (Spring Lake Blitz) a round robin, and round robins and matchs are not let online. When will the round robin and matchs be accepted for rating in the new program?

Since the Spring Lake Blitz will cause some rerating of events, noticed in the tournament received list, the ending date was posted as 10.20.04 it should be 11.20.04. If it is rerated as being on 10.20.04 it will cause a error to re-rated some other tournaments between 10.20.04 and 11.20.04. The 2004 Michigan Amateur Championship (10.24.04), Kalamazoo Quick (10.24.04), and Spring Lake Swiss Open (11.20.04). It would place a error to rerate the 7th Kalamazoo Scholastic Championship (11.6.04). Im not sure if that data error can be fixed before its rated. As the data error is off by one, if its stays at its current error it will place 4 other events into error.

37

I hear the problems with ratings voiced on the forum. I realize USCF is moving and a new office is being set up. I realize a new rating system is in the process of being implemented. But the average member is suffering from these frequent delays. As a tournament director with years of experience, it is getting old.

USCF has delayed ratings often throughout the years and has promised updates. They have used these updates as excuses for inactive periods of rating runs. The bottom line is that the last effective run of rated events was January 11, 2005. It was not all inclusive, and the mini-run that just occurred simply got people accessing the website only to be disappointed again that events they participated in were unrated still.

I submit paper reports. There are reports dating back to October that USCF has received and noted online that they are received. But the players who have participated in these events have not had a rating change for nearly 6 months. That is one half a year to rate events.

USCF staff fiddled with the received dates of these events. They have listed them as received in January when they were posted in November and December. Checks and mastercard statements show that memberships enclosed in the envelopes were processed. The October and November events have been idle through three USCF rating runs. No "problems have been reported back to me about the reports. They just sit.

As USCF exits New York, the staff there will leave a legacy of poor work ethic. I really believe some of that is deserved. The new staff should benefit from a more effective rating system. But a staff member who really cares about their reputation should perform their duties to the last day. Not for the money, not because they are told to, but because it is the right thing to do. Ratings are a basic service. The appearance is that USCF mismanaged the rating system for several months now. They discontinued an old program, prior to having an effective replacement. They stopped services such as the ability to look up ratings on a large scale without being ready with a replacement.

Now there is testing with the basic data base as we go. There are test runs on the active membership. The program is still not ready. Office staff and this forum suggest dates that will mark the end of this period. But the dates are bogus. “If my computer is set up, If I have internet access” Well, if not then the Friday date you suggested will not happen. Hey, it is Saturday, and it did not happen.

I know that dam will break someday, and the ratings will run again. I just do not like the unkept promises. The finger pointing is out of line. The USCF phones that are never answered, the emails never returned, the reports with faked received dates in order to use that as an excuse not to rate multiple events are getting old. If there were a viable alternative to our National Organization, players and organizers would flock to it.

I think your work Mike is vital. For the sake of this organization, get it done. Get the staff trained on it. Get the most important service to our members online and running again.

Sincerely,

Steve Dillard
NTD

38

Steve Dillard, NTD

Do understand the problems with handwritten tournament reports. They have fallen out of use with a number of directors. With more directors going with computer software then pairing cards, online reporting then paper handwritten reporting. Have the feeling, that handwritten reporting of tournaments are and will be in a few years not accepted for ratings.

Do not accepted the standing word, that handwritten tournament reports take so long to rate because they have to be done by hand. Handwritten tournaments were the standard for all directors back in the 1980’s. The reason why handwritten tournament reports are taken so long to be rated, is not the lost skills of the rating department, its the out right behavior to force directors from handwritten reports to disk and now online reporting.

Directors have a new choice, use computer software for pairings and then use online reporting. When its’ received with online report, will be rated in a few hours (when system is 100% online), then posted in the MSA within a day or the first Monday or Tuesday after the event. Or use pairing cards and send in the report as a handwritten report, then hope it will be rated in less then 8 weeks. Or use computer software, and send in the report on computer disk, then hope it will be rated in less then 4 weeks. Since the office is moving to Crossville, only see one person doing the data entry of the handwritten reports.

39

Steve, your assumptions about the number of events rated are wrong.

The last rate on the old system had 467 events, 1115 sections, 22833 players and 49,555 games in it.

That’s larger than only three rate runs in the past 12 months, and all of those were runs just before supplement cutoffs, too. It has been common USCF practice for them to clear out as many events as they can before a supplement, that is what they did in January as well.

That’s also why we picked then as the cutover point, because there has been a tendency for the ratings department to concentrate on getting events ready to rate ahead of the supplement and letting some other things slip, so they usually take a week or two to play catchup and work on things other than entering rating reports. I know that much of what they did during those two weeks this time around was get ready to move the ratings department files to Tennessee.

The first two runs on the new ratings system had a total of 252 events, 515 sections, 9799 players and 22,253 games. About half of the rate batches in the last year were smaller than that.

According to Nancy Evans, as of Monday there were about 350 events waiting to be processed. That’s a large number, but given that the ratings department was not entering events for about two weeks and then had to learn a new data entry program, they’re still not as far behind as they have been at other times in the past year. Nancy feels that they are already able to get reports on diskette ready to rate quicker than on the old system, and they’re picking up speed on entering paper reports as well. Since error-checking is part of the data entry process (try entering one online and you’ll see what I mean), paper events should be in nearly ratable form by the time they’re done entering them.

One of the things we will be adding to the ‘events received’ list on the website fairly soon is the status of the event, ie, whether it has errors in it, and possibly what kinds of errors. I don’t know yet how detailed that will be.

Today I changed the events received list so that it is updated roughly every hour.

We will start training the staff in Tennessee on the ratings system some time in March.

Yes, it did take longer to get the first rate run on the new system, but that’s because it took a LOT longer to get the formula to agree with what the old system did, in no small measure because the old programming made several assumptions that were different than what a literal reading of the ratings formula would produce.

It took an extra two weeks of working with the formula and testing around 1000 events from the last few months to track down the differences, during which time I exchanged dozens of e-mails with the ratings committee. I also uncovered many more problems with our crosstable history due to inconsistent practices over the years.

It’s taking longer to get the 2nd rate done as well, in part because until a few hours ago I had NO access to the systems in New York from the new offices in Tennessee.

Also, there were a few serious questions raised about the first run, and those need to be addressed before I’m ready to do another run. That’ll probably take me a few days, but I think I should be ready to rate by mid-week. I think there will probably be about two more manual runs, maybe a day or two apart, and by next weekend I should be ready to switch over to a batch job that will run rates at least hourly and probably 2-3 times an hour, at least during the day and the evening. I will probably not schedule any rates between roughly 1AM and 5AM, because that’s our system maintenance period and it would slow things down.

There is also a major policy discussion taking place among the Board and senior USCF staff regarding some issues that have been raised by the new online systems and the new ratings system, especially the issue of how tolerant to be with reports with non-members in them and what to do about memberships from other events that haven’t arrived as promised. I’m sure this will impact the development schedule somewhat.

40

Mr. Forsythe,

Whether USCF is trying to force TD’s to switch to diskette or online report submission is moot. There is a premium rate charged now to TD’s that submit reports by hand (mine are from WinTD, I just send a paper report). That premium paid suggest then that USCF will treat the report with equal concern. In other words, I pay a price for submitting written rather than diskette or online reports. Once paid, my reports should be treated equally.

As long as USCF allows written reports mine will be. Even at $5 a game, mine will be. Until the process is simplified to the point that the submission of reports online can be performed without delay at my end, I see no reason to change. I should be able to pop a disk in my computer and send a file. I cannot. So, written reports come from Kentucky.

Steve Dillard