Brian, since you seem to feel that the testing goals are inappropriate, please feel free to suggest different ones.
What are the testing goals?
I thought you knew them, you’ve sure been posting as if you did!
I don’t get your dig(?) about reading comprehension. You said “why bother with such a test” and “an open-book test might be alright, but then it should be timed”. That comment did NOT mention an on-line test and did not limit the subject to just future tests.
You said this knowing that there currently is not an on-line system and there didn’t used to be such a system. Were you denigrating what has been done for decades because currently available technology was not in use in the '80s? Were you complaining that the USCF is only in the starting phases of working on what you proposed recently?
Until a usable time machine is developed there will always be things that were not done the way we might have wanted to in the past, and that past will not be changed.
This whole thread is about developing a question bank so that we can do online testing, study, and self-assessment. I guess you missed that.
When I started the thread, I didn’t realize that the current open book “test” spans several weeks. That is absurd. Why bother? Nobody has to learn the rules to pass that test. They just have to be able to parse the rule book and the questions. That might not be so easy because the rule book is rather poorly written and organized, and for all I know, the tests are the same. So some people might not be able to succeed at the test even over “several weeks”. Those people most probably wouldn’t be much use if they had to do the same on short notice at a tournament, instead of several weeks.
But being able to do it over “several weeks” (and pass the test) does not mean that you know the rules well enough to interpret them during a tournament. It may be that a timed test isn’t really feasible or practical, except online. If so, the proper alternative to a timed test isn’t a nonsensical multiple-week open book test. The alternative to a nonsensical test is: no test.
No I did not miss that. You said the current procedure wasn’t worth bothering with. You didn’t qualify it by saying it wouldn’t be worthwhile after an on-line exam was up and running.
There was somebody who said “to be terrific you must be specific”. A number of your statements in the past have been a bit unspecific and have later been clarified into ways that some people found surprising and objectionable (such as your thread on draws). Try not to be surprised in the future when somebody reads and comprehends a statement in a way that is logical but not what you actually intended.
P.S. Your correction in the prize split thread was valid. Rare, but valid, since such occurences have happened in the past.
And 4 hours required everything I had!!
But since you have neither taken the test nor seen it, you don’t really know what is invovled. Isn’t that what Mike was saying?
That may be one of the reasons why they’re going to 5 1/2 hours.
I think that if I sat through two days of intensive online seminar training, I’d want some time to relax/decompress before taking an exam that might take 4 hours or longer.
I assume the testing goal is to establish that a TD has knowledge of the rules and the ability to reason about them so as to make correct rulings and decisions, based on those rules, at USCF-rated tournaments.
A multi-week open-book test can be passed without any knowledge of the rules at all, provided the person can find the answers in the book. Doing that under time pressure might reflect sufficient knowledge of the rules to apply them in a tournament. Doing it over “several weeks”? No. Over several weeks it becomes a test of research skills, reading comprehension, and ability to reason about this set of rules – none of which is the same as knowledge of the actual specific rules.
Ok, so I’ll bite. How do the questions on the test overcome the problems with it being a multiple week open book test? I haven’t seen the questions. But you have. So tell us how these questions are crafted so as not to be answerable except by someone who actually has a thorough knowledge of the rules, even over several weeks? Those must be some pretty impressive questions.
It seems to me that somewhere along the line, it was decided that USCF TD certification would involve passing a test. However, closed book, proctored tests were not practical, and online testing was not available. So, rather than giving up on the idea of testing, it was decided to do an open-book mailed test, which ended up meaning giving plenty of time. That hollowed-out the test as meaningful at all. But it could be counted as a “test”, and at least it disqualifies people who are completely incapable of comprehending or reasoning about the rules. So that’s the test.
And that assumption is based upon what?
You are also assuming that because someone took several weeks to return the exam that he or she spent several weeks working on it. I suspect that is not the case, and more’s the pity.
As I recall, when I took the local exam (many years ago), I think it arrived in the mail the day before I was leaving on vacation for several weeks, and that was a couple of weeks after I had requested it. When I did take it, I think it took me an hour or two. It may have taken me another week to get the envelope mailed back.
I’m pretty sure I didn’t even look at it until a week or so after we returned.
Common sense. I asked you the goal, and you wouldn’t say, so I am using common sense. If that is wrong, what are the goals of the test? Does the test have any goals, other than being a “test”?
No simple test would be able to measure how a TD actually reacts in a tournament situation. Some people test better than the perform and some perform better than they test. That is why there is also an experience requirement. There are arguments about that experience requirement. Some say it is too low to be able to determine whether or not TDs have actually demonstrated that they are really capable. Some say it is too high and is an onerous bar to limit the number of TDs. Both sides of that debate have a small fraction of their members who think that the other side is clearly wrong, incompetent, and working against the best interests of the USCF by proposing an idea that they feel is an abomination.
As far as being able to pass a test without working knowledge of the rules and simply by researching the rulebook, I’ve run across people that memorized the rulebook (or claimed to have) and some of those would apply the wrong rules to tournament situations (which is something that a test might catch, with the probability rising if there are multiple rules where confusion reigned).
Another method of advancing might be to have an apprenticeship and journeyman program so that a junior TD can demonstrate (to a more qualified TD) that the junior TD really can perform at a higher level. The first obvious problem I see with such a program is that it would quickly get attacked as being subject to favoritism even if it was actually being run in a totally unbiased manner (and there are those who say that even if people honestly believe there is no bias in their decisions, their own life experiences with cause them to have subconcious biases).
Four hundred questions for a test bank? Brian, how many volunteer work-hours would it take to develope this bank?
All the best, Joe
All any written test can do is be a fairly raw measure of knowledge. Unless you have actors, it would be difficult to test a TDs ability to handle a live dispute in a controlled situation.
Many TDs will refer to a rulebook when making a ruling that they don’t make all the time. That is even true of NTDs who have read, re-read, and re-re-read the rulebook, let alone Local and Club TDs. So a test is designed that will measure how somebody answers a question in a normal situation (I’ll grant that it wouldn’t catch somebody who habitually makes rulings off-the-cuff).
I recently had a case where a ruling was being appealed (actually, the section chief knew it would be appealed so he came to me in advance before actually making his ruling). After hearing the specifics from the TD (who was higher than local and very experienced) I knew exactly how I was going to handle it (which he later told me was different than what he had planned), but it still took 3-5 minutes of leafing through the rulebook to find and show him the exact rule I was going to use. We then went back to the players and I verified that the situation really was as it had been explained (another few minutes). I made my ruling and then spent another few minutes making sure both parties understood it (and I did have to show the rule that I had previously located in the rulebook before they accepted that it was a valid ruling).
Simple research isn’t always simple. If a TD is not capable of doing the research when having time then the TD is unlikely to be able to do it during a live situation. One way of looking at the test is to not see it as a test to pass (demonstrating competence), but rather as a test to fail (demonstrating incompetence). The experience requirements are also available to help weed out those who demonstrate incompetence (organizers are less likely to rehire incompetent TDs). Even with that you sometimes cannot determine the competence of a TD until seeing the TD in action. Even then, a TD may be just fine in some circumstances and totally out of the TD’s depth in others. There are some TDs who are quite competent for open tournaments (where most participants are adults and which may even have large prize funds and/or IM/GM participants) but those same TDs may be on the “only-get-if-nobody-else-is-available-and-we-can’t-survive-better-with-nobody” list for a scholastic state or national event (it works the other direction as well).
Actually it’s 3, 6 hour days over 2 weekends. The exam is on the final day of weekend #2. So there is a week to decompress between the first half and the second half.
I thought from the beginning that 4 hours was not sufficient and I tried to put for 6 hours for the test. Comprise was 5 1/2 hours.
So if it is Saturday/Sunday on week one and Saturday on week 2, do you take the exam on Saturday or on Sunday?
If Saturday, then my concerns remain, because that 6 hour day is now potentially an 11 1/2 hour day.
You take the exam on Sunday of weekend #2.
Days don’t go more than 6 hours.
Perhaps the real goal of the TD test is to require the prospective TD to look up lots of stuff in the rule book for the sake of practice – so that, when it comes time to apply the rules in a tournament situation, even if he can’t quote chapter and verse, he knows what he’s supposed to be looking for and where to find it!