This is not an exact quote, but I think Ken Sloan has said (more than once) that the purpose of the local exam is to see if the TD can find rules in the book, the purpose of the senior exam is to see if the TD can figure out how to apply those rules to situations other than the ones expressly given in the book and the purpose of the NTD exam is to see if the TD can figure out what to do when the book doesn’t offer guidance or when the guidance it offers is inappropriate.
If one accepts Ken’s statements as a reasonable, if perhaps not formally declared, definition of purpose, then it seems to me that regardless of whether a successful candidate takes 2 hours or 20 hours to find the (right) answers, that purpose has been achieved at all three levels.
Ken also has a suggestion that he has made several times (I’ve heard it at the Rules workshop more than once, and he has posted it here as well.)
Paraphrasing Ken again, TDs taking the exam (most often referring to the local exam, I think) should FIRST try to answer the questions closed book, then take the time to look up EVERY question and see how many of them he or she got wrong.
Originally, (I am one of the few left, who remember back then in the early seventies), the goal of the program was that a member can go anywhere in the country and get equal and fair treatment in a USCF-rated event. That goes beyond just rule interpretation.
I think the Federation has accomplished and maintained that—a successful program.
Around the time the TDCP (TD Certification Program; I don’t see that acronym any more) just started, I took the LTD test successfully. I was an experienced tournament player and had done some sort of directing activity, but now USCF said I needed a cert to direct in the future so I took their test. I don’t remember if they told me to read the rulebook cover to cover then, but it was much thinner then anyway. Maybe I scanned it.
I don’t know how much it’s changed since then, but it was open-book with a long time available to take the test. I remember having to search the book for a number of answers. I left the experience knowing the book, and my own strengths and weaknesses of rules knowledge, better than when I started.
I think that it would be hard to “reason about the rules” for someone who is not familiar with tournament play, even for someone with good general reading comprehension. If I’m right, a “professional test taker” who doesn’t know anything about chess tournaments would have to work hard to pass the test, and at the end of it he would have learned something about it despite himself. And that is a useful background for working as a TD. Is it enough? Too much? Or maybe it’s about right.
Considering that there are already several tests with 40 questions each, and that probably these replaced other tests, now no longer in use, which the USCF hopefully has not lost, it might not take as long as it seems.
Besides, 400 was not my number. Apparently, there are only 160 questions on current tests (2 Club/Local TD tests, 2 Senior tests, 40 questions per test). That would seem to suffice for a test bank, since it apparently suffices for the mailed tests.
In fact, I am not sure that there are 400 different questions worth of material in the rulebook – at least not 400 different items that wouldn’t be too trivial or picky. So, to get to 400, you might have the same rule knowledge being tested a lot of different ways. How many touch-move scenarios that are fundamentally different can anyone invent? (By the way, that would be another problem for the “random selection” algorithm. You wouldn’t want a test that asked repeatedly about the same rule only with arbitrarily different details.)
This seems to be a pretty low threshold: the certified Local TD can read at some level of comprehension, so that over the course of several weeks he or she can answer questions that are directly answered by a book. Why bother, and why make such a fuss about it? Doesn’t society already have a certificate that is supposed to certify this ability – a high school diploma? How many people applying to be a Local TD, having met the experience requirement, couldn’t do this? What does it have to do with what TD’s do in the tournament room? Some people have said, we want Local TD’s to have read the rulebook and the test is good because it makes them do it. However, as Mike Atkins so politely informed me, the test comes after the experience requirement is met – which means that we have certified Club TD’s at USCF tournaments who have not read the rulebook. They get around to reading the book during the test, and this is put forward as a good reason for having the test.
Rule-reasoning seems a reasonable thing to test using mailed open-book format, though it seems to me that being able to research and to reason about the rules on the time scale of weeks is rather different than the situation faced by TD’s during a tournament.
So, we have a test which you can pass only if you have been around long enough to know the unwritten rules of the TD community. Is a “test” needed for this? It is apparently a club. Don’t the other members of the club know when someone is suitable to join?
Who defines “inappropriate” and where is this defined? What do the Delegates, who adopted the rules, think of the fact that in order to become an NTD, you have to have absorbed knowledge from some nebulous esoteric source about which are the real rules and which were just whimsy of the Delegates and not “appropriate”? See comment above about the “club”.
I disagree that this is sufficient for a test bank (though I’m not disagreeing with you, you’re just stating what is in existence now). 2 tests at each level for an open-book, non-timed, sent out exam is not sufficient nor in the best interests.
Before solid numbers can be thrown out the tested material topics have to be broken down into functional areas. That’s the first step. Then we can talk about how many questions per area we need and how much material is there.
As thick of a rule book as we have, I have to disagree. Again we first have to break down to functional areas and then worry about the questions.
And this is exactly what gets done in other professions. It’s the same concept that gets tested in a lot of different ways.
I miscalculated here. The number of existing questions is 240. Reading back through the thread, I see that there are 3 versions of each of the Local and Senior TD tests, not two. So it wouldn’t take that long to generate 160 more to get to 400, since you could just make variations of the existing 240.
No. I say this as an experienced copy editor. Anyone can read the Associated Press Stylebook. Most people can understand it. Some can retain much of what they discover in it. But the difference between these people and a professional copy editor is that the copy editor has (a) a little trip-wire alarm in his mind that tells him when he should be uncertain about something, (b) the automatic knowledge of where to look to answer the question, and (c) the inclination to do it, while the ordinary person lacks these things.
If you ask the ordinary person the difference between “effect” and “affect,” he may be able to tell you. But he won’t read (or write) a sentence like, “It is important that we effect this proposal immediately,” and pause on the word “effect” to ask himself whether the right one was used. The copy editor will probably know whether the right one was used, but even if he doesn’t know, he knows that it’s an issue, and he’ll take a moment to double-check.
It’s not about what you know. It’s about what you know you don’t know, and what you do when you realize it.
But none of this is what is being tested. What is being tested is the ability to find the answer to a question in a book, given plenty of time and the question being one whose answer is unmistakably in the book, else it would not be on the “Local TD” test (assuming Sloan’s characterization of the tests is correct). Success on the test does not depend on any of the things which you mention, though I agree those are all good things for TD’s as well as for copy editors.
…and USCF needs a low threshold! The rulebook is intimidating in size and detail to new people. We always need “fresh blood” to come into the TD pool. These people start often in a club that needs SOMEONE to submit reports and dues, and to unruffle the feathers of feuding friends during a game.
It is more important that a TD maintains impartially in his rulings, even if there are better decisions than those made. A complaint to USCF can correct technical mishandlings. Just the act of certifying means that the person is willing to accept responsibility for what he does, and accepts USCF’s authority over his actions.
You’re fixating on the word “test.” The test is as much a formative assessment as it is a summative assessment – in plain language, it’s the course as well as the final. Or, to return to my earlier metaphor, taking the test is a process of installing new mental trip wires.
A certification test is summative, no doubt about it. If you pass you gain the rights and privileges appertaining to the honored status of USCF Local TD (or Senior, or whatever). If you don’t pass, you don’t gain those rights. Summative.
If it were formative, you would be given the correct answers and why they were correct. There would be an explanation of why a wrong answer was wrong. Maybe there would be extra questions until you had learned all the material. There isn’t anything like that. Not formative; don’t kid yourself.
I took a test. Anjiaoshi took a test. Probably everyone in this thread has taken a test or more. Except Brian. Who is kidding whom?
Being formative means it teaches something. It doesn’t mean that the method of teaching is to present a list of correctly answered questions with reasons. The method actually used is to present a list of unanswered questions, point the examinee at the rulebook, and motivate him in his research and thinking by telling him that he can only pass if he answers most of the questions right.
I don’t doubt it. I know Ken well enough from online contacts to believe he wouldn’t make such a statement without being pretty sure of what he’s talking about.
But I’m just wondering how he found out. As I understand it, one cannot receive the exam until one has met the experience requirement (he hasn’t passed the test and now just completing his experience for NTD) and he’s probably a very good test-taker, so I’m fairly sure that he passed it if he took it. So – how could he know?