Understanding What Goes On Behind Closed Doors Through

One Judge's Experiences
by Sandy Rodgers

 
As originally published in Needle Pointers, June/July '93

Editor's Note: Sandy Rodgers is a frequent contributor to many aspects of ANG. To name just a few, her donations to the Annual Auction have garnered large sums, she wrote the commentary for the "What is Needlepoint, 2001" on the Web site, worked for over ten years in the administration of the Master Teacher Program, contributes to the Stitch Book and Chapter Program Book, and writes the very popular "Product News" column in NEEDLE POINTERS. Sandy holds numerous certifications from the major needlework organizations, including Senior Master Teacher through ANG, has written four highly acclaimed needlework books, teaches and judges extensively throughout the country, and is highly respected by her students and peers for her embroidery knowledge and teaching skills. 

A highlight of our ANG annual seminars is always the exhibit. We're all inspired by beautifully worked examples of needlepoint, many with award ribbons hung next to them. Those ribbons, and the process of deciding which pieces are worthy of them, are the responsibilities of the judges.

Ever wonder what goes on behind the locked doors of the exhibit on Monday - judging day? It's quite a process, and clearing the mystery of the judging system is more than just interesting. Possibly knowing this procedure will help you decide to enter your work in our next exhibit. ANG has honored me greatly by having me as one of the judges for three of our national exhibits, twice as Head Judge. This is some of what I experienced, and what went on when I was 'behind closed doors' on Monday, Judging Day.

Selection of the three exhibit judges is made by the ANG Board at their winter meeting, usually early February. The judges, and the exhibit judging, come under the auspices of our ANG President. However, the judges are selected by a vote of the Board of Directors.

An effort is made to secure certified judges - not just to ANG Certified Judges, but also those holding judging certification from EGA or NAN are considered. By having the other programs represented, a balanced panel is achieved. Paramount in the selection of judges is that each be outstanding in their knowledge and expertise of color, design, and needlepoint techniques. The Board also aims for a group of three people who complement each other. For instance, one person may be especially strong in knowledge of color and design, another in technique, and possibly the third may have special knowledge of historic design. These areas of specialization are in addition to their proficiency in color, design, and technique. In short, only highly qualified people are selected to bestow the coveted award ribbons.

One decision which doesn't involve the three judges is the recipient of the State Award. This determination is made by an invited dignitary who selects his/her favorite exhibit entry for this special ribbon. The Award, and the announcement of the winner, is presented either at the opening or closing banquet, depending on when the honored person is available to be present.

The judging process actually begins on the Sunday afternoon of seminar. While many attending seminar are still arriving at the hotel and getting settled in their rooms, the judges begin their work with a first review of the exhibit. Prior to being opened to the ANG members anxious to get a first peek at the display, the judges look at each entry. Other than the judges, the only persons in the exhibit room at this time are members of the ANG Board of Directors.

The judges check each piece to determine if it has been entered in the correct category and that it fulfills the ANG definition of 'needlepoint.' They go through the exhibit individually, checking every item to be judged. Should one judge find what seems to be a questionable entry, a consultation among the three of them is requested. If, after discussion, it is determined that a piece is not entered in the proper section, it is moved to the correct category and is not judged. Thus, it is most important for each entrant to read the category requirements very carefully.

One of the saddest things I've ever had to do as a head judge is tell the ANG President that the judging panel had decided some pieces were entered in the wrong category. Even though that piece, or pieces, was then moved to the proper category, such pieces are disqualified from any competition for ribbons. If you have any question about which category your piece should be in, be sure to contact the local exhibit chairman. His/her name is listed on the exhibit registration in the seminar issue of NEEDLE POINTERS. The local chairman will help you enter your piece in the right category. It's so very important; please take the time to ask if you are uncertain. You don't want to enter your piece, then have it not judged just because you slipped up and put it in the wrong section.

Another problem I encountered in my experience of Sunday afternoon reviews was with Princess Grace candidates. The entry rules state that to be considered for this honor a piece must be worked completely in tent stitch. This means that even one French knot. or one straight stitch, will disqualify it from the Princess Grace contest. Although such a piece can no longer be considered for the Princess Grace ribbon, it will still be judged in the category in which it was entered. such as original, adaptation, painted, or charted design. An explanation of any change will be sent the exhibitor

Sometimes the judges find a piece that's all worked in tent stitch. but not listed as a Princess Grace candidate on the registration form. If this happens, the piece is considered for the Princess Grace competition even though it wasn't originally entered in it.

As they review each entry, the judges are also getting a 'feel' of the exhibit and the pieces they will evaluate. They make a mental note of how many are in each category, and the overall size of the full exhibit. Consciously or subconsciously, they observe the general level of expertise displayed. All national exhibits require skill, knowledge and endurance to evaluate, but certainly some more than others. This Sunday afternoon review is sometimes called a 'walk through,' and that's truly a descriptive name. Among other things, it helps the judges prepare themselves for the responsibility ahead.

The real work begins Monday morning. Judging starts early, usually 8:00A.M. and ends late, sometimes as late as 7:30 in the evening. Each judge is assigned a page who is one of the members of the ANG Board. The page will remove pieces from the wall, if necessary, so that the judge may make a closer inspection. All judges I've seen use their own small magnifiers to
examine work, and my personal favorite is a magnifier with a small light attached. Sometimes, however, a piece hanging on the wall will be too far away, even with the use of such enlargers, and must be moved closer.

Pages will, if requested, read the judge the information supplied by the entrant, such as ground fabric, threads or techniques used. At no time is the judge ever allowed to know who worked any entry. At no time does the judge actually read, herself, from the written materials which contain the name of the person who entered it. Nor does she handle the entry in such a manner that she might, even accidentally, see the name of the entrant which is on the back of the piece.

There is a tally clerk present in the exhibit room during judging. We'll discuss her job later. Also present is the ANG President. It's her responsibility to see that all ANG judging procedures are followed precisely.

Three 'trial' or practice judges may be in the exhibit during the judging. These may be people working in the ANG Judging Certification Program. If there are not three people from the ANG program who are ready to trial judge, judging candidates from other organizations will be permitted to do their practice judging. The total number of these trial judges does not exceed three. They actually judge numerous pieces just as if they were the contracted judges. The trial judges do not discuss their opinions with the contracted judges until after all ribbons have been decided. Of course, any determinations made by these trial judges do not enter into the awarding of ribbons. No other people are allowed in the exhibit room during the judging, not even the local exhibit chairman. It's his/her responsibility to sit outside the exhibit, "guarding the door" and making sure no unauthorized person enters during the judging process.

The ANG exhibit is always beautifully hung. Entries are arranged in groups according to their category. Categories of entries are original, adaptation, charted design, painted design, and class project. Complete descriptions are found in the ANG Policies and in the entry rules printed each spring in NEEDLE POINTERS, but in brief the categories are described as follows.

An original is a design created entirely in the mind and fingers of the stitcher. An adaptation is a translation from another medium to needlepoint, either as an exact visual replication of the original or as a new design containing easily recognized elements from the original, and adapted by the person exhibiting it. The exhibitor must be the person who both adapted from the original source and stitched the piece. A charted design is taken from a needlework book, graph, or chart. A painted design is worked on pre-painted or stamped canvas, or on a canvas designed exclusively for the exhibitor. A class project has been worked under the direction of the teacher, either by attending a class or by correspondence. All entries must be worked by the exhibitor and in the adaptation category the transformation from another medium must also be done by the exhibitor. While there are other regulations surrounding the exhibit entries which are explained in NEEDLE POINTERS with the entry form, these are the categories into which a piece may be placed.

Each category is judged against a standard of excellence which has been devised just to fit its own definition. Each judge has 100 points available for every piece, except for those in the class project category. Those 100 points are distributed according to a pre-assigned formula.

For example, when a judge approaches an entry in the original category, she first assesses the design. She assigns somewhere between 0 and 25 points depending on the merit of the overall design. She appraises the colors used and awards a point value between 0 and 25 points for how those colors are manipulated. Then she looks at the workmanship/technique. For this area she has 0 to 20 points. 0-10 points are allotted for how creative the piece is. Creativity involves how fresh and innovative the work is, within the confines of a good design. True creativity will help to elevate needlework as an art form. Suitability of materials and/or stitches is given somewhere between 0 and 10 points. Finally the judge considers the finishing of the piece - the framing and matting, or mounting of a footstool, or perhaps how the seams are worked on a handbag or garment - and gives somewhere between 0 and 10 points.

The number of points given by each judge for a particular entry are totaled by the tally clerk. Then the points all judges have given each entry are combined together. It's easy to see that if a judge gives a piece the highest point values possible in every section, she would give that piece 100 points. If all three judges were to give a piece the highest values possible, the total would be 300 points. Obviously, this is most unlikely.

 
Point distributions that judges work with for the various other categories are:

As you can see, the total for each category is 100 points, except for the class project section. Class projects don't need a full 100 points because they are judged on their workmanship and finishing. It's important to understand that point values and distributions are predetermined by the sponsoring organization. The point system is part of the ANG Policies and not something determined by the judges. What each judge decides is, of those points available, how many she will award to a given entry.

The judges begin their inspections at different categories and work a distance away from each other. That way they are separated, have plenty of room to get close to, or back away from an entry, and one judge does not accidentally intrude on the thoughts of another. Each judge is given tally sheets which list the entry numbers of each piece in the category, and the point distribution system for that section. The judge fills in her own tally sheets, indicating how many points she has awarded in each area for each entry. As each sheet is completed, she signs it. With that signature the judge indicates that these are her evaluations and that she has arrived at them on her own without influence from anyone else. When she completes reviewing each entry in a category, the judge gives her tally sheets to her page, who in turn takes them to the tally clerk.

Each judge works differently One judge's point placement may be very low. Possibly another's is quite high. It is unimportant if she judges high or low, the important thing is that as a trained and competent judge, she is consistent. Thus, one judge's totals may all be fairly low, whereas the other's are all high. When points from the three judges are added together the distribution becomes very clear.

The tally clerk adds up the points each judge has given every entry. The soft whirring of her adding machine can be heard throughout the day. After all three judges have completed a category, and the tally clerk combines all points for those entries, the Head Judge is given a break down of the totals in that group. The entries whose point totals are closest to 300, which would be perfect scores, are reviewed by the judges who then work as a group. They confer and, using the point totals, decide the first, second, and third places, plus honorable mention if so merited. The Head Judge informs the ANG President of their decision, and the appropriate ribbons are hung.

If the point totals in a category are felt to be very low, that is well below the 300 possible even considering one or more judges who tend to judge with low scores, the judges may elect not to give a first place. Then the highest ribbon given might be a second. In the same way, if there is a very large discrepancy between, for instance, first and second, the judges might decide not to give a second place. Instead they may give two third places or just eliminate the second place and give a third. Such variations are at the decision of the judges after much deliberation.

It didn't take long to write this explanation of judging procedure, and it took you even less time to read. But, it takes a long, thoughtful, considerate time with every entry to properly evaluate and assign points to it. Each judge takes short breaks, as she desires, staying in the exhibit room. Such close scrutiny of so many entries takes its toll on the eyes. These rest breaks are essential.

There is, of course, a lunch break. The Head Judge and ANG President determine if the judges eat together, and this is usually the case. Who the judges eat lunch with may not seem important, but it is. This hour away from the continuous judging process is crucial. It helps insure fresh thinking in evaluating all pieces, but at the same time, the judges need to keep focused on what they're doing. They can't get too far away from their work. Conversation, during lunch, about any entries would be unethical, however, just being together keeps the judges in touch with their mission.

As the day progresses, the anticipation and tension begin to mount. More categories are completed by all judges, and the various ribbons are hung. Finally each judge has evaluated all entries, and first, second, and third places have been determined in every category.

It is time to review all entries in the Princess Grace competition and all first place winners. From the latter the Best of Show is selected. The judges gather closely together for their deliberations. Others in the exhibit room - pages, tally clerk and even the ANG President - are at a distance, well out of hearing range. Finally the decisions on the Princess Grace and Best of Show awards are made, and the Head Judge gives that information to the ANG President.

The only ribbons left to be decided are the Judges' Choice. These are fun decisions. Each judge is permitted to give a special ribbon to her favorite design of all those entered in the judged section of the exhibit. She need not have a specific reason for liking the piece, it is only required that it appeal to her.

During the judging process, there may be discussion among the judges. Except for the Princess Grace and Best of Show decisions, some of this dialogue may be within the hearing of the Board members acting as pages or tally clerk, most is not. No ethical judge, nor anyone in the exhibit room during the judging, reveals any of this information outside the exhibit room. Not for any reason.

First, second, third, honorable mention, and judges' choice awards are hung immediately, and their winners known as soon as the members see the exhibit. The Princess Grace and Best of Show awards are held in confidence to be announced by the President at the Friday evening Awards Banquet.

After the judging is completed, the judges meet with the ANG trial judges to answer their questions. If asked, they'll explain why one piece won second place rather than first. The trial judges can compare their determination of the various winners with those of the contracted judges. At no time are confidential discussions which took place among the judges divulged, even to those trial judges.

Finally, all is finished. The President marks several exhibit brochures, showing the various awards. Each judge signs these marked brochures, indicating that she agrees these were the entries which received the ribbons. The judges leave the exhibit room, each with definite, visible signs of the intense strain they've been under since early morning. The exhibit committee quickly straightens the area, making sure that all is in place. ANG members, many of whom are waiting outside, are permitted to enter. Squeals and sighs, both of joy and disappointment, can be heard. Another seminar exhibit has been reviewed and the best entries determined. The judges' decisions are 'written in stone'.

One of the highest honors ANG bestows is the invitation to judge the seminar exhibit. I feel very humble to have been asked to so serve for three years. That means I've been there to see what goes on, not once but three times, and under the charge of three different ANG Presidents.

I can promise you that the ANG exhibit is fairly, honestly judged. I can promise you that every piece you enter will be carefully, thoughtfully examined. You can be certain that anything you enter will be conscientiously considered, will have a fair chance in competition with other entries, and will be evaluated against a predetermined standard of excellence. You can be sure that all ANG guidelines and policies are strictly adhered to. You can enter the ANG exhibit with confidence, knowing it will be judged under a good, impartial system of determining blue ribbon winners.


ADDENDUM: The preceding article was written in 1993. Just as our world is different since then, so have there been changes to the judging procedure of the ANG Exhibit. Most are not of major importance to those who enter pieces. For instance, the day of the judging has been changed. One item that is important to stitchers who enter is that a critique of your entry is now available for a nominal sum. This is a wonderful opportunity to have a written review of your work prepared by one of the exhibit judges.

Even though some things have changed about our ANG Exhibit in the last nine years, one has remained constant. Anyone entering a piece in the exhibit can enter with confidence. You can be certain that it will be evaluated fairly, honestly, and by qualified judges. Reread the last paragraph in my original article. These are the things which I sincerely feel will never, ever, change about the ANG Annual Exhibit.

The distribution of points within a category has evolved in the years since I wrote the 1993 article. This is the current assignment of points with which each judge evaluated an entry: