|
|
Judging journals instead of articles
mike salovesh (T20MXS1@NIU.BITNET)
Sat, 11 Jun 1994 16:03:00 CDT
Let's take a close, INSIDE, look at why personnel practices are what
they are. (First off, I HATE the way they are, I fight against the
system all I can, and I believe that the whole business stinks. But
what I'm doing here is seeking the underlying rationale . . . )
When NIU hired me, that did it for political anthro and Mesoamerican
social anthro. We already have a guy who does that -- me -- so there
is no sense in using our limited chances for new hires in getting
something we already have.
What that means is that when my record is up for review (annually, in
our system), the reviewers are by definition NOT aware of the ins and
outs of my specialties. How, then, are they to judge what I have
done?
What they COULD do, as I try to do, is maintain at least a surface
familiarity with things outside their own specialties. But then, I'm
an old fogey raised on the four-field approach (I count applied as a
fifth). There was a (15-month) year in which I had publications
accepted in four of the five fields. I started to write an article
having to do with archaeology that year, too, but then took hold of
myself and acknowledged that publishing in all five fields, in this
day and age, was just going too far. So I gave the basic idea in a
meeting paper instead.) What I do is READ WHAT THE OTHER GUY WROTE,
apply appropriate standards to the work itself, and make a pro-
fessional judgment as to the value of the work.
Part of why I do that is because I know the history of some landmark
articles in our field, all published in sources so out of it that
they may never have published anthropology before or since. I think
of Sapir's "Time perspective . . . ", Canadian Journal of Mines;
Radcliffe-Brown's "Mother's brother in South Africa . . . ", South
African Journal of Science; V. Gordon Childe's "The urban revolution"
Town Planning Review. I also think of textbooks, normally out of the
running in brownie points, but then there was Linton's Study of Man
(1936), which established the accepted definitions of status and role
for all of the social sciences. Or book reviews, which are so low on
the evaluation scale that they might as well not exist, but does
anyone NOT know of Chomsky's review of Skinner's Verbal Behavior? If
you haven't read it, stop messing around on the net and go out and
read it right now! (Do that even if you reject both Chomsky AND
Skinner. You're sure to learn something worth knowing.)
But people of newer generations are not likely to have had full
training in all four or five subdisciplines of anthropology. They
feel diffident in trying to judge work outside their own fields, in
part because they also feel inadequate. All right, when you can't
or won't make your own judgments, but you must make a recommendation
anyhow, what's left?
The easiest copout is to substitute something that is supposed to
represent a competent judgment that everybody can agree on. That's
why personnel decisions make a big thing about whether an article
is published in a peer-reviewed journal; as a member of a personnel
committee, you can say "Well, I don't know about this stuff, but it
had a peer review from people who do and they accepted it for publi-
cation. THAT is an objective judgment, and I'll go with it." (The
hell you say!) Publish an article in a book, rather than a refereed
journal, and who knows whether there was "adequate" peer review? It
would be safer to conclude that what goes into a book is a minor
effort, and should not count as much as an article in a refereed
journal. And the more severethe refereeing--that is, the higher the
rejection rate, or the reputed rejection rate since we seldom have
the real figures--the better the resulting rating should be, no?
(Good God, NO!) Any publication that did not go through peer review
at all is equivalent to a puff piece in the vanity press.
Now, let's face it, everybody knows there are "good" journals of
high prestige and not-so-good journals of less prestige and really
bad journals of negative prestige. All right, if we're incompetent
to judge the merit of an individual article, we can substitute the
prestige of the journal for the content of the article. That can
be represented as an "objective" process, besides. (The Econ
department at NIU actually publishes a list of journals, classified
at FIVE levels, and assigns points to articles on the basis of the
level of the journal in which it appears. Individual authorship of
an article in a Level One journal counts 80 points; individual
articles in a Level Five journal count--I forget what, but it is
less than five points. Anything published in a journal that is not
on one of the lists counts two points no matter how important that
source might be in some other field. I might add that the approved
list does not include journals in any language but English, and it
has damned few published outside the US. A total lifetime
accumulation of at least 250 points is prerequisite to tenure and
promotion to Associate Prof, and Full Professorship demands a
total life accumulation of 400 points, as I recall. Note that
books do not count toward accumulated point levels! Not only did
they set up this insane abdication of professional judgment, they
have been following it for at least twenty years. One of their
justifications is that "Well, everybody knows what's expected of
them this way, and we've all accepted this system." The "we"
who have all accepted this system, of course, are the tenured
Associate and Full Profs; Assistant Profs are excluded from voting
on criteria for becoming superior beings.)
I refuse to accept even the suggestion that peer reviewing means
the same thing as quality control. I read a lot of journals. Even
those of highest prestige, with severe peer reviewing, publish an
awful lot of sheer crap. The fact that it appears in CA or AA is
no guarantee that it won't be crap. And everybody knows that, too.
I'll take my own judgment over that of peer reviewing every time:
an article that fails to convince me that it has something impor-
tant to say has to be a failure in its own terms, no matter where
it appears. And an important, history-making, magnificent article
is important, history-making, and magnificent even if it appears in
the Town Planning Review or the book review columns of Language.
But people who feel incompetent to make a professional judgment feel
perfectly happy allowing prestige and peer review of the journal to
substitute for reading the content of an article. Besides, once you
know the rules of the game, you can tailor your behavior to meet its
standards much more easily than you can come up with something new or
important or durable. (Once in a while I set myself the task of
writing an article that exactly fits a genre of things published in
a particular journal, without thinking about content very much, just
to prove that they will publish their kind of stuff even if it's
empty. The first time I tried that I had a $50 bet riding on my
ability to get John W. Campbell, Jr. to publish a letter from me in
the old Astounding Science Fiction. I wrote only one letter; it got
published; the $50 paid for a classic brawl in Osaka back when one
dollar bought a helluva lot of yen.)
As I said, I hate the rules of the game, and I fight them wherever
I can. But they're the rules we play by!
mike salovesh <t20mxs1@niu.bitnet> OR <t20mxs1@mvs.cso.niu.edu>
|