Evaluation of Printed Sources

This article originally appeared in Printed Sources: A Guide to Published Genealogical Records, edited by Kory L. Meyerink.

Introduction

Whatever sources are used in research, it is important to evaluate them to determine if they are accurate, complete, correct, logical, and reasonable. While all sources need to be evaluated carefully, the various printed sources have much in common, and there are several general principles to keep in mind when using them. After an overview of general evaluation concepts, this section will discuss the key ways of evaluating the books themselves and their contents.

General Evaluation Concepts

In the process of genealogical research, nothing is more important than the accurate evaluation of the evidence found during the research process. Evidence is one or more statements of fact or pieces of information that appear to pertain to the subject of research. Genealogists often use dozens of sources, each with many pieces of information, as they seek to solve research problems and establish proof of relationships. However, not every source or piece of evidence carries the same weight. Some information found during research is simply more likely to be true than other information. The problem many researchers face, especially beginners, is identifying and understanding the many aspects of each record that must be evaluated.

Unfortunately, the terminology often used to describe a record's reliability is so general that many of these aspects are overlooked, greatly affecting the way a record can be judged and the information it supplies. A more careful description of the different aspects of evaluation allows for better teaching of the principles of evaluation; this, in turn, will lead to better proof of genealogical relationships.

Many researchers find it difficult to comprehend the differences between primary and secondary sources. Some have thus suggested that artificial descriptions of records that serve no real purpose in evaluating genealogical facts or conclusions should be discarded (Drake 1992, 12). The real solution, however, as noted in the previous sections, is to use clear and meaningful definitions. To divide the world of genealogical sources into categories and use descriptive terms for those categories is worthwhile because it allows discussion of the various aspects of evaluation. Furthermore, such descriptions enable teachers and writers to better instruct new genealogists about how they should weigh their findings.

As mentioned in Origin of Information, there are two key issues: the nature of the records themselves (how they were created) and the origin of the information in the record. Simply categorizing a record as primary or secondary avoids the question of how good the information in that record is. Again, a death certificate is an excellent example'ˆmuch of the information in such a record pertains to events that happened years before the event of death.

But it is not enough even to determine if the book is original or compiled, or if the information is primary or secondary. There are at least seven separate aspects of each source to consider when evaluating research findings: the relevance of the record; the origin of the information in it; the nature of the record itself; the record's format; and the evidence, facts, and events given in the record.

Relevance of the Record

It is first necessary to determine, if possible, if the record pertains at all to the family of interest. For example, a marriage record from the right place at the right time is not necessarily that of the husband and wife being sought. Be especially careful when researching common names in densely populated areas. Very seldom are all of the people with the same surname related to each other, and it is very common for persons of a similar age with the same names to be living in the same vicinity. It can be helpful to review other records of the locality to learn just how common the names are in that area. The assumption of right place, right time, right name has probably led to more erroneous connections than any other aspect of record evaluation.

Sometimes the decision about how relevant a record is must be delayed until further evidence is available. It may be possible to make that decision only after all of the potentially applicable records have been evaluated.

Origin of Information

As discussed in Origin of Information, the information in a record should be considered as primary or secondary. Most records contain both primary and secondary information. For example, the 1880 census identifies relationships at the time of the census; that can be considered primary information. However, the age of the adults in the same census is only secondary information, as is the birthplace. Be sure to identify what information was recorded at or near the time of the event (primary) and what was recorded much later than the event described (secondary).

Nature of Records

The nature of the record (that is, how it was created) is a consideration separate from the information in the record. A compilation of marriages taken from original county records still presents primary information; however, the nature of that record (a compilation) allows additional errors to occur. Here is where a consideration of the categories and formats of records is useful in the evaluation process.

As mentioned earlier, genealogical records (those which provide information about individuals, such as names, dates, or relationships) comprise two categories. Original records are those created to record certain events. They are generally created at or near the time of the event they record, and are usually the earliest record of that event. Thus, a baptismal record is an original record, as are obituaries, military pension papers, business account books, and even city directories and most newspaper articles. Every original record contains information written directly into the record by someone who was in a position to have accurate (often firsthand) knowledge of the information being recorded.

Compiled records (sometimes called records of previous research) represent gatherings of information from one or more additional sources (original records, other compiled records, or both). An important characteristic of a compiled record is that its author or compiler interpreted information found in other sources. Examples include family histories, biographies, and local histories. Many, but not all, compiled records are in the form of published books.

These two categories must be evaluated differently. Original records require that the researcher attempt to determine if the information in them was recorded correctly in the first place. When working with compiled records, the researcher must determine if the compiler found all applicable source records, analyzed them correctly, and came to valid conclusions.

Kinds of Formats

The format of a record(actual records versus various kinds of reproductions) is also significant when evaluating the evidence found therein. Recording errors can appear in any source, but the nature of errors changes with the format. The actual document (either an original or a compiled record) is often available to the researcher. Virtually as good as the actual document is a photographic copy; these include microfilm, microfiche, and photocopies. Either form represents the record as it was first made; any errors are the fault of the person who made the record. Copy errors can be introduced if the document is transcribed, extracted, or abstracted, an important consideration in evaluation. If such a copy is a printed copy, the genealogist must account for possible copyist and typographical errors. A manuscript copy can contain copyist errors, but there is also the possibility of misreading the handwriting in it. In brief, the further removed the copy is from the actual document, the more errors are likely to have been introduced.

Directness of Evidence

Evidence is the statement(s) of fact made in a record, or the interpretation of the facts in a record. A direct statement specifically states a fact, such as the date of death. An indirect statement (often called circumstantial evidence) reasonably implies a fact (Stevenson 1979, 181, 186). For example, a marriage record is direct evidence that a person was born but only indirect evidence of the time of birth, as the person may have been born sixteen or sixty years before the marriage. Genealogists usually prefer direct evidence because indirect evidence may be interpreted differently by others.

These first five aspects of evaluation include judgements that can be made by examining the single record (or source) alone. The last two aspects include the necessity of judging the information within the record (1) against itself (internal evaluation) and (2) against information from other sources (external evaluation).

Consistency and Clarity of Facts

Every record includes several facts (or alleged facts). The researcher must judge these facts in conjunction with other facts that may be in the same record or in other records.

First, the facts must be consistent with other known facts. For example, if a child's birth record shows that she was born a year after the known death of the father listed on the birth record, there is a serious disagreement that must be resolved before that record can be accepted as proof of the child's paternity. If the facts in a record conflict with other facts, determine which information, if any, is primary and, if so, if it was recorded correctly.

Second, are the facts clearly stated and easy to interpret? A record may contain difficult-to-read handwriting. In addition, many words were spelled differently in earlier years, and those varied spellings may obscure the meaning. Some records use obscure language, and the meanings of some words may have changed; legal terminology and the formal language used in some records, such as wills and deeds, are examples. Foreign-language records can also be difficult to comprehend. There are four elements to observe when evaluating the clarity of the stated facts:

• Names. Be sure to read them correctly and that any spelling variations are acceptable. Determine if they follow appropriate naming patterns, if any.
• Dates. Interpret the style of dates correctly. If all-numeric dates are used, determine if the month or day is listed first. Learn which calendar (Gregorian or Julian) was in use at the time.
• Places. Among the most difficult parts of a record to read are place names. Often they are unfamiliar, and they may be spelled incorrectly. There may be more than one place with the same name, and a city and county may share the same name.
• Relationships. A record may state relationships directly or may only imply them. Previous relationship terms, such as cousin and in-law, had different meanings at one time then they do today.

Likelihood of Events

Evaluation requires that the researcher consider whether the events, as shown in the record(s), really could have happened. Some events are less likely than others, such as an ancestor joining the military at the age of ten or twelve, being born on the father's birthday, or a probate inventory showing a considerably larger estate for a person than tax lists or census records indicate. For most situations there is a continuum of probability ranging from very likely to highly unlikely to impossible. This continuum will change with different cultures and time periods.

Evaluation and Proof

Every record can be evaluated individually based on the seven aspects described above, but proof is the accumulation of acceptable evidence. The judge (that is, the researcher) is responsible for determining if the accumulated evidence represents clear and convincing proof of a genealogical fact (Drake 1991, 153 and Devine 1992, 131).

When evaluating evidence, it is important to realize that original records are not inherently better (or worse) than compiled records. However, they do need to be evaluated differently. The same is true of the information contained in a record. Primary information is not necessarily more correct than secondary information. Every genealogist knows of cases where clerks made mistakes in recording events in original records. Such errors are sometimes corrected when a compiled record is created, making a better record of events and relationships than the originals they are based on.

The Book As a Product

While the above evaluation guidelines are useful for any genealogical source, printed sources provide additional elements to evaluate. One often-overlooked element is the book itself. Examine the book as a product. How good is the binding? Was the text composed professionally (typeset) or is it a typescript of uneven quality? Is there a preface, and how long is it? Is there a foreword and/or introduction? How large is the index in relation to the text? (Many people feel that the ratio should be relatively high for genealogical books. Many printed genealogies [family histories] have one index page for every ten [sometimes fewer] pages of text.) Is the publisher's name known in genealogical or research fields'ˆor does the book appear to have been self published? What year was it published? Theories may have changed or been disproved in the years since it was published, and new sources that the book does not discuss or the compiler did not have access to when preparing the material may be available.

Look for other items, such as consistency in the style and formatting of the text and of bibliographic citations throughout the book. Are there misspellings? (It is difficult to have much confidence in a book that spells the name of the avocation as geneaology, yet that mistake is not unknown in published books.) These elements suggest the presence or absence of a careful, competent editor. Although no editor is perfect, and there will be mistakes, try to determine the amount and nature of the mistakes in judging a book.

A negative answer to any of these questions does not necessarily mean that the content of the book is useless. However, any concerns raised by them should make the researcher stop and ponder the nature of the book. Good material has been produced on a shoestring, especially in genealogy, but bad material almost always is. On the other hand, a well-produced book by a well-known publisher may not live up to expectations. For example, when the American Library Association published Lois Gilmer's Genealogical Research and Resources: A Guide for Library Use (Chicago, 1988), the book was well received by many libraries and even received good reviews because of its publisher's reputation. However, the information in it was poorly arranged, outdated, and often of questionable value.

Don't place too much credence on the quality of a book's production. Gold edging and leather bindings do not mean the content is without error. Rather, they suggest that the person financing the publication had substantial financial resources. Generally speaking, however, poor production quality often means poor quality of content.

Content

The content of a book can be even harder to judge than the book itself, especially for the person who is not very experienced in genealogy. There are, however, some aspects to pay attention to. Often the researcher will be greatly rewarded for taking the extra time to learn about the author and the subject in order to evaluate a printed source.

The researcher should begin with his own knowledge of the subject. Seldom will there be a complete lack of familiar information in a new source. Regardless of how new a researcher may be to the subject, she brings some knowledge that can be compared with she reads in a printed source. This author often forms part of his judgment about a dictionary or encyclopedia by its treatment of his state or occupation'ˆboth of which he is very familiar with. If the work is careful and balanced on those topics, he feels more comfortable that it has treated other topics carefully. Maps and atlases can be easily evaluated by the detail and clarity with which they treat the location where the user lives or grew up.

Family histories are often taken at face value because they are used to extend ancestral lines, even if their users don't know anything about the families that had not been on their pedigrees before finding the book. Look carefully at the information for a family or generation that is familiar. Is it complete and correct? Use compiled records like family histories only to guide research. Take the time to trace a few families in original sources, such as census or probate records, and note whether the published book gives information that matches period documents.

Evaluate the Author

Evaluating an author may be difficult if he is unfamiliar to the researcher or if a publication is his first. Often the book itself will indicate something about the author, such as his educational level, profession, prior publications, or other achievements. Obviously the book will paint a very favorable picture of the author, so read past the glowing comments. Does it really matter that she is the mother of six children or descends from the kings of England? Do his awards and achievements have anything to do with the subject of the book? While achievements may indicate a certain amount of mental ability or research acumen, does being an army colonel have much bearing on his ability to link families together?

Learning an author's qualifications from outside sources can be difficult. Lists of genealogists who have met certain standards are available from several organizations, such as the Association of Professional Genealogists, The Board for Certification of Genealogists, and the Family History Library of The Church of Jesus Christ of Latter-day Saints (LDS church). Who's Who in Genealogy and Heraldry provides some information about the training, credentials, and background of the persons listed in it. Ask other researchers what they know about an author, and check library catalogs for other books written by her.

Deceased authors can be more difficult to evaluate. Notable authors are often eulogized in the periodicals of the societies they were most closely associated with. Such items are seldom very critical; however, they will attest to the author's activities, and generally only the most competent authors are eulogized.

A deceased author might also be easier to evaluate. The greater the distance between the user of a source and the writer, the easier it is to be constructively critical. (Death certainly places some distance between the reader and the author.) Many authors will have published several works and have a reputation in the field. Therefore, ask colleagues for their opinions. Briefly study an author's other works. Were they carefully compiled? Are sources cited? Are the works generally thorough?

Read Book Reviews

Seek reviews of the book being evaluated. Unfortunately, most book reviews are actually just notices and do not evaluate the book or author in any depth. Even critical reviewers tend to be charitable; their intentions often seem to be to avoid discouraging genealogical publications, because their authors are rarely compensated very well. It is wise not to base an opinion solely on book reviews.

Evaluate Sources

Determine what the author's sources were. There should be a bibliography or source list, usually at the end of the book or at the end of each chapter. Determine if the author used a variety of sources. Did she use original records extensively, or do compiled records dominate? It may be impossible to compile a family history using only original records, but one that uses only compiled material cannot be relied upon as being as accurate as possible. Did the author use more than one source to support each statement made? Reliance on only one source for any point of fact shows an ignorance of research methodology.

Did the author find any contradictory information? If so, how did he deal with it? It is most unlikely that a compiler will not find some facts that conflict with others. Be sure that the author presented evidence that is clear and convincing. Find out how the author collected her data. Did she travel to the place where the events took place, and, if so, were the records she needed still on location'? With the possible exception of using the Family History Library in Salt Lake City, it is very difficult to prepare a family history, or any of several other types of printed genealogical sources, by researching in only one location or institution. If the author corresponded for information, how good were his correspondents? Did he use the original sources, or abstracts, extracts, and transcripts? Often the writer of a history will use whole sections from an earlier history, frequently without acknowledgment. Today this practice is called plagiarism, but in earlier years it was the norm. One way to evaluate sources is to compare the information with that in other printed sources that deal with the same subject, such as a county's history or a family genealogy.

Read the Preface

The author's preface is the key to evaluating any printed source. There she usually reveals why and how she created the book. Thus, the preface may be the most important part of a book (but it is also the most overlooked section). Sometimes the user has to read between the lines, especially if the author is not very willing to reveal his process. The lack of a preface, or a very brief one, suggests that the book was put together in haste, or at least without due consideration of the reader, which may well imply carelessness.

It is important, for example, when reading a book of cemetery inscriptions, to know when the transcriptions were done and by whom. What kind of experience did the person have? What were the weather conditions? Did she carefully clean the hard-to-read stones or just guess at the inscription? Are there other, earlier transcriptions? Are sextons' records available? Were the complete inscriptions copied, or only the genealogically significant material? (This irritating phrase begs the question, Who determines what is genealogically significant?) Why are the names arranged the way they are in the book? Are they arranged alphabetically (ideally not), or by cemetery row and plot? Is there an index and, if not, why? Answers to such questions should be found in a preface.

Read Other Introductory Sections

The preface is usually written by the author(s). However, there are other parts of a book that will help in evaluating the content. The foreword is usually written by another person, often knowledgeable in the field, who the author hopes will lend credibility to the book. The comments will, of course, be positive, but a careful reading will nonetheless reveal more about the book and the author(s).

The introduction may provide information that the preface did not. Like the preface, it is usually written by the author(s) and seeks to explain the book. Even the acknowledgments can be helpful. They help the reader learn who else was involved with the book. Often a teacher, leader, or other mentor may be mentioned, from which something about the author may be inferred. What about assistants? What roles did they play? How much guidance did the editor or publisher give?

Check Publication Information

In evaluating the content of a book, look for other clues. How often has it been reprinted? How many copies were produced? (The publisher may be able to provide this information.) These facts give some clue as to the book's popularity and thus the opinions of other users. Although poor books have been reprinted, such situations are relatively rare. How recent is the edition? Try to use the most recent one. Although a first edition is valuable to book collectors, recent editions suggest changes in the text; they might only be updates, but they may also be corrections. A writer rarely revises a book unless she thinks that she can improve it thereby. Revisions often reflect new thinking, new theories, or even additional proof that the author has found.

Judge the Length

How long is the book? How does it compare in length to other books that cover the same subject? A book's ideal length may be hard to judge, but a brief discussion of genealogical how-to books will illustrate this point.

This author would be very uncomfortable with a one hundred-page booklet on research methodology that claimed to be all the beginner needs to know.' (It may be all the beginner wants to read, but he certainly needs to know more.) Many of the topics discussed in Printed Sources were chosen because relatively little has been written about them. How can this so when hundreds of different how-to books have been written on genealogy? A survey of the literature revealed that many books say the same things, only with different wording and arrangements. Certainly some new sources have come to light and many new indexes are now available, but research principles stay the same, and major record sources have not changed.

Interestingly enough, four of the most widely used and accepted introductory books were first published between sixty-seven and twenty-four years ago (Genealogy as Pastime and Profession, 1930; Searching for your Ancestors, 1937; Genealogical Research: Methods and Sources, 1960; and The Researcher's Guide to American Genealogy, 1973). They have endured, usually in several revised editions, while others have come and gone. Each takes a slightly different perspective and serves a different audience, but they do their jobs well. Yet, at least one hundred other such books have been published. All of them devote a similar amount of content to the major sources, such as census and probate records, and less on other sources (such as tax records). Not until The Source was first published in 1984 were original sources treated in greater depth than they had been in dozens of earlier books. Therefore, the size of The Source (846 pages in the current edition) becomes a measure of its usefulness. Its size equates to greater discussion than ever before of many common sources, and a focus on other sources (such as institutional and business records and city directories) that had seldom been treated in similar works.

In considering some of the topics for Printed Sources, potential users were surveyed, as were previously printed books. Most of the compiled sources treated in part 4 have previously received only brief mention in most books (including The Source). Compiled biographies, for example, received no more than two or three pages of explanatory text in any of a dozen books reviewed. Often only a few paragraphs were used to describe this diverse and sometimes difficult type of source. The same was true for local histories and genealogical periodicals.

This discussion serves to illustrate that length can be a method of evaluating a book. If a book is brief, does it add to the field? Evaluating a book based on its length requires that the researcher become familiar with the topic of the book she is evaluating, or at least that it be evaluated with other books in hand. A bibliography with fifty entries has to be judged against a similar bibliography covering the same topics with five hundred entries. There may be a purpose for the shorter bibliography (longer annotations, narrow selection criteria), but it should be justified.

Unfortunately, it can be difficult to find much of the information needed for evaluation within the book itself. Dr. Horle expresses his concerns about genealogical publications:

Far too often, the reader is not told the qualifications of the transcriber, the repository of the original record, or the methodology used in the edition. Rarely do transcribers provide tips about dating of documents where appropriate, that is, whether the Julian or Gregorian calender was in use, nor do they indicate whether the transcription was made from microfilm or from the original manuscript. Similarly, with reprints of older works, a new introduction should be added that discusses the work itself . . . (1992, 218).

It is important, when evaluating a printed work, to take the time to carefully examine the book before accepting its assertions. Then, check other sources (reviews, other researchers) to learn about limitations of the book. The dozens of how-to books mentioned above are surprisingly brief on evaluation (hence this discussion). For further ideas about evaluation, two books stand out: Family History for Fun and Profit (formerly Genealogical Research: A Jurisdictional Approach), by Vincent L. Jones, Arlene H. Eakle, and Mildred H. Christensen (Salt Lake City: Genealogical Institute, 1972), and Genealogical Evidence, by Noel Stevenson (Laguna Hills, Calif.: Aegaen Park Press, 1979).