chapter 6

The VITEK assessment is intended to provide a quantitative measure of the vital status (retention/change) of TEK that is both (potentially) applicable at a global scale and also appropriate and representative of this dynamic process at the local scale. This means that the assessment method design must be sufficiently comprehensive so as to permit replication and comparison across diverse cultural and ecological contexts while at the same time specifically tailored to the local categories, priorities and codes of conduct held in a particular place. A primary conclusion of the literature review is that the development of the global indicator cannot be based on existing data sources because of the lack of uniformity and comparability between them. The differences in topical focus, field methods, sample design, statistical operations, and data sets are simply too great to expect that they can somehow be converted into a single type of measure. The only viable strategy, in our opinion, is to start from scratch as it were and begin the indicator with the collection of primary data in the field. The goal of producing comparable measurements necessitates the elaboration of a standardized methodological framework. In this section, we elaborate a uniform protocol (i.e. steps and procedures) for collecting data in the field and converting them into quantitative measures. Besides comparability and appropriateness, the main criteria used for the VITEK method design are: reliability, robustness, data quantification, aggregability of measures into larger units of analysis, simplicity of use, time efficiency, low cost, and easy reporting. We believe that these parameters offer the best chance for reaching a workable and meaningful indicator that gains a wide level of acceptance and sustained use over time.

Although emphasis is placed on the importance of a uniform protocol, at the same time we are fully cognizant that no two field situations are alike and that some flexibility will have to be incorporated into the research design in order to achieve global applicability. Flexibility is built into the method in two main ways. First, we maintain a flexible approach to the procedures used to collect the baseline data from which test items will eventually be drawn. The protocol is intended for use and application by multiple other actors who have an interest in assessing TEK vitality and change, such as public agencies, conservation organizations, scientific researchers, and local communities. It should be expected that the specific objectives, means, and criteria of the diverse parties, as well as the situations they encounter, will vary somewhat and therefore exact procedural replication in every place will not be realistic. While we make specific suggestions in regards to the methods that can or should be used to complete different steps, we also allow the field research team some room to either follow them or choose an alternative which will lead to a similar result. We particularly stress the use of participatory rapid appraisal (PRA) techniques, such as focus group discussions and local consultant-generated data sets, because of the general ease, efficiency, emic faithfulness and open-endedness associated with these procedures. At the same time, we recognize that more sophisticated (and costly) techniques, such as informant-indexing measures, cultural consensus analysis, principal components analysis, and quantitative behavioral sampling, may be used as long as the goal is to produce the same kinds of data. Thus, the method is designed to be flexible in a procedural or tactical sense but without compromising the overall structural design. Second, we admit some flexibility with respect to the data coverage. We propose that a core set of domains or topics of data be documented and tested, but we also recognize that it will not always be possible to include all of these either because of non-applicability (e.g. agricultural knowledge among forager groups), cultural privacy sanctions (e.g. taboos on talking openly about ethnomedical practices), or absence or reluctance of key informants.

The proposed VITEK protocol is described and justified in the following sections. The purpose here is to lay out the different steps that should be followed to carry out the assessment. These are outlined as follows:

1. Project description and previous informed consent
2. Constructing the Data Register
   Defining constituent domains of TEK
   Making an inventory of TEK Items
   Assigning weights to domains and categorical items
3. Testing Instrument
   3.1. Test design
   3.2. Sample population selection
   3.3. Test administration
   3.4. Test evaluation and scoring
4. Vitality Index
   4.1. Calculating the Vitality Index
   4.2. Significance Tests
   4.3. Vitality Index Aggregation and Disaggregation

6.1. Project Description and Previous Informed Consent

The VITEK design contemplates the collection of primary data in the field and therefore its implementation should comply with standard ethical and legal codes of conduct related to such activities. As with other kinds of human subjects research, the first step is to inform the participating or candidate communities about the scope and aims of the research operation and obtain their prior informed consent (PIC). Ideally this should be done by both written and oral forms of communication. A written statement should be prepared in which the nature, purpose, methods and procedures, estimated chronology, participating individuals or groups, financial support, confidentiality provisions, possible risks, benefits, and foreseeable outcomes of the project are clearly explained. It is highly recommended that the project description also be communicated orally in the context of a public meeting or some other locally acceptable venue, thus giving the local people a chance to raise questions, express their concerns, negotiate the terms of their cooperation, or decline to participate. In situations where all or some community members speak a local language as their first language, a competent translator should be used to enable effective communication. If the community agrees to participate, they should be asked to sign a voluntary PIC form or to provide some other written notification indicating their consent, such as a letter signed by the community leader(s) or various members. PIC should also be obtained in a similar manner from all individual participants whose knowledge is recorded or tested as part of the project. Although the VITEK is not intended for any sort of commercial application, nevertheless it is recommended that the PIC documentation include contractual agreements specifying rights and responsibilities in regards to intellectual property, benefits sharing, information disclosure, access and use of genetic resources, or any other topic having legal implications. In some places, research permits will have to be obtained from local or national governmental authorities. Established procedures for securing the necessary permissions should be duly followed.

Once the necessary permissions are obtained, the local research team in consultation with community leaders and members should devise a plan and schedule for carrying out the assessment process. The protocol described here stresses a participatory approach in all phases of the process and therefore we strongly recommend that one or more members of the community be incorporated into the research team and trained in all methodological aspects, from data collection to analysis, management and reporting of the results. One of the objectives of the initial assessment is to build local capacity for repeating the assessment test at future dates and using this information for their own purposes.

top

6.2. Constructing the Data Register

The first step toward assessing TEK vitality is to construct a data register or inventory of conceptual categories and items that represent a significant portion of the local body of knowledge. The register will comprise the baseline data from which a testing instrument for measuring TEK status and trends will be developed. The sheer size and complexity of any single knowledge system is so vast that it is likely impossible to achieve a complete inventory of everything everyone knows. Plus the time and cost needed for the assessment should be minimized if the objective of wide application is to be attainable. In view of these limitations, the register will have to consist of a mere sample of key categories and items. The purpose of the register therefore is not to document the full richness and depth of the knowledge system but rather to identify certain representative elements of it upon which the test and measurement will be based. Such elements should serve as key indicators of the larger system of which they are a part.

The basic method for constructing the data register relies primarily on rapid participatory appraisal techniques, especially the consensual consultation with local group members in the context of collective gatherings. Exploratory consulting events can range from larger public meetings to smaller focus group discussions to reviews by a locally designated "council of experts." Although this technique may not be as systematic or controlled as structured surveys or interviews (see sections 4.3.1 and 4.3.2), we consider that the advantages outweigh the disadvantages given the constraints mentioned above. It is more participatory, interactive, and dynamic in the sense that it permits iterative discussion, feedback and negotiation among the participating consultants. It is relatively fast and efficient in that it bypasses a laborious phase of interviewing numerous individuals and performing complex statistical analysis of their answers. It prioritizes local criteria by letting the consultants themselves to decide directly what categories and items are correct, representative, and relevant. Although this method tends to bring out more generalist as opposed to specialist type knowledge, the latter can be incorporated by supplementing the larger meetings with smaller focus groups of resident experts or even individual interviews with recognized experts if they cannot be brought together. In any case, we recommend that the consultation process consist of triangulation of different meetings with different groups. In particular, separate consensus groups by gender will be needed because of the common tendency of women to defer to men in public arenas in many societies and because the test design depends on this division (see sections 6.2.1, 6.2.2, and 6.3).

6.2.1. Defining Constituent Domains of TEK

Even though TEK is frequently portrayed as holistic and totally integrated with virtually all aspects of daily life, empirical investigations of numerous cultural systems of knowledge and practice over the past half century have quite convincedly demonstrated that it is highly structured and organized. As with other aspects of cultural creation, TEK is structured at cognitive and behavioral levels into distinct domains (or subsets) and these may be further differentiated into smaller subdomains and so on. For example, the classification of environmental phenomena can be divided into the domains of plants, animals, rocks, soils, landforms, vegetation types, land use types, and weather patterns. The plant domain can, in turn, be further divided into more circumscribed domains according to some shared distinctive feature(s) of meaning, such as: morphological aspect (e.g. tree, vine, herb, etc.), use category (e.g. food plant, medicinal plant, etc.), or habitat (e.g. garden plant, forest plant, etc.). In a similar vein, subsistence practices can be analyzed in terms of distinct activity sets, such as farming, hunting, fishing, food preparation, curing, toolmaking, etc. Even while the boundaries between domains are not always clear and one may point to multiple connections between them (e.g. the dual significance of many plant species as food and medicine), it is still possible to discern the reality of different domains in thought and practice. Many of the ethnoscientific methodologies developed for the study of TEK phenomena (described in section 4.3.1.1) are, in fact, based on the implicit assumption that structured domains are psychologically real. The domain structure of the knowledge system is a fundamental property that needs to be taken into account if the assessment is to be representative. However, a complete inventory of all knowledge domains is impractical (if not impossible) due to time and cost constraints and therefore limits will have to be set on the total range of possibilities (see below, same section).

As mentioned above, the primary challenge for developing a globally aplicable, locally appropriate indicator is to strike an effective balance between universal and culturally-specific formulations of knowledge. The framework for domain selection will be crucial for achieving this balance. Some cultural domains are purported to be universally recognized (e.g. ethnobiological taxonomies, food taboos) while others are very restricted or culture-specific (e.g. Micronesian navigation techniques). However, this distinction often boils down to differences in the level of abstraction at which the category is defined. Our solution to this problem is to define relevant domains of TEK at two basic scales: cosmopolitan and local. The cosmopolitan scale refers to domain and subdomain categories that are loosely recognizable in many different cultural and ecological contexts around the world based on our reading of the TEK literature. Some of these may be truly universal in the sense of being found everywhere (e.g. plant and animal taxonomies) while others are preponderant but not universal (e.g. soil classification, which may be absent among nonagricultural groups). In any case, it should be understood that the domains identified at this scale are defined according to a global or intercultural frame of reference, are culture-free to the extent that is possible, and therefore are very broadly conceived. At the local scale, we refer to culturally-specific semantic domains and categories as defined and recognized by the local group. The testing and measurement of TEK must be based on emic (i.e. insider) categories of the knowledge system in order to be representative and reliable. It is expected that not all of these categories will be directly comparable across groups nor will there be a perfect one-to-one correspondence to domains defined at the cosmopolitan scale. Therefore a clear procedure for relating one scale to the next is required.

The VITEK method involves preselecting a general set of domains (i.e. the cosmopolitan domains) as a first step and then adapting this set to specific locally recognized domains through consultation with local group members. The process of consultation entails reviewing the list of preselected domains with a sample or group of community members and recording the closest equivalent local term(s) or concept. If no verbal equivalent can be determined, the research team attempts to confirm whether or not the category is somehow salient and understandable for them. In order to balance efficiency with accuracy, we recommend that the consultation process be carried out with one or more focus groups of locally-elected or recognized TEK experts. A more time-consuming alternative would be to conduct the reviews with individuals and then take their consensus opinions. Although we consider that adequate translation from one level to the next is indispensable, we also maintain a flexible position with regard to how this is done.

To allow for the fact that some domains of knowledge and practice are sharply divided along sexual lines, another key aspect of the domain verification and adaptation process is to record which domains are considered to be male- versus female-dominated or are not gender-differentiated. This distinction will be important for constructing appropriate tests for each group. Categorical items within each inclusive domain should also be classified in these terms. To accomplish this task, we recommend that both men and women be included in the initial group consultation process. The results should then be confirmed (or modified) through a second consultation phase involving men-only and women-only focus groups.

The set of cosmopolitan domains is presented in table 6.1. Given the need to establish manageable limits on the size and scope of the indicator, the list gives priority to those which are most closely related to biodiversity, the appropriation and utilization of natural resources, and culturally significant elements of the environment. These criteria are determined by the VITEK's overall goal of providing a cultural indicator that can used and integrated with other indicators of biodiversity to advance global environmental assessment and policy-making. Additional criteria for the selection was broad-based (i.e. multi-site) treatment of these topics or related topics in the empirical TEK literature and demonstrated use of structured methodologies for their study. The resulting list is therefore intended to cover some of the major areas of TEK that have been documented by previous work in this field and that reflect the unique relationship between diverse human groups and their habitat, including knowledge and practices dealing with biological entities (plants, animals, biotic communities), ecological relationships, soil, climate, and land. The organization of the list is explicitly hierarchical and consists of four distinct levels. From higher to lower inclusive levels, these are: component, primary domain, secondary domain, and terciary domain. The terminology given here is arbitrary and merely employed to be able to distinguish clearly one level from the next. This is important because the hierarchical domain structure is maintained throughout all phases of the test design and index calculation (sections 6.3 and 6.4).

The first level of the domain classification made here subscribes to the basic distinction made by Reyes-García et al. between theoretical knowledge and practical knowledge although we refer to the former as conceptual knowledge and the latter as practical skills. Conceptual knowledge is understood here as "know-what" (i.e. referential knowledge about the world encoded in abstract mental concepts) while practical skills is essentially "know-how" (i.e. performative knowledge embedded and expressed through concrete behavioral activity). While this dichotomy may not correspond to any locally recognized formal classification, we feel it is justified by the generalized findings that ideas do not always match actions and that the learning processes and responses to change involving these two forms are somewhat different ( section 4.4.2). We have chosen to refer to this level of categorical contrast as "components" to reflect the fact that it is a distinction made entirely by an outside observer and probably has no conscious meaning for local actors whereas we are expecting that the "domains" identified here should often (but not always) have local salience.

Table 6.1. Cosmopolitan Domain List

I. Conceptual Knowledge

     1. Plant domain
          a. taxonomic names and identifications
          b. cultural use or significance
               i. edible
               ii. medicinal
               iii. construction
               iv. technological
               v. fuel
               vi. commercial
               vii. ornamental-artistic
               viii. spiritual-ritual
               ix. other
          c. characteristics (e.g. morphology, behavioral habits, life cycle traits, habitat)

     2. Animal domain
          a. taxonomic names and identifications
          b. cultural use or significance
               i. edible
               ii. medicinal
               iii. labor
               iv. technological
               v. fuel
               vi. commercial
               vii. ornamental-artistic
               viii. spiritual-ritual
               ix. other
          c. characteristics (e.g. morphology, behavioral habits, life cycle traits, habitat)

     3. Plant-Animal Relationships
          a. type of relationship (e.g. food source, shelter, protection, dispersal agent)
          b. effect of relationship (beneficial/harmful/neutral)

     4. Biotopes/Landscape units
          a. names
          b. characteristics (e.g. elevation, topography, edaphy, architecture, indicator species, disturbance agents, etc.)
          c. cultural use or significance

     5. Soil domain
          a. names
          b. characteristics (e.g. color, texture, fertility)
          c. cultural use or significance
          d. crop suitability

     6. Climate domain
          a. elements (e.g. temperature, precipitation, wind)
          b. seasonal periods and indicators
          c. seasonal activities

     7. Ethnogeography
          a. place names
          b. location
          c. cultural use or significance

II. Practical Skills

     1. Primary resource production or procurement
          a. agriculture
          b. herding
          c. hunting
          d. fishing
          e. collection

     2. Food preparation or processing

     3. Ethnomedical preparations or applications

     4. Craft and tool making

     5. Architecture and construction

The intended use and purpose of the list of cosmopolitan domains in the VITEK is twofold: (a) as a guide or parameter for establishing the selection of local domains and (b) as a set of analytical categories for assessing the variable rates of change of different types of knowledge. As a guide for local domain selection in preparation of the test instrument, we present it as a menu of possibilities. The list may have to be reduced according limiting factors of the local situation encountered. Alternatively, the list may be expanded to address other needs or interests, such as broader testing aimed at identifying special needs prior to implementing ethnoeducational programs or more in-depth testing focused on specialized research questions. But for purposes of the indicator development, it is important to include as many of the core set of domains defined here as possible. As an analytical tool, one of the main reasons for adopting this two-step approach is to permit the disaggregation of different domains or types of TEK and their direct comparison across groups and places. This feature of disaggregative comparability enhances the analytical value and power of the indicator. It can be used to identify what types or areas of TEK are more vulnerable or conservative in the face of change and to test more detailed hypotheses regarding the covariation or causal relationships between different knowledge domains and between domains and change indicator variables (e.g. whether knowledge of biotic communities is dependent upon taxonomic knowledge; how market integration affects agricultural knowledge; see section 6.4.3).

6.2.2. Inventory of TEK Items

After the local domains are established, the next step is to make an inventory of the segregate categories or items pertaining to each one. Given time and cost constraints, it is probably not realistic to attempt to do complete inventories. For example, plant and animal taxonomies can each total several hundred and would require specimen collections and extended interviewing, and thus could take a very long time period to carry out. Therefore we recommend that this exercise be limited to compiling a list of culturally salient and representative items with an upper range limit of 50-100 items per domain. These limited inventories would focus on the most important taxa or categories according to local criteria and they would provide the basic sets of TEK units from which test items are drawn. Once again, we consider that this task is most efficiently carried out, at least initially, in the context of expert focus group discussions although other formats can also serve this purpose. Male and female experts should be consulted separately to come up with gender-specific inventories.

The information collected about each categorical item will vary according to the domain under consideration. For plant and animal domains (I.1. and I.2. in Table 6.1), the local names of taxa can be elicited by the free-listing technique (section 4.3.1.1) with the qualifier that the exercise focus on the organisms that the group (or individual) considers to be most important for their traditional (or current) lifestyle. Every possible effort should also be made to record the scientific name(s) for all of the local taxa in order to clarify their referential ranges and limits. This may require specimen collections (especially in the case of plants), in which case the collaboration of biotaxonomic specialists may be needed, or the use of printed descriptions or illustrated field manuals to make positive identifications. The cultural significance or uses of ethnobiological taxa constitute an important set of subdomains, or secondary domains, that should be incorporated into the VITEK calculation. For plants, nine general use categories, or terciary domains, have been preselected: edible, medicinal, construction, technological, fuel, commercial, artistic-ornamental, spiritual-ritual, and other. For animals, the general use categories are: edible, medicinal, labor, technological, fuel, commercial, artistic-ornamental, spiritual-ritual, and other. Another related field of information deals with characteristics of each taxon, including morphological traits, behavioral habits, life cycle, and habitats.

Plant-animal relationships (I.3 of Table 6.1) can be investigated by the adapted version of the pairwise comparison technique pioneered by Atran and his associates. Because the total number of plant-animal pairs could be extremely large (e.g. 50 plants x 50 animals = 2,500 plant-animal pairs), we recommend that this line of inquiry be limited to no more than 20 plants and 20 animals (i.e. 400 possible relationships). For each pair, two basic kinds of information would be elicited: the type of relationship (e.g. Does plant x give food/shelter/protection to animal y?) and the effect of the relationship (e.g. Does animal y harm/hurt/not affect plant z?). This set of questions should asked for the plant as well as the animal in the predicate position.

Named categories of biotopes or landscape units (I.4 of Table 6.1), can be recorded by free-listing but we suspect that the walk-in-the-woods technique (section 4.3.1.3) will produce more reliable and accurate results. However, it may not be feasible to survey all locally known biotopes on the ground, especially those which are distant from the community site, so we recommend that a combination of these techniques be used. Since it may be difficult to establish clear correspondences between locally recognized biotopes and scientific ones in many places, we consider that translation or further description of the local categories is not a vital part of this exercise. Instead, attention should be focused on eliciting the locally recognized characteristics (e.g. elevation, topography, edaphic characteristics, architecture, indicator species, disturbance agents, etc.) and cultural uses or significance of these areas.

Named classes of soil types (I.5. of Table 6.1) can be recorded by free-listing. Given the variability, complexity and specialized nature of scientific classifications of soil, we do not consider that exact identifications of the local categories will be feasible but it may be useful to collect samples or take pictures for use in the testing phase. The characteristic properties (e.g. color, texture, fertility, etc.) and uses of soil types can be registered through the use of semi-structured questions. An important line of questioning for application among agricultural groups concerns the suitability of soil types for raising particular crop types. This can be done by rating soil types per selected crop type using a Likert scale format (section 4.3.1.1).

Local categories and concepts with respect to climate (I.6. of Table 6.1) can be explored using semi-structured interviews. This line of questioning would cover aspects such as climatic elements (e.g. temperature, precipitation, wind, cloud cover), seasonal periods and their indicators (e.g. plant phenology, animal movements), and seasonally scheduled cultural activities (e.g. pastoral migrations, cultivation cycle tasks).

Ethnogeographic knowledge (I.7. of Table 6.1) can be explored by experimental mental map-drawing exercises. This technique involves asking people to draw maps of their home territory showing culturally salient features of the landscape. The relevant information elicited by this technique includes place names, their relative locations on the map, and the cultural uses or significance of the represented areas.

A list of traditional skills (II.1. of Table 6.1) can be produced through free listing, semi-structured interviews, informal interviewing, observations of local activities and occupations, or some combination of these. The focus here is on practical subsistence and manufacturing skills involving the direct use and manipulation of natural resources, such as primary resource production/procurement activities, food preparation, curing techniques, and craft and tool making. The category of resource production activities is subdivided into the major biotechnological classes of agriculture, herding, hunting, fishing, and collection. This organization will facilitate comparison between groups with similar economic orientations.

6.2.3. Assigning Weights to Categorical Items

Quantitative TEK research has established that the cultural importance values of different biological species are not equal. We make the assumption that this principle also applies to the evaluation of the knowledge and practices associated with such resources. Applying this notion to the VITEK design, we now consider the task of assigning different weights to the individual categorical items that will potentially be included in the vitality assessment. The concept of cultural importance refers to a relative judgment about the relationship between an item or set of items and the cultural lifestyle of a group of people at a given place and time. In this context, weighing refers to the measurement of the importance value of the item according to some predetermined scale. Weighted factors are built into the testing instrument at two levels: (1) contribution of the domain to the test composition and (2) selection of categorical items included within the domain.

The method of weighing adopted here essentially designates the local participants as the ultimate judges of value and allows them considerable freedom to use their own criteria for making such judgments. It is argued that such an approach better reflects the needs and priorities of the people themselves and utilizes a more global and multi-dimensional concept of value. The method also stresses the use of elicitation procedures for extracting value judgements that are appropriate and adaptable to a wide range of cultural contexts - e.g. literate and illiterate, numerate and non-numerate. At the same time, however, we are faced with the limitation that the values must take a quantitative form and be adjusted to a single scale so as to permit aggregation into a single measure.

All domains and subdomains included in the test will be assigned a relative value that is intended to reflect the cultural importance of the domain vis-à-vis all others at the same level (i.e. those included within the same superordinate class). Thus under the category of conceptual knowledge, plant knowledge is rated in comparison to the domains of animal, plant-animal relationships, biotic communities, soils, climate and ethnogeography. Within the plant domain, the subdomain of plant taxonomic knowledge is rated against the uses and characteristics subdomains. In the context of community meetings or focus groups, the participants will be presented with a set of cards representing (by printed word, drawing, picture, or some other symbol) each of the constituent categories within the contrast set and prompted to assign a relative numerical value to each category out of a total of 100 points for the entire set. For illiterate, non-numerate or visually-oriented groups, the "stone-distribution" scoring technique can be used. This method involves giving the group of informants 100 stone pebbles, beans, corn kernels or other small objects and asking them to distribute them among the set of cards proportional to their relative importance. A facilitator is present to explain what they are supposed to do, attempt to clarify any doubts or questions that may arise, and encourage active participation by everyone who is present. Some practice runs of the exercise should be performed to ensure that everyone understands the purpose and procedure correctly. In case the exercise generates disagreements, the participants will be encouraged to talk it over and come up with a consensus position. If no consensus can be reached, the exercise should be performed with smaller focus groups (e.g. by age, gender classes) or individuals and then their answers averaged. After the points are assigned, they are converted to a fraction. A hypothetical illustration of how the scoring is calculated is provided in tables 6.2-6.4.

As the result of a series of group consultations, the following scores are obtained:

Table 6.2. Conceptual Knowledge Component Categorical Valuation (Level 2)

Table 6.3. Plant Domain Categorical Valuation (Level 3)

Table 6.4. Use Subdomain Categorical Valuation (Level 4)

The scores obtained at each succeeding level are multiplied by the corresponding score obtained at the immediately preceding higher level to obtain the overall weight of the category. The relative weights per categorical contrast set (i.e. same hierarchical level) and per test component (i.e. conceptual knowledge component) based on tables 6.2-6.4 are depicted in figure 6.1.

Figure 6.1. Hierarchical Calculation Relative Weights by Domain. The abbreviated letters refer to the domain and subdomain categories appearing in tables 6.2, 6.3, and 6.4. The first number shown below the letter label is the relative weight assigned to the domain within the contrast set (i.e. same hierarchical level), as represented in the tables. The second number shown, which appears in parenthesis, is the relative weight of the domain within the entire set of conceptual knowledge.

Despite the straightforwardness of this technique, it may be anticipated that some groups will experience difficulties in placing different values on the domains contained within a contrast set, especially if all are considered to be equally important or mutually interdependent. In that case, an equal value will be assigned to all categories comprising the set. However, we recommend that the point scoring method at least be tried first in order to give the local participants the opportunity to assign value differentials and thus not to assume ahead of time that they are equal or incomparable.

The second level at which weights are assigned concerns the individual categorical items within a domain. Because the point or stone distribution method does not work effectively for sets containing more than 10 items and some of the domains will be larger than this, we recommend that a simple ranking procedure be used to derive the weight at this level. Lawrence et al. argue that the ranking exercise is a relatively quick and easily understood method for assigning importance values to an array of items which can then be converted into interval scale data amenable to statistical analysis. The method involves asking the group of local consultants to rank all the items in a category according to their global importance. The inverse value of the rank number is taken for its score (e.g. in a ranking of 10 items, the highest ranked item is assigned a score of 10 points and the lowest ranked item is assigned a score of 1 point) and then the score is converted into a relative value by dividing it by the sum total of scores. The calculation of rank score (RS) and relative weight (RW) can be formulated as:

RS_ij = (n_ij + 1) - r_ij,

where n_ij is the number of items in the ranking and r_ij is the rank of item i for domain j, and

An illustration of this calculation is provided in table 6.5.

Table 6.5. Relative Weight of Domain-specific Categorical Items

The overall purpose of assigning weights to domains and their categorical items is to determine the composition of the test upon which TEK vitality will be evaluated. Thus the relative values obtained at the first level will be used to determine the proportion of questions corresponding to the primary, secondary and terciary domains that have been selected through the process of local consultation. Meanwhile the relative values obtained at the second level will be used as a weighted factor for determining the selection of individual items that will appear in the test.

top

6.3. Testing Instrument

6.3.1. Test Design

A standardized test designed to rate the TEK aptitude of individuals in the participating local group will be prepared. The TEK aptitude scores obtained in the test will be used directly for making the vitality assessment (section 6.8). The composition of the test will be based on the results of the local domain and categorical item selection processes as well as the relative weight assignment exercises. Thus questions will be drawn from all of the domains and subdomains defined and selected by the community members as locally recognized and relevant categories of TEK. The weights calculated for the different domains at each level will serve to define the proportion of test questions pertaining to those categories. For example, using the hypothetical results presented in table 6.1, 20% of questions in the conceptual knowledge test component would consist of questions pertaining to the plant primary domain. Of those 20%, 40% (or 8% of the total component) would correspond to questions about the secondary domain of plant uses. Of those 40%, 20% (or 1.6% of the total component) would correspond to questions about the terciary domain of edible plants.

With respect to the selection of categorical items, the method adopted here calls for a random sample selection of items for inclusion in the test but the chances of selection of any given item should be proportional to its relative weight within the class. Thus, for example, an item with a relative weight of .12 would have a 12% probability of being selected while an item with a relative weight of .04 would have a 4% probability of being selected. This means that the sampling procedure would involve a two-step process: (1) construct a population of items in which the proportional representation of each item is equal to its relative weight and (2) randomly select the number of test items corresponding to the proportional representation of the domain in the entire test component.

Using the procedure described above, separate tests should be drawn up for male and female subjects. To the extent that local domain selection, categorical item inventories, and relative weight assignments differ between the two groups, the tests will also be different. A minimum of three different sample tests per gender group should be constructed in this fashion. From this group, one is randomly selected for each subject taking the test.

In order to exercise greater control over the test answers and thus optimize the time needed to take as well as grade the test, we recommend that the questions comprising the test be phrased as multiple choice or true/false questions. However, we also propose that an "I don't know" response option be allowed for every question to eliminate (or reduce) the likelihood of guessing. In the practical skills component, the basic question will amount to a self report of knowledge of the skill type (e.g. "Do you know how to do/make _________?" or "Have you done/made _________ in the last month/year/lifetime?").

A notebook of visual prompts in the form of illustrations, photos or specimen samples (where possible) should be prepared for use in the test. Among oral-based cultural groups, the test should be administered as an interview, preferably by someone from the study community. Among literate groups, it is totally feasible that the entire test can be prepared and administered in a written form, albeit with visual prompts included.

The test will consist minimally of three sections:

(1) Social Data. This section records basic sociodemographic information about the test subject, including his or her name, an identification code for the person, age, gender, community and ethnic group. If the research team wishes to capture other socio-economic variables (e.g. educational achievement of subject, education of parent, language fluency, principal occupation, wealth, marital status, number of children, years living in the place, etc.), that information can be recorded here as well.

(2) TEK Aptitude Test. This section consists of two sections: (a) the conceptual knowledge component and (b) the practical skills component. The two components will be treated separately but equally for scoring purposes. Thus a separate score for each component will be obtained. The two scores can be combined to produce a total score. To avoid participant fatigue, we recommend that the first component consist of no more than 120 questions and the second component be about half that length. A time limit should be set for answering equivalent to one minute per question (e.g. 2 hours for 120 questions). However, this may have to be adjusted according to the test conditions, the need for breaks, and the form of administration, whether oral or visual, communicated directly or through an interpreter.
The conceptual knowledge component of the test is more developed than the practical skills component in large part because knowledge can only be inferred indirectly from the observation of behavior and the measurement of behavioral patterns requires a longer and more sustained investigative process. The test is purposely designed to provide results based on a quick, one-time assessment in order to maximize ease, speed and productivity, and therefore enhance its appeal to a broad audience of potential users. Ideally, the test would be administered periodically among the same group(s) to produce a true time-series based indicator but we still feel that the chances for widespread adoption will hinge upon a fast and efficient, albeit representative and reliable, method. However, time and cost constraints permitting, we propose that the practical knowledge component could be expanded by measuring the actual uses or manipulations of plant and animal taxa by the individuals who have taken the test. This would require a much more intensive data collection effort. The easiest way would be to record self-reports of the diversity and frequency of plant and animal resources harvested by the individual (or the household to which they belong) on a regular basis (e.g. weekly) for an extended period of time (at least a year to capture annual seasonal variations). Scores could be calculated by tabulating the diversity and frequency of taxa harvested per unit time. Individual scores could then be ranked in relation to each or graded in relation to the average score.

(3) Transmission Process. This section is proposed as entirely optional but motivated by our belief that a small amount of data on transmission mechanics can provide a great deal of insight into dynamic processes of change/conservation (see section 5.5). For each subdomain class, two questions would be asked: (1) What method best describes how you learned about these kind of things? (multiple choices being: verbal communication, visual observation of others, active participation in related activities, learned on your own, read about it in book(s), taught at school, other, and do not know), and (2) What person(s) is(are) most responsible for teaching this to you? (multiple choices being: mother; father; grandmother; grandfather; brother; sister; other relative (type:_______); friend; expert; other (describe:________); do not know). Used in conjunction with the results of the aptitude section, analysis of observed patterns of this data set can reveal proximate mechanisms for explaining why TEK is changing or not.

6.3.2. Sample Population Selection (age, gender, community)

In order to provide a reliable and representative measure of knowledge variation and change, the test should be given to a carefully selected sample of the study population. The preferred sampling strategy for the VITEK is to choose a sample stratified according to the social variables that are included in the analysis. Two main social variables are considered as absolutely necessary for making the assessment: age and gender.

Age is the key variable that will be used to gauge whether significant change has occurred or not. Although the most reliable way to capture trends is to apply the test at regular time intervals and measure the differences between outcomes, that is in a longitudinal sense, the method is also designed to produce an instant assessment of change from cross-sectional data based on age-indexed knowledge differentials. Although age is a continuous variable, our focus will be on segmented age sets of the adult population. The assessment will be based on measuring differences within the adult population in order to minimize the confounding effects of normal learning processes during childhood and adolescence. While it may be expected that the learning phases of different areas of TEK will vary in length and timing as a result of difficulty, aged-based social status and roles, and other factors, the available evidence from ontogenetic studies suggests that the vast majority of nonspecialist knowledge or ability to perform subsistence skills is fully acquired by late adolescence-early adulthood. Therefore the sample will be drawn from the fraction of the local population that has reached adult age (~15 years old). This fraction will be subdivided into age cohorts of 15 years each, thus producing for most populations four or five demographic strata (15-29, 30-44, 45-59, 60-74, and 75-89 years old) spanning a total of 60-75 years. The method and index calculations are adaptable to other age intervals, meaning that some flexibility is permitted and shorter intervals could be chosen. However, we also consider that the 15-year interval should be the maximum size of the interval in order to preserve enough variation to be able to discern meaningful trends (i.e. at least three pointsof comparison). Ideally, the segmentation would take place along generational lines but generation is more of a social distinction than a demographic one from a collective point of view and in any case the length of the generation should be calculated on a population-specific basis according to the prevailing demographic structure and life history data. Our interest in age distinction is more chronometric, finding a way to index knowledge variation along a temporal scale and therefore we haven chosen a fixed interval that can be compared directly across sites. The 15-year interval provides sufficient time-series approximation for spotting trends yet does not inflate the overall sample size beyond manageable limits. Finer distinctions (e.g. 5-year or 10-year cohorts) would enhance trend resolution but also make it more difficult to find equal numbers of willing test participants in all age-gender classes. If the local research team is interested in comparing the knowledge of children with adults, then that cohort could be added as well but it remains to be demonstrated whether such differences actually reflect part of a historical trend of TEK shift or the normal learning curve.

Gender is considered to be a core strata for VITEK sampling because some kinds of knowledge and practices are gender-specific and where they are comparative testing, grading and interpretation must also be conscious of this division. Thus it will be necessary to view trends among men and among women as at least potentially separate phenomena. The sample should count on comparable if not equal numbers of men and women in all age groups. This may not be easy to accomplish, given the tendency in many societies for men to dominate public spaces and contacts with outsiders, but every effort should be made to include members of both sexes in all phases of the assessment process.

If the assessment is targeted on an ethnic population or larger unit composed of several or more distinct local communities, then community becomes another primary strata for sample selection. In that case, we recommend that a random sample be used to select the group of communities that will be included in the assessment. Community is a complex variable consisting of multiple social and environmental variables that alone or in combination may affect the pattern of TEK variation and change. Doing a proper stratified sample in this context would require a long and complicated analysis of all of these variables as well as a preliminary study of the range of knowledge variation between communities. A random sample would be much more efficient but may require a larger sample size. Once the sample communities have been chosen, the within-community sample stratified by age and gender should be selected in the same way as for single-community assessments.

Other social variables that conceivably affect the vitality status of TEK could be added to the sample design if there is special interest in doing so (see section 6.6). However, because these will vary considerably from site to site and accounting for and analyzing this data would add considerably to the cost and complexity of the overall assessment, we consider that this should be optional and decided by the local research team.

The first step toward constructing a stratified sample is to conduct a census of the entire local population. If a current or recent census of the community can be found on site or from secondary sources (e.g. official records, publications), we see no reason why it cannot be used if it is reliable and contains information about the social strata of interest. If other socioeconomic variables are to be incorporated into the analysis, then a socioeconomic survey must accompany the census. In order to permit sufficient analysis of variance, at least 5 individuals from each generation-by-sex group must be chosen, or a total of 40 individuals in all in a group characterized by four adult age groups.

6.3.3. Test Administration

The VITEK test is designed for individual testing and every effort should be made to uphold this standard. If the test is conducted orally, it should be held in a secluded place out of visual and hearing range from third parties. Inside or immediately outside a person's home is probably not the best place. However, some respondents may feel uncomfortable in this type of situation and not able to perform to their optimal capacity. We recommend that the research team discuss this matter with community leaders or in a group meeting to come up with a viable solution. It is also preferable that the test be administered by someone who is native or familiar with the community. If some passerby or onlooker does interfere with a test answer (e.g. by volunteering an unsolicited answer), the administrator should be prepared to put a halt to the interference and substitute a backup question on the spot.

A practice run of the test should be performed with one or two subjects prior to actual administration of the test in order to ensure that all the questions are well formed, the stimulus materials are appropriate, and there are no major problems with other aspects of the design. The practice will also be useful to plan and evaluate the pace and interactive dynamic of the testing process. The design calls for a time limit to take the test. We have suggested approximately one minute per question as sufficient but this can be revised upward or downward by the research team according to their experience. It may be necessary to schedule one or more breaks or provide refreshments to overcome fatigue or boredom. The test should be doable in one sitting and therefore the total length in time should not exceed 2-3 hours. However, the most important thing is that the same limits be applied to everyone taking the test to ensure a fair comparison of the results.

6.3.4. Test Evaluation and Scoring (intracultural comparability)

The results of the aptitude test will be tabulated per component, primary domain, secondary domain, and tertiary domain. A hierarchical coding system will be used in which each question is coded according to the different levels of categorical membership to which it corresponds. Thus a question regarding whether a particular tree species has a commercial use would be coded as conceptual knowledge component, plant primary domain, cultural use secondary domain, and commercial use tertiary domain. The scores for all questions within a given category will be calculated separately and then aggregated successively as one moves up from lower inclusive to higher inclusive hierarchical levels. The codes assigned to the two test components as well as all of their constituent domains and subdomains, and the formulas for calculating the aggregate scores for each one, are given in table 6.6.

The modular structure of the test scoring system is purposely designed to permit disaggregation and aggregation of the test results at various levels. The feature of categorical disaggregation will permit more specific comparisons of scores between individuals and groups in regards to different types of knowledge. The feature of successive aggregation will allow for a more composite view of the global status and trends of TEK between groups. All answers will be scored as either 1 point (for right answers) or 0 points (for wrong or I don't know answers). The confidentiality of all persons taking the test and their performance will be protected by assigning everyone a personal code. The identities that go with the personal codes and the test scores by coded individual will be stored in separate databases.

Table 6.6. Categorical Codes and Scoring Formulas

top

6.4. Vitality Index of TEK

Vitality is operationalized here as the rate of retention of knowledge between generational groups. A situation in which vitality is strong is indicated by lack of significant change or even increase in TEK aptitude from one generation (i.e. age cohort) to the next. A situation in which vitality is weak or threatened is one in which a significant decrease of aptitude is confirmed. Whereas the rate of retention or change is measured by taking the difference in average aptitude scores between age groups, the significance of this difference is measured by looking at the means or medians in relation to observed variance within and between these groups. Thus the two sets of measures depend on separate but related statistical procedures.

6.4.1. Calculating the Vitality Index

The vitality index (VI) consists of three related measures: the intergenerational rate of retention (RG), the cumulative rate of retention (RC), and the annual rate of change (CA) (Figure 6.2). All of these rely on very simple statistical calculations and can be done by hand or with a pocket calculator. The first step is to calculate the means of the score results for all age/gender groups included in the aptitude test.

Figure 6.2. VITEK Component Measures

The RG indicates the rate of retention between any successive pair of age groups and is calculated as the ratio of the generation mean to that of the generation immediately preceding it. This calculation is given by:

RG_t = g_t / g_r

where g_t is the mean score of the target age group (i.e. the younger group of the pair)

and g_r is the mean score of the reference age group (i.e. the next ascending group).

The RG_t of the oldest age group is set at 1 based on the logic that no information about the aptitude level of the preceding generation(s) is available and therefore we cannot assume that any differences or changes have occurred in prior time periods.

The rate of retention for men (m) and for women (f) are defined respectively as:

RG_tm = g_tm / g_rm

where g_tm is the mean score of the male target age group and g_rm is the mean score of the male reference age group, and

RG_tf = g_tf / g_rf

where g_tf is the mean score of the female target age group and g_rf is the mean score of the female reference age group.

The rate for the combined samples of both men and women (b) is defined as:

RG_tb = ½ ( RG_tm + RG_tf ).

The cumulative rate of retention (RC) essentially reflects the proportion of the baseline aptitude level retained by each succeeding age group. The formula used for RC is adapted from that used to calculate the Living Planet Index since they have similar purposes (i.e. measuring retention over time based on sample data) although they are measuring different things (i.e. TEK aptitude vs. biological populations). RC is calculated by multiplying the reference RC by 10 raised to the power of the logarithm of the target RG. As with the RG calculation, the RC of the oldest target age group is set at 1. The formula is defined as:

RC_t = RC_r 10^log(RGt)

The same statistic can be calculated for men, women, and combined samples.

A composite index expressing the average rate of retention across all age group pairs tested can also be calculated and is defined as:

The annual rate of change (CA) expresses the average rate and direction of change per year reflected by the target age group and is given by:

where yg_t is the length in years of the target age group interval.

This measurement can be calculated for all age groups combined (CA_a ) by simple addition, as given by:

The same basic measurement can be applied to the male, female, and gender-combined samples.

The composite measure CA_a is especially useful for making direct comparisons of TEK vitality between local populations in space and time. It is adaptable to studies using different age intervals, it gives a single number for the overall population, and it is expressed in a unit of time. As such, it can be used as a reference measure for monitoring trends of TEK retention/change over time through repeated application of the test in the future. If interventive policies aimed at preserving, restoring or innovating TEK are enacted, the impact and performance of these can be directly and unambiguously measured through the CA_a.

A hypothetical example of all of calculations mentioned in this section is presented in table 6.6. The results are plotted on graphs to visualize the prevailing trends (figures 6.3-6.5).

Table 6.6. Vitality Index Calculations by Age/Gender Group

Figure 6.3. Graphic representation of retention rates by age group for male, female, and combined groups.

Figure 6.4. Graphic representation of vitality indices by generation for male, female, and combined groups.

Figure 6.5. Graphic representation of annualized change indices by age group for male, female, and combined groups.

6.4.2. Significance Tests

Significance tests will be used to assess whether the trends calculated by the vitality index signal significant differences (i.e. changes) in knowledge between generations. Given the data characteristics and objectives of the index, we believe that the best measure to use is the Mann-Whitney U (MWU) test. The MWU, also called the two-sample rank-sum test, is a nonparametric statistic which does not assume a normal distribution, does not require equal variances, works well with small sample sizes ( n < 20), and can be used with interval or ordinal data. The calculations are very simple and can be done by hand. The first step is to convert the continuous measurements reported in the test scores to ordinal measurements. The U statistic is calculated as follows:

where samples of size n₁ and n₂ are pooled and R_i are the ranks.

The significance of the U statistic is determined by comparing the obtained value with the value shown for the corresponding sample sizes in a table of critical values. One-tailed tables can be used to test the alternative hypothesis that x values tend to be smaller (i.e. knowledge loss) or larger (i.e. knowledge gain) than y values.

For the composite indices, the preferred test of significance is the Kruskall-Wallis one-way analysis of variance (KW) by ranks. This is a non-parametric method for testing equality of population medians when comparing three or more groups. The method begins by ranking all groups together (i.e. ignoring group membership). The KW statistic is defined as:

where n_g is the number of observations in group g,
r_ij is the rank (among all observations) of observation j from group i,
N is the total number of observations across all groups,

r is the average of all the r_ij equal to (N+1)/2.

The P-value is approximated by P_r (χ²g-1 > K).

More complex (and powerful) tests of significance may be called for if other socioeconomic change indicator variables are included as analytical operators. In that case, age and gender can be used as control variables or as sorts for investigating and testing the effects of these additional variables. For example, the association between formal schooling and TEK aptitude could be tested by age or gender group and then compared to the same association for the entire population sample. This would give us a better idea of the impact of schooling on specific subgroups, particularly those displaying a lower rate of retention. The statistical methods employed can vary according to the type of data collected, the scale and precision of measurement, and the hypotheses being tested. Stepwise regression is one of the techniques that has been used to separate the most significant operators in multivariate situations. The Vitality Index based on age and gender serves as an exploratory state measure for identifying dynamic situations where more detailed studies of the pressures and drivers of TEK change are warranted. Because the Vitality Index can be disaggregated at various levels, it also serves as a generator of hypotheses concerning the impact of different environmental factors on different areas of TEK.

The Vitality Index measures (RG, RC, CA) can be aggregated according to different scales of inclusiveness, from the local community on up to the entire globe, depending on data availability of course. For example, if the index is calculated separately for different communities belonging to the same ethnic unit, these community indices can be combined to produce an ethnic index. Several ethnic indices can be aggregated to form a provincial index and provincial indices can be put together to calculate a country index. Country indices can be aggregated to make a macro-regional index and all of these together would comprise the global index. Instead of aggregating according to spatial-political units, the aggregation could be determined by eco-regions (e.g. landscapes, regions, continents) or levels of demographic density. There is really no limit on its capacity for aggregation other than comparability of method and data across all of the sites included. The same basic formulas for calculating the Vitality Index described in section 6.9 can be used at any scale except that the calculation involves taking the mean index value of all the individual indices included from the lower scale. The mean vitality index is calculated by the following equation:

where VI is used to represent any of the measures RG, RC, or CA and n_t is the number of samples included

The same basic formula can be used to calculate the mean vitality index by scale for males (VI _tm), females (VI _tf), and all generational groups (VI _a) as depicted below.

For males:

For females:

For all generational groups:

Aggregate measures can in turn be disaggregated according to the TEK domains and categories that are included in the test across different sites. This feature is especially important for identifying generalized trends of TEK constancy/change with respect to particular areas or kinds of knowledge or for testing hypothesis about the impacts of specific environmental drivers. For example, this technique could be used to assess the vitality/loss of traditional agricultural practices in different populations of a province, country or region. The vitality of ethnobotanical and ethnozoological knowledge of communities living in areas experiencing high (or low) rates of deforestation and biodiversity loss during the lives of their members could actually be measured. The effects of free (vs. protectionist) trade policies on the TEK-dependent livelihoods of women could be assessed. In sum, this combination of aggregative and disaggregative properties creates in effect a powerful tool for empirically documenting the complex, variable and dynamic trends of TEK at different scales of inclusiveness.

top