The accuracy of selected land use and land cover maps at scales of 1:250,000 and 1:100,000

Geological Survey

Table of Contents iv Accuracy of selected maps

Introduction

A Level II land use and land cover classification system for use with remotely sensed data (Anderson and others, 1976) is being used for maps at scales of 1:250,000 and 1:100,000. The maps were prepared with manual interpretation of high-altitude aerial photographs by the U.S. Geological Survey. Sca~e-stable copies of the topographic map base were used as a mapping base. The land use and land cover classification is shown in table 1. The minimum mapping area according to the specifications is 16 hectares for most categories and 4 ha for Urban or Built-Up Land and such selected categories as Confined Feeding Operations (23), Other Agricultural Land (24), Water (53-54), Strip Mines, Quarries, and Gravel Pits (75), and Transitional Land, if urban (76) (Loelkes, 1977).

Certain of the mapping criteria adhered to, specified by Anderson and others (1976), stated that (1) the land use and land cover maps must be at least 85-percent accurate, (2) the accuracy of the interpretation will be about equal for the several categories, and (3) the results must be repeatable from interpreter to interpreter and from one time to another. Once the maps were compiled, field verification was conducted to assure correct interpretation, and the maps were made available to local users for review. Additional corrections were then incorporated.

Ideally, any land use and land cover classification system designed should have categories that are mutually exclusive. l''one the less, there are some categories in ary system whose identifying signatures can be confused with others or whose function~ so closely resemble others that confusion occurs. For this reason, a classification er·or matrix is useful in recognizing where and why errors occur. For the system employed t ., the Geological Survey, experienced interpreters can often predict which categories will t1~ in error. Usually these errors are expected within a Level I category or between two or mo"e Level II categories that have similar visual signatures or characteristics.

Certain of these predictable errors mE" not affect the useability of the land use and land cover map for some purposes. For exam pie, an agency estimating the forest area on a map might not be concerned with the separation of deciduous and evergreen tree types. When the two categories of industrial and commercial land are intermixed in an in· dustrial park, a planner might not be concerned as to whether the area is classiF~d as Commercial (12) or Industrial (13). Some categories such as Cropland and Pasture (21) and Other Urban or Built-Up Land (17) often differ only in where they are located. The lane cover may be the same. It is helpful, therefore, for the user to know, not only how accurate the map is but what kinds of errors occurre~.

Introduction Figure 1 is a classification error matrix highlighting those categories that are most in error because of the definition of the categories. The degree of difficulty of differentiating among these categories differs from one area of the country to another depe'1ding on the intermixture of land use patterrs and vegetation types.

This research project was undertaken to study quantitatively and objectively the accuracy of the Level II land use and land cover maps and to see if the mapping criteria were being met. From maps of different geographic regions and from maps believed to be characteristic of the map products as a whole, it was possible to determine the map accuracy and to conclude whether or not the mapping accuracy criteria were being met. It was also possible to show through classification error matrices the types of errors that had occurred.

Sample Selection

For this research project to assess accuracy for selected land use and land cover maps across the country, the Tampa, Fla., Portland, Me., Charleston, W. Va., and Greeley, Colo., maps were selected as examples of the 1:250,000 scale. Atlanta, Ga., and Seattle and Tacoma, Wash., maps were_ selected as examples of these maps at 1:100,000 scale. Specifications for compilation were the same for the 1:1 00,000-scc: 'e and the 1:250,000-scale maps, and, so, the only compilation difference was format. Seattle and Tacoma are formatted as 0.5 o (latitude) x 1 o (longitude) maps. The standard 1:250,000 quadrangles are 1 o x 2° maps, whereas the Atlanta map is in a special format '''hich is centered on the city of Atlanta.

Previous analyses demonstrated that the best method for determining accurc~Y was by sample points selected according to a stratified systematic unalined sampling design (Fitzpatrick-Lins, 1978). The land use and land cover map was first stratified into 10- x 10-kilometer blocks coinciding with the Universal Transverse Mercator grid system. Within the 10-km blocks, points at t"le intersection of a 1-km grid cell were S'~lected

Accuracy of selected maps

E2

p = the sample value of p or r/n exeach category to the total number of points for each category sampled from the map.

Another pertinent question less often addressed is how well was the real world depicted by the map, or what were the probabilities of an error of omission? This value can be obtained from the ratio of the number of points correct for each category to the total number of points for each category found in the verification process.

By definition, the sample was designed to select from map data, and, so, we chose to consider primarily the errors of commission. Once these data were analyzed and compared to the verified interpretation, an estimate of the errors of omission was obtained. For each analysis, a two-tailed test was used to find the true range of value for errors of commission and errors of omission. Neither of these estimates can stand alone as the accuracy of the category. It was also possible to compare the c~assification error matrix for each map sheet ·with the error matrix of most likely misinterpretations to see if the interpretation difficulties were predictable.

The maps produced at 1:100,000 scale are easier for the user to read, but the information content is the same as the maps at 1:250,000 scale. For the purposes of the accuracy study, the scale consideration was negligible, as all verification was done at the compilation scale. For the maps discussed, the compilation scale was about 1:125,000. For this reason, no comparison of the results for different scales was performed.

Accuracy Assessment

The Tampa, Fla., land use and land cover map was produced at the same scale and format as the 1:250,000-scale topographic map base. The nonocean part of the Tampa land use and land cover map portrays about 8,981 square kilometers (7,209 km of land categories), extending from the city of Tampa to south of Sanibel Island and including the metropolitan areas of Tampa, Sarasota, and Bradenton. The source material for the compilation was a set of 1:80,000-scale black-and-white transparencies copied from quad-centered color-infrared high-altitude ph,.,tographs. Compilers mapped the land use and land cover at 1:125,000 scale using an anlarged copy of the topographic map as a base with copies of the black-and-white transparencies at 1:125,000 scale. The minimum mapping unit was 16 ha for most categorier and 4 ha for Urban or Built-Up Land (11-17) and such selected categories as Confined Feeding Operations (23), Other Agricultural Land (24), Water (52-54); Strip Mines, Quarries .. and Gravel Pits (75), and Transitional Areas (if urban) '76) (Loelkes, 1977).

In the Tampa area, there was a difficulty in distinguishing Herbaceous Rangeland (31) from Nonforested Wetland (62) using tl1e photographs. Often the only way to distinguish between the two was to determire whether there was standing water by detecting sun glint. Transition areas were also causes of difficulty. Forested Wetland (61) often bordered Nonforested Wetland (62). Tall wetland grasses were mixed with brust'y mangrove areas. The signature of the forested category usually predominated even in areas where the mangrove had died leaving only bare bushes mixed with tall grasses. Another transition area was from Herbaceous Rangeland (31) to Evergreen Forest Land (42). The categories are not always mutually exc•usive, so that a particular interpreter's favoring of one category when drawing a boundary in a transition area was no more correct or incorrect than favoring another category.

As mentioned earlier, points were se•~cted using a stratified systematic unalined rample. A total of 311 points were selected. Wl1en a point fell on a boundary, it was considered as two points. There were 43 boundary points on the Tampa map, making a total of 354 points.

For the Tampa land use and land cov~r map, a computer printout of data from the digitized land use and land cover map was available. It was possible, therefore, to compare the number of points selected with the actual number of hectares in each category. The point sample shown in table 2 prov~d to be representative of the major categories on the map. The percentage of area and of points for each category compare favorably. By applying a Spearman Rank Correlation Test to compare the number of points samp'o.d

Accuracy assessment with the area of each category, we were able to assume an association between the number of points and the area of each category. The null hypothesis of no association was re· jected with less than 0.001-percent probability of error, indicating that this method of selecting a sample is area weighted, which is not at all surprising.

The table shows that only the predominant categories are sampled with sufficient points for adequate analysis. Many of the categories that occur occupy only a small percentage of the map area and that infrequently are not sampled or are sampled with too few points to determine their accuracy. Methods are currently being developed to utilize a computer program to sample all categories adequately for each map. This system will be applicable to any map that has been digitized and the data stored on a computer tape.

Of the 354 points sampled on the Tampa land use and land cover map, 329 (93 percent) were found to be correct. The 95-percent one• tailed lower confidence limit for these data would be 91 percent, and, so, the accuracy of the overall land use and land cover map is

Accuracy of selected maps well above the acceptable accuracr of 85 percent.

It is understandable that water categories, particularly category 54, were almo~t always mapped correctly. Considering that errors were practically limited to the land~based categories, it was interesting to consider only the land area categories. Of the 35.tt points, 65 points were classified as category 54. (This doe not include any large bodies of other than inland water such as the Gulf of Mnxico.) Of the remaining 289 points from the land surface, 25 were in error and 264 were correct. For this land area, the accuracy percentage is 91 percent with a 95-percent one-tailed lower confidence limit of 87 percent, still in excess of the required value of 85 percent. This particular analysis of only land area was necessary to determine if the large portion of the map showing water might have led to an inflated value of accuracy when considering the accuracy of the overall map. Although the map appeared more accurate when the water category was included, the map wa~ still above the specified accuracy of 85 percent in the land area only.

b: 43

Accuracy assessme.,t A classification error matrix of the results for the Tampa land use and land cover map is shown in figure 2.

The major land use and land cover categories on the Tampa map were analyzed for errors of commission (table 3).

The categories providing a reliable estimate of the true percentage correct with a 95-percent lower confidence limit within ± 10 percent are Cropland and Pasture (21), Herbaceous Rangeland (31), and Bays and Estuaries (54). Of these categories, all met the criterion of at least 85-percent accurate.

When the data were examined for errors of omission from the point of view of the number of ground truth points for each category correctly identified on the map, the results differ somewhat (table 4). For instance, the number of points of Residential found during field work were sufficient to provide an estimate of the accuracy with ± 10 percent,

Accuracy of selected maps whereas the ratio of correct points to the map sample points of Residential was too low to provide reliable results. In contrast, Cropland and Pasture appeared to be mapped with less accuracy when the actual occurren~e of this category in the field was compared to the number of occurrences mapped. Whereas 52 of the field-checked points were in Cropland or Pasture, only 47 points were identified as such. From this viewpoint, it is app~rent that some cropland and pasture was ovarlooked.

The Portland, Me., land use and land cover map was produced at the same scale and format as the 1:250,000-scale topogra~~ic map of Portland, Me. The land area is 15,038 km and includes the area from coastal New Hampshire at Portsmouth to north of Portland, Me. At the interior of the map sheet are Sebago Lake, Me., and Lake Winnipesaukee, N.H. The towns of Laconia and Franklin, the city of Concord, and the northern portion of Manchester, N.H., are also included.

The terrain is hilly to mountainou~ with many small lakes. The urban areas are not large, there is relatively little cropland, and most of the area is forested. The most serious interpretation problem encountered by the compilers was one of separating the forest types.

The Portland land use and land cover map was compiled using the 1:250,000-sc-:ale topographic map base enlarged to a scale of 1:128,000. The land use and land cover source materials were black-and-white quad-centered photographs at a scale of 1:80,000 c':ltained in 1973,. 1974, and 1975 along with copies of the same prints reduced to a scale of 1:128,000. Where this coverage was limited, 1~73 color-infrared photographs at an approximate scale of 1:128,000 served as the source material. Two small areas of the map for which conventional source material was not available were compiled at the Survey's Special Mapping Center in Reston, Va. Once the land use and land cover data were compiled, the finished maps were reduced to a scale of 1:~~1),000.

The land use and land cover map was evaluated for accuracy at the scale of compilation using the same photographs used for the original interpretation. The source material for that portion of the map compiled at the Special Mapping Center was not available and so that portion was not evaluated for accuracy.

A total of 456 points were selected according to a stratified systematic unalined sample. Each point was reexamined on the photographs and compared to the original interpretation. Where any interpretation difficulties remained, the actual land use or land cover was verified in the fields.

Of the total 456 points verified in the Portland land use and land cover map, 410 (90 percent) were found to be correct. At the 95-percent one-tailed lower confidence level, the lower limit of true accuracy was 87 percent. This value was well above the acceptable accuracy of 85 percent. When the land use and land cover categories were considered (excluding category 54, which is not part of the land area of the sheet and not usually subject to interpretation errors), the accuracy of the map remained at 90 percent (402 correct of 448 total points) with a 95-percent one-tailed lower confidence limit of 87 percent. From this, it was apparent that the number of sample points in the water category 54 did not bias the results on this map sheet. The classification error matrix for Portland, Me., is shown in figure 3.

The major land use and land cover categories examined for errors of commission for the Portland, Me., sheet are shown in table 5.

Only Cropland and Pasture (21), Evergreen Forest Land (42), and Mixed Forest Land (43) were reliable with ± 10 percent of the sample results. Of these, Evergreen Forest Land (42) fell short of the acceptable accuracy of 85 percent at the lower confidence limit. The most frequent misclassification of thif category was that is was mapped as Mixed Forest Land (43).

When the data were examined for errors of omission rather than errors of commis~ion, the results for these same categories were as shown in table 6.

Those categories having accuracy pP.rcentage with ± 10 percent were Evergreen Forest Land (42) and Mixed Forest Land (43). Residential (11), Cropland and Pasture (21), and Deciduous Forest Land (41) have too wide a confidence interval for reliable result~, Residential appeared to be interpretod more correctly when compared to the actual 31 points of Residential found in the field than when compared to the 36 points of ma"lped Residential. In all probability, Resident al is overestimated on the map. The interprfltation of Evergreen Forest Land also appeare1 to be more precise when compared to the fie d data than when compared to the map data. Much of the land interpreted as Evergreen Fc"est Land was actually Mixed Forest Land.

The Charleston, W.Va., land use and land cover map, at the same scale and formqt as the Charleston, W.Va., 1:250,000-scale topographic map, encompasses 19,352 km in the heart of West Virginia. The land is mostly forested mountainous terrain transecte1 by the Kanawha River, which is visible in the lower left corner of the map. Charleston, the capital of West Virginia and the only major city on the map, is at the intersection of the Kanawha and Elk Rivers. The area of tr~ map

g: 43

Accuracy of selected maps is predominantly forested with small rural settlements.

The major categories on the land use and land cover map are Deciduous Forest and Mixed Forest Lands. Very little Cropland and Pasture is present; special specifications were created to map these areas to a mini- . mum width of 200 meters instead of the usual 400 m, so that the few existing agricultural areas would be mapped.

Another land use type in this region given special consideration is the surface strip mine. Where the surface strip mined area was larger than 4 ha and a minimum of 90 m wide, it was mapped as category 75. Ordinarily, this category is mapped for a minimum of 4 ha but must be at least 200m wide. In evaluating the map for accuracy, these unique situations were taken into account, and care was taken to meet the same specifications.

The land use and land cover map was compiled using 1:125,000-scale high-altitude color-infrared photographs of September 1972 and December 1973. The Kern P.G. 2 stereoplotter was used with the high-altitude photographs and the 1:250,000-scale topographic map base, each at original scale. The stereoplotter allowed for stereoscopic viewing of the photographs and for adjusting this image to the scale of the map base for compilation. The photographs could also be enlarged to two, four, or eight times the original.

After the original land use and land cover mapping was completed, a copy of the map was enlarged to a scale of 1:125,000, the same scale as the original photographs. For quality control, the photographs were registered to the enlarged map for direct comparison without the use of the P.G. 2 stereoplotter. The accuracy check involved this same technique, overlaying the photographs with the map, to verify the interpretation at selected sample points.

The 424 sample points were selected using a stratified systematic unalined sample. The same specifications and special considerations approved for compilation were followed for the accuracy check.

Of the 424 points checked for accuracy on the Charleston land use and land cover map, 393 (93 percent) were found to be corre':t. The 95-percent one-tailed lower confidence limit was 90 percent. The classification error matrix for Charleston is shown in figure 4.

Of the 424 points checked, 353 were in two forest categories, Deciduous Forest (41) and Mixed Forest (43) Lands. The next significant category in the sample was Cropland and Pasture (21) with 53 points. Thus, 406 P">ints (96 percent) represent these three major land use categories. The results for each of these three categories analyzed for errors of commission are shown in table 7. Neither category 21 or category 43 is mapped '"ith the necessary accuracy of 85 percent at th~ lower confidence level.

When the sample was considered for errors of omission, these three categories we ..e found to have the accuracies shown in table

All three categories above exceed th~ required accuracy of 85 percent at the lower confidence limit when examined for errors of omission.

After comparing the tables of errors of omission and of commission, we reexamined the map data to analyze the difference and concluded that the size of smaller polygons

g: 43

a: 54

Accuracy of selected maps of land use were increased even larger than the unique specifications would allow to insure their inclusion on the map, thereby decreasing the errors of omission for these categories but increasing the errors of commission. This practice appeared to be the primary cause of error for Cropland and Pasture (21); frequently, the surrounding Forest Land was included in the polygon mapped as Cropland so that the polygon would meet the minimum mapping size.

Mixed Forest Land (43) was also less than

limit. The primary cause of error in category 43 appeared to be that it was often used for convenience even when the predominant forest type was Deciduous Forest Land (41). There were 16 sample sites where category 41 was mapped as category 43. This error, one of misclassifying one forest type for another, might not be as serious from a users point of view as mistaking Forested Land for Cropland and Pasture.

In spite of these specific error types, the overall accuracy of the Charleston land use and land cover map exceeded the required accuracy of 85 percent at the lower confidence level.

The Greeley, Colo., land use and land cover map, 1979, at a scale of 1:250,000 is representative of the land use and land cover types of Colorado's eastern plains and mountain slopes. The land area of the map is 18,855 km • The western portion of the map includes parts of the southern Rocky Mountains with their forest and alpine vegetation, and the eastern portion of the map is in the eastern plains of Colorado with its cropland, pasture, and rangeland vegetation. In the central portion of the map are the cities of Boulder, Longmont, Loveland, and Greeley, Colo.

The compilers mapped the land use and land cover categories using high-altitude black-and-white photographic transparencies acquired in 1975 and 1976 as the primary source material. Reductions of these transparencies were overlaid with a copy of the topographic map base at a common scale of 1:128,000. The land use and land cover compilation was made directly on the 1:128,001- scale copy of the topographic base map. To assist in the interpretation, the compilers also employed high-altitude color or color-inf~ared transparencies. These photographs wer€ from

materials to the more current black-and-white photographs. No stereo processes were employed during compilation, so registration was a problem in the mountainous regic'ls. Valley bottoms were often correctly registered, whereas the ridgelines were often misregistered. No measure of the resulting misregistration was made during the accuracy check.

Specific problem categories for the interpreters using the black-and-white photo, graphs were in separating Cropland and Pasture (21) from Herbaceous Rangelan1 (31), separating Deciduous Forest Land (41) from Evergreen Forest Land (42), separating t"erbaceous Tundra (82) from Bare-Ground Tundra (83), and separating Herbaceous Rangeland

In the eastern portion of the map, the rangeland may be either a shrub and br ·.•sh rangeland with much sagebrush and sand sages, rangeland of grasses and forbs, or a mixed rangeland. It was often not possible to recognize these vegetation types on the: photographs alone. The graininess of tt'~ film produced more texture than the pattern of vegetation types (Eldon Jessen, personal commun., 1978).

Where range management was practiced, the signature of Herbaceous Range (31) was almost identical to that of Cropland and Pasture (21). For pasture to be classifie1 as category 21, it is required that it be in r~rma nent use and maintenance as pasture. Often, brush control was the only modification, and, therefore, the grazing land would be classified as Herbaceous Rangeland (31). In these areas, once the shrub brush had regrown, the proper category was Mixed Rangeland (33).

Even cropland in the wheat areas is not permanent. Much former cropland has bee-'1 abandoned and reverted to rangeland, and new areas of rangeland have been cultivated in wheat. Often cropland scars are app~rent long after the land has been allowed tc revert to rangeland. Abandoned cropland would be

Accuracy assessment classified as Rangeland; however, fallow cropland would still be in the Cropland and Pasture category.

Because of these difficulties in interpreting the land use and land cover categories in the eastern portion of the map, this area was extensively field checked at the time of compilation. The original field-check photographs were actually more beneficial to an accuracy check than current field work would have been, considering how transitional the land cover is.

In the wes"'"fern portion of the map, the land cover is more static. Aspen was difficult to distinguish from the coniferous trees on the summer black-and-white photographs. The color-infrared photographs were the most useful for separating the Deciduous Forest (41) and Evergreen Forest (42) Lands. Herbaceous Tundra (82) and Bare-Ground Tundra (83) were difficult to separate on the black-and-white photographs. For these areas, the color-infrared photographs were helpful. During the accuracy check, stereo viewing was also beneficial.

According to a stratified systematic unalined sampling design, a sample of 375 points was initially chosen. Of these, 38 fell on a boundary and were considered to be double points, giving a total of 413 points. Each point on the map was checked against the original compilation materials. Where questions occurred, the points were located on the 7.5-minute topographic map and checked against the original precompilation data by those who had spent time in the field.

The result of the accuracy check of the Greeley, Colo., land use and land cover map is that the overall accuracy was 399 points of 413 points total, or 97-percent correct with a 95-percent one-tailed lower confidence limit of 95 percent. This result far exceeds the expected accuracy of 85 percent.

Of those categories with more than the required number of points to provide reliable accuracy data, the results for errors of commis.. sion are as shown in table 9. All these categories exceed the expected accuracy of 85 percent at the lower confidence limit.

When these same categories are considered from the point of view of errors of omission, or the field-identified points that were

Accuracy of selected maps correctly interpreted, the results are as shown in table 10.

Again, these categories exceed t'le expected accuracy of 85 percent at t~e lower confidence limit. From the classification error matrix (fig. 5), it is apparent that Mixed Rangeland (33) is occasionally misinterpreted as Herbaceous Rangeland (31), while Herbaceous Rangeland (31) is misinten"''"eted as either Cropland and Pasture (21) or Mixed Rangeland (33), and Evergreen Fore::.t Land (42) is occasionally misinterpreted as Mixed Forest Land (43). These errors, few as they are, are the very errors expected frc'll the interpretation difficulties on this sheot.

A 1:100,000-scale land use and land cover map was compiled for the Greater Atlanta Region, Ga. The published map covered 20,554 km and was centered on tho city of Atlanta. Part of the area of the map was initially compiled using 1:24,000-scalo orthophotoquads at that scale as a base and were mapped with a minimum mapping unit of 1 ha. The land use and land cover data were then generalized from the 1.:24 ,000-scale

g: 43

a: 54

Accuracy assessment parts to 1:100,000 scale. The minimum map-Ping unit for the 1:100,000 scale land use and land cover maps was increased from 1 ha at 1:24,000 scale to 4 or 16 ha at 1:100,000 scale, according to the specifications (Loelkes 1977, p. 18). The remaining portion of the map was compiled using a stable base copy of a 1:100,000-scale topographic base and black-and-white high-altitude photographs at a contact scale of 1:76,000 and at a reduced scale of 1:100,000. The specifications for the minimum mapping unit remained consistent.

The compilers experienced the greatest difficulties in interpretation of the forest land categories. The black-and-white aerial photographs were obtained in February, April, and May 1974, and the degree of foliation varied with the time of photography. For this reason, the shades of gray on the photographs varied for evergreen and deciduous trees from month to month, so that there was no consistent signature for forest types. The interpreters had the greatest difficulty separating evergreen and deciduous forests on the May photographs, as both forest types had similar gray tones. This problem was as difficult to resolve at 1:24,000 scale as it was at 1:100,000 scale.

These same difficulties were experienced in verifying the land use and land cover interpretation at the 381 sample points when the same photographs were used. As a result of these and other problems, 72 points were examined in the field. Most of these points were in areas of Evergreen Forest, Deciduous Forest, and Mixed Forest Lands.

The number of correct interpretations for the Atlanta land use and land cover map at the 1:100,000 scale was 343 (90 percent) of 381 points sampled as seen in the matrix (fig. 6). The 95-percent one-tailed lower confidence limit is 87 percent.

The results, obtained by a stratified systematic unalined selection of 381 points, represented the complete map at a scale of 1:100,000. Only 13 of the 21 land use and land cover categories on the map appeared in the sample. Of the 13 categories included, only 5 had more than 20 occurrences in the point selection. The results for these five categories examined for errors of commission are as shown in table 11.

Accuracy of selected maps Only Mixed Forest Land (43) exceeds the specified accuracy of 85 percent at the lower confidence limit. Residential (11) ard Cropland and Pasture (21) at 83 pert~ent for the lower confidence limit approact' the criterion of 85 percent accurate. A larger sample for all categories except for Mixed Forest Land would give more reliable results and would narrow the confidence interv~':\1.

When these same data are examined for errors of omission the results were a~ shown in table 12.

From these results, it appears thct the accuracies of four of these categories approach or exceed the criterion of 85-percent accuracy. Category 41 was less accurate, yet the range in accuracy was so great that it would be necessary to test several more points for more precise results. The overall accuracy of the land use and land cover maps exceeded the criterion of 85-percent accuracy at the scale of 1:100,000.

g: 43

a: 54

Accuracy assessment The Seattle and Tacoma, Wash., land use and land cover maps were each produced at a scale of 1:100,000. The area of each sheet is one-quarter of the standard 1:250,000 1 o x 2 o topographic map of the Seattle area. The Seattle 1:100,000-scale map includes the northeast corner of the standard 1:250,000- scale topographic map between Iat 47 °30' and 48° N. and long 122° and 123° W. Most of Puget Sound and all but the southern portion of the city of Seattle is shown. The land area of the Seattle sheet is 3,326 km • Between Iat 47 o and 47 °30' N. just south of the Seattle sheet is the Tacoma 1:100,000- scale map. The Tacoma sheet includes the southern portion of Puget Sound and of the city of Seattle and all of the city of Tacoma. The land area of the Tacoma sheet is 4,110 km • Both the Seattle and the Tacoma land use and land cover maps include major metropolitan areas, but the majority of the mapped area is nonurban agricultural or forest lands.

The maps were compiled at a scale of 1:125,000 using 1: 125,000-scale color-infrared high-altitude photographs acquired during S~ptember 1975. Once the maps were compiled, they were enlarged to a scale of 1:100,000. The minimum mapping units for the 1:100,000-scale maps were the same as the minimum mapping units as 1:250,000 scale.

The forest categories provided the most difficulty on the Seattle and the Tacoma maps. This problem was further complicated by the presence of Western Larch, a deciduous conifer having the same signature as most evergreen trees but correctly classified as Deciduous Forest Land. Later, it was decided that this particular species would be considered correct as either Evergreen Forest (42) or Deciduous Forest (41) Lands.

A total of 196 points were selected on the Seattle sheet and 202 points on the Tacoma sheet. Those points falling on boundaries were considered to be double points and thereby increased the number of points on the Seattle map to 212 and the number of points on the Tacoma sheet to 235. The land use and land cover interpretation was verified from the same photographs used for the original compilation. Only those points providing interpretation difficulties were field verified.

Accuracy of selected maps The results were not consistent between the Seattle and the Tacoma map sheets. Of the 212 points sampled on the Sea•tle land use and land cover map, 206 (97 percent) were correct. The 95-percent one-tailed l~wer confidence limit was 95 percent. Wher the land area alone was considered, 162 po1nts of a total 168 (96 percent) were correct. The 95-percent one-tailed lower confidence limit for the land area data was 93 percent, well above the criterion of 85 percent ac-:curate.

For the Tacoma land use and lar1 cover map, 202 points (86 percent) of the total 235 points were correct. The 95-percent one-tailed lower confidence limit was 82 perc·~nt. When the land area with the exception of category 54 was considered, the accuracy dropped to 83 percent or 167 correct points of the 202 points of land-area categories. The 95-percent one-tailed lower confidence limit fc r the land area only was 78 percent, considerably below the accuracy criterion of 85 percent.

Classification error matrices for Seattle and Tacoma are presented in figures 7 and 8.

The major land use and land cow~r categories for Seattle analyzed for errors of commission are as shown in table 13.

The major land use and land cover categories for Tacoma analyzed for errors of commission are as shown in table 14.

The primary cause of error on the Tacoma map sheet was misinterpretation of category 11, as evidenced by figure 8. Six of the points of residential land were actually Crc~land and Pasture (21). Although farmsteads O" small clusters of buildings may have been visible on the photographs and in the field, su~h areas should not have been classified as Residential according to the mapping specifications.

fE 43

g: 43

a: 54

Accuracy of selected maps Those categories of forest land believed to cause difficulties for the interpreter were actually mapped with a high degree of accuracy.

From figures 7 and 8, it was possible to determine the errors of omission. Major land use and land cover categories of Seattle expressed in terms of errors of omission are as shown in table 15.

The major land use and land cover categories of Tacoma expressed in terms of errors of omission are as shown in table 16.

The most obvious difference in the data for errors of omission and the data for errors of commission was on the Tacoma sheet. Cropland and Pasture (21) on the Tacoma map sheet was often overlooked. Much of tile Cropland and Pasture land was misclaf~ified as Residential (11) or Other Urban and Built-Up Land (17) resulting in large errors of commission for Residential Land and large errors of omission for Cropland and Pasture. rhould these errors be corrected, Cropland and Pasture would become a more significc'lt category on this map sheet.

Combined Results

When all the point data for these ma..,s were merged, a combined classification error matrix was constructed as shown in fi~··Jre 9. A sufficient number of points was accumulated to provide a reliable estimate of t1e interpretability of nine categories. Reside:'ltial (11 ), Cropland and Pasture (21 ), Herbaceous Rangeland (31), Mixed Rangeland (33), Deciduous Forest Land (41), Evergreen Forest Land (42), Mixed Forest Land (43), Lakes (52), and Bays and Estuaries (54) were all 85-perc.ent accurate or better at the lower confidence limit when tested for errors of commisf on. Only Residential (11) was less accurate at the lower confidence limits. The p values calculated from errors of commission for these categories where p is estimated at ± 10 percent of its true value are shown in tablo 17.

For comparison, these same categories calculated from errors of omission are shown below in table 18.

g: 43

a: 54

Both Deciduous Forest Land and Lakes were less than 85-percent accurate at the 95-percent lower confidence limits. However, the value for Deciduous Forest Land is calculated to be 84.33 which is acceptable as a lower confidence limit. Lakes, on the other hand, had too wide a confidence interval to be a reliable estimate of the accuracy. A larger sample of Lakes would be necessary for a reliable comparison of the errors of commission and of omission.

The accuracy for all points combined was 92 percent or 2,282 points of a total of 2,475 points with a 95-percent lower one-tailed confidence limit of 91 percent.

Conclusions

The selected land use and land cover maps interpreted from high-altitude photographs meet the criterion that they be mapped with an accuracy of 85 percent. Table 19 is a summary of these results. The single exception to this for those maps tested was the Tacoma, Wash., map sheet where the accuracy was less than 85 percent at the lower confidence limit. Those categories causing interpretation difficulties were not the same on all map sheets, and the number of errors were not always the highest for these categories. Errors committed in general were predictable errors between categories of similar signatures or within a Level I category. Categories in error for the most part were found to fall within the predictable error types shown in figure 1. Each user must judge the seriousness of any of these error types for his r~eds.

When all points sampled for these rna p sheets were combined, the results yielded sufficient points to provide reliable results for nine categories. All categories sampled with a significant number of points exceed the required accuracy of 85 percent. Only one category on the maps, Residential (11), was less than 85-percent accurate at the lower confidence limit. Although the sample value exceeds 85 percent, the lower confidence limit is 82 percent, indicating there may be P"Oblems in mapping that category. The errc rs of overestimating Residential would be a s-~rious disadvantage to may users, and so extra caution should be employed when mapping this category.

Although the lower confidence limit i~ the lowest value accepted (with 95-percent confidence) as the true accuracy of the map, there is just as much chance that the maps exceed the measured accuracy as fall st,ort of it. The lower confidence limit, therefore, is a conservative estimate of whether the maps meet the accuracy criteria of 85 percent. Those categories whose sample results fall short of 85-percent accuracy at the lower confidence limit may still meet or exceed 85-percent accuracy in the true population.

A more precise accuracy statement w0uld be possible if all categories on the map were sampled with a sufficient number of points to be analyzed reliably with a narrow confidence interval, a procedure which would require a

Where this page came from

This page was imported from U.S. Geological Survey. Published by the U.S. Geological Survey and, as a work of the United States government, in the public domain.

Nobody has written it yet — it is the source material at a new address, which is why search engines are asked to skip it and why no one earns from it. It is up for grabs: take it on, and it is yours to rewrite and to earn from.