Introduction to Geographic Information Systems in Forest Resources

UW Home GIS@UW Search

Syllabus Schedule Class Meetings Assignments Course Data Internet Search Current Grades Contact Us CFR 590 Internet-only section Lab Locations

Discussion:

One of the greatest strengths of GIS is the ability to integrate large amounts of data in different formats from different sources. However, this frequently creates data management nightmares. For any GIS project, no matter how simple, effective data management is absolutely essential.

Some common concerns are:

• Where are the source files?
• What types of files are they?
• What projection/coordinate system are they stored in?
• Are there multiple copies of the files?
  • If so, which are the most current or correct?
• Where are new files automatically placed?
• How do I move ArcView projects between computers?

These types of questions must be able to be answered definitively for every project, regardless of the size of the project.

The major data management problem with ArcView is that it uses two basic system file types (this does not mean data model types, such as raster and vector, but actual system files). The two basic file types are

1. ArcInfo data sets
   coverages
   grids
   TINs
   INFO tables
   
   
2. non-ArcInfo data sets
   ArcView shapefiles
   Image data (e.g., TIFF, BMP, BIL, JPEG)
   CAD drawings (DXF, DWG, DGN)
   SDE data (If Database Access is installed)
   StreetMap data (If StreetMap is installed)
   VPF data
   dBase tables
   delimited ASCII files
   

ArcInfo data sets cause a lot of data management problems due to the file structure of the source data. The files that make up ArcInfo data sets cannot be arbitrarily moved or copied around the file system. If files or directories that comprise ArcInfo data sets are moved, renamed, deleted, or otherwise altered, the result may be completely corrupt, unusable, and unretrievable data. ArcInfo data sets cannot be moved or altered in any way at the system level; they can only be managed by ArcView or ArcInfo.

Non-ArcInfo data sets do not cause the same kind of problems as do ArcInfo data sets. They can be moved around the file system without corruption, using system-level file management tools, such as command interfaces and GUI file management tools, (as long as all related files are moved as a unit).

Setting the working directory
Copying
Renaming
Archiving

Copying & Renaming
Archiving
Dealing with ArcInfo coverages

Managing non-ArcInfo data sets

Non-ArcInfo data sets can be managed using system tools (e.g., DOS commands, Windows Explorer). However, some data sets are composed of multiple single files. If you copy, rename, move, or delete these files, you need to make sure that all files are handled. If you neglect to copy or rename a single file in a multi-file data source, you could be left with corrupt data.

• Setting the working directory

ArcView gives you the ability to set a working directory, where new files created by ArcView are automatically placed. If you use this functionality, you will know where new files are placed. If you take a minute or two at the beginning of creating a project to create and set a working directory, you can save yourself literally hours of cleanup later. It also saves time during the project, as you will not need to navigate through the file system as frequently.

Using the Startup Project will help you later, since it automatically sets the working directory to the Zip drive. That means that new files and temporary files will by default be written to the Zip disk. This will avoid the problem of files being spread across the file system.

• Copying and moving

Non-ArcInfo data sets can be copied or moved freely across the file system. You can use the DOS copy, xcopy , or move commands, or a GUI-based file manager, such as the Windows Explorer, to copy these types of files from one place on the disk to another, or from a fixed drive to removable media.

In ArcView it is advised to use the ArcView Source Manager (File > Manage Data Sources) menu choice to copy, rename, or delete shapefiles, grids, or TINs. This is a more foolproof and easy way to manage these types of files, but shapefiles can be managed at the file system level without file corruption.

The only thing that you need to be aware of is the existence of multiple files. If any data sources are composed of multiple files, be sure to copy all associated files.

For example,

• Shapefile theme sources are composed of at least 3 data files. Consider a shapefile called shape:

1. shape.shp
2. shape.shx
3. shape.dbf
4. shape.sbn (may be present)
5. shape.sbx (may be present)
6. shape.ain (may be present)
7. shape.aih (may be present)
8. shape.prj (may be present)

Files 4-8 are special index files that are not necessary for the data set, but are created automatically for particular needs. A shapefile will be fully functional with files 1-3.

• Image theme sources are composed of at least two files

1. the image file itself
2. the image world file (for georeferencing)
3. the image header file (may be present)
4. the image statistics file (may be present)

See ArcView help on Images for a description of what each file does.

 

• Renaming

Renaming data sets uses the same rules as copying files. For data that are composed of multiple files, make sure to rename all associated files.

• Archiving

Archive files are generally created with such programs as WinZip, PKZip, Stuffit, or the UNIX command-line utilities tar, zip, and gzip. Archives store multiple files in a single archive file, that is usually compressed. These archives can be saved or stored for backup or recording, or used to copy large numbers of files from one machine to another or from one platform to another (such as PC to UNIX).

The same concern for copying and renaming data sources applies to archiving data. Make sure that all associated data files are archived, or when the archive is restored, not all necessary files will be present. You may end up with a nonfunctional data set.

Managing ArcInfo data sets

The following image is a schematic for how ArcInfo coverages and other ArcInfo data sets are stored. The workspace (top-level data container) is the directory jasper. Within jasper there are several other directories, including a special directory called info. The other directories (water, soil, elevatin, and vegatatn) all contain files for each of their particular data sets. The info directory contains all of the attribute tables for each of the data directories. The file structure must be maintained exactly in this format. Moving, renaming, or deleting any files will cause problems.

In general, when managing ArcInfo data sets in ArcView, there are a few rules to follow.

1. Never copy, move, delete, or rename any coverage, grid, or TIN directory using system tools (such as the Windows Explorer) unless you move the entire parent directory, including the info directory.
   
   
2. Always use the File > Manage Data Sources menu choice when renaming, copying, or moving grid or TIN data sources.
   
   
3. Close any existing project (but leave the ArcView application open) before making any alterations. Even if data sets are deleted from the ArcView project, there may be "remnants" of the data sets within the project that will prevent you from moving or renaming these data sets.

There is no support for copying, renaming, moving, or deleting ArcInfo coverage data sources in version 3.2 of ArcView. This is a little strange, as the most common data source type is the ArcInfo coverage.

• Copying and Renaming

Use the File > Manage Data Sources menu choice's dialog box to manage any grid or TIN data sets.

• Archiving

To archive single or multiple data source files, first create a new directory using the operating system, and then use the File > Manage Data Sources menu to copy the data sources to the new directory. Use one of the archiving utilities to create an archive of the entire directory, including the info directory. If you leave out the info directory, you will be left with corrupt data.

• Dealing with ArcInfo coverages

Because ArcInfo coverages are not supported in the Manage Data Sources menu controls, the only method for copying, renaming, or archiving these data sources is first to convert them to shapefiles. For coverages with multiple data types, each different data type must be loaded into a project and converted to a shapefile.

Once shapefiles are created, these can be managed easily. However, because ArcInfo coverages can be composed of multiple feature data sources (e.g., polygon, arc, labelpoint), conversion to shapefile often results in data loss.

Copying and moving projects

If you have worked with an ArcView project and attempted to open it on a different computer, you have most likely faced the "where is x data set" problem. As you should know by now, ArcView projects are simple ASCII files. Projects do not contain the data sets themselves, but merely contain pointers to where data are on the file system.

All data sets used in projects will have their location listed in the project.apr file.

It is possible to browse the file locations in ArcView projects by using system-level text search utilities (e.g., DOS/Windows find, and UNIX grep):

This searches through the project file (which is simply a special type of ASCII text file) and filters for any lines containing the string "Path:". All of the matches in this example refer to some type of data source, and point to a location on the file system. The Path statements that include $ and some capitalized text refer to variable-defined locations. Rather than hard-coding in some locations, operating system variables are used as "shorthand" to point to locations on the file system.

In the following example, the dtm_mesh theme (from the shapefile dtm_mesh.shp) is stored at the path c:/esri/av_gis30/avtutor/3d/site2/dtm_mesh.shp.

If the project opens and cannot find these files in the exact location stated in the project file, ArcView will ask where the data are on the file system. Most people who have used ArcView will have dealt with this frustration.

To move or copy projects from one system to another, there are several steps:

1. Copy all data from one machine to the other, or make sure that the data exist on the other system.
2. Place the data in the exact same directory structure on the target system.
3. Open the project.

or

1. Copy all data from one machine to the other, or make sure that the data exist on the other system.
2. Place the data anywhere on the target system.
3. Edit the project.apr file and replace all the "Path:..." statements with the correct path. This is usually done with search-and-replace functionality in a word processor or other text editor.

Before making any changes to project files, always make a backup copy!!

If projects are moved to different systems with different ArcView versions or extensions, projects may not even open.

Syllabus Schedule Class Meetings Assignments Course Data Internet Search Current Grades Contact Us CFR 590 Internet-only section Lab Locations

The University of Washington Spatial Technology, GIS, and Remote Sensing Page is provided by the College of Forest Resources and the College of Ocean and Fisheries Sciences through Unit-Specific UIF. Site administrator: Phil Hurvitz.

Copyright © Phil Hurvitz, 1998-2003