Flat-file data system

FIGURE 4 A typical directory structure found in a flat-file data system. 

Figure 7 shows the relationship between a raw data channel and its associated metadata. If we were to choose the item highlighted Instrument Method , the embedded relational database would retrieve the exact version of the instrument method that was used to acquire the raw data. All this occurs in a fraction of a second. Imagine how long it would take using a conventional flat-file system (see Figure 8). 

The kind of information managed, whether it is sales data, electronic docnments, clinical trial data, or recipes for a manufacturing execution system, is fairly independent of the database type (althongh no one wonld build a flat file database for any of these). The choice of relational vs. hierarchical vs. network is primarily dependent on business needs. 

The informatics process requires the integration and analysis of datasets in the public domain as well as from proprietary data sources that represent the intellectual property of the research organization. Public flat-file biology data are often curated and delivered from a number of public ftp sites. This information is released periodically, and any private storage of this information must be kept up to date with the public data as they are released. In addition, these data may be available in a variety of physical formats, such as flat file, Extensible Markup Language (XML), and relational. Software systems must support the ability to keep proprietary information physically separate to facilitate both the periodic update of public data and the integrity and security of private intellectual property. 

The Sequence Retrieval System (SRS [19]), initially developed at EMBL and the European Bioinformatics Institute, uses an interesting approach by combining the features of data warehouses and federated database systems. SRS is on the one hand heavily indexing locally stored genomic flat file databases and, on the other hand, it allows one to query DBMS on different sites. An example for a federated approach is the Mouse Federated Database of the Comparative Mouse Genomics Centers Consortium (http //www.niehs.nih. gov/cmgcc/dbmouse.htm). 

If the database is not integrated in a database system, the database is called a flat-frle. As the name indicates, the data arc stored in a file that can be used directly by the user. 

In the bioinformatics realm, SRS (Sequence Retrieval System) [2] is a popular system, which uses a centralized collection of data resources primarily in flat text file form and, more recently, handles XML (Extensible Markup Language) files as well. Data resources are treated in a federated manner since each is maintained in its original form. However, SRS contains a large number of cross-references between corresponding fields in various data sources, so that keyword searches can be done across them. SRS thus performs more structured searches across the information than what a simple text search provides (such as web indexes perform, for example). Even though the data model implicit in the cross-reference tables is not very deep, SRS provides a useful way for users to browse and do simple queries across a large number of data sources as well as to integrate results from some computational methods. 

Figure 9.3. MACCS— the Molecular ACCess System—an early structure indexing system. This program originally used fixed menus for searching, registration, and reporting. Later versions allowed users to customize the menus. The figure shows the result of a 3D pharmacophore search for ACE inhibitors. Out of a database of 115,000 structures, 21 fit the 2D and 3D requirements of the search query. The user could typically browse the "hits" from the search, save the list of structures to a list file, and output the structures to a structure-data file (SDFile). The MACCS database was a proprietary flat database system in which data of a given type, say, formula, was stored in a given file, indexed by the compound ID number.

---