bioDBnet: Documentation: Flexible Database Integrator

Documentation for FDI and download details

What is Flexible Database Integrator (FDI)
FDI is an integration technique developed at ABCC for integrating data stored in relational databases. FDI uses an XML file (fdi.xml) where the relations in database tables are mapped as edges in the FDI network. Network paths are then generated based on the edges defined in fdi.xml. The fdi.xml is also used by various utility functions to generate and execute the appropriate SQL queries during the traversal of the network to get from a specific input to a specific output.

Idea behind FDI
The idea for FDI stemmed from the database infrastructure at the Advanced Biomedical Computing Center (ABCC). At ABCC we were downloading publicly available biological data, parsing and uploading it into our Oracle data warehouse. As the data is not generated in-house we have no control on data modifications or the timing of the updates. So, our first need for the integration approach was that the database layer has to be completely separate from the integration layer. This has lead to the use of an XML file with edge (connection) definitions as the focal point of bioDBnet and we had the initial prototype for FDI.

Advantages of FDI
FDI package
The package Flexible Database Integrator was created as an example software for users who are interested in integrating the data in their data warehouses. It has basic code and examples for creating networks similar to bioDBnet.

Required Software
The sample FDI code provided is designed for the UNIX operating system. As it's a PHP web applications it requires Apache and PHP installed. The code to read/write/update XML files are in Perl and require the XML::Twig package (http://search.cpan.org/~mirod/XML-Twig-3.35/Twig.pm)

Downloading FDI
Please click here to download the FDI package. Once downloaded unzip the folder in your web applications directory. The README_fdi.txt file has notes and explanations on each of the files in FDI. The README file also has instructions on testing FDI and steps for creating your own networks.

Points to remember
The code in FDI is provided to give insight into a flexible database integration process. The code gives basic understanding and can be used as is to serve simple integration needs. The XML schema used as the middle layer is meant to be modified by users to suit more complex requirements.

FDI network generated by using the samples provided in the package sample FDI network diagram