You are here

Home

Import existing data into Drupal

Submitted by Peter on Thu, 2011-09-22 20:50

How do you import existing data and databases into Drupal? This article focuses on Drupal 7 for those converting from other systems and for people who are converting from incompatible previous Drupal installations.

Drupal ID problem

Drupal uses a mixture of identifiers, integers and machine names. Machine names can work across Web sites. Site A has an item labelled "Red paint" and the item is identified internally as "red_paint".

You can copy the item to any Drupal site and the item remains as red_paint. Most items in Drupal are identified by integers, not machine names, and they crash into each other when copied. Site A has Red paint identified as item 1. Site B has Blue paint identified as item 1 and Green paint identified as item 2. Red paint is added to site B as item 3. Now you have the problem of retrieving Red paint. You import other data from A into B and the other data links to 1 instead of 3.

The problem was identified in Drupal 7 and some changes, weird and smart, are planned for Drupal 8. The Drupal 8 changes are not consistent and are aimed at whatever the developers think is configuration data, not data in general. You still have to handle the bulk of the ID problem yourself, one of the reasons I often use my own Import module.

Existing modules

Which existing modules can you use or modify to help you import data into Drupal 7? We look at the following modules.

Feeds

Feeds, drupal.org/project/feeds, is more work than Import and has the advantage that Feeds can be set up to continually add new data. Import runs when you choose and imports everything in the one import. Feeds can run from cron on a schedule and import new data on each occasion. Feeds handles RSS and a bunch of other sources fitting the bit-by-bit pattern.

On the negative side, feeds is aimed at creating nodes. If you want stock updates or anyone of the million other specialised updates required by businesses, you are better off starting with a module focused on your type of data.

Migrate

Migrate, drupal.org/node/415260, looks like a good module with documentation. Migrate is oriented to one off migrations instead of the Feeds style regular updates. The Drush bit is a less fortunate direction.

Back in the 1930s your grandmother or grandfather used a teletype to feed instructions into computers. In the 1960s, Americans replaced the teletype with the punched card. In the 1970s computer companies introduced graphical user interfaces, greatly speeding up everything. In the 1980s Unix abandoned the modern world and focused on emulating the teletype interface, calling their version the command line. Drush is an attempt to force Drupal back to the teletype interface. Mirgrate appears to prefer the teletype/command-line interface over anything from this century.

Node import

Node import, drupal.org/project/node_import, looked good at one stage in Drupal 6 then failed to develop. The developer recommends Feeds as a replacement for Drupal 7.

Power import

Power import, drupal.org/project/pimport is a module for Drupal 6 I never used and never tried to use. I include it here for completeness.

Import (D6)

Import, drupal.org/project/import is a module for Drupal 6 I never used and never tried to use. I include it here for completeness. The devedlopers of this module recommend the Migrate module.

Backup and Migrate

Backup and Migrate, drupal.org/project/backup_migrate, is another way to migrate from one Drupal setup to another or from one Drupal release to another. This module works mainly between identical implementations of Drupal. The latest release includes files. You have some choice of what is included in a backup. There is little or no facility to merge a backup from site A into a current active site B.

Import (My code)

Import, occasionally called ImportANT, is my first choice but it is not on drupal.org because of the difficulty of getting anything on drupal.org. You have to propose modules for use on drupal.org and face a barrage of posts saying CCK or Views will do what you propose, despite no one having ever published something the same using either CCK or Views. If you survive that, you may have to downgrade your code to the ancient Unix C format used at drupal.org.

Import is the name used for several similar projects spanning from the 1980s through to today across several languages and now focused on PHP. ImportANT was a name used by one customer because of a similarity to the ANT application for automating CVS updates. ANT also stood for Automated and New Technology. The name Import is already used at drupal.org for another project. Perhaps ImportANT would fit.

The code handles all sorts of situations and lets you extend the code for tour special cases. One version handles the Drupal ID problems.

Conclusion

The Feeds module is a lot of work and may be worth the work if you require ongoing incremental updates. One of the other modules migth be a better choice when you do not need an update after the initial import.